Book a consult
We want to understand your business needs.
There’s an old saying that is especially applicable to artificial intelligence: from big data comes big insights. Before you begin any AI project, you need to develop data awareness:
Data cleaning can be a massive undertaking, but thankfully we have some strategies and heuristics we can use. By looking at high-level patterns and eliminating problematic entries, we use our expertise to refine data so it’s useful for training the AI.
What data do you have?
How much data do you have?
What does it mean?
With data audits, you also have to know what gaps to look for in order to provide scope for future data cleaning activities.
Some organizations have an issue with what’s called ‘dark data’, in which they have terabytes of data, but don’t know its source or purpose. Dark data can be generated by IoT sensors, or can be security camera footage no one ever sees. But unpacking it, understanding it, and making use of it can provide countless benefits for businesses.
An additional consideration in data audits is compliance with privacy regulations. The General Data Protection Regulation (GDPR) applies to data collected from any users from the European Union. Other data protection regulations may apply, depending on location.
Almost a full AI engagement in and of itself, research and development generally includes a review of applicable data science literature, coding, and evaluation of multiple models to determine suitability.
On one project involving image recognition, we gave the Rorschach test to a number of different models, to find out what they “see”. We wanted to learn if the different models had any kind of bias in interpreting the images. For example, if a model has a 98% belief that it’s seeing a fighter jet, we wouldn’t want to use it in military applications. The result could be an unfortunate tendency to trying to shoot down butterflies (which can look like a jet when close to the camera, or swerving on an angle). Read the full story of what we learned
Empty files and anomalous readings can become a stumbling block for artificial intelligences. That’s why we thoroughly review all datasets to determine what records are useful (including legitimate edge cases) and what records are just noise.
Data cleaning can be a massive undertaking, but thankfully we have some strategies and heuristics we can use. By looking at high-level patterns and eliminating problematic entries, we use our expertise to refine data so it’s useful for training the AI.
Empty files and anomalous readings can become a stumbling block for artificial intelligences. That’s why we thoroughly review all datasets to determine what records are useful (including legitimate edge cases) and what records are just noise.
Data considerations include:
Missing data
Inaccurate data created by sensor issues
Determining the beginning and end of a series of inaccurate data.
The end result is a better system delivered in a shorter timeframe.
After the initial design and planning work has been done in the consultation phase, we start building and testing the working system.
