7
Sat next to a data scientist on a flight last week
He told me most people train their AI models on bad data and don't even know it. Said garbage in equals garbage out but folks skip the cleaning step. Anybody here actually vet their training data before running it through?
3 comments
Log in to join the discussion
Log In3 Comments
lunaf6715d ago
Shane's right about biased labels but the bigger issue is people not checking for class imbalance too... if one category is way more common than another the model just learns to guess that one every time. Even clean data can fool you if you don't look at the distribution first. It's like sorting apples from oranges but mostly apples end up in the training set.
7
eric_knight715d ago
Class imbalance gets brushed aside way too often because people assume more data always helps. You'd think anyone running a classifier would check the distribution first, but a lot of folks just dump in whatever they have and run with it. Does anyone actually look at the ratio before training, or is it just another checkbox people ignore?
9