Stumbled on a stat that 94% of AI training data never gets used

I was reading through a research paper from MIT last night (kill me, I know) and they found that most companies only use like 6% of their collected data for actual model training. The rest just sits on servers burning electricity. That blew my mind because I always figured you'd want as much data as possible, but apparently quality over quantity is even more extreme than I thought. Has anyone else seen numbers like this or found ways to cut down on wasted data storage?

3 comments

3 Comments

taylorc4012d ago

Cut my dataset by half. Model actually got better.

ellis.leo12d ago

That "cut my dataset by half. model actually got better" line really got me thinking @taylorc40. What kind of data did you end up cutting out? Was it like low quality stuff or just random samples? I always wonder if people are trimming noisy data or if they're actually cutting good data that was somehow throwing the balance off.

lunag3012d ago

Yeah I heard someone say garbage data drags everything down more than missing data ever could.