That ML engineer gave me bad advice about training data size

My coworker told me I only needed 500 images to train a decent object detection model for our warehouse project. After 3 weeks of garbage results I finally ran a proper test with 5,000 images and it worked perfectly first try. Has anyone else been burned by someone pushing minimal data when you really need more?

2 comments

Log in to join the discussion

Log In

2 Comments

milaw143mo ago

Start small, iterate fast" nearly wrecked my buddy's drone project. He followed that advice for a weed detection model, used 300 images, and spent two weeks tweaking settings that never fixed the false positives. Finally his advisor told him to try 3,000, and it worked on the first real test. The small dataset just had too much variation in lighting and plant types, so the model kept memorizing weird shadows instead of actual weeds. He still gets mad talking about it.

9

shanelee3mo ago

Have you ever just assumed more data would be overkill and regretted it? I used to totally buy into the "start small, iterate fast" mindset until I wasted a month on a tiny dataset that had garbage accuracy. Now I just throw more data at the problem from the start and it saves me way more time in the long run.

6