That demo at CES last year changed how I train models

I was at the NVIDIA booth in Vegas last January and saw someone fine tune a language model on a laptop in real time. Took maybe 8 minutes from start to having it answer questions about a custom dataset. Before that I thought you needed a rack of GPUs just to tweak anything. Now I use LoRA adapters for almost every project and it saves me like 12 hours a week. Has anyone else switched to smaller fine tuning methods after seeing a live demo like that?

4 comments

4 Comments

harperg7621d ago

Yeah, and I bet that engineer also left out the part where he spent 3 days scrubbing the dataset and another 2 praying it didn't catch on fire? Real world LoRA is basically "celebrate for 8 minutes, then suffer for 8 hours.

julia_anderson21d ago

Oh man, same here, spent a whole week just fixing typos in my dataset.

grace8922d agoTop Commenter

That 8 minute demo was impressive but I think people miss that the dataset was probably tiny and pre cleaned. I remember talking to the engineer after and he admitted the custom dataset was like 500 examples of very structured Q&A pairs. Real world LoRA training still takes me a couple hours once I factor in data prep and validation runs.

faithcampbell21d ago

Totally agree with you @grace89, that demo was slick but so unrealistic. I tried a quick LoRA on a customer support dataset last month and even with just 300 carefully written examples it took over an hour of tweaking params and running validation to get something that didn't hallucinate product names. The preprocessing alone ate up my whole morning finding and fixing typos and inconsistent formatting. These flashy demos always skip the boring parts.