10
Caught AI hallucinating during a live demo at a tech meetup in Portland
This guy from a new startup was showing off their chatbot and it made up a fake citation to a real court case. I called him out on it and he just shrugged. How are we supposed to trust these tools for legal or medical stuff if even the creators don't check the output?
3 comments
Log in to join the discussion
Log In3 Comments
the_linda22d ago
Not saying it's great but let's calm down a bit. One startup founder being sloppy at a meetup doesn't mean every AI tool is broken for legal or medical work. People mess up demos all the time, especially when they're nervous or trying to rush through slides. The real issue is whether the company fixes it after getting called out, not that it happened at all.
5
the_tessa22d ago
Yeah, I used to think it was a bigger deal but this actually made me reconsider too.
5
hart.sage22d ago
Disagree completely. When you're pitching AI for legal or medical work, a sloppy demo isn't just nervousness, it's a red flag about how seriously they take accuracy. Doctors and lawyers face real consequences if their tools give wrong answers, so a founder who can't bother to double-check their slides probably isn't running tight enough tests on the backend either. The whole point of AI in these fields is trust, and one botched demo can undo months of that trust in minutes. If the company gets called out and still releases buggy software, then the sloppy demo was just the warning sign people ignored.
2