Can we talk about how a free AI chat tool made up 4 fake court cases for my cousin's brief?
My cousin is a paralegal in Toledo and she used one of those free AI chat tools to help draft a legal brief last week. The thing invented 4 court cases that do not exist, complete with judge names and dates, and she almost filed it before her boss caught it. I get that these tools are getting scary good at sounding right, but sounding right and being right are two very different things, and people are going to get burned bad if we keep pretending otherwise. What is the actual fix here, better fact checking by users or should these companies be forced to add some kind of confidence warning on stuff like legal and medical answers?