This is pretty old news. Open AI states not to use GPT-3.5/4 generated stuff for serious uses if you do, double-check the work. He took made-up case law that ChatGTP conjured up to argue in court (note the case law sounded convincing). The judge was like I have never heard of such cases, and he had to fess up that he used ChatGPT. It's a good eye-opener for many who rely solely on ChatGPT for serious work. Dudes gonna get disbarred for this; it's kinda hilarious. GPT hallucinates a little too much still, even GPT-4. For context, there is a 10% hallucination rate on GPT-4.