More than ever we're seeing it's not dead.
LLMs just cannot handle the full context of a software project. It's too much information. Too much context. Too many dynamic, moving parts.
They are excellent, above-human level at solving fixed, specific problems, like creating individual functions and methods. Documenting functions. Creating tests for functions. Creating a class with boilerplate code. Anything where you give them exact tasks. Really difficult edge cases they can still struggle with, but generally they can do almost any single task.
What they're not good at is designing programs, adding features and debugging.
Just because they can find stupid easy obvious bugs that noob devs can't see doesn't mean they are good at debugging. They're not. The real bugs that humans struggle with, they don't find.
Basically, relying on LLMs creates an enormous amount of technical debt. Companies that go this route because of over-eager executives jumping on the AI bandwagon are in for a real surprise a year down the line.
The hallucination rate of LLMs is still massive.
O3 was found to hallucinate 33% on PersonQA answers.
And the issue is, the people who are calling for the end of programmers, are NOT programmers. They don't even know the models are hallucinating. They are unconsciously incompetent.
4o hallucinates 61% on SimpleQA, 30% on PersonQA. Even on summarisation it hallucinates 1.5% of the time.
Even 4.5 is still 37% on SimpleQA and 1.2% on summarisation.
Here's some examples of companies that got in shit relying on LLMs.
Air Canada - Chatbot told a bereaved passenger he could buy a full-fare ticket and claim a bereavement refund later—completely false. - Tribunal ruled the airline liable, ordering payment of C$650 plus fees and stressing that a bot is not a “separate legal entity.” -
https://www.theguardian.com/world/2024/feb/16/air-canada-chatbot-lawsuit?utm_source=chatgpt.com
Watsonvillle Chevrolet Dealer - A plug-and-play ChatGPT chatbot agreed to sell a brand-new Chevy Tahoe for US $1 and issued other nonsense (“legally binding—no takesies backsies”).
DPD UK - Support bot, updated to a general-purpose LLM, was tricked into calling itself “utterly useless,” swearing at the customer, and promoting rival Evri. -
DoNotPay - The “robot lawyer” LLM drafted demand letters and LLC agreements that plaintiffs said were nonsensical or legally invalid. - Class-action in California for unauthorized practice of law; June 2024 settlement in principle (terms undisclosed) after FTC scrutiny and a promised $193 k redress.
So much for lawyers being replaced..
Yeah, LLMs work well most of the time. Most of the time you can draft decent legal docs, if it hallucinates just 3% of the time, that means for every 100,000 documents, 3000 of them are invalid. That's a LOT. Human lawyers produce 10's of thousands of documents over a lifetime, with not a single one being invalid.
character.ai - Chatbots aimed at teens produced sexual role-play, violent scenarios, and messages encouraging self-harm.
And this is just easily findable very public issues.
It's a complete disaster waiting to happen for any company stupid enough to think AI is going to replace humans.
LLMs are a tool, like calculators were a tool and spreadsheets were a tool.
They're just more sophisticated.
Humans are more important than ever, and what we're likely to see is MORE dev jobs becoming available, not less.
The barrier to entry for creating software is lower with LLMs. A company can create more, and faster, so they'll want more good devs.
The ROI from a dev is going to 10x. Think about it. If you could make $5 back for every $1 of dev time and now you can make $50 back for $1 of dev time, are you going to fire devs? lol.
Or are you going to HIRE MORE.
It's the same with law.
You aren't going to replace your lawyers with an "AI lawyer". You're now able to get 10x more value from each lawyer. They can work with huge amounts of data. They can generate documents with AI, then review and tweak them. They can generate a document, review it, and ask to re-write but telling the LLM which points weren't right, and what to correct.
They can have the LLM research 1000's of similar cases looking for angles and present them to the HUMAN.
Lawyers become more valuable.
Devs become more valuable.
Humans become MORE VALUABLE.
Economic output skyrockets.
We produce more as a society.
Prices come down because we produce more. Jobs go up.
We just need to pray that no one invents something better than the transformer that replaces humans.
