Claude released a new model Sonnet 3.7

[koxokot890]

Thanks.

I appreciate that. You clearly don't have a simple mind, so I thought you would appreciate the taking of a different direction rather than us continuing to prove each other wrong.

It's not that I don't believe perfect AI isn't possible. I just don't believe LLMs are it. They're going to be extremely useful and very disuptive to the economy.

When I say we're in an AI bubble, that doesn't mean AI is useless. We had a tech bubble, and tech is far from useless.

It's a bubble in the sense that there's too much mania. Investors are throwing money at garbage and I believe the main problem is that with LLMs instead of focusing on AGI, we should be focused on fine-tuning AI to augment human work.

I just don't think AI augmentation when it comes to development is healthy for a human because we *need* the experience and time solving problems otherwise we never become the 30+ year master devs.

What interests me more is building AI products that augment other areas in work. Research, assessment, error checking, fact checking, planning and in general doing things with large data that is hard for humans.

For example, let's say you need a consultant to do some sort of compliance work once a year for your company. You may have to pay for 40 hours of that consultant's time. With AI we can fine-tune models to do most of that and then have the consultant do 2 to 4 hours of work instead. Mostly just checking.

Doctors can make good use of fine-tuned LLMs to help them diagnose. They can test theories by conversing with LLMs and ask for extra information that would be helpful.

There are many great uses of it.

The best AI content checker is pangram.com btw. It has the lowest false-positives AND the lowest false-negatives. Originality.ai has low false-positives, but it has really high false-negatives, and often mistakes human text for AI.

View attachment 425261

I agree with what you're saying; it's the right approach—fine-tuning and cutting costs. I agree we're in a bubble, isn't that just natural?

That's how the system operates. Let's not forget that the marketing department has a job to do... I sense there are patterns here that have been observed in other contexts.

 

[nakamura]

So far I didn't have any problem with Sonnet 3.7. Works absolutely fine with me, especially when it comes to debugging things in my SwiftUI / iOS apps, and I am not even using their "deep thinking" feature. I can definitely feel and see a different to Sonnet 3.5.

 

[4440]

It seems like Anthropic is doubling down on coding.

While I don't believe LLMs can ever replace humans and this approach is futile, I do believe they make fantastic assistants when guided properly and will make dev teams much more productive.

Anthropic has a team dedicated to coding.

Coding is the easiest and most promising AI use cases as you can validate the response of AI coding automatically.

 

[splishsplash]

Anthropic has a team dedicated to coding.

Coding is the easiest and most promising AI use cases as you can validate the response of AI coding automatically.

How?

This would be the holy grail in all dev. To be able to validate that you have working bug-free code.

I just don't see how that's possible from an LLM when they struggle to add new features without breaking old ones.

They break down with anything more than hobbylist level CRUD/API websites. They can't even make real applications right now. They just make react boilerplate with a few bells and whistles.

Try having claude 3.7 create a usb device driver for linux, or a packet sniffer, with the ability to deconstruct packets at every OSI layer, edit, rebuild and reinject, or have it create netflix, or slack, or the software that runs this forum, vBulletin.

There is a world apart from hobbyist CRUD/API react apps and real world engineering.

Everything people are making with LLMs right now is just simple CRUD, like todolist apps, or API wrappers. It's barely a couple of steps beyond boilerplate. What LLMs do right now is more like sophisticated boilerplate with the ability to add in a few basic features.

The features they add in are so basic and have no thought.

Imagine you've got an nextjs app that needs to manage logistics for a trucking company. You couldn't even attempt this with claude 3.7. The architecture alone would require a human to sit down and spend hours working with stakeholders to figure out requirements.

This can't just be a frontend, with a few tables and a mysql database.

Something like this takes expert level human devs months to create and they maintain it for years.

Ask claude 3.7 to add the ability to log messages throughout the microservicse architecture to a rabbitmq exchange, making sure it sets up the appropriate queues, exchanges and bindings. Different types of log messages classified into different classes should be routed to different queues, and a log monitor micro service should be setup to route critical messages to the appropriate handlers.

It won't even know where to begin. It's so vastly beyond anything it's capable of. There is SO much to consider when setting this up. It requires incredibly careful planning and consideration of not only the microservices architecture, but the business logic of the logistics app.

Then, who's going to handle redundancy and backups in the system? An LLM?

Claude 3.7 is great at setting up a nice looking website with pregenerated content, tables, some CRUD and maybe an API call or 2 to openai and a submit button, but beyond that.. This is not what companies are doing in the real world, and it's why no company is using LLMs in their production code. Only hobbyists.

 

[4440]

How?

This would be the holy grail in all dev. To be able to validate that you have working bug-free code.

I just don't see how that's possible from an LLM when they struggle to add new features without breaking old ones.

They break down with anything more than hobbylist level CRUD/API websites. They can't even make real applications right now. They just make react boilerplate with a few bells and whistles.

Try having claude 3.7 create a usb device driver for linux, or a packet sniffer, with the ability to deconstruct packets at every OSI layer, edit, rebuild and reinject, or have it create netflix, or slack, or the software that runs this forum, vBulletin.

There is a world apart from hobbyist CRUD/API react apps and real world engineering.

Everything people are making with LLMs right now is just simple CRUD, like todolist apps, or API wrappers. It's barely a couple of steps beyond boilerplate. What LLMs do right now is more like sophisticated boilerplate with the ability to add in a few basic features.

The features they add in are so basic and have no thought.

Imagine you've got an nextjs app that needs to manage logistics for a trucking company. You couldn't even attempt this with claude 3.7. The architecture alone would require a human to sit down and spend hours working with stakeholders to figure out requirements.

This can't just be a frontend, with a few tables and a mysql database.

Something like this takes expert level human devs months to create and they maintain it for years.

Ask claude 3.7 to add the ability to log messages throughout the microservicse architecture to a rabbitmq exchange, making sure it sets up the appropriate queues, exchanges and bindings. Different types of log messages classified into different classes should be routed to different queues, and a log monitor micro service should be setup to route critical messages to the appropriate handlers.

It won't even know where to begin. It's so vastly beyond anything it's capable of. There is SO much to consider when setting this up. It requires incredibly careful planning and consideration of not only the microservices architecture, but the business logic of the logistics app.

Then, who's going to handle redundancy and backups in the system? An LLM?

Claude 3.7 is great at setting up a nice looking website with pregenerated content, tables, some CRUD and maybe an API call or 2 to openai and a submit button, but beyond that.. This is not what companies are doing in the real world, and it's why no company is using LLMs in their production code. Only hobbyists.

Sorry. my statement is too vague that may get you carried away.

Let me rephrase.

"Coding is the easiest and most promising AI use cases as you can validate the response of AI coding automatically."

What I mean is, the code generated by AI can be tested by automated test assertions without human involved. Give it a bunch of input and see if the code generates the desired and correct output. And you can benchmark the quality of code by a lot of non-human involved standards.

So it is a lot easier to train for AI's coding ability.

For other use cases like writing, reasoning, most of the time you need human involved to judge and rate the quality of the AI output.

I didn't mean "AI is able to validate that the code it generated is bug-free."

 

[splishsplash]

Sorry. my statement is too vague that may get you carried away.

Let me rephrase.

"Coding is the easiest and most promising AI use cases as you can validate the response of AI coding automatically."

What I mean is, the code generated by AI can be tested by automated test assertions without human involved. Give it a bunch of input and see if the code generates the desired and correct output. And you can benchmark the quality of code by a lot of non-human involved standards.

So it is a lot easier to train for AI's coding ability.

For other use cases like writing, most of the time you need human involved to judge and rate the quality of the AI output.

I didn't mean "AI is able to validate that the code it generated is bug-free."

Test driven development is a hotly debated topic. Some people find it stupid, others like it.

https://medium.com/@csjcode/why-tdd-sucks-650eebd62d8c

https://geometrian.com/projects/blog/test_driven_development_is_stupid.html

https://www.quora.com/How-could-anyone-actually-believe-that-test-driven-development-is-a-good-idea

It only catches the "obvious" stuff. It doesn't really catch many bugs. It's mostly good for libs to make sure there's no obvious weird stuff going on.

You can take TDD to the next level using actual UI test based scripts using selenium. This is a step up, but it's still not going to catch everything.

I'm always talking here about how LLMs suck for code, so let me show you a real example.

Back in 2023 I created this thread:

https://www.blackhatworld.com/seo/j...h-ai-fine-tuning-only-advanced-level.1501959/

One of the things I had to do was write a program called "get_structure_for_classify_webpage.py"

I shared the full code here - https://github.com/tbelfort/ai-seo-tools/blob/main/get_structure_for_classify_webpage.py

It works perfectly to this day. You can literally just copy and paste the code in, install the requirements(Sorry I didn't make a requirements.txt!) and run

python get_structure_for_classify_webpage.py github.com

And it'll run it on github.com's homepage, or any other site you want. (It only won't work on ones that have scraping protection, but that's not important and easy to fix by just using the scrapfly API).

This gives us a huge output with stuff like this



It is a complete representation of a web page, which I used to train models to do cool things that I use in one of my SaaS products.

Here's the hardest part:-


'template_string': 'a,a,h2,a,a,ul(li(ul(li(a),li(a),li(a),li(a),li(a),li(a),li(a),li(a)),ul(li(a),li(a),li(a),li(a))),li(ul(li(a),li(a),li(a), li(a)),ul(li(a),li(a),li(a),li(a)),ul(li(a),li(a),li(a),li(a),li(a)),a),li(ul(li(a),li(a),li(a),li(a),li(a)),ul(li(a),li(a),li(a),li(a),li(a),l i(a))),li(ul(li(a)),ul(li(a)),ul(li(a),li(a),li(a))),li(ul(li(a)),ul(li(a),li(a),li(a))),li(a)),h1,ul,a,h1,p,h1,h2,p(a),a,a,a,a,a,h1,p,a,h2,ul( li,li,li,li,li),p,h2,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,h2 ,p,img,h3(a),a,img,p,a,img,p,a,img,img,img,img,h3,p,a,h3,p,a,h3,p,a,h3,p,a,h2,p,img,h3,a,img,p,a,img,p,a,img,p,a,img,h3(a),h3(a),h2,p,img,img,h 3,a,blockquote,img,img,img,img,h3,p,a,h3,p,a,h3,p,a,h3,p,a,h2,a(img,img,p),a(img,img,p),a(img,img,p),a,a,h2,p,a,h2,ol(li(p(a,a)),li(p(a))),a,h2 ,a,h3,p,a,h3,ul(li(a),li(a),li(a),li(a),li(a),li(a),li(a),li(a),li(a)),h3,ul(li(a),li(a),li(a),li(a),li(a),li(a)),h3,ul(li(a),li(a),li(a),li(a) ,li(a),li(a),li(a)),h3,ul(li(a),li(a),li(a),li(a),li(a),li(a),li(a),li(a),li(a)),ul(li,li(a),li(a,a),li(a),li(a),li,li),ul(li(a),li(a),li(a),li (a),li(a),li(a),li(a))',


This template string represents the html structure as a tree, but in a string format to fine-tune an LLM.

You see at the start we have a,a,h2. That's just <a></a><a></a><h2></h2>

If we have

<h2></h2><p>blahblah <b><a></a></b></p>

This is a tree with 3 layers down, p-->a-->b

This is represented as

'template_string': 'h2,p(b(a))'

This code was HARD to do. It's not a trivial problem.

I had claude 3.7 try it, in extended thinking mode, by showing it the exact output from my program and it completely failed. Not only did it fail at generating the correct template_string, but it had bugs in the output and some parts were wrong.

Total fucking mess.

No one is going to be able to fix that garbage code it's produced and they can reprompt it as many times as they like. It won't be able to do it.

Maybe gpt 5 will, but this is only an example of a non-trivial real world problem.

Imagine the example of a full logistics app. It can't even handle 1 simple non-trivial real world function let alone an entire microservices architecture for a real world logistics company with 1000's of trucks.

I'm going to create another thread and show the full code/output from 3.7 so people can inspect and see I'm not just making this up.

This is why I stick with my assessment that LLMs are only useful for hobbyist tools.

It's like creating a little circuit with flashing lights vs building a nuclear power plant.

 

[koxokot890]

Test driven development is a hotly debated topic. Some people find it stupid, others like it.

https://medium.com/@csjcode/why-tdd-sucks-650eebd62d8c

https://geometrian.com/projects/blog/test_driven_development_is_stupid.html

https://www.quora.com/How-could-anyone-actually-believe-that-test-driven-development-is-a-good-idea

It only catches the "obvious" stuff. It doesn't really catch many bugs. It's mostly good for libs to make sure there's no obvious weird stuff going on.

You can take TDD to the next level using actual UI test based scripts using selenium. This is a step up, but it's still not going to catch everything.

I'm always talking here about how LLMs suck for code, so let me show you a real example.

Back in 2023 I created this thread:

https://www.blackhatworld.com/seo/j...h-ai-fine-tuning-only-advanced-level.1501959/

One of the things I had to do was write a program called "get_structure_for_classify_webpage.py"

I shared the full code here - https://github.com/tbelfort/ai-seo-tools/blob/main/get_structure_for_classify_webpage.py

It works perfectly to this day. You can literally just copy and paste the code in, install the requirements(Sorry I didn't make a requirements.txt!) and run

python get_structure_for_classify_webpage.py github.com

And it'll run it on github.com's homepage, or any other site you want. (It only won't work on ones that have scraping protection, but that's not important and easy to fix by just using the scrapfly API).

This gives us a huge output with stuff like this

View attachment 425601

It is a complete representation of a web page, which I used to train models to do cool things that I use in one of my SaaS products.

Here's the hardest part:-


'template_string': 'a,a,h2,a,a,ul(li(ul(li(a),li(a),li(a),li(a),li(a),li(a),li(a),li(a)),ul(li(a),li(a),li(a),li(a))),li(ul(li(a),li(a),li(a), li(a)),ul(li(a),li(a),li(a),li(a)),ul(li(a),li(a),li(a),li(a),li(a)),a),li(ul(li(a),li(a),li(a),li(a),li(a)),ul(li(a),li(a),li(a),li(a),li(a),l i(a))),li(ul(li(a)),ul(li(a)),ul(li(a),li(a),li(a))),li(ul(li(a)),ul(li(a),li(a),li(a))),li(a)),h1,ul,a,h1,p,h1,h2,p(a),a,a,a,a,a,h1,p,a,h2,ul( li,li,li,li,li),p,h2,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,img,h2 ,p,img,h3(a),a,img,p,a,img,p,a,img,img,img,img,h3,p,a,h3,p,a,h3,p,a,h3,p,a,h2,p,img,h3,a,img,p,a,img,p,a,img,p,a,img,h3(a),h3(a),h2,p,img,img,h 3,a,blockquote,img,img,img,img,h3,p,a,h3,p,a,h3,p,a,h3,p,a,h2,a(img,img,p),a(img,img,p),a(img,img,p),a,a,h2,p,a,h2,ol(li(p(a,a)),li(p(a))),a,h2 ,a,h3,p,a,h3,ul(li(a),li(a),li(a),li(a),li(a),li(a),li(a),li(a),li(a)),h3,ul(li(a),li(a),li(a),li(a),li(a),li(a)),h3,ul(li(a),li(a),li(a),li(a) ,li(a),li(a),li(a)),h3,ul(li(a),li(a),li(a),li(a),li(a),li(a),li(a),li(a),li(a)),ul(li,li(a),li(a,a),li(a),li(a),li,li),ul(li(a),li(a),li(a),li (a),li(a),li(a),li(a))',


This template string represents the html structure as a tree, but in a string format to fine-tune an LLM.

You see at the start we have a,a,h2. That's just <a></a><a></a><h2></h2>

If we have

<h2></h2><p>blahblah <b><a></a></b></p>

This is a tree with 3 layers down, p-->a-->b

This is represented as

'template_string': 'h2,p(b(a))'

This code was HARD to do. It's not a trivial problem.

I had claude 3.7 try it, in extended thinking mode, by showing it the exact output from my program and it completely failed. Not only did it fail at generating the correct template_string, but it had bugs in the output and some parts were wrong.

Total fucking mess.

No one is going to be able to fix that garbage code it's produced and they can reprompt it as many times as they like. It won't be able to do it.

Maybe gpt 5 will, but this is only an example of a non-trivial real world problem.

Imagine the example of a full logistics app. It can't even handle 1 simple non-trivial real world function let alone an entire microservices architecture for a real world logistics company with 1000's of trucks.

I'm going to create another thread and show the full code/output from 3.7 so people can inspect and see I'm not just making this up.

This is why I stick with my assessment that LLMs are only useful for hobbyist tools.

It's like creating a little circuit with flashing lights vs building a nuclear power plant.

Narratives sell, systems don't.

I don't know why I'm getting the feeling that people will become dumber in a world full of LLMs that solve trivial problems.

If you stop to think for a moment, there are people who only learn React and nothing else, and React job offers abound everywhere. Even Svelte had to come up with a silly justification to bring you closer to patterns similar to React in a desperate attempt to bite off a larger piece of the pie. Perhaps the future that awaits us is that there will be an immense shortage of well-prepared engineers, since the machine is going to make people stupid.

You have to flee from the path the flock is taking, but that only applies depending on the role that has been assigned to you in this video game.

 

[splishsplash]

Narratives sell, systems don't.

I don't know why I'm getting the feeling that people will become dumber in a world full of LLMs that solve trivial problems.

If you stop to think for a moment, there are people who only learn React and nothing else, and React job offers abound everywhere. Even Svelte had to come up with a silly justification to bring you closer to patterns similar to React in a desperate attempt to bite off a larger piece of the pie. Perhaps the future that awaits us is that there will be an immense shortage of well-prepared engineers, since the machine is going to make people stupid.

You have to flee from the path the flock is taking, but that only applies depending on the role that has been assigned to you in this video game.

View attachment 425612

Yes, this is the problem. When you rely on LLMs to solve all the trivial problems, your brain literally starts to rot.

I just tested that problem above with gpt 4.5 since I saw it's available.

It got it right.

First LLM to manage that one.

But.. $150/1M output cost. That's potentially more expensive than a senior dev now, and all it's proven that it can do is build a string representing an html tree.

It still won't be able to build the large logistics app, because it's not a problem you can just put it into a context window.

I can see that using LLMs in the way that they use the computer on star trek could be good though. I don't know if you watch star trek, but they don't say to the computer "Build me a design for a new star ship". They say things like "Calculate the spatial density of the black hole relative to the warp drive at a hypothetical speed of Warp 4.5."

Then they get something back, and ask it to do something else.

I still think the first real AGI we'll see will be like a commander data that will have to interact slowly with the world like we do. It's hard to "speed up" learning in the real world the way you can with just throwing textual data. Interaction/feedback is limited by the speed of reality.

Unless of course you can simulate the universe inside a computer, then you could potentially create an AI inside there that would learn fast.

Then you might ask, are we that AI inside a simulated universe? ;-)

 

[koxokot890]

Yes, this is the problem. When you rely on LLMs to solve all the trivial problems, your brain literally starts to rot.

I just tested that problem above with gpt 4.5 since I saw it's available.

It got it right.

First LLM to manage that one.

But.. $150/1M output cost. That's potentially more expensive than a senior dev now, and all it's proven that it can do is build a string representing an html tree.

It still won't be able to build the large logistics app, because it's not a problem you can just put it into a context window.

I can see that using LLMs in the way that they use the computer on star trek could be good though. I don't know if you watch star trek, but they don't say to the computer "Build me a design for a new star ship". They say things like "Calculate the spatial density of the black hole relative to the warp drive at a hypothetical speed of Warp 4.5."

Then they get something back, and ask it to do something else.

I still think the first real AGI we'll see will be like a commander data that will have to interact slowly with the world like we do. It's hard to "speed up" learning in the real world the way you can with just throwing textual data. Interaction/feedback is limited by the speed of reality.

Unless of course you can simulate the universe inside a computer, then you could potentially create an AI inside there that would learn fast.

Then you might ask, are we that AI inside a simulated universe? ;-)

I'm not as optimistic about achieving AGI, as it would be the end of civilization. Can you imagine? The first country to develop it would use it for military purposes. You could ask it, "How do I conquer country Z?" and it would lay out the entire strategy, coordinating everything with strategies the enemy can't even imagine. The paranoia of the rival country would be so great that a preemptive EMP attack against the other country wouldn't be an outlandish idea. But even AGI would foresee that scenario.

Before we reach AGI, it's most likely that civilization will collapse. This would explain the Drake equation and why we don't have machines visiting us. After all, they are immortal and could travel enormous distances in space. It would also be consistent with the idea that the simulation (the universe) operates in cycles.

At a high level, to avoid adding complexity here, let's imagine for a moment: If stars undergo a cycle of birth, growth, and death, why wouldn't a civilization follow a similar cycle?

The end of the species isn't so bad, after all. Look at the exponential development humanity has had. So far, we estimate the age of the universe to be about 13.5 billion years, and that of Earth to be about 5 billion years. Civilization itself is only between 7,000 and 10,000 years old, and observe the technological advancements. How long have we had electricity? And well, the end will be brief, like a movie. Perhaps this is what's running within the simulation's game loop.

I suspect that Elon knows this and that's why he's desperate to leave the planet.

I don't think it's likely that we can simulate the universe in its entirety, since the universe itself is a simulation, lol. And even though we're in a "virtual box," the funny thing is that the universe doesn't allow itself to be observed; it's not there when no one is observing it (double-slit experiment).

The most likely scenario, as you say, is that AI will evolve as an assistant, and the quality of the response will depend on the user's input, and of course, on fine-tuning. Just as there are thousands and thousands of books, and ways to do things, perhaps there will be many personalized assistants that solve trivial problems.

Now, someone with advanced knowledge might be able to solve difficult problems with the help of a diverse group of LLMs that have been fine-tuned, and that's possible. However, I highly doubt it will be available to the general public, and it remains to be seen if it is profitable from a business perspective. Moreover, depending on the domain of the problem, the personalized solution would also vary; here we would be speculating a lot. That said, I find it extremely improbable that a single person, even with unlimited resources, could build an aircraft carrier using many assistants.

In practice, the effectiveness of these tools will depend heavily on how individuals use them. Some will use the LLMs correctly, while others will shoot themselves in the foot and start losing skills, since these aren't reinforced unless there's constant repetition and error correction. Memories degrade, you forget what you don't practice constantly. We already know that history repeats itself or rhymes, so there's no need to do what the majority is doing.

Even the simulation is ironic, and it seems that almost no one is exempt from the laws of Pareto. A minority will use the LLMs correctly, while the majority will gradually become more stupid and lazy—nothing new under the sun.