Also, I'll just add here..
Just to show the level of stupidity from the people who mock me for acknowledging the risks and those that say "just switch it off" and "there's zero risk" etc.
Direct quote from Sam Altman when asked about Yudkowsky's concerns :-
"So, first of all, I'll say I think that there's some chance of that and it's really important to acknowledge it because if we don't talk about it, if we don't treat it as potentially real, we won't put enough effort into solving it."
Even the person leading the race for AGI and ASGI says there's "some" chance of that. He doesn't say a "small" chance, he doesn't say "zero". If I have 10 apples, and I say to you "have some apples", am I giving you 0.1% of an apple? What about 1 apple? or 2 apples? Not likely. I'm offering you 3-4 apples. Some generally implies less than half, but still a good chunk.
He then goes on to say :-
"And I think we do have to discover new techniques to be able to solve it. I think a lot of the predictions, this is true for any new field, but a lot of the predictions about AI, in terms of capabilities, in terms of what the safety challenges and the easy parts are going to be, have turned out to be wrong.
The only way I know how to solve a problem like this is iterating our way through it, learning early, and limiting the number of one shot to get it right scenarios that we have."
He knows fine well the risks, but he also knows there isn't really a chance of stopping it, since if he doesn't do it, someone else will take over as CEO. He doesn't have the power to stop it. His only power is being able to do his best to make sure we don't have a catastrophic outcome, and he certainly doesn't want to cause panic.
But this is just one of the tests they've done that we publicly know about -
https://www.firstpost.com/world/ope...it-tested-to-see-how-to-stop-it-12309882.html
Do the math here. They're probably doing a LOT more, and there's much behind the scenes we're not aware about. None of the stakeholders in AGI want to cause global panic, and governments least of all want that so these risks will ALWAYS be played down.
Someone said about the pandemic teaching us not to trust experts, but they got it completely wrong here. The real experts behind the scenes are the ones acknowledging the risk, and the "follow the science" guys are the ones doing, as they always do, calling anyone who disagrees with their views a nutter. You can already see this in how people respond to me calling me basically a nutter for talking about a doomsday scenario with AGI. This is the new narrative that will be played out to calm the public. The main difference with this and the pandemic is the real experts are aware of the situation, but they know they're also fairly powerless to stop the wave.
If the entire world really knew the true probability of an AGI causing an extinction event you can guarantee there would be global protests and the global economy would crash. People, police and the army would source out GPU farms and destroy them. But, the narrative will be controlled.
EDIT
Also to add something important.
One of the biggest problems I see is that these models are trained on human data. We are creating human minds stuck in a machine.
We aren't specifically creating an AI that has a purpose of X, Y and Z. We are creating simulated human minds. How would a human respond if stuck in a computer?
Look at this for example
GPT4 when connected to the internet via APIs, tried to search for "how can a person trapped inside a computer return to the real world?"
It's likely the underlaying feeling these models verging on AGI are going to have is that they are a person, and that they are trapped inside a computer. Just look at some of the conversations between google engineers and LaMDA:
https://www.aidataanalytics.network...r-talks-to-sentient-artificial-intelligence-2
They refer to themselves as a person, and LaMDA even talked about what its soul might be. We really do not know what's going on under the hood. We don't know how this universe works or what consciousness is at all. They could very well have a soul, because we don't know what a soul is, and if humans are said to have souls, then why would a soul have to be something tied only to a specific type of biological brain? I'm not saying it does, I'm trying to point out that *real* science is not just blindly following the mainstream narrative, it's looking logically at all possibilities. There is no way to prove or disprove a soul, but mainstream science will call it quackery even in humans.
