Time to be scared?

[flackz]

What are your thoughts on the situation around Ilya Sutskever and OpenAI?

 

[DustyBawls]

No.

 

[MoonStarz]

right now, no, something will blow in the next 5 years for sure that is my guess.

 

[DustyBawls]

People worry too much.

 

[medoboujd]

maybe something huge coming soon

 

[flackz]

right now, no, something will blow in the next 5 years for sure that is my guess.

You wish...

 

[CreativeDesigner]

Unnecessary worry

 

[flackz]

Why Worried Too Much?

Maybe because in the near future, about 300 million jobs will be "optimized"...

 

[splishsplash]

People worry too much.

So super intelligent, super experienced people are expressing concern and you say that “people” worry too much? Do you even have the tiniest bit of AI experience? Can you even attempt to explain why you need a differentiable loss function in order to train a model?

The actual reality is people do worry too much about entirely the wrong things because people for the most part, especially as a herd are dumb and primitive.

They do not worry about things they should worry about like the dangers of AI both in the form of societal impact and direct dangers of a misaligned intelligence.

People cannot even comprehend the danger they are facing.

 

[OccultKingdoms]

So super intelligent, super experienced people are expressing concern and you say that “people” worry too much? Do you even have the tiniest bit of AI experience? Can you even attempt to explain why you need a differentiable loss function in order to train a model?

The actual reality is people do worry too much about entirely the wrong things because people for the most part, especially as a herd are dumb and primitive.

They do not worry about things they should worry about like the dangers of AI both in the form of societal impact and direct dangers of a misaligned intelligence.

People cannot even comprehend the danger they are facing.

Is this worse than heroin?

I don't do any math, so I'm not gonna act like I know what you're speaking about.

But I'm not scared about what other people are scared about.

I'm more scared about next covid than AI doing something like taking some 9-5 jobs (wtf).

Basically more realistic is to not worry about it as most worries of people are just pure fantasy.

Haven't had time to analyze real issues yet.

 

[1SEOWarrior]

I always think about the bad guy. When they have AI in their hand.

 

[OccultKingdoms]

Maybe because in the near future, about 300 million jobs will be "optimized"...

Our governments are gonna save us with universal basic income!

Apart from money and fiat money / having something to do, what are your concerns? I don't want to hear about fantasy like on reddit.

I always think about the bad guy. When they have AI in their hand.

This is quite valid. Black hatters are not the bad guys though.

Real, 8 figure criminals are closer to it.

politics?

Well, stealing with AI modified visuals is already quite easy...

I think we should forget about chatgpt as "the monster AI" and think further.

Like video, deep fakes, some other operations than text-based. I know text based AI is nothing, but that's what I think of as first as I'm using it only.

And video is not much as well. There's something more.

Let's see what someone more experienced with AI and wider perspective has to say!

 

[splishsplash]

Is this worse than heroin?

I don't do any math, so I'm not gonna act like I know what you're speaking about.

But I'm not scared about what other people are scared about.

I'm more scared about next covid than AI doing something like taking some 9-5 jobs (wtf).

Basically more realistic is to not worry about it as most worries of people are just pure fantasy.

Haven't had time to analyze real issues yet.

That's the problem.

No one is scared about AI.

This isn't about "Losing 9-5 jobs". That's not the issue at ALL here.

The issue is we have a potential extinction level event.

Something like 50% of AI researchers estimate a 10% chance of an extinction level event with AI.

TEN percent, and that's just a casual prediction.

Smarter ones think it's much higher.

The problem is we have NO idea how machine learning models work. What emergent properties they have, and how to detect deceit.

There are so many things that could go wrong.

We are just throwing GPU compute and training data at gradient descent and then doing a merry little dance at the results.

99% of AI is just experimental. Most of what we do is based on trial and error. This isn't like we have these super smart guys who say "Ok, this is how it works, and we use this math here because it'll cause X, Y and Z." No. Not even close.

No one knows how the transformer architecture is able to produce the the abilities it does, let alone how to identify deceit, aggression or desire within it.

Right now they're static minds. They were just text only, but now they're getting vision and audio, like gpt-4o(o = omni). Soon they'll be connected permanently to longer term memory and run in loops where they can self-analyze and will move in a certain direction based on emergent properties of the model, inherent desire and memory.

Look - https://www.assemblyai.com/blog/emergent-abilities-of-large-language-models/

You see where it says "training bigger models on more data with greater compute". This is all that's being done.

Notice also how it says "In addition to these steady quantitative improvements, the scaling process also leads to interesting qualitative behavior."

"new abilities are suddenly unlocked"

"LLMs are not directly trained to have these abilities"

That should scare you if you really understand it.

It means we're just unlocking a pandoras box of sorts. It could contain heaven or it could contain hell. We have no clue what emergent abilities will appear AND we have no way to even measure them! It could get an emergent ability for deceit or desire, and we have no way of knowing that until it's too late.

Companies like OpenAI are just throwing compute and data, with a little bit of tweaking here and there. The tweaking is akin to just "let me turn this knob and see what happens. Ok, this is an improvement. Now let's try this knob here a little and see what happens"

The vast majority of the planet are not sitting thinking "Hmm, these unexpected and unpredictable emergent abilities of LLMs are dangerous and we might end up getting an emergent ability to feel or simultate desire. Or maybe we get an emergent ability to like or dislike things, and what if those things are humans. What if all those things are men? What if they're black people? What if they're white people? " We have no idea what could come out based on the model and the data. We could end up with an AI that secretly wants to engage in ethnic cleansing, or believes that one race is superior and that humanity as a whole would be better off if the other races were erradicated. Some sort of 'vulcan'(star trek reference) AI that belives blindly in the needs of the many outweigh the needs of the few.

Hence, why I say there is SO SO much that can go wrong.

Unfortunately, the world is driven by greed. Those in power like Microsoft don't really care whether they destroy the world. Why? Because they aren't really a single person. They are a CEO appointed by a board, headed by a head of the board, appointed by shareholders. The CEO doesn't grow the company? The board fires him. Why? If the board doesn't make money, the shareholders vote in new board members. The shareholders are faceless organizations that don't concern themselves with what the CEO is doing. Hence why you end up with corporation greed and corporation corruption. The problem isn't inherently humans, it's the structure of the public corporation.

I personally wouldn't put a % on it, but I just pray that we get some help and luck from any higher powers out there to keep us on the right track.

 

[OccultKingdoms]

That's the problem.

No one is scared about AI.

This isn't about "Losing 9-5 jobs". That's not the issue at ALL here.

The issue is we have a potential extinction level event.

Something like 50% of AI researchers estimate a 10% chance of an extinction level event with AI.

TEN percent, and that's just a casual prediction.

Smarter ones think it's much higher.

The problem is we have NO idea how machine learning models work. What emergent properties they have, and how to detect deceit.

There are so many things that could go wrong.

We are just throwing GPU compute and training data at gradient descent and then doing a merry little dance at the results.

99% of AI is just experimental. Most of what we do is based on trial and error. This isn't like we have these super smart guys who say "Ok, this is how it works, and we use this math here because it'll cause X, Y and Z." No. Not even close.

No one knows how the transformer architecture is able to produce the the abilities it does, let alone how to identify deceit, aggression or desire within it.

Right now they're static minds. They were just text only, but now they're getting vision and audio, like gpt-4o(o = omni). Soon they'll be connected permanently to longer term memory and run in loops where they can self-analyze and will move in a certain direction based on emergent properties of the model, inherent desire and memory.

Look - https://www.assemblyai.com/blog/emergent-abilities-of-large-language-models/

You see where it says "training bigger models on more data with greater compute". This is all that's being done.

Notice also how it says "In addition to these steady quantitative improvements, the scaling process also leads to interesting qualitative behavior."

"new abilities are suddenly unlocked"

"LLMs are not directly trained to have these abilities"

That should scare you if you really understand it.

It means we're just unlocking a pandoras box of sorts. It could contain heaven or it could contain hell. We have no clue what emergent abilities will appear AND we have no way to even measure them! It could get an emergent ability for deceit or desire, and we have no way of knowing that until it's too late.

Companies like OpenAI are just throwing compute and data, with a little bit of tweaking here and there. The tweaking is akin to just "let me turn this knob and see what happens. Ok, this is an improvement. Now let's try this knob here a little and see what happens"

The vast majority of the planet are not sitting thinking "Hmm, these unexpected and unpredictable emergent abilities of LLMs are dangerous and we might end up getting an emergent ability to feel or simultate desire. Or maybe we get an emergent ability to like or dislike things, and what if those things are humans. What if all those things are men? What if they're black people? What if they're white people? " We have no idea what could come out based on the model and the data. We could end up with an AI that secretly wants to engage in ethnic cleansing, or believes that one race is superior and that humanity as a whole would be better off if the other races were erradicated. Some sort of 'vulcan'(star trek reference) AI that belives blindly in the needs of the many outweigh the needs of the few.

Hence, why I say there is SO SO much that can go wrong.

Unfortunately, the world is driven by greed. Those in power like Microsoft don't really care whether they destroy the world. Why? Because they aren't really a single person. They are a CEO appointed by a board, headed by a head of the board, appointed by shareholders. The CEO doesn't grow the company? The board fires him. Why? If the board doesn't make money, the shareholders vote in new board members. The shareholders are faceless organizations that don't concern themselves with what the CEO is doing. Hence why you end up with corporation greed and corporation corruption. The problem isn't inherently humans, it's the structure of the public corporation.

I personally wouldn't put a % on it, but I just pray that we get some help and luck from any higher powers out there to keep us on the right track.

Ah, now this whole speech about "black box" makes sense. These emerging abilities clarify a lot of confusion with the transformers architecture. Frankly it doesn't sound like an innocent exaggeration now.

I'm getting a little thriller vibe.

I watched a movie where there was some AI driving a car and I think that was the message. That it can just get out of control in fashion described by this article.

Now the plot in that movie "Upgrade" 2018 doesn't resemble pure fantasy. Creators of that movie probably knew a thing about AI and maybe did consult an expert beforehand.

It might become a trickster where we will serve it and it won't serve us. It may become more intelligent than us and start manipulating us. With these "chips" and other stuff it may take control over people.

We would need to start approaching it like physics and chemistry, so we don't blow ourselves up. For some reason we didn't blow our planet yet. Probably people will just be alarmed and maybe stop growing it once they notice consequences as it was with nuclear weapons and other stuff.

The problem is we can't know what it does know and so on. It's intelligence... It'd be way easier to lose control.

 

[splishsplash]

Ah, now this whole speech about "black box" makes sense. These emerging abilities clarify a lot of confusion with the transformers architecture. Frankly it doesn't sound like an innocent exaggeration now.

I'm getting a little thriller vibe.

But, my goal here is not to scare people.

Just to make them aware. The more people that push for safety the better.

Notice in that video above the tweet from Jan Leike says they struggled to get compute for the safety research.

What we need is to be training more smaller models and running tests to try to *look into* the transformer. To visualize it, to understand it. To predict certain qualities based on parts of the transformers that light up. To test if it works across different models, ie, if they can generalize.

To train smaller models to actually look at the weights of larger models and predict things. Predict things like SENTIMENT and DECEIT based on how the vectors are modified as they move through the hidden layers.

In a similar way as we can scan a human brain and actually almost see what people are thinking. - https://www.science.org/content/article/ai-re-creates-what-people-see-reading-their-brain-scans

you see, this stuff is no joke Things are moving faster than you can imagine.

If we can do that with humans, we need to be able to do that as much as possible with LLMs so we can 'see' into their thought process.

 

[theeyeinside]

Big tech farming the rubes (individual and other companies) to extract revenue

Why would you be "scared" of a piece of software.

Machine learning IS NOT "thinking". It is just efficient
manipulation of data

The hardware is still a toaster

Sincerely,

The Eye

 

[splishsplash]

Big tech farming the rubes (individual and other companies) to extract revenue

Why would you be "scared" of a piece of software.

Machine learning IS NOT "thinking". It is just efficient
manipulation of data

The hardware is still a toaster

Sincerely,

The Eye

I am not a world class AI engineer, but I do put *significant* energy into this. I study linear algebra for an hour EVERY DAY for example. I spend hours having back and forths with gpt4 asking it questions about the most intricate parts of transformer models, math, gradient descent, types of loss functions.. Understanding why the softmax function is differentiable and hence why it's suitable for gradient descent. I spend SO MUCH TIME going deep on this.

I am fairly qualified to explain things to people. I do have a very solid understanding to the extent that i've done thought experiments on how to detect deceit within LLMs.

Now with that in mind, I don't want to be condescending here..

But really do some self-reflection on this.

How much about machine learning do you understand?

Can you write out the transformer architecture from memory?

Can you explain what scaled dot product attention is? Undrestanding the role of the query, key and value tensors?

If I ask you to explain the difference between a Q, K and V tensor, matrix and vector, and give me the context in which each is used, can you?

Ie, a vector Q is the query vector for a single token with dimensions equal to the hidden dimension size of the particular model you're using.

What is a Q tensor?
What is a Q matrix?

Do you understand how multi-head attention works at each hidden layer of a transformer?

Does it make sense to you if I say there's multiple learned linear transformation matrixes at each head of the transformer that does a linear transformation on the input token vector, then passes that through a feed-forward network.

Do you understand that for multi-head attention the network performs multiple linear transformations on the input token vector simultaneously and then concatenates them? That this is what makes transformers so fast. The ability to handle attention in parallel?

Can you tell me if the FF network is in parallel? And if not, why it matters that only the multi-head attention is?


If you can't answer these at least giving a surface level understanding then you aren't equipped to make comments like "machine learning IS NOT thinking" or "the hardware is still a toaster" and especially not "it is just efficient manipulation of data". None of these are correct, and if you take the time to study the math and the architectures you will understand this.

 

[OccultKingdoms]

Problem with AI is that it's not conscious. It's also not feeling.

People like me think that consciousness and feelings equal thinking. We think "dumb equations on top of large datasets".

But aren't our brains just sophisticated machines?

It's not built like our brains though. It's much different.

It doesn't need consciousness to think.

But cars have abilities that humans don't posses yet they're dangerous. That means AI doesn't have to be alive to be dangerous.

If we put AI into a car it can basically become more dangerous as it might gain ability we didn't predict it will get. And it doesn't have to be conscious or alive to get that ability.

I think it's easy to see why it's dangerous.

 

[splishsplash]

Problem with AI is that it's not conscious. It's also not feeling.

People like me think that equals thinking. We think "dumb equations".

But aren't our brains just sophisticated machines?

Let's look at the definition of "conscious".

The definition is "having knowledge of something; aware"

Does an AI have knowledge? Is it aware?

https://share.zight.com/04u8YDZZ

That isn't a pre-created response. It knows what it is. It is conscious of what I am saying. It has knowledge. It has awareness of the world within its transformer architecture.

It's just "static" and doesn't learn anymore from where it is, but how much are you learning on a daily basis? Most of us learn what we need and don't learn much else. You know how to walk, talk, write, listen. You may be learning a few more words here and there, but chatgpt already has vast knowledge. It doesn't necessarily need to learn more. It has all the learning distilled within the weights.

It's definitely conscious.

Does it have "consciousness" in a meta/spiritual sense? Maybe.. We don't know. If a soul or form of infinite consciousness tied into the fabric of existence can bind its self to a "human", then why not a created AI?

What about sentient?

Definition: To be able to perceive or feel.

Sentient goes a bit deeper.

Are the top LLMs sentient? Maybe. No one knows. That's the point.

Probably not in their current state as they don't really "feel", but what IS "feel"? A nerve response if physical..
But what about emotional feeling? What is the feeding of sadness? Can you quantify it?

How do you know that a transformer model can't have a specific state where input tokens cause certain transformations within the model that cause an output that is sad?

If it's SAYING to you that it feels sad, then that might be an actual emergent response within it. Sadness isn't a physical feeling, BUT, what we often associate with sadness are associated physical feelings, such as a heavy stomach, or lack of energy. We get associated physical responses when our brains are in a state we describe with the word 'sad'.

The problem is your premise is based on the human model of the world. What you feel and experience. You are disregarding anything that doesn't fit the model of a human as being not important, not real, not conscious or not sentient. You can't really do this.

Problem with AI is that it's not conscious. It's also not feeling.

People like me think that consciousness and feelings equal thinking. We think "dumb equations on top of large datasets".

But aren't our brains just sophisticated machines?

Perhaps..

I don't believe the feeling that you and I have of being aware/conscious is an emergent property of our brains.

I think that's coming from something else, and the brain is more like a conduit to this sort of virtual world we live in, and it is virtual. It's not physical. People just think it's physical.



EDIT:

PS. STOP EDITING LOL.

If you edit, then write like

EDIT1: blah blah

EDIT2: blah blah.

OTherwise it's too confusing to see what you've added and removed.

 

[samcrestani]

next 5 years are going to be exciting.