I completely agree with
@courses_retailer
BTW I am deep learning engineer
and I can guarantee that every AI bot depends on mathematical algos
Still both wrong.
Machine learning models are not a mathematical algorithm.
It's not correct to say that the model its self is a mathematical algorithm.
BUILDING the model, ie, training it uses mathematical algorithms to minimize the loss function by gradient descent, which is done using calculus. Specifically partial derivatives and the chain rule.
It's akin to saying that a car engine uses a wrench to get from A to B. It doesn't.
A wrench was used to tune the engine. Once the engine is tuned and ready, it doesn't use the spanner.
Likewise, a machine learning model does not use any mathematical algorithms AT ALL once. You BUILD the machine learning model with algorithms.
This is the fundamental difference between machine learning models and traditional programs.
Traditional programs use algorithms and conditional statements to perform a task. A machine learning model does not. There's not a single algorithm you can find in a neural network.
A machine learning model will produce totally different outputs for the same input each time you run it. (Providing of course the hyperparameter temperature is high enough). But a machine learning model returns multiple options at each step and chooses one to use. This is also what the decoder hyperparameters top-k and top-p are for. To tweak what tokens are chosen. top-p is tokens who have a probability above the top-p and top-k is the top k tokens.
This isn't possible with an algorithm. An algorithm will produce 1, and only 1 answer for every input, every time.
It's really not correct to describe a model as an algorithm, let alone "just an algorithm".
The term 'algorithm' is heavily used, but it's often incorrectly used or misunderstood.
At the most fundamental level an algorithm is a set of rules that are followed.
You could in a way say that a transformer is a "set of rules", and it is.
But within that set of rules is a black box that NO ONE understands.
No one knows why machine learning models are doing what they're doing.
machine learning is mostly experimental science. So reducing it to an 'algorithm' is highly misleading.
The problem is when one is talking about 'deep learning algorithms' what's being referred to are the algorithms for training and the algorithms for deploying the transformer its self.
Take a look at
https://arxiv.org/ftp/arxiv/papers/1506/1506.01195.pdf
Look at how they use the term 'algorithm'
"CNN algorithm has two main processes: convolution and sampling ."
The 'algorithmic' components are the outer processes. It's not the part within the neural network its self after it's trained.
https://share.getcloudapp.com/OAuond7W
Normally you don't have to make this subtle distinction, but when someone's trying to reduce the power of what a deep neural network is.
again here
The 'algorithms' are the processes of that particular model and the training. It's not the actual brain of the AI, which is not just simply an algorithm.
https://www.spiceworks.com/tech/art...cles/how-does-ai-learn-through-ml-algorithms/
Notice how it says "machine learning algorithms are the computer programs that can learn from data". The algorithms are in the training. Not the final result.
