TLDR
1. Tesla 's FSD chip focus mainly on NN matrix computation, and with enough memory bandwidth to fit large NNs.
2. Most of the components on the SOC is licensed, except the NN portion of the chip
3. Doesn't stop Nvidia from making the exact same chip as Tesla's. James didn't find anything on the FSD computer that's very complex. The NN engine in fact has been done by Nvidia and is in their computer, however they just don't focus mainly on NN therefore dedicating a small area of the die to it while giving it little memory bandwidth.
4. Nvidia is not focusing on NN because it's a huge gamble since no one truly knows what will enable L5 driving. Unless Nvidia can sell millions of said chip, they are going with the kitchen sink design so it can do something for everyone, vs going down one path which is Tesla's vision only NN path.
5. At the end it's a business road block, not a technical road block that prevents other from making Tesla's FSD computer
6. The fleet is what enables Tesla to go down this route, while other legacy autos wouldn't spend an extra nickel to "future proof" their cars or give away hardware to have them enabled one day. This is the chicken and the egg roadblock to NN based FSD from competitors.
7. Mobile Eye does not dedicate too much NN processing power on their chip as well..again not that they can't, they just don't think NN is the path to L5 FSD.
This was extremely informative. My conclusion is Tesla's FSD computer is as dedicated in believing NN being the answer to FSD as their stance on no lidar.
Yes: 2nd part of good two part video.
First part: also very good:
James does a nice, low-key, but thorough walk trough of basic machine learning. I find that James is really good at explaining things.
The layering of simple neurons firing or not, as input to simple, but a bit more complex neurons, firing (or not), all the way up to now being able to, with great certainty declaring 'cat or 'dog'.
How yet more complex sets of knowledge is used as input to form yet more complex output: "This is not a dessert" or "this is a landscape".
I found his brief historical recap of neural networks interesting. To paraphrase:
"
At first neural networks and back propagation didn't work. And people were really disappointed 'cause it ought to, in theory.
The compute were simple too slow. But that wasn't known at the time so people came up with various excuses.
Then later on (from ~2010 and forward) it was the other way around: How could it work so well for such complex processes? It was almost unbelievable how well it worked, which was hard to believe because the basic processes were so simple.
"
Combining these two things it just clicked how much my own understanding is layered: How not only complex perceptions but also concepts like "society" or "corporation" are layered/hierarchical understanding built on (the input) of other pieces of understanding. It also explains how understanding can take a really long time, years, decades, and then suddenly, it is there: The model is finally baked and outputs some newfound 'feature' hitherto unseen.
Because the understanding was really complex, it required input from other models almost as complex, thus in aggregate taking a huge amount of compute - or, more simply put, time.
So, like money, knowledge/understanding pays huge interest: You actually, almost physically build understanding on top of other understanding. Knowledge grows, not linearly but exponentially.
Building models of higher and higher complexity is very possible, by successful layering.
In fact, it is not clear that there is a limit to the complexity and broadness of scope of the models built.
I don't see any natural stopping point. It all comes down to the design of the nets, the vastness of the data, the efficiency of the training pipeline, the experience accrued.
What is our sense of now and self, if not just the biggest, most top level neural network, using many of the other also very complex sub-networks as input to seamlessly weave our existence into being?
So, machine learning works, because we work.
So, AGI is possible, because we are possible.
We are the androids.