Speculate: what the heck happened to Chris Lattner?

[chillaban]

SPECULATION:
The timeline is especially interesting considering how quickly he left after the rollout of the supposed "silky smooth" version, which people have reported as only a marginal improvement. Either Elon is delusional about the level of improvements, or perhaps Lattner made a command decision and removed one of the features/behaviors in the "silky smooth" version for testing, stability, or other reasons without first telling Elon, and that was the last straw.

I've been speculating about this too. I had originally assumed this was just Elon overstating things as usual, but wow, Chris Lattner's resume mentions Silky Smooth and > AP1 capability on two bullet points. And speaks as if it's already released (umm, it's not. Unless you went to a SC you don't have it).

That really makes me wonder who was delusional, and if it was both of them. But if both of them were delusional, sounds like they should've gotten along a ton better.

 

[NerdUno]

So they're all bullsh*tters. I feel better now.

 

[anticitizen13.7]

One camp had Vorlons and the other camp had Shadows.

Culture clash.

 

[S4WRXTTCS]

Your previous post states "His compiler and computer languages expertise made him perfect for Tesla's efforts to design their own chips", which is easy to misinterpret..Regardless, you don't need either of those when designing a new chip, unless so also want to design a new ISA, in which case you need someone like Jim more than someone like Chris. In reality, a new ISA is completely unnecessary, and most companies with their own silicon (e.g. Apple with their Ax chips, Qualcomm's Snapdragon, Nvidia Tegra, Huawei Hisilicon...) all license reference designs and/or ISA from ARM. They then amortize costs over millions in sales.

On the GPU side, you'd likewise pick CUDA from NVidia or AMD Mantle, or OpenGL/Vulkan to address programmability. None of these require reinventing the wheel, and in fact, since AP2 is NVidia DrivePX2, Tesla is implicitly committed to CUDA anyway. There's a wealth of CNN/DNN work leveraging CUDA (cuDNN), and most recent deep learning and computer vision projects leverage NVidia gear, starting from AlexNet, which used 2x GTX580s (trained for 2 weeks!) for image recognition using ConvNets. Tesla could create their own chips, but it's a pointless capital sink when NVidia builds kit like Drive PX2, and its follow-on, the Xavier SOC.

In my opinion, this entire discussion about building compilers, languages, chips etc is pointless. The hard problem of deep learning for computer vision in ADAS is a) accurate image recognition and discrimination and b) real time results to car controls. Lots of things can be recognized with great accuracy today, given adequate training and adequate time to convolve choices with high confidence and then drive the car accordingly. For that, you need deep learning experts to help you train the cars accurately using the wealth of shadow-collected data from the cars. That's where Karpathy comes in. Tesla should have hired someone like him, Yann Le Cun or another well known member of the computer vision community much sooner.

No, this is what I said.

"His compiler and computer languages expertise made him perfect for Tesla's efforts to design their own chips. So he could lead the development efforts for the compilers and tools needed to use those chips."

I'm not entirely sure how someone like you who clearly knows about these things missed that. But, okay.

You seem to be assuming that Tesla is going to be building a chip like what's on the NVidia TX2. Where there is an ARM plus the GPU, but I don't believe that's the case. My understanding is they plan on building a dedicated chip like the google tensor processor where the architecture is optimized for the kinds of deep neural nets for self driving cars. Something that might need a fair amount of software tools to fully optimize for it. As to whether it's a good idea for Tesla to build their own chips? I'm with you on that one. I just don't get the logic of it.

In any case I agree that this discussion is pointless. The only reason it came up was I firmly believe there was a place at Tesla for someone of his talents. From what's been said it seems like it's a lot less to do with his talents, and more about the Elon/Tesla culture. So I'm not sure if it would have mattered had he headed up general software instead. It might have since it's less visible, but it probably still would have led to head butting.

I am bummed by this because I feel like the Tesla SW has far too many bugs, and glitches. Where it's too a point where people are scratching their heads wondering Tesla can pull off self driving when they can't get the small stuff to work.

 

[suraj1194]

You seem to be assuming that Tesla is going to be building a chip like what's on the NVidia TX2. Where there is an ARM plus the GPU, but I don't believe that's the case. My understanding is they plan on building a dedicated chip like the google tensor processor where the architecture is optimized for the kinds of deep neural nets for self driving cars. Something that might need a fair amount of software tools to fully optimize for it.

Building dedicated chips is very capital intensive. Everyone who does this, does so on the back of a large market of cellphone/tablet sales to generate revenues. The talent needed just to build a regular chip is expensive. Even more so when building a tensor processor. I would know how well it pays, since this is my line of work They need 100s of people to handle the design and validation effort, and that does not come cheap.

Google has its search revenue to bankroll their DeepMind effort. Tesla is quite stretched cash wise already. They're in carmaking, historically not something a lot of companies survive in without going bankrupt. Chipmaking similarly is a difficult business that bankrupts companies. Tesla would find it a losing bet to re-invent the wheel. In any case a TPU is not some spectacular new creature. It's a GPU with greater FP8 support and no rasterization/shading/blender logic. NVidia already has FP8 support, and it's probably not hard for them to redo a unified core to skip graphics specific blocks.

Tesla trying to do the same thing would result in them duplicating effort *and* probably losing NVidia's goodwill, and correspondingly all the CUDA based DNN work available so far. It is not trivial to port existing DNN learning data from one platform to another.

In summary, building chips is pointless duplication of effort. Tesla would blow billions on something it doesn't need to do. Their value lies in their huge amount of AP data, and the ability to train learning algorithms from that. For that, hiring a computer vision expert is the right thing to do. A hardware guy can help them build the interface from an off the shelf PX2/Xavier platform and the car itself.

 

[Matias]

"in the past seven months, three executives overseeing the Autopilot software have left the company: Sterling Anderson, David Nistér, and now Lattner."


Elon Musk was the reason one of Apple's most famous developers left Tesla after only 6 months

 

[supratachophobia]

One camp had Vorlons and the other camp had Shadows.

Culture clash.

Great series. Are you saying that Elon makes progress through chaos then?

 

[Electroman]

"I effectively stopped regretted attrition from the team"

I am having difficulty parsing this sentence. The word 'regretted' in that sentence simply doesn't make sense. Incoherent.

 

[anticitizen13.7]

Great series. Are you saying that Elon makes progress through chaos then?

I can't claim to know exactly what goes on inside Tesla, but from everything I've read about jobs at Tesla and SpaceX, the grueling pace of work, insane deadlines/expectations, and high turnover could foster chaos.

I'm sure that the changeover from AP1 to AP2 was a mess. How much advance info would front line engineers have had about the Tesla/Mobileye relationship going south? Miscommunication in service and sales is another clue towards organizational chaos. Finally, rapid iteration and evolutionary growth seems to be part of Tesla and SpaceX organizational DNA.

I think it fair to say that Elon is closer in philosophy to the Shadows than the Vorlons, even though the Shadows were sort of the "bad guys" in B5. In fairness the Vorlons acted just as bad in the end.

 

[anticitizen13.7]

I am having difficulty parsing this sentence. The word 'regretted' in that sentence simply doesn't make sense. Incoherent.

"Regretted Attrition" is IMO corporate speak for losing talent that was beneficial to the company.

In plain English: "I stopped valuable employees from leaving the Autopilot team."

 

[smac]

For now.

However, I suspect that Tesla sees this as a future growth opportunity. Imagine a whole suite of Tesla Apps available on your primary screen (a la Apple products). From driver compatible software (voice activated, or Siri-like read outs, etc....) or even just "regular" apps for future use with FSD (Netflix like streaming?).

A large driver for Apple profitability is their closed ecosystem that people buy into. Imagine that in your car, then put that capability and make it portable into your phone via your Tesla App.

This was suggested to happen years ago... I was hugely excited about the prospect of being able to write some apps for my Tesla prior to it's delivery in 2014. ... Any chance that's likely to happen now, is passed.

It makes no sense anyway. They would be far far far better off biting the bullet, and working with Apple or Google and make the world's best implementation of CarPlay /AndroidAuto.

The budget Google/Apple have on nav and voice recognition blows away anything Tesla are likely to be able to muster in any reasonable timescale.

Even today I sometimes give up using the built in nav, and fall back to Google on my phone. I can't use Spotify offline, and as for Tesla's voice recognition, it sucks bad compared to Siri/Google.

That's before you start talking about the HUGE library of third party apps.

I'd love to be able to watch Netflix whilst parked up, especially during a charge (surely I'm not the only person who has had to sheepishly explain the screen can't play video?)

It seems to me this is all just re-invention of the wheel, and I just don't get it. We are already starting to see auto journalists pick up on the lack of CarPlay, and it's only going to get worse.




Any way back on topic. Why do I think Chris left, because he was brought in to fix a floundering project of huge importance, that was over promised and undeliverable in the timescales. Clearly they had a problem.

Him mentioning staff attrition is interesting to me. I've had to get involved in some high profile failing projects, and no matter what you do technically people want to distance themselves. They go into self protection mode and the atmosphere gets really toxic. People start blaming each other, especially cross departments. Ultimately people then leave.

To break what can turn into such a vicious circle, you need a really strong leader constantly fighting political battles while trying to keep your team all still rowing along. He was a good fit in this regard IMHO.

The fact he's being followed out the door by a number of other fairly senior team members really doesn't bode well

 

[BigD0g]

I am having difficulty parsing this sentence. The word 'regretted' in that sentence simply doesn't make sense. Incoherent.

That's the technical term in HR when you dont want to lose the person, vs someone quits and you are glad they are gone.

 

[calisnow]

@smac - good points. I hope the project succeeds - that Elon will reach deep and do what it takes to succeed with this project - if that means changing the way he manages/deals with people then when or if his back is finally against the wall I hope he'll change himself if he must rather than fail.

On the other hand this is a speculation thread and for now I'll just keep reminding myself that AP2's rate of progress keeps trucking along release after release. Plus they just started the video data gathering program.

 

[supratachophobia]

@smac - good points. I hope the project succeeds - that Elon will reach deep and do what it takes to succeed with this project - if that means changing the way he manages/deals with people then when or if his back is finally against the wall I hope he'll change himself if he must rather than fail.

On the other hand this is a speculation thread and for now I'll just keep reminding myself that AP2's rate of progress keeps trucking along release after release. Plus they just started the video data gathering program.

The progress is not impressive from an owner standpoint because Tesla has, once again, done a poor job of managing expectations.

And Elon will never change, for good or for ill. He's the hero we need, not the one we want or deserve necessarily.

 

[stefan.bc]

I'd love to be able to watch Netflix whilst parked up, especially during a charge (surely I'm not the only person who has had to sheepishly explain the screen can't play video?)

Maybe this is why 3 has the screen rotated. I know some don't like it, but they went with the practical solution.

 

[S4WRXTTCS]

Building dedicated chips is very capital intensive. Everyone who does this, does so on the back of a large market of cellphone/tablet sales to generate revenues. The talent needed just to build a regular chip is expensive. Even more so when building a tensor processor. I would know how well it pays, since this is my line of work They need 100s of people to handle the design and validation effort, and that does not come cheap.

Google has its search revenue to bankroll their DeepMind effort. Tesla is quite stretched cash wise already. They're in carmaking, historically not something a lot of companies survive in without going bankrupt. Chipmaking similarly is a difficult business that bankrupts companies. Tesla would find it a losing bet to re-invent the wheel. In any case a TPU is not some spectacular new creature. It's a GPU with greater FP8 support and no rasterization/shading/blender logic. NVidia already has FP8 support, and it's probably not hard for them to redo a unified core to skip graphics specific blocks.

Tesla trying to do the same thing would result in them duplicating effort *and* probably losing NVidia's goodwill, and correspondingly all the CUDA based DNN work available so far. It is not trivial to port existing DNN learning data from one platform to another.

In summary, building chips is pointless duplication of effort. Tesla would blow billions on something it doesn't need to do. Their value lies in their huge amount of AP data, and the ability to train learning algorithms from that. For that, hiring a computer vision expert is the right thing to do. A hardware guy can help them build the interface from an off the shelf PX2/Xavier platform and the car itself.

There is absolutely nothing about what you wrote that I disagree with. Like I said in my comment I don't get the logic of it, and what you wrote is a good summary of why it doesn't make sense.

But, from what I've read it seems like Tesla does have ambitions in building their own hardware. Or at least they did 6 months ago. Hopefully they've changed directions.

 

[neroden]

I suspect Elon thought we wanted a proper software development cycle until reality hit - such a proper process slows things down quite a bit. It produces a more reliable software on a more predictable schedule, but that gets in a way of running as fast as you possibly can and releasing bleeding edge technology. Who needs requirements and QA when you have Elon and customers with OTA updates.

I think you're right. They need a proper software development cycle, though. It seems like before Lattner they didn't even have proper bug tracking or regression testing.

 

[neroden]

The purely rational part in me would agree, but otoh, why then would so many people (myself included) buy a Samsung smartphone for example?

The problem is that Tesla isn't up to Samsung / Android standards of software quality either.

 

[neroden]

Latter wanted to change coding style, replace their development environment and version control system.

Which was badly needed.

He had his priorities backwards

Sounds like Tesla had its priorities backwards to me.

and Tesla sacked him, gave him 5 year salary in advance while barring him from commenting on his work at Tesla or working for competition.

 

[neroden]

Elon Musk: "I don’t believe in process. In fact, when I interview a potential employee and he or she says that “it’s all about the process,” I see that as a bad sign....process becomes a substitute for thinking."

Hmm. So Musk doesn't understand the degree to which processes are essential, which is very stupid of him. He'll never make it to Mars with that attitude.