Connect with us

Hi, what are you looking for?

NVIDIA A100 GPU chip
Image courtesy of NVIDIA

Autonomous Vehicles

NVIDIA: Tesla’s AI-Training Supercomputers Powered By Our GPUs

This actually isn’t breaking news — I’ve been meaning to write about it for weeks, ever since I discovered it on NVIDIA’s blog. It also echoes what Danny Shapiro, NVIDIA’s Vice President of Automotive, told me in a podcast interview in February. The short story is that NVIDIA GPUs power the supercomputers that Tesla is using to train its neural networks for better and better autonomous driving.

In the blog, Shapiro noted that Andrej Karpathy, Tesla’s senior director of AI, gave a presentation on June 20 about Tesla’s autonomous driving work. Shapiro notes that, in the presentation, Karpathy “unveiled the in-house supercomputer the automaker is using to train deep neural networks for Autopilot and self-driving capabilities.” Karpathy says that it is possibly the 5th most powerful supercomputer in the world.

NVIDIA was happy to use the opportunity to point out that it powers that #5 supercomputer (and I would not be surprised if it has its GPUs inside of some of the other 4 supercomputers). “The cluster uses 720 nodes of 8x NVIDIA A100 Tensor Core GPUs (5,760 GPUs total) to achieve an industry-leading 1.8 exaflops of performance.”

Image courtesy of Tesla

If you follow the Tesla FSD Beta testers closely, you can see that an enormous amount of progress has been made on Tesla’s FSD firmware, with the progress increasingly coming from Tesla AI teaching itself — well, presumably that’s the case, but we’ll hear more on this later today during Tesla’s AI Day presentation. In the meantime, here are some more words from NVIDIA on why Tesla is using its hardware in its superdojo:

“NVIDIA A100 GPUs deliver acceleration at every scale to power the world’s highest-performing data centers. Powered by the NVIDIA Ampere Architecture, the A100 GPU provides up to 20x higher performance over the prior generation and can be partitioned into seven GPU instances to dynamically adjust to shifting demands.”

Shapiro further summarizes how Tesla AI is working to improve autonomous driving in Tesla cars, and I don’t think I can do a better job than him, so here’s more on that:

“Tesla’s cyclical development begins in the car. A deep neural network running in ‘shadow mode’ quietly perceives and makes predictions while the car is driving without actually controlling the vehicle.

“These predictions are recorded, and any mistakes or misidentifications are logged. Tesla engineers then use these instances to create a training dataset of difficult and diverse scenarios to refine the DNN.

“The result is a collection of roughly 1 million 10-second clips recorded at 36 frames per second, totaling a whopping 1.5 petabytes of data. The DNN is then run through these scenarios in the data center over and over until it operates without a mistake. Finally, it’s sent back to the vehicle and begins the process again.

“Karpathy said training a DNN in this manner and on such a large amount of data requires ‘a huge amount of compute,’ which led Tesla to build and deploy the current generation supercomputer with high-performance A100 GPUs.

You can gather more from the Karpathy presentation above, but a final highlight is that about 20 engineers work together on each neural net team. For much more on the Tesla Autopilot team in general, based on a chat I had with Technoking Elon Musk last year, see this piece: “Tesla Autopilot Innovation Comes From Team Of ~300 Jedi Engineers — Interview With Elon Musk.”

And here’s the chat I had with Danny Shapiro earlier this year:

Don't want to miss a cleantech story? Sign up for daily news updates from CleanTechnica on email. Or follow us on Google News!

Have a tip for CleanTechnica, want to advertise, or want to suggest a guest for our CleanTech Talk podcast? Contact us here.
Written By

Zach is tryin' to help society help itself one word at a time. He spends most of his time here on CleanTechnica as its director, chief editor, and CEO. Zach is recognized globally as an electric vehicle, solar energy, and energy storage expert. He has presented about cleantech at conferences in India, the UAE, Ukraine, Poland, Germany, the Netherlands, the USA, Canada, and Curaçao. Zach has long-term investments in Tesla [TSLA], NIO [NIO], Xpeng [XPEV], Ford [F], ChargePoint [CHPT], Amazon [AMZN], Piedmont Lithium [PLL], Lithium Americas [LAC], Albemarle Corporation [ALB], Nouveau Monde Graphite [NMGRF], Talon Metals [TLOFF], Arclight Clean Transition Corp [ACTC], and Starbucks [SBUX]. But he does not offer (explicitly or implicitly) investment advice of any sort.


You May Also Like


Peugeot e-208 is the 2022 best seller in France!


Following what many considered a rough 2022 for Tesla, investors are warming up to the idea of a big 2023 for the automaker, especially...


Tesla’s stock has remained a polarizing topic, especially as the company’s stock price dropped immensely throughout the last year. Bears and bulls are trying...


Following yesterday’s earnings call, shareholders of Tesla stock are pleased to see a “comeback” after a tough few months. Shares surged over 10% in...

Copyright © 2023 CleanTechnica. The content produced by this site is for entertainment purposes only. Opinions and comments published on this site may not be sanctioned by and do not necessarily represent the views of CleanTechnica, its owners, sponsors, affiliates, or subsidiaries.