Connect with us

Hi, what are you looking for?


Autonomous Vehicles

Tesla’s Andrej Karpathy Talks Neural Networks & Tesla Autopilot

As you read this, Tesla’s vehicles are out there on the world’s roads, learning, learning. Like precocious children, they’re steadily building a mental image of the world around them, a body of knowledge that forms the heart of Tesla’s Autopilot suite of features. 

Originally published on EVANNEX.
By Charles Morris

As you read this, Tesla’s vehicles are out there on the world’s roads, learning, learning. Like precocious children, they’re steadily building a mental image of the world around them, a body of knowledge that forms the heart of Tesla’s Autopilot suite of features.

One of the chief architects of this 24/7 educational endeavor is Andrej Karpathy, Tesla’s Director of Artificial Intelligence and Autopilot Vision. Karpathy is a specialist in machine learning and image recognition, two fields that are seen as the keys to autonomous driving. He recently hosted a workshop on Neural Network Multi-Task Learning, where he gave a talk entitled Multi-Task Learning in the Wilderness. (For those who prefer the executive summary, YouTuber Mother Frunker helpfully condensed his talk — scroll below to a nine-minute video.)

Karpathy begins by describing some of Autopilot’s current features: Navigate on Autopilot, which can pretty much drive your Tesla on the highway, executing lane changes and passing slower vehicles automatically; and Summon, which can bring your Tesla to you from a dark and rainy parking lot. “Then you get in like royalty,” says Karpathy. “It’s the best … when it works.”

Summary of recent presentation on neural network multi-task learning from Tesla’s Andrej Karpathy. (YouTube: Mother Frunker)

In terms of functionality, the self-driving systems being tested by potential competitors may seem the same as Tesla’s, but under the hood, they are very different. As Karpathy explains, most other autonomy systems use LIDAR to build high-definition maps, whereas Tesla’s Autopilot uses neither of those things. It relies primarily on eight cameras that provide a 360-degree view around the car. The system parses the video inputs using neural networks to create an image of the surrounding scene.

As safety is the prime goal, nothing less than 99.999% accuracy will do. Like a student determined to earn a scholarship, the system must study and study, constantly building on what it’s already learned. It’s an iterative multi-step process: build a dataset; train your network; deploy it and test it. When you notice that the network is “misbehaving,” you incorporate the errors into the training set. As Karpathy puts it, “you spin this data engine loop over and over,” and the system learns from its mistakes until it reaches the desired level of accuracy.

The future for Tesla’s Autopilot will eventually be hands-free. Image credit: Kyle Field, CleanTechnica

When you break down the number and variety of “data points” that (good) human drivers effortlessly process, it’s quite amazing. Static objects, moving objects, road signs, overhead signs, traffic lights, lane lines and road markings, curbs, crosswalks — each of these “tasks” actually has multiple sub-tasks (e.g., a moving object could be a car, a bus, a pedestrian, a cat, etc., etc.) and all must be correctly identified in real time. The vehicle passes through many different types of environments, from residential neighborhoods to bridges to tunnels to tollbooths.

As Karpathy explains, it’s not all just ones and zeros. Humans have to do a certain amount of “massaging” of the data to ensure that the system is building a useful map of the real driving environment. The end result is a system that’s already transforming the way Tesla owners drive, and is growing more capable by the day.

Karpathy’s presentation: Multi-Task Learning in the Wilderness

I don't like paywalls. You don't like paywalls. Who likes paywalls? Here at CleanTechnica, we implemented a limited paywall for a while, but it always felt wrong — and it was always tough to decide what we should put behind there. In theory, your most exclusive and best content goes behind a paywall. But then fewer people read it! We just don't like paywalls, and so we've decided to ditch ours. Unfortunately, the media business is still a tough, cut-throat business with tiny margins. It's a never-ending Olympic challenge to stay above water or even perhaps — gasp — grow. So ...
If you like what we do and want to support us, please chip in a bit monthly via PayPal or Patreon to help our team do what we do! Thank you!
Sign up for daily news updates from CleanTechnica on email. Or follow us on Google News!

Have a tip for CleanTechnica, want to advertise, or want to suggest a guest for our CleanTech Talk podcast? Contact us here.

Written By

We publish a number of guest posts from experts in a large variety of fields. This is our contributor account for those special people, organizations, agencies, and companies.


You May Also Like

Clean Transport

Elon Musk spent much of last year selling off Tesla stock, though he remains the company’s largest individual stakeholder. However, despite a disdain for...


1 in 4 new passenger vehicles in France had a plug in February.


Most people considering an electric vehicle are most worried about running out of charge and being stranded. Basically, they premature anxiety about what is...


When it comes to charging, many owners charge their Teslas at home. But when you’re out on a road trip or you simply need...

Copyright © 2023 CleanTechnica. The content produced by this site is for entertainment purposes only. Opinions and comments published on this site may not be sanctioned by and do not necessarily represent the views of CleanTechnica, its owners, sponsors, affiliates, or subsidiaries.