Tesla Fast Tracking Full Self-Driving Development With Advanced Neural Net Training Techniques

Sign up for daily news updates from CleanTechnica on email. Or follow us on Google News!

---

Autonomous driving has been one of the fundamental pillars of Tesla’s push to electrify transport, and by all accounts, the California company is leading the pack in production deployments of autonomous driving technology.

The team of engineers at Tesla working on AI are some of the brightest minds in the space and continue to roll out new, innovative ways of not only processing and interpreting computer vision, but in developing new methods to train its AI. It’s the digital equivalent of building the machine that builds the machine, the virtual equivalent to taking a step up the chain from designing automobiles to designing the manufacturing machines, processes, and systems that build them.

Senior Director of AI at Tesla Andrej Karpathy recently took on the task of frontman for Tesla’s AI team for a day as he presented Tesla’s methods for training its AI at the Scaled ML Conference in February. Along the way, he shared a ton of new updates about the company’s approach to cracking the Full Self Driving nut once and for all.

Laser-Focused on the Objective

Tesla calls its fully autonomous driving solution Full Self Driving, and that has always been the ultimate goal of Tesla’s Palo Alto–based AI team. Along the way, Tesla has built the foundational computer vision systems that have empowered its vehicles with the most advanced active safety systems in existence, in any production vehicles in the world. Combined with its best-in-class structural engineering, all of Tesla’s vehicles have taken the crown as the safest vehicles in their respective classes.

It’s no small feat for a 15 year old automotive startup that has built its business on the premise of disrupting the entire automotive market. But that’s a longer story for another day. Full Self Driving has been foundational to Tesla’s identity and to the future of the business as the key deliverable to the company unlocking the Tesla Network.

A vehicle capable of driving anywhere in the world by itself with zero human intervention will revolutionize the safety of automotive transportation, and Tesla has already made significant progress with safety. Tesla’s vehicles already have the best safety record of any vehicles out there, and as more active safety systems are deployed to them via over-the-air updates, this will only improve. Achieving a solution capable of fully autonomous driving is the holy grail when it comes to safety, and because of that, Tesla’s Full Self Driving solution is a key deliverable for the company.

Reality is a Difficult Nut to Crack

The real world, it turns out, is more irregular and unpredictable than it might seem. Something as innocuous as a standard stop comes in all sorts of shapes, sizes, configurations, and locations. On top of that, a layer of weather, trees, plants, humans, and other objects conspire against standardization to hide them from view. A fully autonomous vehicle must be smarter than this wide range of possible manifestations of stop signs to perform safely.

Following the white rabbit further down this particular hole, Tesla has leveraged its deployed fleet in a way that is not immediately intuitive in order to train its Autopilot neural net. The company has tasked vehicles driving around the real world with finding examples of stop signs or likely stop signs. The vehicles then send the resulting data surrounding the encounter back to the Tesla mothership.

Tesla is then able to analyze the ~10,000 images in the sample set and use them to not only see first hand what some of the outliers look like, but to use them directly to train the autonomous driver neural net for its vehicles. This is but one of the ways Tesla is able to tap into the power of its deployed fleet of well equipped, intelligent, connected vehicles around the world.

Scaling this up to additional use cases like street lights, lane markings, curb types, speed limit signs, and more allow Tesla to rapidly, efficiently train its neural net for an impressive range of real-world situations. The more potential situations the model trains on, the more abstract scenarios it will be able to handle. Much like human drivers, it doesn’t have to have seen everything to be able to handle new situations, but the broader its experience is with any particular application, the more robust the resulting solution will be.

Chip in a few dollars a month to help support independent cleantech coverage that helps to accelerate the cleantech revolution!

360 Degree Visibility

Understanding spacial location relative to objects around the vehicle is critical for low-speed, high-density situations like parking lots, parallel parking, and urban driving. To improve the vehicle’s self awareness, Tesla’s Autopilot team integrated the views from the onboard cameras to enable seamless 360 degree visibility. “You have to somehow project this out into a bird’s eye view for this to make sense,” Karpathy said.

It’s a non-trivial exercise that requires immense processing power and image manipulation. Remember, this is all happening in realtime. “The raw detections, we project them out into 3D and stitch them up into what we call an Occupancy Tracker,” Karpathy told the audience. “The Occupancy Tracker takes all the raw detections from 2D and we project them out into the world and we stitch them together across cameras and we stitch them together across time.”

Extrapolating flat 2D video images out into 3D leverages the massive computing power of Tesla’s third-generation hardware to build out a dynamic, geometric world that can then be used for navigation. It’s not a simple physics problem from there on out, but creating true spacial awareness makes the task much easier.

Tesla’s progress with computer vision has enabled it to develop what Karpathy calls pseudo-LIDAR. The system uses complex calculations to estimate the depth of each pixel in the frame to replicate the functionality of a laser-based LIDAR system. The calculations are performed in realtime. The full 360 degree camera views give Tesla an unparalleled view of the world around the vehicle.

A Hybrid of Static & Dynamic Code

The evolution of Tesla’s Autopilot solution is fundamentally shifting the solution from a black and white 1.0 code written by humans to a new and improved dynamic world of neural nets. “Roughly, you can think of two code bases hidden inside this software stack,” Karpathy said.

1.0 code is written by a human and formed the foundation for Tesla’s early Autopilot builds, but Tesla is moving beyond this with the increased development, training, and utilization of neural nets in production builds. “Then you have what I call software 2.0 code where the code is the outcome of an optimization,” Karpathy said. “It’s a compiler that takes your data set and a neural network code that runs in the car.”

Tesla is increasingly migrating functionality one code base at a time from human code to dynamic neural nets. “You can take functionality that runs from the 1.0 code base and put it in 2.0 code base,” he said. “So this boundary is fluid and you can actually engulf more and more of the software stack.”

Neural nets are able to dynamically respond to real-world situations much more fluidly than static hardcoded solutions are. The result is an inevitable march towards neural nets and the transition is only going to accelerate as Tesla continues to build out Full Self Driving functionality. “The neural nets have expanded in how much of software 1.0 and they’ve taken over and the trend is upwards.”

If CleanTechnica has helped you learn about Tesla or Tesla’s Energy products, feel free to use my Tesla referral code — https://ts.la/kyle623 — to get 1,000 free Supercharging miles with the purchase of a new Tesla vehicle or a $250 award after activating a new Tesla solar system. If you’re anything like me, the award serves as a nice bonus after doing something great and feels a lot like finding a toy in a box of cereal, back when that was still a thing.