Case Study EXPLAINED: How TESLA is using Neural Networks?

6 min readMar 4, 2021

“Tesla, Inc. is an American electric vehicle and clean energy company based in Palo Alto, California. Tesla’s current products include electric cars, battery energy storage from home to grid-scale, solar panels, and solar roof tiles, as well as other related products and services.” ~Wikipedia

Tesla, The revolutionary invention and the world’s largest and success Electric Car were developed, created & architected by one of the world’s billionaires, Elon Musk. The man behind Tesla. Everything which God does not create, is created by Humans we know this fact, but this is also a fact that Tesla car can go on the roads without having a driver inside the car. Is this magic? Obviously not. This is science, and according to this article topic, this is Data Science. Data Science has a category or subject called Deep Learning, or generally, it’s a subclass of Artificial Intelligence. Deep Learning has a fascinating topic called Neural Networks. In this article, we’ll see how Tesla is using Neural Networks behind the scenes.

Components of Autopilot AI

Tesla develops and deploys autonomy at scale. The company believes that an approach based on advanced Artificial Intelligence (AI) for vision and planning, supported by efficient use of inference hardware is the only way to achieve a general solution to full self-driving.

Hardware
Neural Networks
Autonomy Algorithms
Code Foundations
Evaluation Infrastructure

The Role of Neural Networks

Apply cutting-edge research to train deep neural networks on problems ranging from perception to control. Tesla said that their per-camera networks analyze raw images to perform semantic segmentation, object detection, and monocular depth estimation. Their birds-eye-view networks take video from all cameras to output the road layout, static infrastructure, and 3D objects directly in the top-down view. Their networks learn from the most complicated and diverse scenarios in the world, iteratively sourced from their fleet of nearly 1M vehicles in real-time. A full build of Autopilot neural networks involves 48 networks that take 70,000 GPU hours to train 🔥. Together, they output 1,000 distinct tensors (predictions) at each timestep. Want to visualize? Have a look at the work as shown in the video below

Summarized Explanation of Patent by Tesla, Inc. titled with “US20200210832 — SYSTEM AND METHOD FOR ADAPTING A NEURAL NETWORK MODEL ON A HARDWARE PLATFORM”

(Content is originally taken from the mentioned patent)

One embodiment is a method implemented by a system of one or more processors. The method may include: obtaining neural network model information comprising a plurality of decision points associated with a neural network, wherein one or more first decision points are associated with a layout of the neural network; accessing platform information associated with a hardware platform for which the neural network model information is to be adapted; determining, based on the platform information, constraints associated with adapting the neural network model information to the hardware platform, wherein a first constraint is associated with a processing resource of the hardware platform and wherein a second constraint is associated with a performance metric; and generating a candidate configuration for the neural network via execution of a satisfiability solver based on the constraints, wherein the candidate configuration assigns values to the plurality of decision points.

Another embodiment is a system comprising one or more processors and non-transitory computer storage media storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations including: obtaining neural network model information comprising a plurality of decision points associated with a neural network, wherein one or more first decision points are associated with a layout of the neural network; accessing platform information associated with a hardware platform for which the neural network model information is to be adapted; determining, based on the platform information, constraints associated with adapting the neural network model information to the hardware platform, wherein a first constraint is associated with a processing resource of the hardware platform and wherein a second constraint is associated with a performance metric; and generating a candidate configuration for the neural network via execution of a satisfiability solver based on the constraints, wherein the candidate configuration assigns values to the plurality of decision points.

Yet another embodiment is a non-transitory computer storage media storing instructions that when executed by a system of one or more processors, cause the one or more processors to perform operations including: obtaining neural network model information comprising a plurality of decision points associated with a neural network, wherein one or more first decision points are associated with a layout of the neural network; accessing platform information associated with a hardware platform for which the neural network model information is to be adapted; determining, based on the platform information, constraints associated with adapting the neural network model information to the hardware platform, wherein a first constraint is associated with a processing resource of the hardware platform and wherein a second constraint is associated with a performance metric; and generating a candidate configuration for the neural network via execution of a satisfiability solver based on the constraints, wherein the candidate configuration assigns values to the plurality of decision points.

Behind the System

FIG. 1 is a schematic representation of an example model configuration system.

As shown in FIG. 1, the model configuration system 100 can include: a hardware platform 102, a neural network model 105, a model configuration platform 110, a traversal module 120, a constraints module 130, a constraint satisfaction solver 140 (e.g., SMT solver 140), a datastore 150, a configurations module 160, and a performance module 170.

The Method

FIG. 2 is a flowchart representation of the object detection method.

S210 includes traversing a neural network model to identify decision points, as described above with respect to the traversal module 120.
S220 includes identifying variable constraints, model constraints, and performance constraints, as described above with respect to the constraints module 130.
S230 includes executing an SMT solver for the neural network model, as described above with respect to the SMT solver 140.
The method can optionally include, S240, which includes receiving candidate configurations, as described above with respect to the configurations module 160.
S250 includes determining that the candidate configurations are satisfiable, as described above with respect to the configurations module 160.
S260 includes determining a configuration that satisfies one or more target performance metrics, as described above with respect to the performance module 170.

Embodiments of the system and/or method can include every combination and permutation of the various system components and the various method processes, wherein one or more instances of the method and/or processes described herein can be performed asynchronously (e.g., sequentially), concurrently (e.g., in parallel), or in any other suitable order by and/or using one or more instances of the systems, elements, and/or entities described herein.

Additional Embodiment

All of the processes described herein may be embodied in, and fully automated, via software code modules executed by a computing system that includes one or more computers or processors. The code modules may be stored in any type of non-transitory computer-readable medium or other computer storage device. Some or all the methods may be embodied in specialized computer hardware.

Read the original patent from here.

***

Thanks for Reading !!!

This content is originally written, edited, and published by Shobhit Sharma. (Some Rights reserved; Patent & Diagram/Figures rights are totally reserved by Tesla, Inc.)