Tesla is working to get an autonomous driving system as soon as next year. They have patented a data pipeline and deep learning system that could help them develop it faster. A new patent application by Tesla was filed for the ‘Data Pipeline and Deep Learning System for Autonomous Driving’.
Tesla’s data pipeline has data from a fleet of hundreds of thousands of vehicles equipped with a large suite of sensors.
Tesla explains the problem that its system is addressing:
“Deep learning systems used to implement autonomous driving typically rely on captured sensor data as input. In traditional learning systems, the captured sensor data is made compatible with a deep learning system by converting the captured data from a sensor format to a format compatible with the initial input layer of the learning system. This conversion may include compression and down-sampling that can reduce the signal fidelity of the original sensor data. Moreover, changing sensors may require a new conversion process. Therefore, there exists a need for a customized data pipeline that can maximize the signal information from the captured sensor data and provide a higher level of signal information to the deep learning network for deep learning analysis.”
Tesla’s solution:
“A data pipeline that extracts and provides sensor data as separate components to a deep learning network for autonomous driving is disclosed. In some embodiments, autonomous driving is implemented using a deep learning network and input data received from sensors. For example, sensors affixed to a vehicle provide real-time sensor data, such as vision, radar, and ultrasonic data, of the vehicle’s surrounding environment to a neural network for determining vehicle control responses. In some embodiments, the network is implemented using multiple layers. The sensor data is extracted into two or more different data components based on the signal information of the data. For example, feature and/or edge data may be extracted separate from global data such as global illumination data into different data components. The different data components retain the targeted relevant data, for example, data that will eventually be used to identify edges and other features by a deep learning network. In some embodiments, the different data components function as containers that store data highly relevant for identifying certain targeted features but do not themselves identify or detect the features. The different data components extract data to ensure accurate feature detection at an appropriate stage of a machine learning network. In some embodiments, the different data components may then be pre-processed to enhance the particular signal information that they contain. The data components can be compressed and/or down-sampled to increase resource and computational efficiency. The different data components are then provided to the deep learning system at different layers of the system.
The deep learning network is able to accurately identify and detect the features associated with the targeted data (e.g., edges, objects, etc.) of the data component using the signal information retained during extraction as input. For example, feature and edge data is provided to the first layer of the network and global data to a later layer of the network. By extracting different data components that each retain their respective targeted signal information, the network more efficiently processes the sensor data. Instead of receiving the sensor data as an initial input to the network, the network is provided with the most useful information at the most appropriate layer of the network. In some embodiments, a more complete version of the captured sensor data is analyzed by the network since the different data components can fully utilize the image resolution of their respective components for their intended purposes. For example, input for features and edges can utilize the entire resolution, bit range, and bit depth for feature and edge data whereas input for global illumination can utilize the entire resolution, bit range, and bit depth for global illumination data.”
See the full patent below:
[scribd id=441072526 key=key-Mao5JoLi07Oi5n3bjffX mode=scroll]
