Search⌘ K
AI Features
Home
Courses
Grokking the Machine Learning Interview

Architectural Components

Understand the high-level architecture of self-driving cars by exploring how different convolutional neural networks perform tasks like object detection and drivable region detection. Learn how recurrent neural networks use these outputs to predict vehicle actions in real-time, and dive into the training and prediction process of semantic image segmentation models that enable safe autonomous navigation.

We'll cover the following...

Overall architecture for self-driving vehicle

Let’s discuss a simplified, high-level architecture for building a self-driving car. Our discussion entails the important learning problems to be solved and how different learning models can fit together, as shown below.

High-level architecture (CNN: Convolutional neural network, RNN: Recurrent neural network, LSTM: Long short-term memory)
High-level architecture (CNN: Convolutional neural network, RNN: Recurrent neural network, LSTM: Long short-term memory)

The system is designed to receive sensory inputs via cameras and radars, which are fed to the visual understanding system consisting of different convolutional neural networks (CNN), each for a specific subtask. The output of the visual understanding system is used by the action predictor RNN or LSTM. Based on the visual understanding of the environment, this component will plan the next move of the vehicle. The next move will be a combination of outcomes, i.e., applying brakes, accelerating, and/or steering the vehicle.

📝 We won’t be discussing input through Lidar here. However, it can also be used for scene analysis similar to a camera, especially for reconstructing a ...