In conjunction with the CES keynote address by NVIDIA, Audi is demonstrating the intelligence of the Q7 deep learning concept on a specially designed, variable open area for piloted driving. The car orients itself by means of a front camera with 2 megapixel resolution, and the camera communicates with an NVIDIA Drive PX 2 processing unit, which in turn controls the steering with high precision. The high-performance controller is specially engineered for piloted driving applications.
Serving as the core of the software are deep neural networks that experts from Audi and NVIDIA have trained specifically for autonomous driving and recognition of dynamic traffic control signals. Beginning with a human driver at the wheel, the Audi Q7 deep learning concept gained a limited familiarity with the route and the surroundings, by means of observation and with the help of additional training cameras. That established a correlation between the driver’s reactions and the occurrences detected by the cameras. So during the subsequent demonstration drives the car is able to understand instructions, like from a temporary traffic signal, interpret them right away and act as the situation requires. When a corresponding signal appears, the concept car immediately changes the driving strategy and selects either the short route or the long one. The design of the system is so robust that it can even cope with disturbance variables such as changing weather and light conditions. It masters its tasks day and night, and even in direct sunlight or harsh artificial light.
The learning methods used for the Audi Q7 deep learning concept are essentially very much like those of deep reinforcement learning. This method was the underlying principle behind the Audi presence at the Conference and Workshop on Neural Information Processing Systems (NIPS), an AI event held in Barcelona in December. There, the neural networks – which are similar to the human brain – were also trained for a particular application. While the 1:8 scale model car at NIPS learned how to park through trial and error, during the training runs the network of the Audi Q7 deep learning concept receives concrete data it finds relevant – in other words, it learns from the driver.
Artificial intelligence is a game-changing key technology for piloted driving – of this Audi is convinced, which is why it is working closely with the leaders in the electronics industry. Together with its partners, Audi is evaluating various approaches and methods for machine learning. The aim is to always find the optimal method for the specific application being studied. Collaborative efforts by companies in the IT and automotive industries are also of tremendous value for future implementation in concepts and production cars.
With its impressive systems expertise, NVIDIA is considered the worldwide semiconductor industry’s biggest, most capable player. Audi has been working with the manufacturer since 2005. The Audi A4 was using an NVIDIA chip as early as 2007, and two years later NVIDIA technology allowed the Audi A8 to achieve a new dimension in visual displays. The Modular Infotainment Platform (MIB), which was introduced in 2013, featured the Tegra 2 processor from NVIDIA. And the MIB2 followed in the Audi Q7 in 2015, running with an NVIDIA T 30 processor.
The platform’s next level of development is the MIB2+ – which is premiering this year in the new generation of the Audi A8. Its key element is the Tegra K1 processor, which makes new functions possible and has the impressive computing power needed to support several high-resolution displays – including the second-generation Audi virtual cockpit. Onboard and online information will merge, making the car part of the cloud to a greater degree than ever. Together with the MIB2+, the central driver assistance controller (zFAS) in the new Audi A8 is also making its series debut. The K1 processor is also on board and in future the X1 processor from NVIDIA. Audi and NVIDIA are planning to intensify their long-standing partnership by combining NVIDIA’s development environment expertise for AI applications with Audi’s wealth of experience in the area of vehicle automation.
Another Audi key partner is Mobileye, whose image processing chip also is integrated in the zFAS. The high-tech Israeli company is the world leader in the field of image recognition for automotive applications. Mobileye is already supplying a camera for use in a range of Audi models – the Audi Q7, the A4/A5 series and the new Q5 – and the product’s image processing software can recognize a large number of objects. These include lane markings, vehicles, traffic signs and pedestrians. Today, defining the characteristics needed to clearly classify objects is still done manually.
In the new Audi A8, Audi and Mobileye are demonstrating the next level of development – with image recognition that uses deep learning methods for the first time. This significantly reduces the need for manual training methods during the development phase. Deep neural networks enable the system to be self-learning when determining which characteristics are appropriate and relevant for identifying the various objects. With this methodology the car can even recognize empty driving spaces, an important prerequisite for safe, piloted driving.
The traffic jam pilot function will be offered in a series production model for the first time in the new A8. This is the first piloted driving function in series production that will enable the driver to let the vehicle take over full control at times. With this step the stage is set to begin the next decade with higher levels of automation in a growing number of driving situations.