Considerations for Advanced Driver Awareness Systems why use an All Programmable SoC

Road safety has benefited significantly from Moore’s law, increases in processing capability and the development of CMOS Image Sensors (CIS) and other sensor technologies have enabled vehicle manufacturers to introduce Advanced Driver Awareness Systems (ADAS). ADAS enhances the driver’s awareness of the environment around them reducing the chances of collision. Some systems are also capable of monitoring the driver and alerting them, should they become sleepy for instance.
Increasingly ADAS also takes control (or provides information to autonomous driving systems), providing assistance to the driver with capabilities like parking assist, lane assist and adaptive cruise control.
It is no surprise therefore the ADAS market is predicted to be worth $42 Billion a year by 2021 and is currently experiencing a 10% Compounded Annual Growth Rate (CAGR) (Source: https://www.marketsandmarkets.com/Market-Reports/driver-assistance-systems-market-1201.htm).
ADAS use a wide spectrum of sensors encompassing embedded vision, RADAR and LIDAR, often to extract the information required they utilize a sensor fusion approach combining information from several sensors. Within the embedded vision sphere, ADAS can be split further into two categories, external monitoring which addresses aspects like lane departure, object detection, blind spot detection and traffic sign recognition. While internal systems monitor aspects such as driver drowsiness and eye detection. Both internal and external ADAS applications bring with them challenges to address in implementing the image processing algorithms.
These challenges range from the ability to implement the algorithms required for the application, to complying with the correct automotive standards. Many ADAS applications also require sensor fusion to combine the inputs from several sensors, significantly increasing the required processing power. Sensor fusion can be homogeneous where the multiple sensors of the same type are used, or heterogeneous where different sensor types are used to extract the information required.

Many applications utilize an All Programmable SoC or FPGA to implement the system due to the flexibility provided. Both to implement the required algorithms but also due to the ability to interface with different sensors types and networks.
Along with performance ADAS applications also come with several system challenges, which may not be obvious at first thought, vehicle manufacturers have stringent pollution standards to achieve and therefore the weight and power consumption of the overall solution is important. The cost of the solution is also critical due to the many tens / hundreds of thousands of vehicles being produced. While security and safety of the system is very critical and governed by a number of standards, using a SoC or FPGA can help us address a number of these.

System Architecture
The development of an embedded vision ADAS which monitors both external and internal cameras can be seen to be one of the more challenging ADAS implementations. This system needs to be able to interface to several cameras located around the vehicle, process the images and provide the information to the occupants. Many camera solutions use point to point LVDS wiring to transfer the data however, this brings with it additional cost and weight in the cabling required. There are however alternative approaches which are increasingly gaining ground, this pushes some functionality into the camera itself. If the image output by the camera is compressed and not the raw image, then network based architectures are possible. These networks could be based around commonly used automotive buses such as MOST (Media Oriented Systems Transport) – A high speed network which can be implemented in either optical or electrical physical layers IDB-1394 – High speed network implemented over an electrical physical layer, this is implemented in a daisy chained topology.

Ethernet AVB – Ethernet Audio Visual Bridging provides the ability to route image data and other data around the vehicle as needed Should we elect to use a network the system architect must ensure the necessary bandwidth is available to transfer the image data between the camera and ADAS core with the required latency for its application.
The data generated by the ADAS system may need to be shared with other systems within the car for instance adaptive cruise control, or parking assist. Therefore, the ADAS must be able to interface to other commonly used automotive interfaces like CAN or FlexRay.
At the architectural level the use of an All Programmable Zynq® SoC based approach provides several advantages, if point to point wiring is used to interface to the cameras the camera receivers can be implemented in the programmable logic prior to the image processing chain. If an All programmable SoC were to be used the flexibility of the processor system side would allow for the easy inclusion of a CAN, Ethernet and other protocols like FlexRay when combined with logic in the programmable logic and an external PHY if necessary. The combination of dual core processors and programmable logic allows for a very low power per pixel, as the system is very tightly integrated.

All Programmable SoC Architecture
Where the SoC comes into its own is the utilization of the Programmable Logic (PL) to implement camera interface and image processing chain. While the Processing System (PS) of the SoC can provide the communication, control and additional algorithmic processing as required. The image processing pipeline can be generated using a number of provided IP libraries and more specialist IP cores supplied by specialist vendors reducing the time to market.
Should we want to develop additional algorithms we can utilize the suite of High Level Synthesis tools like Vivado® HLS, SDSoC™ and Matlab to accelerate the development. Rather than developing the IP using a traditional Hardware Description Language like VHDL or Verilog we can use a higher level language like C or C++ reducing the time to market.
Further acceleration of the algorithm can be achieved using open source frameworks like OpenCV, algorithms developed using this framework can be mapped into a HLS video library supported by both Vivado HLS and SDSoC. This eases the transition from proof of concept and demonstration to the algorithms running in the target hardware for characterisation and qualification.
Many implemented architectures will use the processor DDR memory as a frame buffer, this enables the processor to access the images as necessary for further onwards transmission if using an Ethernet based system or PCIe for example. We can also use the power of the processor system to perform additional image processing algorithms upon the images stored within the DDR before re-insertion into the image processing chain is necessary.
This provides a very interesting capability that the SoC itself can form its own prototype and demonstration platform. Using common embedded vision development frameworks like OpenCV running on the cores within the processor, this provides for a down to size prototype system. This system can then be optimised for performance using the programmable logic side of the SoC.

Safety and Security
The very nature of ADAS is that it contributes to automotive safety, and as such systems must be developed in line with a standard like many other high reliability systems, for ADAS the applicable standard is ISO 26262. This standard defines a number of Automotive Safety Integrity Levels (A-SIL) which defines the time to failure in hours. There are four levels of A-SIL with D being the highest and most difficult to achieve and A the lowest. Achieving these requirements requires an integrated engineering delivery lifecycle to ensure not only the target A-SIL is achieved, but the data pack generated also demonstrates this by containing the necessary evidence.
Automotive applications are subject to harsh environments, and as such the developers need to ensure they use automotive grade components, for instance one certified to AEC-Q100, which has been manufactured and qualified to a higher standard than commercial components.
They must also consider the security of the system that is preventing un-authorised people from making modifications to the system, as this could potentially lead to catastrophic results.
Using an All Programmable SoC provides the ability to use its secure boot capability to prevent unauthorized programmes and bit streams from being configured into the system. To achieve this, they can utilize the AES Decryption, HMAC and RSA verification combined with the ability to use TrustZone technology. TrustZone allows the system designers to implement orthogonal SW environments which limits access to hardware functions within the SoC including programmable logic peripherals.
The health of the system can be monitored using the XADC which allows monitoring of the SoC supply voltages and die temperature. They can also use external SoC pins to monitor other analogue signals within the wider system providing for further health and usage monitoring of the system.
As the SoC provides tightly integrated processors and programmable logic coupled with an internal ADC, the designers are able to benefit significantly from a reduction in parts count. In turn this decrease in component count also significantly helps towards our achievement of the desired failure rate.
Wrapping it all Up
At both the system and device level the use of an All Programmable SoC offers a number of benefits providing for greater levels of integration, allowing for a more weight and power efficient system. Modern high level synthesis tools enable the rapid development of image processing functions which can be used within many open source commonly used frame works like OpenCV.

For more information, please visit our site.