VTT Marilyn, a Finnish robot car, has taken a leap towards automatic 24/7 driving – also sees through fog

VTT’s robot car, Marilyn, sees better than humans in foggy, and even snowy, conditions, and can now navigate without stopping – including in bad weather. It can also see a human through fog and avoid accident automatically. This is enabled by the LiDAR mounted on the car’s roof, which can see wavelengths that are beyond the human senses. As the technology evolves, this represents a big step towards a safe automated vehicle that is not even stopped by foggy conditions.

Most likely, for the first time, robot cars from different manufacturers will demonstrate their driving scenarios in the same arena at the RobustSENSE event in Ulm, Germany, where Marilyn will head with six other robots on 16 May.

The latest additions to Marilyn include optical component wavelengths via the new 1550 nanometre LiDAR and additional intelligence for its software, which improves sensor capabilities. Furthermore, software modules have been built in for the filtering of point clouds and the assessment of scanner reliability. This is to ensure the vehicle’s ability to function, including in fog and powdery snow, under which conditions the LiDAR radar, which ‘sees’ in the visible and near infrared ranges of the spectrum, even enables the robot car to see people better.

“Although Marilyn’s vision is limited to roughly 30 metres in thick fog, the new LiDAR type allows the car to be driven slowly rather than having full stop,” says Project Manager Matti Kutila of VTT’s RobotCar Crew team.

The car also has traditional automotive radars and LiDAR, but their detection of non-metallic obstacles and resolution is limited, particularly when trying to recognise shapes.

“Marilyn can also combine radar and LiDAR technologies by optimising the best aspects of the different sensors. This makes the automatic vehicle safer than a car driven by a person – although there are still lot of obstacles in development path, a major leap has been taken in the right direction,” Kutila emphasises.

“We still have a long way to go on the journey towards 24/7 automated driving, but we are now a big leap closer to achieving our dream. If we think of this as a 42km marathon, we are now perhaps 10km closer to our goal,” he comments.

More and more driving scenarios involved

More and more scenarios (such as cities, main roads, snow, exit ramps), which robot cars can manage, have been added. Marilyn will present her own special capabilities at the RobustSENSE event, along with six other robot cars.

“Marilyn will drive through a bank of fog created in a tent, which is so thick that the passengers cannot see through it. After passing through the chamber, the car will automatically avoid an obstacle in front, in this case a dummy. Among Marilyn’s sensors, the 905 nanometre LiDARs cannot see through mist – the new 1550nm LiDAR is the only sensor based on which a decision to swerve can be made,” says Kutila, describing the driving scenarios of his protege.

This is probably the first time in Europe that six robot cars will drive through their different scenarios in the same campus area, in which also pedestrians are also moving around. Other cars will negotiate crossroads automatically, engage in route planning, and change between manual and automated mode, which makes this exceptionally fascinating day.

Marilyn, on the other hand, will profile the ‘see through’ type of scenario, by driving through fog which even the observer cannot see through.

The LiDAR and sensor fusion technology of VTT Technical Research Centre of Finland, SICK, Oplatek and Modulight are co-created in Marilyn.

The project in overall is joint effort of Daimler AG • AVL • Bosch • Centro Ricerche Fiat • EICT GmbH • Fico Mirrors S.A. • Fraunhofer FOKUS • FZI Forschungszentrum Informatik • Modulight, Inc. • Oplatek Group Oy • SICK AG • University of Ulm • VTT.

“Members of the public will also be able to hop into a robot car at an event which could feature the world’s first robot car traffic jam, but hopefully not chain collisions,” Kutila adds.

Marilyn will next try out her skills in an automated parking exercise in the summer, based on commands from outside the car. Her husband, VTT Martti, will also be making the news in the north during the summer and in November, when he makes route selections based on friction data and LiDAR.