By giving the user a sense of touch, haptic feedback adds a new dimension to current user interfaces – enabling exciting applications like human-machine interfacing and remote surgery. Touchless user interfaces, such as haptic and voice, will play a critical role in the low-touch economy which is expected to emerge in the post-COVID-19 era. At the heart of the first-generation of haptic based interfaces are bulky piezoelectric ultrasound transducers, which are cumbersome in use, impede a very fine interaction and limit end-product integration possibilities. To meet these shortcomings, imec’s Denis Marcon and Xavier Rottenberg introduce compact micromachined ultrasound transducers to widen the field of applications. Two different technology platforms – Si-CMOS and display-manufacturing based large area – are being developed and offered to companies.
A sense of touch
Ultrasound are sound waves with frequencies that are higher than what is audible to the human ear, i.e. larger than 20kHz. Technologies making use of ultrasound waves have been used for many years, mainly for sensing applications. The best-known example is diagnostic imaging, also known as sonography, which was introduced as a diagnostic tool back in the 1940s. In this application, pulses of ultrasound are sent into tissue, and the echo pulses are displayed as an image that reveals internal body structures.
Few people are aware that the ultrasound technology can also be used to equip smart systems with haptic feedback, giving the user a sense of touch. In such an application, the technology is implemented as an actuator rather than as a sensor. The actuator emits ultrasound energy – a type of mechanical energy characterized by pressure waves that propagate through air.
Interesting applications emerge when the ultrasound energy can be perceived by the receptors on our fingers. This can enable enhanced human-machine interfaces for automotive, advanced display applications, or allow a finer interaction with robots in a manufacturing environment.
In one implementation, the user can be ‘physically’ touched by the smart system.
Alternatively, non-contact approaches are being explored, where ultrasonic waves generate a local pressure field causing the user to feel a light sensation. This ‘mid-air haptics’ allows users to interact with objects without touching them.
Or to interact with consumer applications in a more hygienic way – a measure which is receiving great attention due to the recent COVID-19 outbreak.
Figure 1: Mid-air haptics allows users to interact with smart systems without
touching them.
This mid-air haptic feedback can be combined with gesture recognition techniques. This way, the system can follow and track your hand, and beam back to the precise position of your fingers. Think about remote surgery, where the combination of the two techniques can give the surgeon a precise ‘feeling’ of what he touches from a distance.
Mid-air haptic feedback: how does it work?
To turn ultrasound into touch, you need an array of multiple ultrasound sources. Each of these ‘drums’ launches acoustic energy in all directions, like a spherical wave going out. The energy of one source generates the same feeling at different positions around the sphere and can give an imprecise user interaction. But by properly organizing phase shifts between the different sources, ultrasound energy can be focused on one or more spots, while being cancelled at other spots. This way, the ultrasound energy cloud can be shaped in three dimensions, like a full acoustic hologram.
Ultrasound energy with frequencies above 20kHz can however not be ‘felt’ by our finger receptors. In a next step, the frequency of the acoustic hologram needs to be down-modulated to arrive at frequencies below 500Hz in order to be perceived by our fingers.
The precision with which we feel the acoustic hologram depends on the carrier frequency of the ultrasound wave. The higher this frequency, the finer the interaction with our finger receptors. For example, for a carrier frequency of 40kHz, the resolution of the acoustic hologram is in the centimetre range. This improves to millimetre range when the carrier frequency is in the MHz range. At these higher frequencies, however, ultrasound waves are more easily absorbed in air. When trading off resolution with absorption, it turns out that frequencies around 400kHz are most interesting for mid-air haptic feedback sensations at about 1cm distance from the source.
From bulky to micromachined ultrasound transducers
On the technology side, haptic feedback can be generated by ultrasound transducers which convert electrical signals (AC voltage) into ultrasound (and vice versa). So far, first implementations have relied on classical piezoelectric transducers, using bulky ceramic piezoelectric elements. These elements change size and shape when a voltage is applied. AC voltage makes them oscillate at the same frequency and produce ultrasound. The thickness of the piezoelectric elements determines its dominant resonant frequency. First applications making use of these classical ultrasound transducers (or UTs) have already been commercialized, but since they are thick and bulky, the frequency is limited to about 40kHz, impeding a precise interaction with the user.
During the last decade, the idea of using micromachined ultrasound transducers (or MUTs) has become very attractive.
MUTs use miniaturized MEMS-based structures for emitting the ultrasound energy. Having a smaller dimension, a wider frequency range and a larger integration potential, these MUTs promise to outperform classical ultrasound transducers. They favour the development of large 2D arrays of transducers and enable close integration with the supporting electronics.
Two families of MUTs have been proposed: the piezoelectric MUT or pMUT – which is the micromachined equivalent of the classical piezoelectric transducer – and the capacitive MUT or cMUT. In the latter solution, electrostatic forces cause vibration of a membrane that is part of a parallel plate capacitor.
Technology platforms for MUTs
Imec is developing technology platforms for MUTs, leveraging its long-term track record in developing micro-electromechanical systems (MEMS).
In the past, the focus was on developing CMOS-compatible cMUT technologies. This has resulted in a high-performance cMUT technology platform.
Today, imec is focusing on developing generic pMUT technologies that can be tuned towards haptic feedback applications. Two different platforms for pMUTs are being pursued: Si-CMOS-compatible and display-manufacturing-compatible technology platforms. Each of these platforms brings its own advantages and challenges.
Figure 3: Operation principle of cMUT and pMUT technology.
A Si-CMOS-compatible technology platform for pMUTs
pMUTs can be post-processed directly on top of Si-CMOS. The main benefit of this approach is the ability to smoothly integrate the pMUT array with the driving and read-out electronics, either by wire-bonding or by using through-silicon vias – resulting in single-chip solutions. By using mature Si-CMOS-based processes in combination with Si-based pMUTs, highly integrated, efficient, high-resolution and high-performing pMUT arrays can be build. The footprint of the final pMUT chip is limited to typically 2x2cm2.
Following this approach, imec develops with a foundry partner a generic Si-CMOS-compatible pMUT platform, with (scandium doped) aluminum nitride as the piezoelectric ‘vibrating’ material. As such, ultrasound waves are emitted in the 3-15MHz frequency range. To be useful for haptic feedback applications, imec and its foundry partner are tuning this generic platform towards lower frequencies, i.e. 200-600kHz – by using larger membranes (0.8-1.5mm diameter) with higher aspect ratio.
The pMUT process can be adapted to specific use cases and is available for low- and high-volume production.
Figure 4: Microscopic picture of a pMUT array.
Towards a display-manufacturing-compatible technology platform for pMUTs
A second family of pMUT technologies implements polymer-based pMUTs that are fabricated in a display-manufacturing-compatible, large area technology. The true advantage of this technology is the ability to beam ultrasound waves from a large area array (in the order of a few to several tens cm2), resulting in a potentially larger power output and a fine non-contact haptic experience.
This could be used for creating mid-air haptic feedback sensations from a greater distance, e.g. to enable interactive posters for enhanced digital advertising.
Applications can go beyond display-based integrations – think about mid-air haptic feedback experiences generated from the windows of your house.
This technology is currently under development, and first proof of concepts have emerged. At imec, for example, a 4cm x4cm large 64×64 array of 480µm diameter pMUTs were fabricated. The 64 by 64 array consists of four 32 by 32 arrays. First generation devices are made with a piezoelectric polymer, while the next generations under investigation will use piezoelectric ceramics.
Figure 5: 64×64 array of polymer-based pMUTs.
Imec ultimately aims at integrating these pMUT arrays with thin-film transistor backplanes (as the driving electronics) and fabricating them in unprecedented array sizes in display-manufacturing-compatible technology platforms. An attractive approach is to install these MUT capabilities in display fabs that have recently shifted to newer display technologies and make their older capacity available to more diverse products. This is very comparable to the experience in CMOS where capacity became available in 200mm facilities that were re-used for MEMS development and eventually for MUT development.
The broad potential of MUTs: from haptic feedback to brain stimulation
First commercial haptic feedback products have already entered the market, mainly in the form of plug-and-play modules that allow to enrich the interface of existing entertainment installations with virtual buttons or magical sensations. These applications typically implement bulky classical ultrasound transducers, limiting the ultrasound frequency (to typically 40kHz) and as such the haptic feedback experience.
As already mentioned, micromachined ultrasound transducers are expected to open entirely new avenues, by enabling compact and portable solutions with finer sense of touch. In this implementation, haptic feedback will enhance remote control of machines and create advanced human-machine interfaces for automotive or advanced displays. For example, it can be used to enlarge the small touch screen of your smart watch into a full hand interface, or to enhance interactive digital advertising through large ‘haptic’ posters. It may also become a key part of virtual reality/augmented reality systems, adding sense of touch to previously visual-only interfaces. It will enable touch from a far distance, like a virtual handshake. Or, in medicine, it can be of great value for remote surgery, making the surgeon feel what he remotely touches – assisted by gesture recognition.
But the potential of micromachined ultrasound goes beyond haptic feedback. First, the ability to beam ultrasound energy to one focal point is being explored in medicine, for neuromodulation, brain surgery or cancer treatments. Ultrasound transducers can also be used to generate direct sound: tightly focused ultrasound energy can be beamed (and distorted) to become audible to the listener’s ear only.
Figure. 6: Schematic representation of some of the possible applications
enabled by ultrasound.
Second, micromachined ultrasound transducers can be used as a sensing instrument, for imaging applications. Following a Yole Development 2018 report, cMUTs and pMUTs are opening new possibilities in existing markets such as medical ultrasound and fingerprinting, while also creating new markets such as gesture recognition – where ultrasound can provide an alternative to radar- or light-based gesture recognition solutions. Examples of medical ultrasound imaging are portable echography, cardio patches or structural brain imaging – allowing to image body structures with waves that are free of radiation.
Movie: https://vimeo.com/252350041
Figure 7:
[caption movie:] Ultrasound
is the sole technology to enable
low power and non-contact gesture
recognition and haptic feedback.
Want to know more?
- Imec can help you tap into the unexplored potential of ultrasound – for haptic feedback or beyond applications. We are interested in collaborating with partners and clients in the life sciences field, infotainment, or automotive industry and in other applications requiring industrials sensors and actuators that challenge our technologies. Our Si-based technologies are ready to be tailored to specific applications and with our extensive inhouse expertise and infrastructure, we can perform all development steps in-house. Small- and high-volume production is offered through collaboration with our foundry partner. Interested? Feel free to contact imec’s senior business develop manager Denis Marcon.
- You can access the 2018 Yole Development analysis on ultrasound sensing technologies via this link.
About Denis Marcon
Denis Marcon received a M.S. degree from the University of Padova in 2006. Subsequently, he received the PhD degree of doctor in Engineering from the KU Leuven and imec in 2011. He is leading author or co-author of more than 50 journ
al papers or conference contributions. Currently, he is a Sr. business development manager in imec, Belgium, and he is responsible for the partnerships with imec in the field of GaN power electronics and on dedicated development projects of Si-based device and sensors.
About Xavier Rottenberg
Xavier Rottenberg is scientific director and group leader wave-based sensors and actuators at imec. He received the MSc degree in Physics Engineering and a supplementary degree in Theoretical Physics in 1998 and 1999 from “Université Libre de Bruxelles”, Belgium. He received further his PhD degree in Electrical Enginee
ring in 2008 from KU Leuven, Belgium. He worked one year at the Royal Meteorological Institute of Belgium in the field of remote sensing from space and joined imec in 2000.
