Sonar technology promises to turn almost any environment into a keyboard for smart devices, meaning you don’t even have to touch your phone to work it.
Electrical engineers at the University of Washington are using sonar to track finger movements around smart devices without the user needing to touch the device at all.
“You can’t type very easily onto a smartwatch display, so we wanted to transform a desk or any area around a device into an input surface,” said lead author and doctoral student Rajalakshmi Nandakumar.
“I don’t need to instrument my fingers with any other sensors — I just use my finger to write something on a desk or any other surface and the device can track it with high resolution.”
The system, called FingerIO, turns a smartwatch or smartphone into a sonar system using the device’s own speaker to emit an inaudible sound wave. That signal bounces off the finger, and those echoes are recorded by the device’s microphones and used to calculate the finger’s location in space.
The team demonstrated that FingerIO can accurately track two-dimensional finger movements to within 8 mm, which is sufficiently accurate to interact with modern mobile devices. And because sound waves travel through fabric and do not require a line of sight, users can even interact with a phone inside a front pocket or a smartwatch hidden under a sweater sleeve.
“I don’t need to instrument my fingers with any other sensors – I just use my finger to write something on a desk or any other surface.”
Nandakumar said as mobile and wearable devices such as smartwatches become smaller, it is getting tougher for people to interact with screens the size of a matchbook.
The researchers employed a type of signal typically used in wireless communication called Orthogonal Frequency Division Multiplexing and demonstrated that it can be used to achieve high-resolution finger tracking using sound. Their algorithms leverage the properties of OFDM signals to track phase changes in the echoes and correct for any errors in the finger location to achieve sub-centimetre finger tracking.
Next steps for the team include demonstrating how FingerIO can be used to track multiple fingers moving at the same time.
