Blog Post

Gesture in the Air to Control Your Devices

Gestures to control a device is a technology that is gaining a stronger foothold today, owing to the coming age of Internet of Things (IoT). While pressing few buttons to get things going is what IoT is looking forward to, researchers are exploring the lesser explored concept of gesture control to negate the touch of buttons too. And the tech base powering it is Radar technology.

Radar technology is leveraged to track vehicles, satellites, airplanes, etc. Researchers are now looking to deploy this to track micro motions such as twitches or swishes of a hand that will signal an instruction to a wearable, IoT or computing device.

Advanced Technology and Projects group (ATAP) at Google and Infineon recently unveiled a wireless gesture recognition technology named ‘Soli’ that involves a millimeter-wave radar chip to detect very fine gestures with fingers and hands. This is a miniature radar chip that can fit into wearable devices, pick up virtual hand or finger movements in real-time and alter signals accordingly.

How it works?

Radar-based hardware, which is designed into a single solid-state component, is turned into a gesture sensor. The sensor works with Soli software architecture consisting of a generalized gesture recognition pipeline. This pipeline extracts real-time signals from the hardware and outputs high precision position and motion data. Since Radar signals can weave through various type of material, it makes it a suitable technology that can be embedded into devices to track precise motions.

Motion signals detected by the radar chip are transformed into multiple representations, including range Doppler, which helps map the location of a hand by its velocity and distance from the sensor. Features such as "range Doppler distance" are extracted and passed on to a machine learning algorithm, to approximate hand motions based on the signals received.

Virtual hand or finger movements may include rubbing the tip of the finger and thumb to turn a dial, rubbing a finger along the length of a thumb can indicate moving a slider, say for volume control, while a swish of the hand may indicate a change of window or turning of a page.

Soli’s radar sensor solves a long-lived issue when it comes to gesture-recognition technology, such as gestures detected by camera sensors, which have difficulty understanding the overlap of fingers and capacitive touch sensing, which struggles to interpret motion in a 3D context.

Advanced Technology and Projects group (ATAP) is now putting this technology on store shelves to work with consumer product companies to put that whole stack into a shipping product. Further, it seems that this technology has a wide future scope to be used in many applications from sensing cars at a stop light without needing pressure plates to communications including more precise sign language interpretation to medical applications with more fine control for surgical robotics.
A patent application US 20160041617 A1 by Google titled ‘Radar-Based Gesture Recognition’ discloses a technology that recognizes gestures made in three dimensions, such as in-the-air gestures.

Source: Patent application US 20160041617 A1

These gestures can be made from varying distances. Another patent application WO2016170011A1 titled ‘Gesture recognition with sensors’ and assigned to Resmed Sensor Technologies, discloses a radar sensor that employs a transmitter configured to emit radio frequency waves and a receiver to receive and process reflected waves. The received reflected waves are processed by a processor that activates the sensor for gesture or motion recognition.

Source: Patent application WO2016170011A1

Texas Instruments has also forayed into this space and filed a patent application US20160252607 titled ‘Gesture Recognition using Frequency Modulated Continuous Wave (FMCW) Radar with Low Angle Resolution’. The patent application discloses gesture recognition using a FMCW radar to process digital intermediate frequency (IF) signals from radio frequency signals to determine whether or not a gesture was performed.

Source: Patent application US20160252607

Technologies such as this are bringing to the fore the true digital era where a human hand becomes the universal input device for interaction, with no physical control devices or interfaces.