A wrist-mounted device has been touted as a breakthrough in wearable sensing, which could benefit applications in healthcare as well as communications.
A team from Cornell University (Ithaca, NY) and the University of Wisconsin–Madison have developed a lightweight bracelet— FingerTrak—that can continuously track the entire human hand in 3D. It can sense and translate hand positions, including 20 finger joint positions using several miniature low-resolution thermal cameras that read contours on the wrist.
“It’s the first system to reconstruct your full hand posture based on the contours of the wrist,” says Cheng Zhang, an assistant professor of information science at Cornell. He notes that existing wrist-mounted cameras have been considered “too bulky and obtrusive for everyday use,” and most could reconstruct only a few discrete hand gestures; FingerTrak is small enough to still allow free movement.
The new technology employs a combination of thermal imaging and machine learning to virtually reconstruct the hand. The cameras take multiple silhouette images to form an outline of the hand. According to the study, a deep neural network stitches the silhouette images together to reconstruct the virtual hand in 3D. This has implications in healthcare, namely in monitoring and better understanding disorders and diseases that affect fine-motor skills, such as Parkinson’s and Alzheimer’s.
In addition to healthcare, FingerTrak could be used in virtual reality, mobile health, and human-robot interaction. Researchers also cite sign language translation as the most promising application, ousting existing technologies that can be cumbersome. Reference: F. Hu et al., Proc. IMWUT (2020); doi.org/10.1145/3397306.
