Turning this expression of frustration into code: a new human-machine interface
A coder created a new computer interface that uses more than subtle facial expressions and used the software to perform a seemingly real-life job interview code test. The biggest innovation, however, might be giving “duckface” a legitimate reason to exist.
Fletcher Heisler, Director of Developer Enablement at an Application Security Company Veracodeposted a video demonstrating how he built and tested his interface, called Cheeky Keys. It integrates facial recognition and expression tracking.
The coding seems to be light, the controls seem tiring but the story itself is very entertaining. A lot of work is going on in this area, some of which is worth more than an eye roll.
A article on Hackaday.com recounts how Heisler conceived of his innovation as a new dad wondering if there was a way to hold his newly arrived sleeping baby while coding.
His first move was to learn Morse code, which he could transmit with blinks (and, eventually, jaw movements) accompanied by a few facial gestures and interpreted by biometrics-based software.
Heisler used Google’s machine learning software called MediaPipe, primarily designed for tasks like animating facial image filters, to track nearly 500 facial landmarks.
Thus armed, he devised commands and modifiers, including the infamous but ubiquitous duck face pose women and girls strike in ironic Instagram fashion poses.
Heisler’s dry delivery, especially when he takes his phone interview, is fun, but there’s no denying that watching an adult perform a stationary facial kabuki — and pass his test — is a treat.
biometrics | Cheeky Keys | expression recognition | facial biometrics | gesture recognition | machine learning | Research and development