Tracky Mouse
Hands-free mouse control
Hands-free mouse control
Install the desktop app to control your whole computer with just a webcam.
Precisely control the mouse cursor by moving your head.
Click by hovering over a control for a set amount of time. Cancel by moving away.
Click by winking or opening your mouth. Hold the pose to drag.
Regain control of the mouse by simply moving it. Head tracking will pause temporarily.
Adjust the sensitivity and smoothing of head tracking.
Tracky Mouse combines state-of-the-art face detection with tried-and-true optical flow tracking to precisely control mouse movement with your head.
Enable Viacam
uses optical flow to track the movement of points on your face,
but it has trouble finding your face in the first place.
Project Gameface
uses state-of-the art machine learning model FaceMesh to find your face,
but cursor movement is very jittery because it uses the face position information directly.
Tracky Mouse works like Enable Viacam, tracking points on your face precisely with optical flow, but it uses state-of-the-art face detection to place those points.
Since the face detection is so good, it can place the points not just on your face, but at locations on your face that are ideal for tracking, and it can prune away the points when they stray from those specific locations on your face.
The result is a system that's more accurate than either.
People who have ALS or other motor neuron diseases, spinal cord injuries, arthritis, repetitive strain injuries, or other conditions that make it difficult to use a mouse, can greatly benefit from hands-free computer control.
As a programmer, Tracky Mouse helps me edit code faster because I my hands don't have to leave the keyboard.
It's also just fun! It feels like controlling the computer with your mind.
You can try out Tracky Mouse right here in your browser.
Just click to allow camera access, and then adjust the settings until it feels comfortable pointing to any part of the screen.
If you scroll down, there is an area where you can practice pointing at targets.
A demo of Tracky Mouse should appear here.
Most of the functionality of Tracky Mouse is available as a JavaScript library.
You can use it to add head tracking and dwell clicking to your own web application.
Check out the API documentation for more information.
JS Paint uses Tracky Mouse for its Dwell Clicker and Head Tracker features. Click here to launch JS Paint in Head Tracker mode.
Open source on GitHub. MIT-licensed.
Please email me with any feedback or questions, or open an issue on GitHub.