Motion Detection with Optical Flow

I would love to write this up in more detail later, but for now I just dug it up on my old hard disks.

I got bored one day and decided to try and make a webcam gesture controller. The “vision” (ba dum tiss) here was to try and just threshold the image and use motion detection to provide for a very basic workspace switch on my Linux desktop manager.

If I had more time I wanted to make the algorithm more robust and allow for watching a room. Could I point at a television just to turn it on? Could I make an X with my hands to turn it off?

Here I started with plain OpenCV to do motion detection with optical flow algorithms. But then I enhanced it by using a very simple CNN in PyTorch to detect more of the gestures that I wanted to use.

Here is a very basic demo video where one could see a swipe of my hand that led to switching the workspace… in the swipe direction! Magic!