Build a Cool Face Detection App in Your Browser

In my previous post I gave an overview of facial recognition, its advantages and ethical considerations. Today I want to show you how to put this technology in practice. We are going to build a web application that detects your face and does some basic feature analysis.

Let’s start with a demo.

Source code: https://bit.ly/3g39PZY

https://gitlab.com/langnerd/nlp-by-example/-/tree/master/facial-recognition

There are many tutorials of this kind out there. Most of them assume you want to attach an external camera. I intentionally reached for a web app leveraging a built-in camera, just to make the whole setup easier.

The example we are going to build comprises the following tech stack:

• HTML5 Canvas to capture your face through a built-in camera
• Back and forth communication with the server over WebSocket
• Face detection and all the heavy lifting is done by OpenCV, a Python library for Computer Vision
• Docker for orchestration and a seamless installation of all dependencies

I’ve organised this post into sections for a better readability.

• Initial Setup and Dependencies
• Face Detection in the Browser
• Efficient Communication Channel
• OpenCV Face Detector
• Enrichment – Gender and Age Recognition
• Summary

Initial Setup and Dependencies

Let’s start with server-side dependencies declared in requirements.txt

As you can see I use Flask as a web server. Since the web client continuously captures my face and keeps sending video frames, I leverage Socket IO and AIOHTTP for low-latency bi-directional async communication. OpenCV takes care of the actual image processing supported by NumPy, a universal library for all kinds of scientific applications.

A Docker file is next. I like to keep the image minimalistic and lean.

Now, when all the dependencies are taken care of, I can focus on the interesting stuff. Here is what I am going to tackle next:

• HTML5 Canvas to continuously capture a face by a built-in camera
• Communication channel between the web client and the server
• Image processing by a designated OpenCV component
• Additional enrichment – gender and age recognition

Face Detection in the Browser

The objective of the code running in the browser is to use the built-in camera to record your face and send the captured video frames to the server. HTML5 tags video and canvas work in tandem to achieve the required result.

The video tag sports all kinds of fancy controls, but we ain’t gonna need any of them. Simply adding the tag into the DOM allows you to activate the webcam with a tiny bit of Javascript (works in modern browsers). It works as long as you grant your browser access to the camera. Once you do, you are ready to stream video right off your webcam.

Proof ..

Efficient Communication Channel

Having a live webcam at your disposal presents you with another challenge. How do you stream it to the backend for further processing without overwhelming the server? Canvas to the rescue!

Canvas can be attached to the video content, capture individual frames and send them as images at a defined rate. Check the demo, feel free to tweak the code and experiment.

You should see something like this:

The last missing part is exchanging the data with the server back and forth. The input frames are individually sent to the server, processed and the result – the very same image enhanced with a facial frame overlay – is displayed in the browser. All of that happens asynchronously, so that the frames can be sent continuously without a single glitch. Sounds like a perfect job for WebSockets:

• two-way communication channel
• event driven instead of polling by the client
• client side – WebSockets JS API
• server side – AIOHTTP + Socket IO (Python)

Source code: https://bit.ly/3g39PZY

https://gitlab.com/langnerd/nlp-by-example/-/tree/master/facial-recognition

OpenCV Face Detector

OpenCV includes hundreds of computer vision algorithms. Using the library for the purpose of face detection is just a matter of using an appropriate model and a few straightforward API calls.

As for the model, there are numerous publicly available and free of charge. You don’t even need to go too far in your efforts. OpenCV ships with various models identifying both the face as well as other body parts (eye, smile, upper body), or things such as a number plate etc.

OpenCV supports an efficient method of face recognition via Haar cascades, and that’s indeed the model I opted for.

Key points:

• The server receives base64-encoded raw image data. Before any processing can happen, the received data need to be converted into an OpenCV compliant image. That’s what the static method __read_image on line 32 deals with.
• While we want to update the original image with the detected faces, the face detection itself only works with grayscale images – see line 11.
• The face detection algorithm is configurable – line 14. I advise you to consult the documentation and experiment.
• Finally, the original image gets updated with an overlay framing all of the detected faces – line 23.

Enrichment – Gender and Age Recognition

Adding recognition of additional features, such as gender, age or mood is relatively straightforward. You don’t to be an expert either. It’s a matter of downloading the respective model and a bit of configuration. I’ve reached out to the use of CNN For the gender and age detection. Check this article if you are interested in details.

Summary

In this post we’ve explored face detection in practice. I hope this blog post inspired you to experiment with OpenCV or any other framework on your own. Let me know if you found it useful, or any feedback you might have. Always appreciated. Thanks for reading thus far and hope to see you next time.

Source code: https://bit.ly/3g39PZY

https://gitlab.com/langnerd/nlp-by-example/-/tree/master/facial-recognition