The globalization of the cyber world is increasing at an exponential fold. In past couple of years, this shift towards digitization has given access to almost every possible service one can imagine on fingertips. However, the flip side of it is, we end up releasing more information than we should, and it can be noxious. It spawned to increase in research of security, especially in mobile phones, computers, and other digital devices. The evolution of security is visible in past decade from passwords to fingerprint to the most recent and advanced face recognition system deployed in mass consumer devices.


Face recognition is a system which first detects the face (where) in the frame and then recognizes it (who). It is one of the biometrics used to authenticate a person’s unique identification using a database of stored templates. Developing face based application system requires following components.


  1. Face Detection(Localization) – In an image or video frame, first it is essential to localize the face to recognize it efficiently. It makes the face recognition algorithm efficient as it limits the search space for identifying the face which usually requires high computation. Face detection applications are useful in surveillance, object tracking ( human tracking, car tracking, etc.) motion tracking, etc.


Source: Face Detection Paper


  1. Face Feature ExtractionAfter detecting the face, we have to convert the raw pixels in quantized form(vectorization) to compare it with templates in the database.


Facial keypoint extraction is one of the most common and useful algorithms used for 3D modeling of faces used by advanced face recognition systems(Face ID by apple).


Source: Eigen Faces Paper


  1. Feature Matching – Once a face feature extracted in some quantized form understandable by machines, it is used to do feature matching with similar features stored as templates in the database.


  1. Score Analysis – Finally, after a confidence score assigned to every template in the database and the one with the most significant score is usually picked as the recognized face by the system.


Recently, many face based applications algorithms have introduced the use of facial keypoint extraction. Facial features vary vastly from one individual to another, and for every individual, there is a tremendous amount of variation due to 3D pose, size, position, viewing angle, and illumination conditions. Facial key points help to take into account all these real-life scenarios to enhance the accuracy of face recognition systems.


Face Recognition systems developed by major companies like Microsoft (Azure Face API), Amazon Face API(Rekognition), Google Vision API, IBM Watson Face API(Visual Recognition API), etc. are showing different kinds of business applications. Face detection is core part of these systems. All of these Face API’s are proprietary and served through API calls only.

The developers are using open source Deep Learning frameworks like Keras, Tensorflow, etc. to develop face based applications. Keras and Tensorflow Developers have developed face based applications using Keras deep learning framework. There is a book Tensorflow Deep Learning with Applications Using Python: Chatbots and Face, Object, and Speech Recognition With TensorFlow and Keras’ which can be used to get hands-on experience on building real-world applications like chatbots, face and object recognition, etc. developed using these frameworks.


Face recognition has surpassed human accuracy, but still, research is going on to make it invariant and accurate under all constraints.


Do you want to develop Face based Applications?

Please read this book to learn steps to develop Face based Applications
















A curious person who loves to solve problems mainly based on mathematical and computational models. In short, A senior Data Scientist passionate about machine learning algorithms


Leave a Reply

Your email address will not be published. Required fields are marked *