Deploying Facial Recognition Technology at the Enterprise Level

Article contributed by Scylla Co-Founder and product lead for Facial Recognition PhD Dr. Armen Ghambaryan .

Hey there—happy Sunday! Exciting news: we're enhancing our newsletter with guest voices and cool new topics. Also, a big shout-out to my co-founder, PhD Dr. Armen Ghambaryan, who's rocking our biometrics line. Check out his latest piece on facial recognition tech. Plus, as he is huge in video gaming, we're spicing up our look with some fun Japanese gaming vibes. Stay tuned and enjoy!

Facial recognition has long been used by law enforcement to generate investigative leads and identify victims of crimes and by border control to streamline the verifying process. With a massive advance in AI and deep learning, it is now possible for Facial Recognition Technology to be applied more broadly. Automated biometric authorization is increasingly deployed worldwide for smart enterprise management, secure access, and improved surveillance. But how much do we know about deploying the technology in different environments?

How do modern deep learning-based face recognition systems work?

Deploying modern deep learning-based face recognition systems is primarily about building an efficient pipeline of several models, which include the face detection model, optional face landmark detection, and the recognition model itself with corresponding associated pre-and post-processing stages1. The details of the pipeline depend on the deployment environment and the type of system it is being integrated with (access and attendance control, person search, thermal screening, etc.).

The system can be deployed either as a single microservice or as two microservices (with detection and recognition deployed separately). In the latter case, HTTP or gRPC2-based serving is used to connect different modules because of their flexibility and easiness of control. The modules are typically generalized to be used by different services in parallel. To ensure versatility, they can come packaged in Docker containers with corresponding orchestration tools.

The deployment architecture itself varies based on the use case, the existing system/network distribution, the available analytical hardware, etc. One option is to have the system deployed in a central war room which has connections to the surveillance cameras. Alternatively, local micro deployments can be installed on edge devices and maintained by remote service. Also, cloud-based deployments are increasingly popular, though the spread is still hindered by privacy-related concerns and restrictions.

On edge devices like Nvidia Jetson Nano3, the deployment strategy differs depending on hardware limitations. Lightweight models for both detection and recognition are being deployed within a single Dockerized service. All the face-matching procedures are done on the device to make the system work almost as fast as on an ordinary server. Lightweight models are also important when there is a need for the face recognition application to be integrated for large-scale video stream processing purposes (watchlist, person search, etc.).

Enrollment of the faces and the database information is an integral part of the process. This uses a desktop application that captures, filters out, and sends high-quality face images to the face recognition microservice. After the preprocessing of the face images, an average face feature vector is stored in the database. There is no need to store the actual image or any other sensitive information because the vector is sufficient for the system to work and constantly be updated with the new faces while it runs. The feature vector can’t be reverse-engineered to get the initial image, in addition, it is encrypted to increase security.

The topic of adversarial attacks is also crucial in terms of ensuring the overall security of the system. The attacks are handled by designing appropriate system training procedures as well as adding security checking models at the beginning of the video processing pipelines4. The nature of these gate models is again conditional on the deployment environment, and hardware limitations should be taken into account while designing them. There is ongoing research on adversarial attacks on deep learning models, and the face recognition systems are always kept current on that topic.

Face recognition deployment types

Based on the points mentioned above, one could design two general types of deployments for face recognition. The first scheme illustrated below (figure 1) is used when there is a need to integrate facial recognition within another application like thermal screening.

In this particular example, face detection and recognition are separated from each other and the rest/gRPC-based API asynchronously receives cropped faces from the main application. The face enroller is used to update the face database on the fly. After the update, the API receives the corresponding information and starts to recognize all the faces that were successfully enrolled. Once a person is recognized, the information is sent back to the main application, which is subsequently visualized on the dashboard.

The other common deployment strategy is used when one needs only the face recognition functional itself, as is in the case of attendance/access control, person search, or watchlist. When a standalone deployment strategy is used (figure 2) the role of the main application is given to the recognition service itself.

The detection and recognition services are pipelined together in this scenario. The face enroller logic is similar to the case of an integrated application, since it again sends the information to the database and triggers the application to update itself to take into account recently enrolled faces. After the recognition is done, the data is sent both to the dashboard for visualization and to the database with detailed information on when and where the recognition has registered.

Facial re-identification-based solutions

Somewhat different from the above-described use cases are so-called facial re-identification-based solutions. In this case, the biometrics of the person appearing for the first time in front of any camera of the system are deciphered, enrolled into the dataset, and assigned a unique ID. When the same person appears in front of that or any other camera in the system, he/she is re-identified and a corresponding record is created.

This approach can be used to track the movement of customers or visitors on the premises without compromising their privacy, detect a returning customer, time-gate duration of visits, etc. Similar to the previous cases, re-identification can be deployed either in centralized or edge-distributed architecture. In either case, the solution is essentially privacy-agnostic as it does not store any images or identities of people sighted in the views of cameras, but merely understands impersonalized visitor whereabouts patterns and sequences that provide insights to managers for better service.

Security benefits of a central server architecture

Coming back to the topic of system security, it is important to mention that usually enterprises are already equipped with an existing CCTV network and a centralized workstation where all camera feeds are directed, so adding face recognition to this opens up the opportunity to leverage the camera network. This deployment model also offers some benefits over cluster deployments. A centralized solution installed in a dedicated server room is easier to protect from infringements and intrusions, adding one more layer of protection to the one with gated models against adversarial attacks.

Final Takeaway

We know how useful face recognition-related technology can be for people and for society overall. It can make everyday life safer and more secure, for example, facial recognition can ensure that only the right person gets access to a private area or sensitive information meant just for them. It can also be used for tremendous social good; there are organizations using face recognition to fight against the trafficking of minors or finding lost people.

However, before deploying a facial recognition system, we should keep in mind that the success of deploying the system depends on how it’s planned or designed to serve the purpose of the business. This means selecting the right architectural approach that aligns with the organization’s existing hardware and software, security needs, and use case requirements. Edge/hybrid cloud, centralized server, and clustered architectures all offer benefits and drawbacks that need to be carefully considered.

Materials used:

1. Deep Face Recognition: A Survey”. Accessed 6 May. 2021.
2. Documentation | gRPC” 16 Dec. 2020. Accessed 6 May. 2021.
3. NVIDIA Jetson Nano Developer Kit | NVIDIA Developer”, 19 Mar. 2019. Accessed 6 May. 2021.
4. cleverhans v2.0.0: an adversarial machine learning library – arXiv …”. Accessed 6 May. 2021.