Shakibul Islam Akash; Md. Omar Faruk Faisal; Md. Rafiqul Islam

Latency Bias in Face Recognition Systems: Measurement, Statistical Evidence, and Mitigation Challenges for Eyeglass Wearers

PDF (600KB), PP.44-54

Views: 0 Downloads: 0

Author(s)

Shakibul Islam Akash ^1,* Md. Omar Faruk Faisal ^1,* Md. Rafiqul Islam ^1,*

1. American International University-Bangladesh (AIUB), Dhaka, 1229, Bangladesh

* Corresponding author.

DOI: https://doi.org/10.5815/ijeme.2026.05.04

Received: 16 Apr. 2026 / Revised: 18 May 2026 / Accepted: 5 Jun. 2026 / Published: 8 Oct. 2026

Index Terms

Face Recognition, Algorithmic Fairness, Inference Latency, Tail Latency, Label-Free Auditing, Real-Time Systems

Abstract

The face recognition systems (FRS) can have latency bias, with some of their inputs or subsets having disproportionate inference delays, which result in unfair quality-of-service despite apparently similar predictive performance. Although much research on fairness in face recognition research has been done on accuracy-based disparities, there has been little work on runtime behavior. This study seeks to understand whether there are differences in performance in real-time face recognition systems based on visually complex sub-groups, and whether these differences can be considered as operational fairness concerns. In this work, we investigate the latency problem of real-time FRS, with especially great attention to tail inference as the first-class fairness measure. We utilize a label-free auditing framework for conducting runtime fairness assessment, and propose the tail inference latency metric for evaluating the fairness level of the deployment. Instead of focusing on prediction scores as in prior work, our work focuses on the fairness level of our deployment. The MeGlass dataset (noted as a publicly available Kaggle release) is used as the base of experiments, which is an image-only face dataset to test the strength against eyeglasses, and the MeGlass dataset is a base to assess the reality of deployment through the Kaggle publicly available release. Using eyeglass wearers as a sample visually complex group, we conduct a label-free Responsible AI audit in the conditions when we do not have labels of the sensitive attributes. We observed that average performance difference in mean is not statistically significant (p = 0.36) by the permutation test, while mean difference in p99 is 16.85 ms between cohorts, bringing up fairness differences not captured by averages. Doing systems-level regression, we attribute such variances to pipeline-level attributes (e.g. detection confidence, input complexity) as opposed to the use of these attributes directly as causal factors. We also evaluate mitigation policies and determine trade-offs among the latency, fairness and recognition fidelity. The findings here show that fairness inequities can arise at worst-case runtime behavior, not in average latency, suggesting that the use of metrics focused on the tail of a run-time fairness distribution should be considered in fairness assessments of run-time deployed face recognition systems.

Cite This Paper

Shakibul Islam Akash, Md. Omar Faruk Faisal, Md. Rafiqul Islam, "Latency Bias in Face Recognition Systems: Measurement, Statistical Evidence, and Mitigation Challenges for Eyeglass Wearers", International Journal of Education and Management Engineering (IJEME), Vol.16, No.5, pp. 44-54, 2026. DOI:10.5815/ijeme.2026.05.04

Reference

[1]Delimitrou, B., Kozyrakis, C.: QoS-aware scheduling in heterogeneous datacenters with Paragon. ACM Transactions on Computer Systems 31(4), 1–34 (2013). https://doi.org/10.1145/2556583
[2]Yuan, M., Zhang, L., Duan, D., Zeng, L., Song, M.-H., Li, Z., Xing, G., Li, X.: Mitigating tail latency for on-device inference with load-balanced heterogeneous models. IEEE Transactions on Mobile Computing 24(12), 13090–13105 (2025). DOI: https://doi.org/10.1109/TMC.2025.3588430
[3]Dean, J., Barroso, L.A.: The tail at scale. Communications of the ACM 56(2), 74–80 (2013). The tail at scale | Communications of the ACM
[4]Grother, P.J., Ngan, M., Hanaoka, K.: Face Recognition Vendor Test (FRVT) Part 8: Summarizing demographic effects. NIST Interagency Report 8429, National Institute of Standards and Technology (2022). https://doi.org/10.6028/NIST.IR.8429.ipd
[5]Phillips, P.J., Grother, P., Micheals, R.J., Blackburn, D.M., Tabassi, E., Bone, J.M.: Face recognition vendor test 2002: overview and summary. NIST Interagency Report 6965, National Institute of Standards and Technology (2003). https://doi.org/10.6028/NIST.IR.6965
[6]Guo, J., Zhu, X., Lei, Z., Li, S.Z.: Face synthesis for eyeglass-robust face recognition. In: Proceedings of the 13th Chinese Conference on Biometric Recognition (CCBR 2018), Lecture Notes in Computer Science (LNCS), pp. 275–284 (2018). https://arxiv.org/abs/1806.01196
[7]Lyu, J. et al.: Portrait eyeglasses and shadow removal by leveraging 3D synthetic data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3419–3429 (2022). https://doi.org/10.1109/CVPR52688.2022.00342
[8]Sharma, S., Khan, M.A., Mir, H.M., Sharma, S.: A comprehensive survey on masked face recognition techniques using deep learning: Motivations, research progress, and future challenges. ICT Express 12(1), 55–75 (2026). https://doi.org/10.1016/j.icte.2025.12.007
[9]Neto, P.C., Pinto, J.R., Boutros, F., Damer, N., Sequeira, A.F., Cardoso, J.S.: Beyond masks: on the generalization of masked face recognition models to occluded face recognition. IEEE Access 10, 86222–86233 (2022). DOI: 10.1109/ACCESS.2022.3199014
[10]Wang, Y. et al.: TABI: An efficient multi-level inference system for large language models. In: Proceedings of the ACM European Conference on Computer Systems (EuroSys), pp. 233–248 (2023). https://doi.org/10.1145/3552326.3587438
[11]Kallus, N., Mao, X., Zhou, A.: Assessing algorithmic fairness with unobserved protected class using data combination. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* '20), p. 110 (2020). https://doi.org/10.1145/3351095.3373154
[12]Kenfack, P.J., Ebrahimi Kahou, S., Aïvodji, U.: A survey on fairness without demographics. Transactions on Machine Learning Research (TMLR) (2024). https://openreview.net/pdf?id=3HE4vPNIfX
[13]Rathgeb, C., Drozdowski, P., Frings, D.C., Damer, N., Busch, C.: Demographic fairness in biometric systems: What do the experts say? IEEE Technology and Society Magazine 41(4), 71–82 (2022). DOI: 10.1109/MTS.2022.3217700. https://www.researchgate.net/publication/366125778_Demographic_Fairness_in_Biometric_Systems_What_Do_the_Experts_Say
[14]Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM Computing Surveys 54(6), 1–35 (2021). https://doi.org/10.1145/3457607
[15]Khelifa Saif Eddine, K., Bagaa, M., Ouahouah, S., Ouameur, M.A., Ksentini, A.: Tail-latency aware scheduler for inference workloads. In: 2025 International Wireless Communications and Mobile Computing (IWCMC), pp. 679–684 (2025). https://doi.org/10.1109/IWCMC65282.2025.11059653
[16]Guo, J.: MeGlass dataset. Kaggle (2020). https://www.kaggle.com/datasets/mantasu/meglass

International Journal of Education and Management Engineering (IJEME)