Vol. 8 No. 1 (2022): Special Issue on Embedded Systems for Computer Vision
Special Issue on Embedded Systems for Computer Vision

Susceptibility to Image Resolution in Face Recognition and Training Strategies to Enhance Robustness

PDF
  • Martin Knoche,
  • Stefan Hörmann,
  • Gerhard Rigoll
Martin Knoche
Technische Universität München, Arcisstrasse 21 80333 München, Deutschland
Stefan Hörmann
Technische Universität München, Arcisstrasse 21 80333 München, Deutschland
Gerhard Rigoll
Technische Universität München, Arcisstrasse 21 80333 München, Deutschland
DOI: https://doi.org/10.4230/LITES.8.1.1

Published 2022-11-16

Keywords

  • recognition,
  • resolution,
  • cross,
  • face,
  • identification

How to Cite

[1]
Knoche, M., Hörmann, S. and Rigoll, G. 2022. Susceptibility to Image Resolution in Face Recognition and Training Strategies to Enhance Robustness. Leibniz Transactions on Embedded Systems. 8, 1 (Nov. 2022), 01:1–01:20. DOI:https://doi.org/10.4230/LITES.8.1.1.

Copyright (c) 2022 Martin Knoche

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Many face recognition approaches expect the input images to have similar image resolution. However, in real-world applications, the image resolution varies due to different image capture mechanisms or sources, affecting the performance of face recognition systems. This work first analyzes the image resolution susceptibility of modern face recognition. Face verification on the very popular LFW dataset drops from 99.23% accuracy to almost 55% when image dimensions of both images are reduced to arguable very poor resolution. With cross-resolution image pairs (one HR and one LR image), face verification accuracy is even worse. This characteristic is investigated more in-depth by analyzing the feature distances utilized for face verification. To increase the robustness, we propose two training strategies applied to a state-of-the-art face recognition model: 1) Training with 50% low resolution images within each batch and 2) using the cosine distance loss between high and low resolution features in a siamese network structure. Both methods significantly boost face verification accuracy for matching training and testing image resolutions. Training a network with different resolutions simultaneously instead of adding only one specific low resolution showed improvements across all resolutions and made a single model applicable to unknown resolutions. However, models trained for one particular low resolution perform better when using the exact resolution for testing. We improve the face verification accuracy from 96.86% to 97.72% on the popular LFW database with uniformly distributed image dimensions between 112 × 112 px and 5 × 5 px. Our approaches improve face verification accuracy even more from 77.56% to 87.17% for distributions focusing on lower images resolutions. Lastly, we propose specific image dimension sets focusing on high, mid, and low resolution for five well-known datasets to benchmark face verification accuracy in cross-resolution scenarios.

PDF

References

  1. Omid Abdollahi Aghdam, Behzad Bozorgtabar, Hazim Kemal Ekenel, and Jean-Philippe Thiran. Exploring factors for improving low resolution face recognition. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 2363-2370. IEEE, 2019.
  2. Zhiyi Cheng, Xiatian Zhu, and Shaogang Gong. Low-resolution face recognition. CoRR, abs/1811.08965, 2018. URL: http://arxiv.org/abs/1811.08965.
  3. Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4690-4699, 2019.
  4. Berk Dogan, Shuhang Gu, and Radu Timofte. Exemplar guided face image super-resolution without facial landmarks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 0-0, 2019.
  5. Shiming Ge, Shengwei Zhao, Chenyu Li, and Jia Li. Low-resolution face recognition in the wild via selective knowledge distillation. IEEE Transactions on Image Processing, 28(4):2051-2062, 2018.
  6. Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In European conference on computer vision, pages 87-102. Springer, 2016.
  7. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
  8. Chih-Chung Hsu, Chia-Wen Lin, Weng-Tai Su, and Gene Cheung. Sigan: Siamese generative adversarial network for identity-preserving face hallucination. IEEE Transactions on Image Processing, 28(12):6225-6236, 2019.
  9. E. G. Huang, G. B. Learned-Miller. Labeled Faces in the Wild: Updates and New Reporting Procedures. Technical Report UM-CS-2014-003, University of Massachusetts, Amherst, May 2014.
  10. Robert Keys. Cubic convolution interpolation for digital image processing. IEEE transactions on acoustics, speech, and signal processing, 29(6):1153-1160, 1981.
  11. Vahid Reza Khazaie, Nicky Bayat, and Yalda Mohsenzadeh. Ipu-net: Multi scale identity-preserved u-net for low resolution face recognition. arXiv preprint, 2020. URL: http://arxiv.org/abs/2010.12249.
  12. Yonghyun Kim, Wonpyo Park, Myung-Cheol Roh, and Jongju Shin. Groupface: Learning latent groups and constructing group-based representations for face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5621-5630, 2020.
  13. Pei Li, Loreto Prieto, Domingo Mery, and Patrick J Flynn. On low-resolution face recognition in the wild: Comparisons and new techniques. IEEE Transactions on Information Forensics and Security, 14(8):2000-2012, 2019.
  14. Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 212-220, 2017.
  15. Ze Lu, Xudong Jiang, and Alex Kot. Deep coupled resnet for low-resolution face recognition. IEEE Signal Processing Letters, 25(4):526-530, 2018.
  16. Fabio Valerio Massoli, Giuseppe Amato, and Fabrizio Falchi. Cross-resolution learning for face recognition. Image and Vision Computing, page 103927, 2020.
  17. Stylianos Moschoglou, Athanasios Papaioannou, Christos Sagonas, Jiankang Deng, Irene Kotsia, and Stefanos Zafeiriou. Agedb: the first manually collected, in-the-wild age database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 51-59, 2017.
  18. Sivaram Prasad Mudunuri, Soubhik Sanyal, and Soma Biswas. Genlr-net: Deep framework for very low resolution face and object recognition with generalization to unseen categories. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 602-60209. IEEE, 2018.
  19. Nasrabadi NM et al. Identity-aware deep face hallucination via adversarial face verification. In IEEE International Conference on Biometrics Theory Applications and Systems, 2019.
  20. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211-252, 2015.
  21. Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815-823, 2015.
  22. Soumyadip Sengupta, Jun-Cheng Chen, Carlos Castillo, Vishal M Patel, Rama Chellappa, and David W Jacobs. Frontal to profile face verification in the wild. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1-9. IEEE, 2016.
  23. Maneet Singh, Shruti Nagpal, Richa Singh, Mayank Vatsa, and Angshul Majumdar. Magnifyme: Aiding cross resolution face recognition via identity aware synthesis. arXiv preprint, 2018. URL: http://arxiv.org/abs/1802.08057.
  24. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015.
  25. Veeru Talreja, Fariborz Taherkhani, Matthew C Valenti, and Nasser M Nasrabadi. Attribute-guided coupled gan for cross-resolution face recognition. arXiv preprint, 2019. URL: http://arxiv.org/abs/1908.01790.
  26. Su Tang, Shan Zhou, Wenxiong Kang, Qiuxia Wu, and Feiqi Deng. Finger vein verification using a siamese cnn. IET Biometrics, 8(5):306-315, 2019.
  27. Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5265-5274, 2018.
  28. Qiangchang Wang, Tianyi Wu, He Zheng, and Guodong Guo. Hierarchical pyramid diverse attention networks for face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8326-8335, 2020.
  29. Zhifei Wang, Zhenjiang Miao, QM Jonathan Wu, Yanli Wan, and Zhen Tang. Low-resolution face recognition: a review. The Visual Computer, 30(4):359-386, 2014.
  30. Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In European conference on computer vision, pages 499-515. Springer, 2016.
  31. Erfan Zangeneh, Mohammad Rahmati, and Yalda Mohsenzadeh. Low resolution face recognition using a two-branch deep convolutional neural network architecture. Expert Systems with Applications, 139:112854, 2020.
  32. Dan Zeng, Hu Chen, and Qijun Zhao. Towards resolution invariant face recognition in uncontrolled scenarios. In 2016 International Conference on Biometrics (ICB), pages 1-8. IEEE, 2016.
  33. Kai Zhang, Wangmeng Zuo, and Lei Zhang. Deep plug-and-play super-resolution for arbitrary blur kernels. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1671-1681, 2019.
  34. Kaipeng Zhang, Zhanpeng Zhang, Chia-Wen Cheng, Winston H Hsu, Yu Qiao, Wei Liu, and Tong Zhang. Super-identity convolutional neural network for face hallucination. In Proceedings of the European conference on computer vision (ECCV), pages 183-198, 2018.
  35. Tianyue Zheng and Weihong Deng. Cross-pose lfw: A database for studying cross-pose face recognition in unconstrained environments. Beijing University of Posts and Telecommunications, Tech. Rep, 5, 2018.
  36. Tianyue Zheng, Weihong Deng, and Jiani Hu. Cross-age lfw: A database for studying cross-age face recognition in unconstrained environments. arXiv preprint, 2017. URL: http://arxiv.org/abs/1708.08197.
  37. Ruofan Zhou and Sabine Susstrunk. Kernel modeling super-resolution on real low-resolution images. In Proceedings of the IEEE International Conference on Computer Vision, pages 2433-2443, 2019.