Shiming Ge
ABSTRACT
Is it possible to train an effective face recognition model with fewer shots that works efficiently on low-resolution faces in the wild? To answer this question, this paper proposes a few-shot knowledge distillation approach to learn an ultrafast face recognizer via two steps. In the first step, we initialize a simple yet effective face recognition model on synthetic low-resolution faces by distilling knowledge from an existing complex model. By removing the redundancies in both face images and the model structure, the initial model can provide an ultrafast speed with impressive recognition accuracy. To further adapt this model into the wild scenarios with fewer faces per person, the second step refines the model via few-shot learning by incorporating a relation module that compares low-resolution query faces with faces in the support set. In this manner, the performance of the model can be further enhanced with only fewer low-resolution faces in the wild. Experimental results show that the proposed approach performs favorably against state-of-the-arts in recognizing low-resolution faces with an extremely low memory of 30KB and runs at an ultrafast speed of 1,460 faces per second on CPU or 21,598 faces per second on GPU.
- Ankan Bansal, Anirudh Nanduri, Carlos D Castillo, Rajeev Ranjan, and Rama Chellappa. 2017. UMDFaces: An Annotated Face Dataset for Training Deep Networks. In IEEE International Joint Conference on Biometrics. 464--473.Google Scholar
- Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and Andrew Zisserman. 2018. VGGFace2: A Dataset for Recognising Faces Across Pose and Age. In IEEE International Conference on Automatic Face and Gesture Recognition. 67--74.Google ScholarCross Ref
- Bowen Cheng, Ding Liu, Zhangyang Wang, Haichao Zhang, and Thomas S Huang. 2018a. Visual Recognition in Very Low-Quality Settings: Delving into the Power of Pre-Training. In AAAI Conference on Artificial Intelligence. 8065--8066.Google Scholar
- Zhiyi Cheng, Xiatian Zhu, and Shaogang Gong. 2018b. Low-Resolution Face Recognition. In Asian Conference on Computer Vision . 605--621.Google Scholar
- Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019. ArcFace: Additive Angular Margin Loss for Deep Face Recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 4690--4699.Google Scholar
- Li Fei-Fei, Rob Fergus, and Pietro Perona. 2006. One-Shot Learning of Object Categories. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 28, 4 (2006), 594--611.Google ScholarDigital Library
- Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In International Conference on Machine Learning. 1126--1135.Google ScholarDigital Library
- Victor Garcia and Joan Bruna. 2018. Few-Shot Learning with Graph Neural Networks. In International Conference on Learning Representations .Google Scholar
- Shiming Ge, Shengwei Zhao, Chenyu Li, and Jia Li. 2019. Low-Resolution Face Recognition in the Wild via Selective Knowledge Distillation. IEEE Transactions on Image Processing , Vol. 28, 4 (2019), 2051--2062.Google ScholarDigital Library
- Weifeng Ge and Yizhou Yu. 2017. Borrowing Treasures from the Wealthy: Deep Transfer Learning through Selective Joint Fine-Tuning. In IEEE Conference on Computer Vision and Pattern Recognition. 1086--1095.Google ScholarCross Ref
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems. 2672--2680.Google Scholar
- Manuel Günther, Peiyun Hu, Christian Herrmann, Chi-Ho Chan, Min Jiang, Shufan Yang, Akshay Raj Dhamija, Deva Ramanan, Jürgen Beyerer, Josef Kittler, et almbox. 2017. Unconstrained Face Detection and Open-Set Face Recognition Challenge. In IEEE International Joint Conference on Biometrics. 697--706.Google Scholar
- Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. 2016. MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition. In European Conference on Computer Vision. 87--102.Google ScholarCross Ref
- Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2014. Distilling the Knowledge in a Neural Network. In Advances in Neural Information Processing Systems Workshop .Google Scholar
- Junjun Jiang, Yi Yu, Jinhui Hu, Suhua Tang, and Jiayi Ma. 2018. Deep CNN Denoiser and Multi-Layer Neighbor Component Embedding for Face Hallucination. In International Joint Conference on Artificial Intelligence. 771--778.Google Scholar
- Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. 2015. Siamese Neural Networks for One-Shot Image Recognition. In International Conference on Machine Learning Workshop .Google Scholar
- Soheil Kolouri and Gustavo K Rohde. 2015. Transport-based Single Frame Super Resolution of Very Low Resolution Face Images. In IEEE Conference on Computer Vision and Pattern Recognition . 4876--4884.Google Scholar
- Erik Learned-Miller, Gary B Huang, Aruni RoyChowdhury, Haoxiang Li, and Gang Hua. 2016. Labeled Faces in the Wild: A Survey. In Advances in Face Detection and Facial Image Analysis. 189--248.Google Scholar
- Pei Li, Loreto Prieto, Domingo Mery, and Patrick J Flynn. 2019. On Low-Resolution Face Recognition in the Wild: Comparisons and New Techniques. IEEE Transactions on Information Forensics and Security , Vol. 14, 8 (2019), 2000--2012.Google ScholarDigital Library
- Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. 2017. SphereFace: Deep Hypersphere Embedding for Face Recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 212--220.Google Scholar
- David Lopez-Paz, Léon Bottou, Bernhard Schölkopf, and Vladimir Vapnik. 2016. Unifying Distillation and Privileged Information. In International Conference on Learning Representations .Google Scholar
- Ze Lu, Xudong Jiang, and Alex Kot. 2018. Deep Coupled Resnet for Low-Resolution Face Recognition. IEEE Signal Processing Letters , Vol. 25, 4 (2018), 526--530.Google ScholarCross Ref
- Ping Luo, Zhenyao Zhu, Ziwei Liu, Xiaogang Wang, and Xiaoou Tang. 2016. Face Model Compression by Distilling Knowledge from Neurons. In AAAI Conference on Artificial Intelligence. 3560--3566.Google Scholar
- Sivaram Prasad Mudunuri and Soma Biswas. 2016. Low Resolution Face Recognition Across Variations in Pose and Illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 38, 5 (2016), 1034--1040.Google ScholarDigital Library
- Sivaram Prasad Mudunuri, Soubhik Sanyal, and Soma Biswas. 2018. GenLR-Net: Deep Framework for Very Low Resolution Face and Object Recognition with Generalization to Unseen Categories. In IEEE Conference on Computer Vision and Pattern Recognition Workshop. 489--498.Google ScholarCross Ref
- Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, et almbox. 2015. Deep Face Recognition. In British Machine Vision Conference .Google Scholar
- Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, and Kaiming He. 2018. Data Distillation: Towards Omni-Supervised Learning. In IEEE Conference on Computer Vision and Pattern Recognition. 4119--4128.Google Scholar
- Sachin Ravi and Hugo Larochelle. 2017. Optimization as a Model for Few-Shot Learning. In International Conference on Learning Representations .Google Scholar
- Joseph P Robinson, Ming Shao, Yue Wu, and Yun Fu. 2016. Families in the Wild (FIW): Large-Scale Kinship Image Database and Benchmarks. In ACM International Conference on Multimedia. 242--246.Google ScholarDigital Library
- Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. 2015. Fitnets: Hints for Thin Deep Nets. In International Conference on Learning Representations .Google Scholar
- Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. FaceNet: A Unified Embedding for Face Recognition and Clustering. In IEEE Conference on Computer Vision and Pattern Recognition. 815--823.Google Scholar
- Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical Networks for Few-Shot Learning. In Advances in Neural Information Processing Systems. 4077--4087.Google Scholar
- Yi Sun, Yuheng Chen, Xiaogang Wang, and Xiaoou Tang. 2014a. Deep Learning Face Representation by Joint Identification-Verification. In Advances in Neural Information Processing Systems. 1988--1996.Google Scholar
- Yi Sun, Ding Liang, Xiaogang Wang, and Xiaoou Tang. 2015a. DeepID3: Face Recognition with Very Deep Neural Networks. arXiv preprint arXiv:1502.00873 (2015).Google Scholar
- Yi Sun, Xiaogang Wang, and Xiaoou Tang. 2014b. Deep Learning Face Representation from Predicting 10,000 Classes. In IEEE Conference on Computer Vision and Pattern Recognition. 1891--1898.Google ScholarDigital Library
- Yi Sun, Xiaogang Wang, and Xiaoou Tang. 2015b. Deeply Learned Face Representations are Sparse, Selective, and Robust. In IEEE Conference on Computer Vision and Pattern Recognition. 2892--2900.Google ScholarCross Ref
- Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. 2018. Learning to Compare: Relation Network for Few-Shot Learning. In IEEE Conference on Computer Vision and Pattern Recognition. 1199--1208.Google ScholarCross Ref
- Yaniv Taigman, Ming Yang, Marc'Aurelio Ranzato, and Lior Wolf. 2014. DeepFace: Closing the Gap to Human-Level Performance in Face Verification. In IEEE Conference on Computer Vision and Pattern Recognition . 1701--1708.Google ScholarDigital Library
- Antti Tarvainen and Harri Valpola. 2017. Mean Teachers are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results. In Advances in Neural Information Processing Systems. 1195--1204.Google Scholar
- Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et almbox. 2016. Matching Networks for One Shot Learning. In Advances in Neural Information Processing Systems. 3630--3638.Google ScholarDigital Library
- Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. 2018a. CosFace: Large Margin Cosine Loss for Deep Face Recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 5265--5274.Google Scholar
- Hui Wang, Hanbin Zhao, Xi Li, and Xu Tan. 2018b. Progressive Blockwise Knowledge Distillation for Neural Network Acceleration. In International Joint Conference on Artificial Intelligence. 2769--2775.Google Scholar
- Zhangyang Wang, Shiyu Chang, Yingzhen Yang, Ding Liu, and Thomas S Huang. 2016. Studying Very Low Resolution Recognition Using Deep Networks. In IEEE Conference on Computer Vision and Pattern Recognition. 4792--4800.Google Scholar
- Bichen Wu, Alvin Wan, Xiangyu Yue, Peter Jin, Sicheng Zhao, Noah Golmant, Amir Gholaminejad, Joseph Gonzalez, and Kurt Keutzer. 2018. Shift: A Zero Flop, Zero Parameter Alternative to Spatial Convolutions. In IEEE Conference on Computer Vision and Pattern Recognition. 9127--9135.Google ScholarCross Ref
- Fuwei Yang, Wenming Yang, Riqiang Gao, and Qingmin Liao. 2018. Discriminative Multidimensional Scaling for Low-Resolution Face Recognition. IEEE Signal Processing Letters , Vol. 25, 3 (2018), 388--392.Google ScholarCross Ref
- Dong Yi, Zhen Lei, Shengcai Liao, and Stan Z Li. 2014. Learning Face Representation from Scratch. arXiv preprint arXiv:1411.7923 (2014).Google Scholar
- Xin Yu, Basura Fernando, Richard Hartley, and Fatih Porikli. 2018. Super-Resolving Very Low-Resolution Face Images with Supplementary Attributes. In IEEE Conference on Computer Vision and Pattern Recognition. 908--917.Google Scholar
- Xin Yu and Fatih Porikli. 2017. Hallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders. In IEEE Conference on Computer Vision and Pattern Recognition. 3760--3768.Google ScholarCross Ref
- Chenrui Zhang and Yuxin Peng. 2018. Better and Faster: Knowledge Transfer From Multiple Self-Supervised Learning Tasks via Graph Distillation for Video Classification. In International Joint Conference on Artificial Intelligence. 1135--1141.Google ScholarCross Ref
- Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016. Joint Face Detection and Alignment using Multitask Cascaded Convolutional Networks. IEEE Signal Processing Letters , Vol. 23, 10 (2016), 1499--1503.Google ScholarCross Ref
Index Terms
- Fewer-Shots and Lower-Resolutions: Towards Ultrafast Face Recognition in the Wild
Recommendations
A novel approach inspired by optic nerve characteristics for few-shot occluded face recognition
AbstractAlthough there has been a growing body of work for face recognition, it is still a challenging task for faces under occlusion with limited training samples. In this work, we propose a novel framework to address the problem of few-shot ...
Face recognition under varying illumination using gradientfaces
In this correspondence, we propose a novel method to extract illumination insensitive features for face recognition under varying lighting called the Gradientfaces. Theoretical analysis shows Gradientfaces is an illumination insensitive measure, and ...
Age-Invariant Face Recognition
One of the challenges in automatic face recognition is to achieve temporal invariance. In other words, the goal is to come up with a representation and matching scheme that is robust to changes due to facial aging. Facial aging is a complex process that ...
Comments