A Biomimetic Covering Learning Method Based on Principle of Homology Continuity

Xin Ning; Yuebao Wang; Weijuan Tian; Liang Liu; Weiwei Cai

doi:10.52810/TPRIS.2021.100009

Authors

Xin Ning Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China https://orcid.org/0000-0001-7897-1673
Yuebao Wang Cognitive Computing Technology Joint Laboratory, Wave Group, Beijing 10008, China https://orcid.org/0000-0003-0567-4269
Weijuan Tian Cognitive Computing Technology Joint Laboratory, Wave Group, Beijing 10008, China https://orcid.org/0000-0001-6722-7056
Liang Liu Cognitive Computing Technology Joint Laboratory, Wave Group, Beijing 10008, China https://orcid.org/0000-0003-2168-1602
Weiwei Cai Central South University of Forestry and Technology, Changsha 410004, China https://orcid.org/0000-0001-6795-6152

https://doi.org/10.52810/TPRIS.2021.100009

Keywords:

Brain-inspired, covering neuron models, deep learning, homology continuity, optimal coverage

Abstract

The Principle of Homology Continuity (PHC) based covering learning method is an effective method to solve the pattern recognition problem. However, PHC and the existence of optimal coverage are not mathematical proven. To address this issue, we firstly give the mathematical description and theoretical proof of PHC. On this basis, the theoretical definition of optimal coverage is introduced. Optimal coverage can determine the internal connections among samples as prior knowledge and use covering neurons to learn prior knowledge. Finally, we propose a kind of covering neuron model, and the effectiveness of which is demonstrated through extensive experiments conducted on the CIFAR-10, LFW, and YTF datasets.

Downloads

Download data is not yet available.

Author Biographies

Xin Ning, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China

Xin Ning received the B.S. degree in software engineering from Xinjiang University, Wulumuqi, China, in 2012, and the Ph.D. degree in electronic circuit and system from the Institute of Semiconductors, Chinese Academy of Sciences, in 2017. He is currently an Associate Professor with the Laboratory of Artifificial Neural Networks and High Speed Circuits, Institute of Semiconductors, Chinese Academy of Sciences. He has authored or co-authored more than 40 papers in journals and refereed conferences. His current research interests include pattern recognition, computer vision, and image processing.

Yuebao Wang, Cognitive Computing Technology Joint Laboratory, Wave Group, Beijing 10008, China

Yuebao Wang received a B.S Degree in mathematics and Applied Mathematics from North University of China in 2013, and received M.S degree in operations research and cybernetics Currently in 2018. He worked as an algorithm research fellow in the Cognitive Computing Technology Joint Laboratory of Wave Group. His current research interests include machine learning, function approximation theory and TDA.

Weijuan Tian, Cognitive Computing Technology Joint Laboratory, Wave Group, Beijing 10008, China

Weijuan Tian received her M.S. degrees from School of Electronic Engineering in Xidian University in 2017. Currently, she worked as an algorithm research fellow in the Cognitive Computing Technology Joint Laboratory of Wave Group. Her research interests include machine learning, DNNs, neuron modeling, video and image processing.

Liang Liu, Cognitive Computing Technology Joint Laboratory, Wave Group, Beijing 10008, China

Liang Liu Received dual B.S. degrees from Wuhan University and Wuhan University of Technology, and the Ph.D. degree in North China Electric Power University. He worked as an algorithm research fellow in the Cognitive Computing Technology Joint Laboratory of Wave Group. His current research interests include DNNs, computer vision, and image processing.

Weiwei Cai, Central South University of Forestry and Technology, Changsha 410004, China

Weiwei Cai is currently pursuing the master’s degree with the Central South University of Forestry and Technology, Changsha, China. Prior to that, he worked in IT industry for more than ten years in the roles of an Enterprise Architect and a Program Manager. His research interests include machine learning, deep learning, and computer vision.

References

J. Somers, “Is ai riding a one-trick pony?” Technology review, vol. 120, no. 6, pp. 29–36, 2017.

Yuille, A. L., & Liu, C. (2021). Deep nets: What have they ever done for vision?. International Journal of Computer Vision, 129(3), 781-802.

Almahairi, A., Ballas, N., Cooijmans, T., Zheng, Y., Larochelle, H., & Courville, A. (2016, June). Dynamic capacity networks. In International Conference on Machine Learning (pp. 2549-2558).

Ning, X., Li, W., Tang, B., & He, H. (2018). BULDP: biomimetic uncorrelated locality discriminant projection for feature extraction in face recognition. IEEE Transactions on Image Processing, 27(5), 2575-2586.

Ning, X., Nan, F., Xu, S., Yu, L., & Zhang, L. (2020). Multi‐view frontal face image generation: A survey. Concurrency and Computation: Practice and Experience, e6147.

Ning, X., Li, W., & Liu, W. (2017). A fast single image haze removal method based on human retina property. IEICE TRANSACTIONS on Information and Systems, 100(1), 211-214.

Ning, X., Li, W., & Xu, J. (2018). The principle of homology continuity and geometrical covering learning for pattern recognition. International Journal of Pattern Recognition and Artificial Intelligence, 32(12), 1850042.

Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020). Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(8), 2011-2023.

Woo, S., Park, J., Lee, J. Y., & Kweon, I. S. (2018). Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV) (pp. 3-19).

Li, X., Wang, W., Hu, X., & Yang, J. (2019). Selective kernel networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 510-519).

Sabour, S., Frosst, N., & Hinton, G. E. (2017, January). Dynamic Routing Between Capsules. In NIPS.

Cheng, X., He, J., He, J., & Xu, H. (2019). Cv-CapsNet: Complex-valued capsule network. IEEE Access, 7, 85492-85499.

Zhu, X., Su, W., Lu, L., Li, B., Wang, X., & Dai, J. (2020). Deformable DETR: Deformable Transformers for End-to-End Object Detection. arXiv preprint arXiv:2010.04159.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., ... & Houlsby, N. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.

Seung, H. S. (1998). Learning continuous attractors in recurrent networks. In Advances in neural information processing systems (pp. 654-660).

Seung, H. S., & Lee, D. D. (2000). The manifold ways of perception. science, 290(5500), 2268-2269.

Shou-Jue, W. (2002). Bionic (topological) pattern recognition-a new model of pattern recognition theory and its applications. Acta Electronica Sinica, 10.

Shoujue, W., & Jiangliang, L. (2005). Geometrical learning, descriptive geometry, and biomimetic pattern recognition. Neurocomputing, 67, 9-28.

Zhao, X. R., He, Q., & Shi, Z. Z. (2006, May). Hypersurface classifiers ensemble for high dimensional data sets. In International Symposium on Neural Networks (pp. 1299-1304). Springer, Berlin, Heidelberg.

Kamgar-Parsi, B., Lawson, W., & Kamgar-Parsi, B. (2011). Toward development of a face recognition system for watchlist surveillance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(10), 1925-1937.

Muraina, I. D., Alobaedy, M. M., & Ibrahim, H. H. (2017). A framework for preserving data integrity during mobile device forensic in open source software environment. In Proceedings of the Free and Open Source Software Conference (FOSSC), Muscat, Oman (pp. 22-26).

Hinton, G. E., Sabour, S., & Frosst, N. (2018, February). Matrix capsules with EM routing. In International conference on learning representations.

Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 815-823).

Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019). Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4690-4699).

Huang, G. B., Mattar, M., Berg, T., & Learned-Miller, E. (2008, October). Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on faces in'Real-Life'Images: detection, alignment, and recognition.

Wolf, L., Hassner, T., & Maoz, I. (2011, June). Face recognition in unconstrained videos with matched background similarity. In CVPR 2011 (pp. 529-534). IEEE.

Taigman, Y., Yang, M., Ranzato, M. A., & Wolf, L. (2014). Deepface: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1701-1708).

Wen, Y., Zhang, K., Li, Z., & Qiao, Y. (2016, October). A discriminative feature learning approach for deep face recognition. In European conference on computer vision (pp. 499-515).

Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., & Song, L. (2017). Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 212-220).

Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., ... & Liu, W. (2018). Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5265-5274).

Shi, Y., Yu, X., Sohn, K., Chandraker, M., & Jain, A. K. (2020). Towards universal representation learning for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 6817-6826).

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).

Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020, August). End-to-end object detection with transformers. In European Conference on Computer Vision (pp. 213-229).

Zhao, Y., Zeng, Y., & Qiao, G. (2021). Brain-inspired classical conditioning model. Iscience, 24(1), 101980.

Kosiorek, A. R., Sabour, S., Teh, Y. W., Hinton, G. E., & STAFFORD-TOLLEY, M. J. (2019). Stacked capsule autoencoders. Advances in Neural Information Processing Systems, 32.