Towards Labeling-free Fine-grained Animal Pose Estimation
Authors: 
Dan Zeng, Yu Zhu, Shuiwang Li, Qijun Zhao, Qiaomu Shen, Bo TangAuthors Info & Claims
Dan Zeng, Yu Zhu, Shuiwang Li, Qijun Zhao, Qiaomu Shen, Bo TangAuthors Info & ClaimsPages 2545 - 2553
Abstract
In this paper, we are interested in identifying denser and finer animals joints. The lack of standardized joint definitions across various APE datasets, e.g., AnimalPose with 20 joints, AP-10k with 17 joints, and TigDog with 19 joints, presents a significant challenge yet offers an opportunity to fully utilize annotation data. This paper challenges this new non-standardized annotation problem, aiming to learn fine-grained (e.g., 24 or more joints) pose estimators in datasets that lack complete annotations. To combat the unannotated joints, we propose FreeNet, comprising a base network and an adaptation network connected through a circuit feedback learning paradigm. FreeNet enhances the adaptation network's tolerance to unannotated joints via body part-aware learning, optimizing the sampling frequency of joints based on joint detection difficulty, and improves the base network's predictions for unannotated joints using feedback learning. This leverages the cognitive differences of the adaptation network between non-standardized labeled and large-scale unlabeled data. Experimental results on three non-standard datasets demonstrate the effectiveness of our method for fine-grained APE.
Supplemental Material
MP4 File - Towards Labeling-free Fine-grained Animal Pose Estimation
Video showing our animal pose estimation paper which learns fine-grained APE with free full annotation.
- Download
- 15.15 MB
References
[1]
Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele. 2014. 2d human pose estimation: New benchmark and state of the art analysis. In Proceedings of the IEEE Conference on computer Vision and Pattern Recognition. 3686--3693.
[2]
Ahmet Arac, Pingping Zhao, Bruce H Dobkin, S Thomas Carmichael, and Peyman Golshani. 2019. DeepBehavior: A deep learning toolbox for automated analysis of animal and human behavior imaging data. Frontiers in systems neuroscience, Vol. 13 (2019), 20.
[3]
Benjamin Biggs, Oliver Boyne, James Charles, Andrew Fitzgibbon, and Roberto Cipolla. 2020. Who left the dogs out? 3d animal reconstruction with expectation maximization in the loop. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XI 16. Springer, 195--211.
[4]
Jinkun Cao, Hongyang Tang, Hao-Shu Fang, Xiaoyong Shen, Cewu Lu, and Yu-Wing Tai. 2019. Cross-domain adaptation for animal pose estimation. In Proceedings of the IEEE/CVF international conference on computer vision. 9498--9507.
[5]
Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. 2020. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 702--703.
[6]
Luca Del Pero, Susanna Ricco, Rahul Sukthankar, and Vittorio Ferrari. 2015. Articulated motion discovery using pairs of trajectories. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2151--2160.
[7]
Jacob M Graving, Daniel Chae, Hemal Naik, Liang Li, Benjamin Koger, Blair R Costelloe, and Iain D Couzin. 2019. DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning. Elife, Vol. 8 (2019), e47994.
[8]
Yixiao Guo, Jiawei Liu, Guo Li, Luo Mai, and Hao Dong. 2021. Fast and flexible human pose estimation with hyperpose. In Proceedings of the 29th ACM International Conference on Multimedia. 3763--3766.
[9]
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, Vol. 31 (2018).
[10]
Le Jiang, Caleb Lee, Divyang Teotia, and Sarah Ostadabbas. 2022. Animal pose estimation: A closer look at the state-of-the-art, existing gaps and opportunities. Computer Vision and Image Understanding (2022), 103483.
[11]
Dong-Hyun Lee et al. 2013. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, Vol. 3. Atlanta, 896.
[12]
Chen Li and Gim Hee Lee. 2021. From synthetic to real: Unsupervised domain adaptation for animal pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1482--1491.
[13]
Chen Li and Gim Hee Lee. 2023. ScarceNet: Animal Pose Estimation with Scarce Annotations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17174--17183.
[14]
Guangrui Li, Yifan Sun, Zongxin Yang, and Yi Yang. 2022. Decompose to Generalize: Species-Generalized Animal Pose Estimation. In The Eleventh International Conference on Learning Representations.
[15]
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6--12, 2014, Proceedings, Part V 13. Springer, 740--755.
[16]
George Martvel, Nareed Farhat, Ilan Shimshoni, and Anna Zamansky. 2023. Catflw: Cat facial landmarks in the wild dataset. arXiv preprint arXiv:2305.04232 (2023).
[17]
Alexander Mathis, Thomas Biasi, Steffen Schneider, Mert Yuksekgonul, Byron Rogers, Matthias Bethge, and Mackenzie W Mathis. 2021. Pretraining boosts out-of-domain robustness for pose estimation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 1859--1868.
[18]
Eduardo Mendoza, Pierre R Martineau, Elliott Brenner, and Rodolfo Dirzo. 2011. A novel method to improve individual animal identification based on camera-trapping data. The Journal of Wildlife Management, Vol. 75, 4 (2011), 973--979.
[19]
Jiteng Mu, Weichao Qiu, Gregory D Hager, and Alan L Yuille. 2020. Learning from synthetic animals. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12386--12395.
[20]
Xun Long Ng, Kian Eng Ong, Qichen Zheng, Yun Ni, Si Yong Yeo, and Jun Liu. 2022. Animal kingdom: A large and diverse dataset for animal behavior understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 19023--19034.
[21]
Bingbing Ni, Teng Li, and Xiaokang Yang. 2017. Learning semantic-aligned action representation. IEEE transactions on neural networks and learning systems, Vol. 29, 8 (2017), 3715--3725.
[22]
Hieu Pham, Zihang Dai, Qizhe Xie, and Quoc V Le. 2021. Meta pseudo labels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11557--11568.
[23]
Moira Shooter, Charles Malleson, and Adrian Hilton. 2021. SyDog: A synthetic dog dataset for improved 2D pose estimation. arXiv preprint arXiv:2108.00249 (2021).
[24]
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. 2020. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems, Vol. 33 (2020), 596--608.
[25]
Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. 2019. Deep high-resolution representation learning for human pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5693--5703.
[26]
Jianbo Wang, Kai Qiu, Houwen Peng, Jianlong Fu, and Jianke Zhu. 2019. Ai coach: Deep human pose estimation and analysis for personalized athletic training assistance. In Proceedings of the 27th ACM international conference on multimedia. 374--382.
[27]
Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. 2020. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10687--10698.
[28]
Jinhai Yang, Hua Yang, and Lin Chen. 2020. Coarse-to-fine pseudo-labeling guided meta-learning for few-shot classification. arXiv preprint arXiv:2007.05675 (2020).
[29]
Yuxiang Yang, Junjie Yang, Yufei Xu, Jing Zhang, Long Lan, and Dacheng Tao. 2022. Apt-36k: A large-scale benchmark for animal pose estimation and tracking. Advances in Neural Information Processing Systems, Vol. 35 (2022), 17301--17313.
[30]
Hang Yu, Yufei Xu, Jing Zhang, Wei Zhao, Ziyu Guan, and Dacheng Tao. 2021. Ap-10k: A benchmark for animal pose estimation in the wild. arXiv preprint arXiv:2108.12617 (2021).
Index Terms
- Towards Labeling-free Fine-grained Animal Pose Estimation
Recommendations
Animal pose estimation: A closer look at the state-of-the-art, existing gaps and opportunities
AbstractOver the past few years, research on animal pose estimation in computer vision field has grown in many aspects such as 2D and 3D pose estimation, 3D mesh reconstruction, and behavior prediction. Promoted by deep learning, more and more ...
Highlights- Collecting and categorizing animal pose estimation work from 2013 to 2021 into 2D and 3D pose estimation, model reconstruction, and behavior prediction, ...
Corpus-level fine-grained entity typing
Extracting information about entities remains an important research area. This paper addresses the problem of corpus-level entity typing, i.e., inferring from a large corpus that an entity is a member of a class, such as "food" or "artist". The ...
Rigid body pose estimation based on the Lagrange-d'Alembert principle
Stable estimation of rigid body pose and velocities from noisy measurements, without any knowledge of the dynamics model, is treated using the Lagrange-d'Alembert principle from variational mechanics. With body-fixed vision and inertial sensor ...
Comments
Information & Contributors
Information
Published In
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
- General Chairs:
- Jianfei Cai,
- Mohan Kankanhalli,
- Balakrishnan Prabhakaran,
- Susanne Boll,
- Program Chairs:
- Ramanathan Subramanian,
- Liang Zheng,
- Vivek K. Singh,
- Pablo Cesar,
- Lexing Xie,
- Dong Xu
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- Guangxi Natural Science Foundation
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Acceptance Rates
MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 126Total Downloads
- Downloads (Last 12 months)126
- Downloads (Last 6 weeks)75
Reflects downloads up to 02 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in