Individual Behavior Recognition in Laboratory Rats: a Comparative Analysis of Computer Vision Methods

Dmitrii Krasnov orcid (Login required)
ITMO University, Saint-Petersburg, Russian Federation

Aleksey Shmonin orcid
Federal Scientific and Clinical Center for Resuscitation and Rehabilitation
Pavlov First Saint Petersburg State Medical University, Moscow region, Solnechnogorsk, Russian Federation

Maxim Volynsky orcid
ITMO University, Saint-Petersburg, Russian Federation

Nikita Margaryants orcid
ITMO University, Saint-Petersburg, Russian Federation
Pavlov First Saint Petersburg State Medical University, Saint-Petersburg, Russian Federation

Alexandr Gusev orcid
ITMO University, Saint-Petersburg, Russian Federation

Maria Maltseva orcid
Pavlov First Saint Petersburg State Medical University, Saint-Petersburg, Russian Federation

Elena Korotkova orcid
Pavlov First Saint Petersburg State Medical University, Saint-Petersburg, Russian Federation




DOI: 10.18287/JBPE26.12.010306

Abstract

Computer-vision methods have been applied to automated behavior recognition in laboratory rodents. Namely, we explored the possibility of classifying certain behavior classes from still images; compared keypoint-based methods with approaches based on visual embeddings; studied the feasibility of transferring models between rats and mice; and evaluated the relevance of the number and the accuracy of the detected keypoints. We collected a dataset of a freely moving Wistar rat to train six pose-based classifiers with Long Short-Term Memory (LSTM) using six sets of keypoints produced by two detectors and four Convolutional Neural Network (CNN) classifiers using images and optical flow frames. The results demonstrated the highest mean average precision (mAP) of 65.7% for the CNN-based methods and 34.3% for the LSTM classifiers, the feasibility of recognizing visually distinct classes (rearing and body grooming) from still images, and the applicability of a keypoint detector trained on mice. The results of this study can be applied to the design of a computer vision system for automating long-term monitoring of laboratory rodent behavior

Keywords

сomputer vision; deep learning; machine learning; rat behavior; rodent behavior; action recognition; keypoints; convolutional neural network; Wistar rat

Full Text:

PDF

References


1. F. Rotaru, S. I. Bejinariu, H. N. Costin, R. Luca, and C. D. Nita, “Captive animal behavior study by video analysis,” Sensors 23(18), 7928 (2023).

2. Himanshu, Dharmila, D. Sarkar, and Nutan, “A review of behavioral tests to evaluate different types of anxiety and anti-anxiety effects,” Clinical Psychopharmacology and Neuroscience: The Official Scientific Journal of the Korean College of Neuropsychopharmacology 18(3), 341–351 (2020).

3. F. Grieco, B. J. Bernstein, B. Biemans, et al., “Measuring behavior in the home cage: Study design, applications, challenges, and perspectives,” Frontiers in Behavioral Neuroscience 15, 735387 (2021).

4. K. Luxem, J. J. Sun, S. P. Bradley, K. Krishnan, E. Yttri, J. Zimmermann, T. D. Pereira, and M. Laubach, “Open-source tools for behavioral video analysis: Setup, methods, and best practices,” eLife 12, e79305 (2023).

5. M. Perez and C. Toler-Franklin, “CNN-based action recognition and pose estimation for classifying animal behavior from videos: A survey,” arXiv:2301.06187 (2023).

6. S. İşık, G. Ünal, “Open-source software for automated rodent behavioral analysis,” Frontiers in Neuroscience 17, 1149027 (2023).

7. C. J. Gabriel, Z. Zeidler, B. Jin, C. Guo, C. M. Goodpaster, A. Q. Kashay, A. Wu, M. Delaney, J. Cheung, L. E. DiFazio, M. J. Sharpe, D. Aharoni, S. A. Wilke, and L. A. DeNardo, “BehaviorDEPOT is a simple, flexible tool for automated behavioral detection based on markerless pose tracking,” eLife 11, e74314 (2022).

8. M. Kabra, A. A. Robie, M. Rivera-Alba, S. Branson, and K. Branson, “JAABA: interactive machine learning for automatic annotation of animal behavior,” Nature Methods 10(1), 64–67 (2013).

9. M. Lorbach, R. Poppe, E. A. Van Dam, L. P. J. J. Noldus, and R. C. Veltkamp, “Automated Recognition of Social Behavior in Rats: The Role of Feature Quality,” in Image Analysis and Processing – ICIAP 2015, V. Murino and E. Puppo (Eds.), Springer International Publishing, 9280, 565–574 (2015).

10. C. Segalin, J. Williams, T. Karigo, M. Hui, M. Zelikowsky, J. J. Sun, P. Perona, D. J. Anderson, and A. Kennedy, “The Mouse Action Recognition System (MARS) software pipeline for automated analysis of social behaviors in mice,” eLife 10, e63720 (2021).

11. A. I. Hsu, E. A. Yttri, “B-SOiD, an open-source unsupervised algorithm for identification and fast prediction of behaviors,” Nature Communications 12, 5188 (2021).

12. E. A. Van Dam, L. P. J. J. Noldus, and M. A. J. Van Gerven, “Deep learning improves automated rodent behavior recognition within a specific experimental setup,” Journal of Neuroscience Methods 332, 108536 (2020).

13. J. Nie, L. Yan, X. Wang, and J. Chen, “A Novel 3D Convolutional Neural Network for Action Recognition in Infrared Videos,” in 2021 4th International Conference on Information Communication and Signal Processing (ICICSP), IEEE, 420–424 (2021).

14. C. Feichtenhofer, A. Pinz, and A. Zisserman, “Convolutional two-stream network fusion for video action recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1933–1941 (2016).

15. Q. Wang, J. Du, K. Yan, and S. Ding, “Seeing in flowing: Adapting CLIP for action recognition with motion prompts learning,” Proceedings of the 31st ACM International Conference on Multimedia (ACM), 5339–5347 (2023).

16. Z. Bai, Y. Li, Z. Ru, S. Li, W. Li, and F. Duan, “Rats Behavior Recognition Based on Improved YOLOv8,” in 2024 IEEE International Conference on Robotics and Biomimetics (ROBIO), IEEE, 1082–1086 (2024).

17. K. M. Hatton-Jones, C. Christie, T. A. Griffith, A. G. Smith, S. Naghipour, K. Robertson, J. S. Russell, J. N. Peart, J. P. Headrick, A. J. Cox, and E. F. Du Toit, “A YOLO based software for automated detection and analysis of rodent behaviour in the open field arena,” Computers in Biology and Medicine 134, 104474 (2021).

18. N. G. Nguyen, D. Phan, F. R. Lumbanraja, M. R. Faisal, B. Abapihi, B. Purnama, M. K. Delimayanti, K. R. Mahmudah, M. Kubo, and K. Satou, “Applying Deep Learning Models to Mouse Behavior Recognition,” Journal of Biomedical Science and Engineering 12(02), 183–196 (2019).

19. J. Carreira, A. Zisserman, “Quo Vadis, action recognition? A new model and the Kinetics dataset,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4724–4733 (2017).

20. D. I. Krasnov, M. A. Volynsky, and A. A. Gusev, “Preprocessing of skeletal keypoints trajectories in the task of laboratory animal behavior recording automation,” Scientific and Technical Journal of Information Technologies, Mechanics and Optics 25(2), 295–302 (2025). [in Russian]

21. D. I. Krasnov, M. A. Volynsky, and A. A. Gusev, “State estimation of an agent with stochastic dynamics using recurrent filters,” Informatics and Automation 24(5), 1355–1378 (2025). [in Russian]

22. N. L. Goodwin, J. J. Choong, S. Hwang, K. Pitts, L. Bloom, A. Islam, Y. Y. Zhang, E. R. Szelenyi, X. Tong, E. L. Newman, K. Miczek, H. R. Wright, R. J. McLaughlin, Z. C. Norville, N. Eshel, M. Heshmati, S. R. O. Nilsson, and S. A. Golden, “Simple Behavioral Analysis (SimBA) as a platform for explainable machine learning in behavioral neuroscience,” Nature Neuroscience 27(7), 1411–1424 (2024).

23. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778 (2016).

24. K. Hara, H. Kataoka, and Y. Satoh, “Learning spatio-temporal features with 3D residual networks for action recognition,” Proceedings of the IEEE International Conference on Computer Vision Workshop (ICCVW), 3154–3160 (2017).

25. G. Farnebäck, “Two-Frame Motion Estimation Based on Polynomial Expansion,” in Image Analysis, J. Bigun and T. Gustavsson (Eds.), Springer Berlin Heidelberg, 2749, 363–370 (2003).

26. C. Lyu, W. Zhang, H. Huang, Y. Zhou, Y. Wang, Y. Liu, S. Zhang, and K. Chen, “RTMDet: An empirical study of designing real-time object detectors,” arXiv:2212.07784 (2022).

27. J. Huang, Z. Zhu, F. Guo, and G. Huang, “The devil is in the details: Delving into unbiased data processing for human pose estimation,” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 5699–5708 (2020).




Bookmark and Share


Сontact

34 Moskovskoe shosse, Samara, 443086, Russian Federation
Email: j-bpe@ssau.ru
Phone: +7-846-267-4550
© 2014-2025 J-BPE