Using Deep Learning to Quantify Mouse Orofacial Behavior
Machine vision algorithms, especially in neuroscience, have changed how researchers analyze behavior via application to tracking body movements in video frames. One advanced tool, Social LEAP Estimates Animal Poses (SLEAP), utilizes deep learning to track animal pose by predicting body part locations. Throughout the summer, I trained two SLEAP models: one for videos captured from a side-facing camera and another from a lower camera angle. Each model was trained using 11 videos, with 2 videos held out for pose prediction, which is called inference. After inference was run on the test videos, I applied Kalman and Gaussian filters to smooth out noise, and selected filter parameters to remove noise without also removing true movement signal. While all of the filters smoothed large pose deviations caused by errors in model predictions, it remained unclear which filter provided the best tradeoff between maintaining signal and rejecting noise. To resolve this, we evaluated each filter on inference output from the lower camera model by comparing the variance in snout tip distances between a mouse included in the training data and one excluded. The snout moves rigidly so the distance across the snout should be constant over time. High variance would indicate that the data was smoothed too much or too little. The mean value should accurately match the true width of the snout. The results suggest that the Kalman filter was most effective in reducing variance, particularly between the center snout and jaw tips of a mouse not in the training set. A similar evaluation was used for the side camera data. While the project is still a work in progress, my goal is to develop robust models for future research on the neural control of orofacial movements. The project will also explore alternative approaches to quantifying animal behavior like motion energy analysis or Lightning Pose.
Comments