Comparing multi-task strategies for multiple dataset learning in bioacoustics

Multi-task learning describes a training paradigm in which models learn several tasks together, with the underlying assumption that the knowledge of one task can be leveraged to better learn other tasks and produce more robust, task-independent features. This approach is well-suited in bioacoustics, where diverse datasets are currently publicly available but generally used in isolation. In this work, we explore multi- task learning as a means to leverage these datasets simultaneously and train more generalisable models. We compare three multi-task learning approaches to single-task models for the species, individual, and vocali- sation type classification tasks. Then, we evaluate the generalisation of all the resulting models on the BEnchmark of ANimal Sounds (BEANS). Our results show that single-task models generally outperform multi-task models, with the exception of one multi-task model which generalises bet- ter on the BEANS benchmark than one single-task model. This indicates that while multi-task learning may surpass single-task learning in some cases, the latter displays better results overall. We also show that the success of multi-task training depends on the relative task weightings, and find limited benefit from automatic task weighting (GradNorm).