Analysis
We evaluated Perch 2.0 utilizing a few-shot linear probe on marine duties, resembling distinguishing totally different baleen whale species or totally different killer whale subpopulations. Its efficiency was in contrast towards pre-trained fashions which are supported in our Perch Hoplite repository for agile modeling and switch studying. They embrace Perch 2.0, Perch 1.0, SurfPerch, and the multispecies whale mannequin.
For underwater information analysis, we used three datasets: NOAA PIPAN, ReefSet, and DCLDE.
- NOAA PIPAN: An annotated subset of the NOAA NCEI Passive Acoustic Information Archive from the NOAA Pacific Islands Fisheries Science Heart recordings. It consists of labels utilized in our prior whale fashions in addition to new annotations for baleen species resembling frequent minke whale, humpback whale, sei whale, blue whale, fin whale, and Bryde’s whale.
- ReefSet: Developed for SurfPerch mannequin coaching, this dataset leverages information annotations from the Google Arts and Tradition challenge: Calling in Our Corals. It consists of a mixture of organic reef noises (croaks, crackles, growls), particular species/genera courses (e.g., damselfish, dolphins, and groupers), and anthropomorphic noise and wave courses.
- DCLDE: This dataset is evaluated utilizing three totally different label units:
- Species: For distinguishing between killer whales, humpbacks, abiotic sounds, and unknown underwater sounds (with some uncertainty in killer whale and humpbacks labels).
- Species Recognized Bio: For sure labels of killer whales and humpbacks.
- Ecotype: For distinguishing between killer whale subpopulations (ecotypes), together with Transient/Biggs, Northern Residents, Southern Residents, Southeastern Alaska killer whales, and offshore killer whales.
On this protocol, for a given goal dataset with labeled information, we compute embeddings from every of the candidate fashions. We then choose a hard and fast variety of examples per class (4, 8, 16, or 32), and practice a easy multi-class logistic regression mannequin on high of the embeddings. We use the ensuing classifier to compute the world below the receiver-operating attribute curve (AUC_ROC), the place values nearer to 1 point out a stronger capacity to tell apart between courses. This course of simulates utilizing a given pre-trained embedding mannequin to create a customized classifier from a small variety of labelled examples.
Our outcomes present that extra examples per class enhance efficiency throughout all of the fashions, besides on ReefSet information, the place efficiency is excessive even with solely 4 examples per class for all fashions, besides the multispecies whale mannequin. Notably, Perch 2.0 is constantly both the highest or second-best performing mannequin for every dataset and pattern measurement.
