Are you wearing a mask? Improving mask detection from speech using augmentation by cycle-consistent GANs

The task of detecting whether a person wears a face mask from speech is useful in modelling speech in forensic investigations, communication between surgeons or people protecting themselves against infectious diseases such as COVID-19. In this paper, we propose a novel data augmentation approach for mask detection from speech. Our approach is based on (i) training Generative Adversarial Networks (GANs) with cycle-consistency loss to translate unpaired utterances between two classes (with mask and without mask), and on (ii) generating new training utterances using the cycle-consistent GANs, assigning opposite labels to each translated utterance. Original and translated utterances are converted into spectrograms which are provided as input to a set of ResNet neural networks with various depths. The networks are combined into an ensemble through a Support Vector Machines (SVM) classifier. With this system, we participated in the Mask Sub-Challenge (MSC) of the INTERSPEECH 2020 Computational Paralinguistics Challenge, surpassing the baseline proposed by the organizers by 2.8%. Our data augmentation technique provided a performance boost of 0.9% on the private test set. Furthermore, we show that our data augmentation approach yields better results than other baseline and state-of-the-art augmentation methods.

@inproceedings{ristea2020wearing,
  title = {Are you wearing a mask? Improving mask detection from speech using augmentation by cycle-consistent GANs},
  author = {Nicolae-Cătălin Ristea and Radu Tudor Ionescu},
  year = {2020},
  booktitle = {Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH)},
  eprint = {2006.10147},
  archivePrefix = {arXiv},
}