Abstract
Automatic Music Transcription (AMT) is an important task for music information retrieval. Analogous to Automatic Speech Recognition (ASR) in speech domains, where the speech inputs (audio) are converted into text output, AMT aims at converting music audio into symbolic representations such as sheet music or midi files. These symbolic representations have a variety of applications such as music analysis, music indexing, and music generation. Existing AMT models use mostly mel spectrograms and focus only on piano music using supervised learning. In this thesis, we first studied different input features such as short-time Fourier transform (STFT), constant-Q transform (CQT), and mel spectrogram for AMT. After finding the best input feature for AMT, we explored different training schemes such as semi-supervised and unsupervised learning. These training schemes are useful when labeled data is limited, which is a common situation for AMT datasets. We proposed ReconVAT as our semi-supervised AMT framework. This framework uses virtual adversarial training (VAT) as the training method, which can utilize both labeled and unlabeled data for training. ReconVAT maintains a competitive transcription accuracy even when the amount of labels is limited. We also explored the use of classifier-free diffusion guidance as a novel generative AMT model (DiffRoll). Unlike traditional discriminative AMT, DiffRoll is capable of both music transcription and music generation with unsupervised pre-training capability. Finally, we extend single instrument AMT into multi-instrument AMT using a model called Jointist, which outperforms the state-of-the-art AMT, MT3, developed by Google Magenta.