Music Synthesis from MIDI with RNN/LSTM

This paper gives an overview on music synthesis based on recurrent neural networks (RNNs), especially Long-Short-Term-Memory based RNNs, that will be trained with data extracted from MIDI files and can generate MIDI data on the output side.

This student project explores the use of machine learning in the generation of music. Neural networks will be used to learn from real composed music. These same networks can be used to write their own music after a period of training. The project was a group effort of four students that worked together to discuss and implement the contents of this report. This report discusses the way the project was assembled, from the tools that were used to the way data is converted from an industry standard to a more training friendly format. The different designs and iterations of the network itself will be described. Finally, the results and conclusion that stem from these experiments will be presented.

The objective of our project is to generate music in the form of MIDI files using neural networks. To achieve this objective, we want to train a Recurrent Neural Network (RNN) with large quantities of MIDI files. The result should be a playable MIDI file that is pleasant to listen to by a human and that should reflect the style of music that was fed into the network. Directly using notes in a file instead of transcribing the music from a sound file helps training the network for correct dependencies on music notes. If successfully trained with LSTMs, checking other RNN variants (GRUs or bidirectional LSTMs) and comparing their performance to LSTM-based neural networks might be additional work in this project. We might also consider neural network types that were not tested for music generation in other research, like multiplicative LSTM. The project also includes creating a robust way to convert MIDI data to a format that can be ingested by the neural network, as well as the conversion back to MIDI from the predictions the neural network creates.

pkalytta

Philipp Kalytta studiert Technische Informatik an der TH Köln und beschäftigt sich auch als Hobby mit IT-Themen