Neural Network Particle Choir

Interactive application (2020)

The Neural Network Particle Choir is a research project dealing with how a neural network can be trained and used to create complex and realistic audio.

The DDSP: Differentiable Digital Signal Processing system by Jesse Engel,

Lamtharn (Hanoi) Hantrakul, Chenjie Gu, Adam Roberts was used first to train a model and later to perform a timber transfer from the model’s sonic qualities to external audio recordings in order to create new realistic choir voices.

In-depth explanation on the DDSP can be found here:

This project made use of the demo examples available in the project’s git repository at:

In the project’s last phase, selected audio files was employed through FMOD studio into Unity in order to create an immersive interactive application.

Here, the user - through a first person view - is able to explore and experience the choir as generative spatial audio within a moving particle system.

The particles represents the different choir voices from which audio will be emitted if two or more particles are connected with each other. A connection will only happen if the particles are within a certain distance. Which audio file that will be played is determined by the numbers of particles that it’s connected to.

interactive media - machine learning - sound design - programming - visual