Neural Network Particle Choir
Interactive application (2020)
The Neural Network Particle Choir is a research project dealing with how a neural network can be trained and used to create complex and realistic audio.
The DDSP: Differentiable Digital Signal Processing system by Jesse Engel,
Lamtharn (Hanoi) Hantrakul, Chenjie Gu, Adam Roberts was used first to train a model and later to perform a timber transfer from the model’s sonic qualities to external audio recordings in order to create new realistic choir voices.
In-depth explanation on the DDSP can be found here:
This project made use of the demo examples available in the project’s git repository at:
In the project’s last phase, selected audio files was employed through FMOD studio into Unity in order to create an immersive interactive application.
Here, the user - through a first person view - is able to explore and experience the choir as generative spatial audio within a moving particle system.
The particles represents the different choir voices from which audio will be emitted if two or more particles are connected with each other. A connection will only happen if the particles are within a certain distance. Which audio file that will be played is determined by the numbers of particles that it’s connected to.