Supplementary Materials
This page collects the supplementary material accompanying the paper: a video of the prototype instrument in performance, the trained model and patches needed to reproduce it, and the audio stimuli and example responses from the user experience study.
Accelerometer and gyroscope data from a pair of Nintendo Switch controllers is mapped to the dimensions of the RAVE decoder's latent space, which is run in real time in nn~.
40 s · MP4 · 98 MB — download
Trained with stock RAVE v2 in causal mode
(rave train --config v2 --config causal) at 44.1 kHz on a 326-hour subset of the
Cornell Bird Challenge 2020 dataset. No further config overlays — no noise, snake, adain
or augmentations. Taken at epoch 373 and exported streaming with a truncated latent of 16
dimensions (--fidelity 0.99).
The prototype device, which loads the model in nn~ and maps incoming OSC control data to its latent dimensions.
m4l-instrument-prototype.amxdReads motion data from the Nintendo Switch controllers and forwards it over OSC to the Max for Live device.
github.com/leoauri/switch2oscThe six example sounds played to participants, who were instructed to make the best imitation or interpretation of each that they could with the instrument.
Participants were given the nine factors of music-induced emotion from the Geneva Emotional Music Scale in randomised order, and asked to make a short recording that would evoke the named emotion in a listener. Three example responses are given below.