Development Update – "The Wingnuts"
Hello! I’m Jon from Runner Duck, and I’m writing the Development Update this time.
The Citadel is the main base of the player’s faction, a sanctuary, relatively safe from the dangers of the Badlands around it. The player will spend a fair amount of time in the Citadel, equipping their crew and building their vehicle, so it was important to us to make it a vibrant, lively and interesting place. Missions out in the Badlands can be brutal and tense, so allowing the player to return to an oasis of relative comfort and humanity between missions is important for balance. With these things in mind, we came up with the idea of the Citadel Band, “The Wingnuts”.
[previewyoutube][/previewyoutube]
Unlike the rock soundtrack of the missions, The Wingnuts play cheerful folk and skiffle influenced music. I reached out to a musician I knew in Brighton, who performs with a number of bands in a range of styles, who I thought would be perfect for this soundtrack. He wrote the songs, and put together a small group to perform and record them. The songs were so good that we soon realised we would have to step up our presentation of the band within the game in order to do them justice!
The real life musicians behind “The Wingnuts”
I have always found that when there are musicians animated in a game (or cartoon), it really adds to the sense of immersion if their animation closely matches the music they are actually playing. We were using a MIDI data import system to drive the drumming animation of crewmates in War Drums stations, so I extended this system to allow us to animate The Wingnuts.
[h2]MIDI DATA TRACKS[/h2]
All of the music for The Wingnuts was performed live, so it did not have any pre-existing MIDI data to work from. - however, it was recorded along to a click track, so it is not too tricky for me to create ‘silent MIDI tracks’ which mimic what is being played by the actual musicians. For this, I use Ableton Live:
‘Silent’ MIDI tracks matched to live performance audio in Ableton Live
One MIDI track is created per instrument. The guitar MIDI track is the most complex, as it has both the pitch of the note (which affects the fretboard hand’s position) and has midi notes which trigger different animations for picking up, picking down, big strums up, and big strums down. There is an additional MIDI track for voice and harmonica, which animates the band’s mouths. I create these tracks by ear, listening to the song and adding notes as appropriate. Authoring a full set of silent MIDI data for a track generally takes me around an hour.
Once finished, I save out the .mid files, and our code in Unity converts them into a list of animation events.
[h2]CHARACTER ANIMATION[/h2]
Characters were first given base ‘groove’ animation loops, making them nod and bob along with the music - we take the beats per minute of the current track and use this to modify the speed of this animation so they all stay in time with the music.
To make the characters appear to be playing the correct pitch on their instruments, we created a set of animations in which their ‘fretting’ hands were placed at a certain position to play a certain pitch; we took these sets of pitch animations and assigned them to a Blend Tree; a feature of game engine animation systems which allows us to blend their motion together using a simple value input from the pitch tracks of the MIDI data.
We then layer animations for the instrument playing/plucking hands on top of this, along with some incidental variations of the base ‘groove’ animations and there we have it - fully animated musicians!
Adjusting the bass player’s ‘pitch’ blend tree live in the editor
The animation state machine for the guitarist’s strumming arm
[h2]FINISHING TOUCHES...[/h2]
There are some other elements, outside of the animation which we had to consider. The band is placed on stage in the Citadel, so to sound natural, the music should sound as if it is coming from there. However, it can quickly become annoying if the music is only coming from one side while you are working on your vehicle (especially with headphones)! To address this, we use a dynamic spatial blend. When the camera moves or changes, we begin to blend towards the music being spatialised (so the position and direction of the band is taken into account). After a short while of the camera being in a new location, we blend away from the music being spatialised, to just being a standard stereo stream. We also combine this with some reverb and filtering, so that when the camera is far from the band, the music is less clear and contains the natural reverb you would expect from the cavernous space between the walls of the Citadel.
We’ve been listening to The Wingnuts play on in the background of the Citadel as we work developing the game; despite hearing the same set of tracks for hundreds if not thousands of hours, we haven’t got sick of it at all - in fact we find ourselves whistling the tunes when going about our lives outside of work! Testament to what an incredible job the real musicians did when writing and recording the music!
A soothing bedtime ballad
Thanks for reading this far. I hope you enjoyed this small insight into our game!
Jon
The Citadel is the main base of the player’s faction, a sanctuary, relatively safe from the dangers of the Badlands around it. The player will spend a fair amount of time in the Citadel, equipping their crew and building their vehicle, so it was important to us to make it a vibrant, lively and interesting place. Missions out in the Badlands can be brutal and tense, so allowing the player to return to an oasis of relative comfort and humanity between missions is important for balance. With these things in mind, we came up with the idea of the Citadel Band, “The Wingnuts”.
[previewyoutube][/previewyoutube]
Unlike the rock soundtrack of the missions, The Wingnuts play cheerful folk and skiffle influenced music. I reached out to a musician I knew in Brighton, who performs with a number of bands in a range of styles, who I thought would be perfect for this soundtrack. He wrote the songs, and put together a small group to perform and record them. The songs were so good that we soon realised we would have to step up our presentation of the band within the game in order to do them justice!
The real life musicians behind “The Wingnuts”I have always found that when there are musicians animated in a game (or cartoon), it really adds to the sense of immersion if their animation closely matches the music they are actually playing. We were using a MIDI data import system to drive the drumming animation of crewmates in War Drums stations, so I extended this system to allow us to animate The Wingnuts.
[h2]MIDI DATA TRACKS[/h2]
All of the music for The Wingnuts was performed live, so it did not have any pre-existing MIDI data to work from. - however, it was recorded along to a click track, so it is not too tricky for me to create ‘silent MIDI tracks’ which mimic what is being played by the actual musicians. For this, I use Ableton Live:
‘Silent’ MIDI tracks matched to live performance audio in Ableton LiveOne MIDI track is created per instrument. The guitar MIDI track is the most complex, as it has both the pitch of the note (which affects the fretboard hand’s position) and has midi notes which trigger different animations for picking up, picking down, big strums up, and big strums down. There is an additional MIDI track for voice and harmonica, which animates the band’s mouths. I create these tracks by ear, listening to the song and adding notes as appropriate. Authoring a full set of silent MIDI data for a track generally takes me around an hour.
Once finished, I save out the .mid files, and our code in Unity converts them into a list of animation events.
[h2]CHARACTER ANIMATION[/h2]
Characters were first given base ‘groove’ animation loops, making them nod and bob along with the music - we take the beats per minute of the current track and use this to modify the speed of this animation so they all stay in time with the music.
To make the characters appear to be playing the correct pitch on their instruments, we created a set of animations in which their ‘fretting’ hands were placed at a certain position to play a certain pitch; we took these sets of pitch animations and assigned them to a Blend Tree; a feature of game engine animation systems which allows us to blend their motion together using a simple value input from the pitch tracks of the MIDI data.
We then layer animations for the instrument playing/plucking hands on top of this, along with some incidental variations of the base ‘groove’ animations and there we have it - fully animated musicians!
Adjusting the bass player’s ‘pitch’ blend tree live in the editorThe animation state machine for the guitarist’s strumming arm[h2]FINISHING TOUCHES...[/h2]
There are some other elements, outside of the animation which we had to consider. The band is placed on stage in the Citadel, so to sound natural, the music should sound as if it is coming from there. However, it can quickly become annoying if the music is only coming from one side while you are working on your vehicle (especially with headphones)! To address this, we use a dynamic spatial blend. When the camera moves or changes, we begin to blend towards the music being spatialised (so the position and direction of the band is taken into account). After a short while of the camera being in a new location, we blend away from the music being spatialised, to just being a standard stereo stream. We also combine this with some reverb and filtering, so that when the camera is far from the band, the music is less clear and contains the natural reverb you would expect from the cavernous space between the walls of the Citadel.
We’ve been listening to The Wingnuts play on in the background of the Citadel as we work developing the game; despite hearing the same set of tracks for hundreds if not thousands of hours, we haven’t got sick of it at all - in fact we find ourselves whistling the tunes when going about our lives outside of work! Testament to what an incredible job the real musicians did when writing and recording the music!
A soothing bedtime balladThanks for reading this far. I hope you enjoyed this small insight into our game!
Jon