Dynamic music mechanism

Hi!

I've been thinking some time ago about creating a dynamic music system to make game music more immersive and to make it respond to what is happening on the screen.

For those who don't know - dynamic music makes the game not only play tracks fro m start to end, but also "recompose" the music live depending on what is happening in the game. If you're silent - the music is silent too, when you're raging - it keeps up with you.

Now, the idea I had was based on a grid of patterns:

  dynamic music 01.png (Size: 7.85 KB / Downloads: 145)

The A, B, C, D are parts of the song like intro, verse, chorus, bridge, solo.
The 1, 2, 3, 4 are versions ("V" for short) of different tension and loudness (1 is the most calm, and 4 is the most hectic version.

The song is programmed using the letters. For example a sequence: "ABABCBCDCD".
Game knows what tempo the track is in, and where in the beat grid all the parts are located (A1, A2, A3, A4, B1, B2, B3 etc...) so it can play any part anytime.
Now the game needs to perform some tests like:
- how many enemies are chasing the player?
- how much health and armor he has?
- is he's about to steal a flag?
- is he about to return a flag (score)?
- is he going to win the match for his team?
- is he a looser?
Based on the result, the game calculates a number for "tension" ("T" for short).
T should be in range between 0 and 1 (a fraction).
The T is being mapped for music versions (if the track has 4 versions we map the T factor from 1 to 4).
Now the game can switch between the parts hard way, once the T*V gets above certain level, the game waits until current top part ends and drops in another version of next part on the playlist.

Or it can mix the parts together, to make the transisions smoother performing cross-fades, but I haven't figured that out yet, and it will be harded to make.

The audio system should have two "decks" - and use it just like a DJ.
At the beginning it drops the first part in the current V (probably 1 at the beginning)on the first deck, and when the first part ends, it will start next one on the second deck, letting the first one end with a natural reverb/delay tails etc.
This way we have a fluent, natural sound with no strange cuts, and no seams.

I've got a simple audio file with A1, A2, A3, B1, B2, B3 parts:
http://unfamusic.com/5h1+/unfa-IME.ogg
The tempo is 110 BPM, meter is 4/4.
Every part lasts 8 bars:
first 4 bars are actual music and second 4 bars are the fadeout, that needs to be mixed for natural sound, but the next part needs to be played every 4 bars.
The song structure is simply "AABB".

I've tried to implement this system with Python, but I coudn't find out the way to play audio :/

Is there anybody who'd like to code a simple version of this algorithm using my composition?

PS: This is my 1,000th post! Yay!

ok to tell the truth i didn't read your post, just the PS,

but HEY, 1000 POSTS, nice man!

If dynamic music is implemented in the game, then it needs to be done VERY carefully as other games that have tried it (UT3 for example) have botched it up pretty badly.

Thanks, kuniu!

If dynamic music is implemented in the game, then it needs to be done VERY carefully as other games that have tried it (UT3 for example) have botched it up pretty badly.

I didn't hear about problems with dynamic music in UT3! (EDIT: until now):
http://forums.epicgames.com/threads/5865...amic-music
They seem to complain about music shutting off in inaproperiate moments. I think this can be tuned and the Xonotic's music could be "slightly less dynamic" than that.

I found another (slightly different) apprach on implementing such a feature:
http://gamedev.tutsplus.com/tutorials/im...for-games/
Here they made a set of loops that are meant to be playing together.
This is a nice option, becasue it gives more freedom, but should be also combined with song parts. The example delivered plays a minimalistic track that would shortly get boring I guess.

Anyone played Total Overdose?

If some of the ideas shown in that second link were implemented into Xonotic, then that may add another addicting element to the game.

Hmm, I dunno. This would fit for a game that has varying moods throughout the game, like very silent, cautious sneaky movements and heavy constant fire action. I feel that in Xonotic gameplay it is constantly an overdose of speed and action - which would make a mood dependent sound system redundant. When would you play chilling music or sounds that build up tension? Whenever I play Xonotic my player acts like a hyperactive nuclear rackjabbit on speed, and this throughout the entire game (Though I myself experience some kind of almost meditative concentration while doing so, but that's another story).

Hmm, I dunno. This would fit for a game that has varying moods throughout the game, like very silent, cautious sneaky movements and heavy constant fire action. I feel that in Xonotic gameplay it is constantly an overdose of speed and action - which would make a mood dependent sound system redundant. When would you play chilling music or sounds that build up tension? Whenever I play Xonotic my player acts like a hyperactive nuclear rackjabbit on speed, and this throughout the entire game (Though I myself experience some kind of almost meditative concentration while doing so, but that's another story).

The music wouldn't necassarily ever be silent or calm
But we could make it react to player's actions in a different way.
How about special parts or instruments in the music being played only under some tight conditions? Music rewards you for achievements?

Think of this in a different way:

- getting enemy's flag (extra lead part in harmony to the main one)
- being stacked (different, fuller bassline)
- under attack (drums get harder, other instruments back off a bit)
- killing spree (different drumbeat)
- nearly dead (pads come in, the rest is LP-filtered until death or rising health)
- fragged a teammate (alternative lead part sounding like an alarm)

I can think of music as a way of boosting player's experience, we could use some extra DSP options like realtime filtering or attenuating different parts (layers) of the music.

And the music doesn't ever need to be silent or calm! I think that it could be crazy cool if well made.

If there's a coder, I can prepare music for this system - so we could work on a working example.

That would be an interesting experiment, I admit - though it would require to develop Xonotic into some sort of music mixing tool similar to a sequencer (playing different patterns in varying domination depending on what happens). While it would be an interesting experiment, I am personally uncertain as to whether it would actually succeed in enhancing the gameplay experience... but if there are volounteer coders that would like to implement such a system it would surely be interesting.

If I remember correctly Tribes Ascend does have a somewhat dynamic music system responding to events like picking up the flag and such. And I must say, it is thrilling to grab a flag and hearing the music pump it up to 11 while being chased by the opponents.

That would be an interesting experiment, I admit - though it would require to develop Xonotic into some sort of music mixing tool similar to a sequencer (playing different patterns in varying domination depending on what happens). While it would be an interesting experiment, I am personally uncertain as to whether it would actually succeed in enhancing the gameplay experience... but if there are volounteer coders that would like to implement such a system it would surely be interesting.

Agreed, if anyone was up to the task, it would be very interesting to experiment with.

I can think of two or three alternative finishes of the track.
When the game ends and players watch the scoreboard, the music can play a final part, which could be different in any of the following cases:
- you've won (glory!)
- you've lost (shame!)
- draw game (sunspension)

I'd love to compose such a track (if we succeed with one track, then we may make more).
There could be a experimental server set up to tests this feature and gain feedback from players, if the feature is implemented.

We need a coder!

I've put together a (poorly) working example using bash and sox that will work using my Ogg file.
It has some problem with calculating times, as I get some strange overlapping, but I didn't find what's wrong yet, anyway it cuts up the file and plays different parts letting each one fadeout naturally.
Alsa is responsible for mixing the sounds together.

Code:

#/bin/bash

# constants
tempo=110 # BPM / tempo of the track
meter=4 # quarternotes / meter of the track
main=4 # bars / main part of each block
fade=4 # bars / fadeout part of each block
file="IME 02.ogg" # file used for playback
#blocks of the track: A1,A2,A3,B1,B2,B3
#indexes of above:    0  1  2  3  4  5

#variables
bar=$(echo "60/($tempo/$meter)" | bc -l) # seconds one bar lasts for given tempo and meter

# function that play selected block with index starting from 0
playblock() {
    ## calculate trim and sleep times
    # trim start time
    start=$(echo "$1*($bar*($main+$fade))" | bc -l)
    # trim end time    
    end=$(echo "($1+1)*($bar*($main+$fade))" | bc -l)
    # sleep time
    wait=$(echo "$bar*$main" | bc -l)
    #play the sound    
    play "$file" trim $start, $end &
    #halt the program until if finishes main part    
    sleep $wait
}

# now let's play a simple sequence
playblock 0
playblock 3
playblock 1
playblock 4

if you want to test it:
- make sure you're running a linux system with bahs shell availabile (Ubuntu will do)
- make sure you have sox installed (it's the audio tool used for playback)

Code:

sudo apt-get install sox -y

should do
- make sure you save this code as a textfile (name it "play.sh") in a folder with "IME 02.ogg"
- change the file's execution permissions with

Code:

chmod +x play.sh

command
- run it with

Code:

./play.sh

command

Get such error :

Code:

play FAIL trim: usage: {[=|-]position}

Get such error :

Code:

play FAIL trim: usage: {[=|-]position}

Maybe you have a different version of sox installed?

Code:

unfa@unfa:~$ sox --version
sox: SoX v14.3.2