Sound Arts Year 2

Just quickly updating my blog from a song I made back a few weeks before we received any final information about the game. At the time the music references they gave us were Cuphead which had a lot of Jazz Bepop style of music.

I originally attempted to use Ableton to create some music but felt it was corny. Instead of using midi instruments, I translated to what I know how to use which is sampling. I used my SP404MK2 which is a sampler with lots of effects built within.

I then loaded some jazz drum brush loops and got a pattern going. Went through some jazz records I own at home and recorded through my turntable into my sampler and started chopping guitar sounds to layer over the drums. Then processed with filters and other bits to create a new analogue dirty hissing sounding loop.

Again now it won’t be used as we have gone with a sci-fi futuristic sort of composition which I now will be making music for.

For reference here is the machine I use.

And here is the song below

I’m continuing my research into games and chose Minecraft as the last one. I’ve spent hundreds of hours in this game and find the use of music and sound effects have been executed very well. The music comes on and off without ques. As well as the sound effects being really unique and fitting for the game world.

The music as shown above has many differences in it, from tracks that have heavy reverb to showcasing themes of wonder and exploration. To more sad and horrible music in the netherworld (Minecraft hell). I want to explore more into how they thought to use the music like this. As well as potentially what synths they used? Or instruments?

As well as the sound effects too, they have great variation as shown in this video. They are not high quality but distinctive. I think the music and sounds could fit into our game so I’m curious how they made them. Are they samples like other games, or synthesis? Or a combination.

So after watching the above video I understand the concepts of sound design in Minecraft a lot more.

Music should come in randomly, adding music to a film as they say is easier. You know exactly when the audio is going to play and what is happening. But in a game, it is more difficult to predict what’s happening. The score plays randomly, with lots of silence in between each song to build anticipation.

Minecraft is very laidback with its music as you are usually doing something very relaxing. In the specific biome warped forest in the nether, music doesn’t even play. Just the ambience of glitchy electronic sounds distorting. Some of the nether sounds were made using balloons to create these stretching sounds of horrible tension.

Minecraft at night shifts into monsters trying to kill you instead of a peaceful building game. Dark tunnels field with monsters is also a common occurrence. They make the enemies loud so in dark environments you can hear the enemies approaching even before you see them. This gives the player cues, they also play scary music when you are close to darkness this is to just scare you.

They also speak about not creating any dialogue for the character you play, as they want you to feel like it’s you playing not a character they’ve created. Villagers don’t speak either as they don’t want to make people feel offended.

Overall I think it’s interesting the anticipation thing they speak about. As well as the no music for anticipation. I did discuss with Ingrid that an idea I had was about music not being on a constant loop but using FMOD to have multiple tracks stacked up on the randomiser function so it chooses a random song based on the selection with a few silent tracks as well to have a pause sometimes between songs. Something I definitely want to implement if possible.

Another game I want to research is Halo Infinite. I remember when it was coming out all the trailers for it. They had a lot of videos explaining the sound design. At the time we were doing the specialising module and I was working on foley and sound effects for a short film. The video showed them doing very unique things to record sound to manipulate into monsters and weapons for the game. I want to do further research into it and perhaps use what I’ve learnt for the current game I’m working on.

Firstly I watched this video that went through a few sound design field recordings they made. It started with rockets being fired outside, to then using electromagnetic microphones from LOM on an Xbox console and recording those sounds. Recording a pug eating for monster sounds, hydrophones on glass, and making the glass break. Recording old machinery bears in a zoo, the manufacturing of the Xbox remotes.

In this video they use a piano for sound effects, starting by placing a huge bass speaker on the top and recording it. Then breaking the piano, hammering it. Tightening the strings until they snap, cutting the strings as well. They even place dry ice on the piano strings, you can hear the dry ice fizzle like a screaming alien. Really cool stuff!

I also watched this interview with the sound design team at Halo Infinite.

They start by speaking about the guns, one of the most important parts of the game, since it’s first-person it’s the main character. The sound designers speak about hey they design the weapons in layers. Firstly the thud indicates the power and impact of a weapon. Secondly, the mechanical layer which offers tactile feedback with satisfying click clanks of metal. Thirdly, the tail end creates an atmosphere with the gun bouncing of the environment this gives it a sense of where the gun is fired, is it an interior or outside space.

Adaptive sound design is something they use in this game, they tailor sounds to any outcome. This gives feedback on their actions to the player, for example, the mechanical sounds of a weapon get louder as the gun begins to run out of ammo. And sounds of an empty clip are a typewriter and explain communication to the player they have nothing left to shoot.

Car sound design was grounded on real-world vehicles, they recorded real helicopters and dune buggies, as well as a 1918 vintage tractor as they didn’t want a futuristic tank to sound like a current one. 

“Making appropriate sounds from inappropriate objects.”

They also spoke on sound fatigue, something I hadn’t really considered. Because unlike a film players will be putting hundreds of hours into it, they need the sounds to not have too much high frequency or low frequency so the mix doesn’t become muddy.

Following this I want to record more outside and spend a full day attempting sound effects for our current game, there is so much room for cool ambient sounds and technology or other sounds. I want to take this further than it already is.

I’ve decided to do further research into other games I like to understand a bit more behind the process of sound/music creation. To expand my knowledge and learn more techniques and ways of working. As well as potentially attempt them for my game.

Old School Runescape is a 2007 version of Runescape. The game is something I played in 2007 and to see them re-release the same version is amazing. The music is very nostalgic and uses old school midi instruments and sound effects sound like synths. I want to do further research just to find out how it was made. For reference here is an example of the music in the game.

I find the music really beautiful and captivating. Maybe it’s the nostalgia but I find the compositions really fitting for the game. And I want to learn what they used. Especially for the sound effects, I think they almost could fit in the game we are working on. For the machine sounds or ambient sounds, they sound a bit crushed or low bit rate. See below for examples.

I searched online but only seemed to find research about the current Runescape version, not the older version. Or any information was mainly just stating it was midi. I found a Reddit post that Ian Taylor replied to explaining how he made the audio.

I also managed to find a tweet where Ian shows the original synthesizer they created to make all the Runescape sound effects.

I also found another Reddit post that explains the Runescape 2 sound effects were mainly additive synthesis using this above synthesis engine they created for their making their own personal sounds.

I want to attempt to explore additive synthesis and see what I can create. Explore the sound design elements and create some sound effects of machinery for the child being created and the ambience of the factor in the game.

I’ve been trying to use Unity to record a full playthrough of the game for us to use as a video for backup in case adding things through Unity doesn’t work. I’ve had numerous issues and every time I figure out the problem another occurs.

The first was the game not even running or opening. I figured out I was using the wrong editor version and had to use the same one that the MA developers used while creating it. The editor version was 2019.4.11f1

Then when I opened the file I managed to get it running by clicking the play button. Now, whenever I open the game it seems like the image is out of place and not synched up correctly. As well as this, the password for the game doesn’t progress to the next screen which is frustrating. I can also see in the debug part on the bottom left of the screen it says “correct” meaning I’ve got the password correct and it should progress. I’m going to speak to Jingya and find out if she has managed to get it working and if we can record a playthrough for video use.

Otherworld VR was truly spectacular. I have used VR before at a friend’s place and I found it good but it was truly behind what the ideas were. I felt like space was a big thing it needed. At Otherworld VR they had this. I found the fact I could walk around a space really beneficial for the experience.

The beginning section where we all went down an elevator and the portal opened up was very immersive. I think the sound design for that really helped give the idea of falling into an abyss. I also noticed the directionality in the space, with sounds moving around me and where I looked the sound would be in this bubble. As if it’s ambisonic.

I spent most of my time playing a zombie game and found it enjoyable, although the movement was weird, you couldn’t walk you had to hold a button and teleport into an area. Still fun though. All the reading I have done about immersion felt to replicate itself in my experience. I found myself truly immersed and totally forgot about where I was, in this circular pod. I ended up finding the overall experience really worth it. Having that experience has definitely helped me understand the scope of current VR technology and where it can go and where it is.

Currently, I am almost at the hand in. I used FMOD in some experiments and I get why it’s useful. I want to attempt to use Unity for sound editing perhaps if it works. And use FMOD for music.

I need to make Another music track or two perhaps? And record and make sound effects for the atmosphere and the robot’s brain being built.

We have been sent a trailer of the game but not an entire run-through, we have received the actual unity project but I can’t get the game playing? I need to discuss this with my peers in my group to see what we can do.

Finally, in the mixing process, are we going to make a video and edit the sounds on Final Cut Pro or are we going to use a video of the game or implement them?

I feel like currently, I will make the extra song, and the sound effects and use Final Cut Pro to edit the sound to video as it might be difficult to actually implement the new sounds. We can still keep the old sounds they used and just layer in the new ones. I also don’t mind mixing and mastering the audio.

here people worked on the audio, one audio director, one composer and one designer. A small team in comparison to other studios.

They speak on how they wanted the sound design to be a storyteller instead of text and that they read up each section of the world the player interacts with. And made sounds that would exist in those sections, how can we indicate information through the sound they say.

They also spoke on the idea of ambient sounds within the world, they wanted the players to feel immersed in this world instead of focusing on the small character running around. They created a fake fizzle sound when touching a cloud in the game to reflect the dreams we all have to do in real life. I think that’s interesting because in real life clouds are a vapour, they make no noise or I assume so.

They recorded foley outside for the footsteps, water streams and rainfall. It is cool to see a studio use field recordings in their practice! As well as this they also had a studio where the majority of the foley was recorded.

It was interesting to note that they said the music is created before a lot of things in the game, the story and concepts are made around music in this instance which I haven’t heard happen before, as well as music changing around actions in the game, flying, running etc. 

I also found it interesting that they created a serrate mix for each system, with separate EQs and reverbs.

It speaks on that using VR for journalism is a difficult thing to do ethically of course and that the producers of it must take the ethical issues seriously,

“With virtual reality, rather than telling a story, you are putting someone inside a story – and usually involving them in it,” 

I think this makes a great point about the difference, instead of showing it you are making someone experience it and immerse them within the experience.

There have been a few projects in VR published so far containing sensitive issues, the Guardian’s first VR project, 6×9, let viewers experience solitary confinement, Within’s Clouds Over Sidra followed the life of a 12-year-old refugee, and the BBC’s Easter Rising: Voice of a Rebel took viewers back to the streets of Dublin to witness the 1916 rising that saw the attempted rebellion against British rule in the midst of World War One.

The article then goes on to note that you don’t know in what context the participant is using these headsets, are they suffering from PTSD. Do they know exactly what they are Going to see? Have they been involved in this before? Because it’s a completely different medium that involved simulation rather than representation you have to be careful when engaging with this content.

The article gives us five tips.

1. Make a risk analysis, and understand the risks of your content. Who might it affect, how many you reduce this, and what can the outcomes be?
2. Test your material as you go, really test the experience as people will have never experienced your content before, and have test subjects use it before release.
3. Co-create with your audience, She explains it’s similar to a theme-park roller coaster, before you start the ride you are given rules and safety advice and that VR experiences should be the same.
4. Get active feedback from your audience, you want active consumers and advocate for them to give you feedback, with that you can grow your content and understand more about how it affects users.
5. Diversify your teams, having a diverse team will allow you to have integrity. Diversity could be gender, age, ethnicity, or experience level. 

Rachel Simpson is a game designer, composer and trumpet player. She began by showing us a showreel of multiple games.

She’s been working in games for 16 years, she wasn’t a gamer and how she ended up working in it was very unorthodox. She’s been working 10 years in house for companies and 6 years as a freelance artist. She’s had a different time working freelance from the beginning to how she works now. She released 32 games shipped, Singstar, guitar hero, and she’s worked on countless other games. 

So she’s worked in all manner of team sizes, tiny indie studios from 4-5 people to massive teams, as well as over one hundred people in a room when working on Guitar Hero 5. She’s worked in multiple genres, but mainly more fun relaxed experiences than dark first-person shooters, she’s worked on word games, driving games, f2p games, and AR.

She’s really interested in UI, and other aspects of it such as how to make a button seem real and have tactile feedback. She’s done composition for music as nowadays smaller studios want someone that can do everything. She’s also done narration recording and casting.

Foley as well is something she really enjoys, finding things that make cool mad noises to use for sound effects. She’s also done a lot of implementation. The interesting thing about implementation is that it’s technical, she’s not been a technical person by nature but she manages to make it work.

She’s a musician and always knew she wanted to do something with music or sound. She studied jazz and then returned back home to Ireland. Did a course in classical music and transferred back to goldsmiths in London. She applied for a job being a singer/keyboardist for Singstar. Transcribing music for the game. She worked her way up after 6 months and became a lead transcriber for the game. She worked for around two years and felt she had to go back and finish her degree.

Kuju/ Zoe mode was the first studio she worked at and was surrounded by like-minded people like herself which made it hard to leave. She worked for a while on creating midi notes on screen for the pitch detection system on Singstar.

She went on to work as a producer and she was in the centre of the team, controlling time management and other things, now she is freelance and working across other projects she learnt a lot of that. But she says she would never do that again and wanted to go back into being an audio designer.

After university, she went back to KUJU/ZOE MODE and was thrown straight into the deep end doing sound effects. She hadn’t actually done sound effects before and found herself learning and asking questions.

Unfortunately, the studio had to downsize which is something that happens in the industry and she found a job in London working on Facebook games on Sims which something she loved to do. She had access to the sims language and would edit the sounds to create sentences. She did feel it was a darker stage f her career as she was working on a free to play game and it was all about exploiting money from users. 

She says it was also a more diverse studio, and it did reflect in the game and its design as well. From an audio perspective, it was a live game. She comes from a console stage, where you have a release date and you ship and it’s done. Live games are constantly evolving, and every two weeks they release new content for the players. So they had a week to make audio, and a week to test. So it was high pressure. The studio finally after two years got shut down by EA and her manager thought it would be smart to ask to purchase the music gear as she was sure the equipment would be thrown away. After purchasing the equipment for cheap, she then had the gear to go freelance.

She found herself really enjoying being a freelance sound designer. And learning that being alone means you get the freedom to purchase the sound tools that she needs without waiting for the studio to do it.

She worked on a game called CLAY JAM and another called PEAK. She found herself taking any job she could get because she didn’t have the connections or experience yet to find good work. She moved to Scotland and found a job going on in her city. She ended up working on a free to play mobile game studio. She got the job and went on to become a lead sound designer and lead teams on games. 

She didn’t totally love the games at Outplay, she instead enjoyed the stability of working in a studio, and she didn’t have a studio she was working in an office with headphones. She wanted to get back into freelance work and luckily she got contacted by an old friend for an AR game for Lego. Working on lego was interesting for her, she had to learn Repear alongside two other sound designers. She then left and became a freelancer again and found herself being emailed all the time for work. Her advice is that as you get old you get jobs with more connections.

She then eventually went on to do a project with sound installations and after this, she was contacted by a game studio to be a sound designer for a game. For people with autism in VR.

She taught herself FMOD as this is what they’re using, she’s studied VR work and how to use audio in this context. That everything should come from a source or it defeats the immersion. 

She gives us a tutorial on her FMOD usage which was beneficial. She also says you need to have inspiration beyond video games, and find your friends. Get noisy and chat with people and find really nice people. As well as online groups that can help you figure out what the problem is.

Post Lecture Reflection

I found Rachel to be really friendly in her presentation and helpful in showing how one can get into the Games industry. This is something I’ve considered due to my love for sound design and sound effects. She also was very giving with explaining the exact way things happened. Her experience of not knowing a lot and learning as she goes does also give me confidence. I find I know enough about audio but perhaps implementation and other aspects scare me a little. It’s confusing but knowing there are ways to learn, like recently teaching myself FMOD it does seem doable. I also found her describing the differences between bigger audio teams and smaller indie teams and being the only audio designer interesting and the dynamics between them. Benefits and negatives of the situation.