Category: Collaboration

here people worked on the audio, one audio director, one composer and one designer. A small team in comparison to other studios.

They speak on how they wanted the sound design to be a storyteller instead of text and that they read up each section of the world the player interacts with. And made sounds that would exist in those sections, how can we indicate information through the sound they say.

They also spoke on the idea of ambient sounds within the world, they wanted the players to feel immersed in this world instead of focusing on the small character running around. They created a fake fizzle sound when touching a cloud in the game to reflect the dreams we all have to do in real life. I think that’s interesting because in real life clouds are a vapour, they make no noise or I assume so.

They recorded foley outside for the footsteps, water streams and rainfall. It is cool to see a studio use field recordings in their practice! As well as this they also had a studio where the majority of the foley was recorded.

It was interesting to note that they said the music is created before a lot of things in the game, the story and concepts are made around music in this instance which I haven’t heard happen before, as well as music changing around actions in the game, flying, running etc. 

I also found it interesting that they created a serrate mix for each system, with separate EQs and reverbs.

It speaks on that using VR for journalism is a difficult thing to do ethically of course and that the producers of it must take the ethical issues seriously,

“With virtual reality, rather than telling a story, you are putting someone inside a story – and usually involving them in it,” 

I think this makes a great point about the difference, instead of showing it you are making someone experience it and immerse them within the experience.

There have been a few projects in VR published so far containing sensitive issues, the Guardian’s first VR project, 6×9, let viewers experience solitary confinement, Within’s Clouds Over Sidra followed the life of a 12-year-old refugee, and the BBC’s Easter Rising: Voice of a Rebel took viewers back to the streets of Dublin to witness the 1916 rising that saw the attempted rebellion against British rule in the midst of World War One.

The article then goes on to note that you don’t know in what context the participant is using these headsets, are they suffering from PTSD. Do they know exactly what they are Going to see? Have they been involved in this before? Because it’s a completely different medium that involved simulation rather than representation you have to be careful when engaging with this content.

The article gives us five tips.

1. Make a risk analysis, and understand the risks of your content. Who might it affect, how many you reduce this, and what can the outcomes be?
2. Test your material as you go, really test the experience as people will have never experienced your content before, and have test subjects use it before release.
3. Co-create with your audience, She explains it’s similar to a theme-park roller coaster, before you start the ride you are given rules and safety advice and that VR experiences should be the same.
4. Get active feedback from your audience, you want active consumers and advocate for them to give you feedback, with that you can grow your content and understand more about how it affects users.
5. Diversify your teams, having a diverse team will allow you to have integrity. Diversity could be gender, age, ethnicity, or experience level. 

Rachel Simpson is a game designer, composer and trumpet player. She began by showing us a showreel of multiple games.

She’s been working in games for 16 years, she wasn’t a gamer and how she ended up working in it was very unorthodox. She’s been working 10 years in house for companies and 6 years as a freelance artist. She’s had a different time working freelance from the beginning to how she works now. She released 32 games shipped, Singstar, guitar hero, and she’s worked on countless other games. 

So she’s worked in all manner of team sizes, tiny indie studios from 4-5 people to massive teams, as well as over one hundred people in a room when working on Guitar Hero 5. She’s worked in multiple genres, but mainly more fun relaxed experiences than dark first-person shooters, she’s worked on word games, driving games, f2p games, and AR.

She’s really interested in UI, and other aspects of it such as how to make a button seem real and have tactile feedback. She’s done composition for music as nowadays smaller studios want someone that can do everything. She’s also done narration recording and casting.

Foley as well is something she really enjoys, finding things that make cool mad noises to use for sound effects. She’s also done a lot of implementation. The interesting thing about implementation is that it’s technical, she’s not been a technical person by nature but she manages to make it work.

She’s a musician and always knew she wanted to do something with music or sound. She studied jazz and then returned back home to Ireland. Did a course in classical music and transferred back to goldsmiths in London. She applied for a job being a singer/keyboardist for Singstar. Transcribing music for the game. She worked her way up after 6 months and became a lead transcriber for the game. She worked for around two years and felt she had to go back and finish her degree.

Kuju/ Zoe mode was the first studio she worked at and was surrounded by like-minded people like herself which made it hard to leave. She worked for a while on creating midi notes on screen for the pitch detection system on Singstar.

She went on to work as a producer and she was in the centre of the team, controlling time management and other things, now she is freelance and working across other projects she learnt a lot of that. But she says she would never do that again and wanted to go back into being an audio designer.

After university, she went back to KUJU/ZOE MODE and was thrown straight into the deep end doing sound effects. She hadn’t actually done sound effects before and found herself learning and asking questions.

Unfortunately, the studio had to downsize which is something that happens in the industry and she found a job in London working on Facebook games on Sims which something she loved to do. She had access to the sims language and would edit the sounds to create sentences. She did feel it was a darker stage f her career as she was working on a free to play game and it was all about exploiting money from users. 

She says it was also a more diverse studio, and it did reflect in the game and its design as well. From an audio perspective, it was a live game. She comes from a console stage, where you have a release date and you ship and it’s done. Live games are constantly evolving, and every two weeks they release new content for the players. So they had a week to make audio, and a week to test. So it was high pressure. The studio finally after two years got shut down by EA and her manager thought it would be smart to ask to purchase the music gear as she was sure the equipment would be thrown away. After purchasing the equipment for cheap, she then had the gear to go freelance.

She found herself really enjoying being a freelance sound designer. And learning that being alone means you get the freedom to purchase the sound tools that she needs without waiting for the studio to do it.

She worked on a game called CLAY JAM and another called PEAK. She found herself taking any job she could get because she didn’t have the connections or experience yet to find good work. She moved to Scotland and found a job going on in her city. She ended up working on a free to play mobile game studio. She got the job and went on to become a lead sound designer and lead teams on games. 

She didn’t totally love the games at Outplay, she instead enjoyed the stability of working in a studio, and she didn’t have a studio she was working in an office with headphones. She wanted to get back into freelance work and luckily she got contacted by an old friend for an AR game for Lego. Working on lego was interesting for her, she had to learn Repear alongside two other sound designers. She then left and became a freelancer again and found herself being emailed all the time for work. Her advice is that as you get old you get jobs with more connections.

She then eventually went on to do a project with sound installations and after this, she was contacted by a game studio to be a sound designer for a game. For people with autism in VR.

She taught herself FMOD as this is what they’re using, she’s studied VR work and how to use audio in this context. That everything should come from a source or it defeats the immersion. 

She gives us a tutorial on her FMOD usage which was beneficial. She also says you need to have inspiration beyond video games, and find your friends. Get noisy and chat with people and find really nice people. As well as online groups that can help you figure out what the problem is.

Post Lecture Reflection

I found Rachel to be really friendly in her presentation and helpful in showing how one can get into the Games industry. This is something I’ve considered due to my love for sound design and sound effects. She also was very giving with explaining the exact way things happened. Her experience of not knowing a lot and learning as she goes does also give me confidence. I find I know enough about audio but perhaps implementation and other aspects scare me a little. It’s confusing but knowing there are ways to learn, like recently teaching myself FMOD it does seem doable. I also found her describing the differences between bigger audio teams and smaller indie teams and being the only audio designer interesting and the dynamics between them. Benefits and negatives of the situation.

The article begins by speaking on music’s power to influence us, by changing moods and inducing feelings of nostalgia, fear, or excitement. Retailers have used music to drive shoppers to higher price points and casinos play music to encourage gambling. I wasn’t aware that these things actually operated like this, next time I go shopping I want to look out for this music and maybe understand what about the music makes you want to spend more money?

It also speaks on changing music’s role from being background music as it is far more important than just that, it has more uses than simply being the background for the experience. It can actively be part of the experience and help convey messages and themes, it’s important for this to be understood if we are going to figure out its place in VR content.

The challenges mainly are the creators, as the technology for this in games is already there. It’s the mentality that music isn’t that important which holds back creators to use it in ways to influence or enhance emotions in a situation. I think this is perhaps a waste of music in this context, to always use it in the same context.

I started by loading up FMOD and creating an asset library folder of sounds I recorded previously on the modular setup at LCC. I then created an extra parameter of distance, going from 0 to 50 meters. Even though the video game I am working on is 2d and doesn’t allow much room for movement and attenuation I thought it would be good practice to understand FMOD a little further after the previous lectures.

I then thought about what sort of experience and characteristics I would want this sound effect to have, this one, in particular, is more factory electronic sounds and I thought it would be interesting to emulate a 3d environment and have the sounds slowly filter and get quieter after a distance. I read in (Stevens and Raybould, 2017) that a typical error most novice sound designers have with implementing attenuation is that they assume volume is enough to propose distance to the listen/player. When in actuality our environment handles and effects sound over distance in a very specific way. The first is the atmosphere muffles higher frequencies significantly more than lower frequencies and that over distance the higher frequencies cut off allowing us to hear conversations and sounds but for them to be unintelligible. I decided to create an automated parameter with the multiband EQ effect on FMOD and make the EQ slowly allow fewer frequencies over time until it’s completely silent by fifty meters or so. Essentially a low pass filter.

I then began to want to include effects within this practice FMOD session. I decided to incorporate reverb for practice. I first thought about positionality within this imaginary game I’m doing this for, as mine is 2d. Where the player would stand and what the environment looks like, what are the walls made of? Are there any walls? I wanted to create an imaginary unreal environment where when the player went close to the wall the audio was dry and had little reverb, then slowly as they went further it increased to a huge amount. I did this by automating the dry level of the reverb and making it automate to 0 when at 50 meters and increasing the reverb time to the maximum at 50 metres and starting at 0 on the reverb time at 0 meters.

I then started using the pan and degree plugin and automating that, alongside the 3d preview to move the sound.

Overall I feel like FMOD has a lot of uses within a game and it’s interesting to see the middleware software and how it can be used in context. I want to try and use this for my hand in if possible and create the songs as an event and have them randomise rather than playing the same song. Also for the atmospheric sounds, layer up a few and make the work in random selection so it’s not the same every time. Perhaps even put a few empty sounds into the randomiser.

“If you are there and what appears to be happening is really happening, then this is happening to you!”

This quote speaks on the idea that virtual reality is real. That it’s part of our experience that we are feeling right now, with our physical body. And the experience is truthful and transparent and makes us feel that it’s actually happening.

XR as new media.

In the 90s we had a rise in personal computers. Companies started investing and virtual reality became more viable. It did not become a thing at that time potentially because of economic reasons and technology needs financial support to develop. As well as the fact that technology wasn’t there, it does take time to develop these sorts of hardware. Hardware at the time was clanky and heavy, it would reach high temperatures and it wasn’t viable to create such products.

The rest of the lecture spoke on immersion and the theory behind how one can induce a participant into an immersive experience. As well as how to guide presence. The sound was spoken on but not a considerable amount, I will be taking the theory into my own interpretation. What it did help with was the larger ideas around virtual reality and what it takes to develop content for it.

Leslie Deere

Exploratory nature is key to knowing Leslie’s work. Leslie always says that her work is in progress and the approach of always being in process and this allows her to be always reaching further. She is also a stranger to complacency as this allows her to be always moving forward and see the possibilities of technology. It comes from her work being interdisciplinary and this allows Leslie to see the gaps between these disciplines and combine them.

We then watch a video of Leslie’s work called Array Infinite, Leslie Deere, is into gestural visual performance art. And found herself using the Kinect for such pieces. The next logical step for her was using VR to take her work forward.

Array infinitive. It is a piece that requires gestures to create audiovisuals. The hands control the sound particle trails in the VR world live as It happens. She’s designed parameters of reverb and delays to the XYZ Gestures. This allows the performer to explore and create new soundscapes using gestures. Her own group practice of online meditation influenced this project, She’s part of VR groups that meditate online together, which are speech led and she’s doing the opposite in this project she’s doing group meditation through audiovisual experience. They don’t walk around in the virtual world, instead, they’re on a journey through colour and sound.

The performer becomes the guide and the direct connection between the world-changing and the audio being manipulated for themselves and the other participants sitting in the room.

The sound is amplified into the space instead of headphones to add an anchor into reality into the group mentality and this is part of the research she is currently doing. She created seven scenes that related to the chakra system and related the scenes towards the relation to the colours. Green is associated with air, orange with water, and blue is about voice. She uses a three-part vocal female harmony here. Purple is ether and the sound is more into a distant realm.

Leslie used unity to develop the program and local data to connect the headsets to each other. She’s worked with a team to create this. The piece uses Ableton, MAXMSP and unity to combine everything together. 

She also recorded multiple musicians for this work and used their pieces for every colour and the audio experience associated with it. 

She beings by speaking about how big of a project it is, and predominantly her work is collaboration. She finds it an enriching way of working and you can find out so many different things, it can lead to discovering speciality industries and a niche way of working and she enjoys a collaborative practice.

She is making a project about audiovisuals and gestural work. During her PhD annual review sessions, her examiner eventually started saying, what is the context of how she got to the project?

Her background is in dance, she knew about John cage, and she knew about experimental sound but through the context of dance. She didn’t know much about it. She knew of it but not about it too much so this was her way into the practice. During her last few years in New York, she got into tap dancing where the performer would trade 16 counts, it was very musical, the tap dancers would improvise with musicians, bands, and drummers. Dance that makes sound is what she got into.

She moved to London to do a course on the sonic arts at Middlesex, she didn’t come from a DJ or band background so she had no studio knowledge, or on signal flow. She wanted to make sounds similar to music concrète and collage them in an artistic way. There were two pathways she could take, one more performative way being laptop performances and an installation route. She went the installation route and felt it has a similarity with dance, working with space, the body.

So she created this series titled The Amplified Series, its sculptures that don’t make a sound. They pick sound up. There’s a microphone on a yellow box with a headphone output, so when you listened you could hear the pendulum swinging and the room as well. And she was interested in the idea of the room and the environment being part of the piece. Deep listening to what’s around you became part of her work.

When you have headphones on, and record someone you can hear subtle details that you wouldn’t hear otherwise, and she made a piece with two sets of headphones and it’s supposed to bring the room and a conversation into the piece. She also made a similar idea but with a hydrophone in a tank of water and the participant would listen and adjust the mixer to their liking.

This gave her an interest in how sound affects different spaces and the relationship between them.

She modified the mixers on her earlier pieces, she would paint them and this was to demystify the technology, for an average person this would be intimidating and the idea was to make it more playful. The idea of her work is to filter our environment, what we hear and what we don’t hear. What do we hear in an amplified room?

So at this point, she still hasn’t performed with sound. And her journey into sound performance became an interest. Live sound-making was what she wanted to do next. She received a residency in London and she developed a project that used a Kinect camera to pick up her movements and play sounds. Her motion would trigger sounds. It was called Modern Conjuring for Amateurs. This became the thematic backbone of what she does. The basic idea was that when she moved it would create sound and the silhouette would echo her movement. 

The title of her piece came from a book with the same title. She was interested in the turn of the 20 century and the technology surrounding it, electricity, radio, and the phonograph. A lot of technology was available and spiritualism was the theme of this era. Clara Rockmore and her theremin would have been futuristic for the time and considered weird and she was into the idea of this. Her project tried to embody these ideas that had sprouted inside her head.

In her performance with the Kinect when she closes her hands and it loads in a new bank of sounds. It went through different sonic landscapes. The idea was to create Long-form drones with her performance and make the audience zone out and sink into the immersion she was creating. She performed her piece in numerously different places, she acknowledged that Kinect in live situations is not perfect and she picked a difficult technology to use during performances. But agrees it was a good learning curve. 

After this project, she felt she wanted to take this into VR. Her PhD project was called array infinitive. She was thinking of Infinitive as a form of speech 

To make one feel, to activate something. An infinitive. A Similar concept to gestural audio-visuals but she wanted to concentrate on a group experience and the zoning out. Is it possible to gain an altered state experience in a group VR performance?

Erika Fischer-Lichte speaks of a performance as an art event, and through the visceral experience of performance, the audience is transformed. Jonathan Weinel’s work also speaks about the developments of electronic music and art and the different opportunities for creating different consciousness. Can art that uses this as an idea affect these mind states?

Mark Grimshaw speaks on how the design of sound contributes to player immersion in games. The perception of sound and the real vs the virtual world and the rise of immersion. Are we comparing and contrasting sounds? Are they supposed to sound real or fake in the game world? Does this add or take away from the game world? What were the intentions of the composition or the design, was it to promote a state of immersion? Listening across disciplines, listening for science or politics. What are the ideas behind combining listening? Lastly Altered States by Dr David Luke, doing trials of psychedelics. This all helped her with a pilot test on the first function of her piece.

So what were her inspirations? 

Catherine Yass Works with films and photography. She has a piece called the lighthouse and it doesn’t have sound but it was projected on the whole wall of a gallery. The shots were sweeping swooping shots that play with perspective and it was mesmerising and she could tell it had an effect on An audience. And this was one of the first times she saw a piece of art affect an audience. She works with colours and the psychology of space as well.

Ann Veronica Janessens,

Minimal work and she considers light to be her primary medium. She perceives her work as experiential and very hard to document. Her work is about the experience and also uses colours, movement and sound. To heighten our awareness of colours and sound and movement is a large theme of her work and this sensory experience of reality and to play with our senses.

James Turrell,

Again light and perceptional phenomena and there’s a reflective meditative theme in his work. He talks about the emotional effects of luminosity. Seeing ourselves seeing and understanding ourselves as an entity of being.

Hilma af Klint,

Considered one of the first abstract painters, she was into channelling. Born in 1862, again the time period of technology, radio, phonograph, spiritualism. She was into mediumship and every painting she did she was channelling from somewhere else. Also abstract and uses colours.

Agnes Pelton

Similar themes, born in 1862, same time period. Born in Germany and relocated to the US and has spiritual themes including, communicating, and translating. She depicted the spiritual reality in meditation.

Jeff Cornelis.

She found his work through her research and he made this film called Colours of the Mind. He’s a dutch filmmaker. His film Colours of the Mind explores the social and spiritual role of self-induced trances. Rave culture as well and does well to link everything together. He connects raves with the transcendental acts that we engage in when we enter these spaces.

For sound, the inspirations begin with Eliana Radigue.

Her work is minimal which is a similar theme to the other artists she is interested in. Her tape loops and the idea of playing loops that get out of synch and that when doing so new elements start to emerge. Duration, Longform and seeing where the journey takes you. Unending music. She became a Buddhist later in life and this became an important part of her practice.

William Basiniki.

The disintegration loops, she likes that he has a cosmic idea towards things. Taking a sound from out of this earth. Taking something cold and transforming it.

Maryanne Armacher,

Sound objects and the idea of the third ear. The idea is that we have a third ear and when we listen other elements pop out. As well as discussing the feeling of sound and ritualistic notions of sound. As well as listeners’ position in the room, how close are you to the sound source, what if you move into a different place. How does this alter your perspective and listening experience? 

For her PhD, she recorded musicians in Glasgow, Processed bowls, Cello, and a choir. To make the work was a huge undertaking, a big thing to try to pull off. Ross flight designed the interface that she used for the Kinect camera, he’s a sound engineer and works a lot with theatre, so they worked together again on her PhD project and modified the earlier version. Stuart Cupitt kindly sponsored her work in the development of the project. Chris Speed is an amazing DJ who makes music and also helped out in the Art.

Last summer she had an official pilot test and had 25 participants, the ethical clearance for that during covid was insane but it was a great test. She had an age range of 17-80 years old. A mixed audience and quite a lot of people afterwards would sit down and chat, some people would state their neurodivergent and would like to see where it goes. It was also interesting to see people experience it that know nothing about technology. She also noticed that gamers want to do things and want to get up and move around.  

The idea of the piece is that there are four people sitting down and one performing and the group experience was key to her as they travel through the colour spectrum. Unfortunately in the pilot stage none of them noticed the group element, most likely due to the overload of technology. She noticed the piece was interpreted in different ways. The sound is also amplified into the room and she didn’t use headphones intentionally. She wanted the sound to be experienced similar to a gig. She was unsure if this would ruin people’s experience as you could hear her moving around and this would take them out of the immersion but she found out people enjoyed it. And found that people felt more grounded.

After the pilot test, she did a public Beta and for the first time she had a mixed audience, in and out of VR and she found it super interesting. It creates a different dynamic and the people outside of VR can see what she’s doing. And there exists dynamic feedback between her in VR and the audience observing.

Post Lecture Reflection

I found Leslies’ work to be very forward-thinking in terms of the cross-disciplinary aspect of it. It made me reflect on our current process in collaboration with the games design MA students and how that process has been, which is fruitful. We have in turn explained how we do sound design to the MA students and the process behind it and also how they explained the process of making a game. As well as using each other’s strengths. Alongside this in a smaller context within just my own group how we have managed to learn from each other within this collaborative project and understand each other’s workflows and ways of thinking has been refreshing and thoroughly insightful.

I also found Leslies’ process of research and how it actively reflects her own work and the process of creating different versions to finally reach an endpoint perhaps to be an ongoing process. From her first piece of Kinect and the original sound installations she did which eventually led to her creating her latest project. One can look back and easily understand the process of it. I also enjoyed her use of others in this project, she openly admitted she knew very little about the sound world but got the right people around her to help. I think it’s important to understand that sometimes even the people with the skills might not have the ideas to create projects such as these. That it might take an outsider of the discipline to enter with a fresh thought process, not under influence of the norm.

To look at my group’s work from a sound arts perspective I can say we’ve definitely done some things correctly others not. I do believe we have given ourselves enough time working together and the ideas have flourished and everyone has done their part. But the active reflective process that comes with collaboration wasn’t entirely there. The constant feedback was very little and did make it difficult to really go through the same stages that Leslie for example did in her work. I purpose that going on forth I’m unsure if the mix is all individually or as a group? But if it is an individual or group I want to ground our sounds in theory as they have some at the moment. But I feel there could be a greater theoretical influence on our decisions, as well as this understanding of how sound can be used outside of the MA games design understanding. They only asked for sounds that they needed and now we have an opportunity to completely decorate this game world. For students such as ourselves where sound is our everything we can enhance and work alongside the visuals and present immersion in ways that visuals cant do alone.

Since the crit we had as a group we have been working towards implementing the feedback provided.

One thing that’s working well was that the music and sound effects were really good and fit the idea of the game really well. We thanked them for their feedback, they said the music as well had a lot of layers which felt nice and didn’t clash too much with the game. As sometimes overly complex music can make a video game distracting to play as your brain is listening to the music rather than concentrating on the current action.

One thing to add or change. One student said a variation of sound effects, they enjoyed the music and the sound effects Jingya had made, but they offered the feedback that perhaps we could change them in between levels. Or create more variations of the sound effects we have created already, to allow an element of immersion that comes from variation in sound.

One positive overall. A student noted that what we did was a good job and they were very impressed with the music compositionally and creatively. They enjoyed our presentation and our take on what the MA students wanted us to do. They thought our ideas fit the game’s aesthetic perfectly and were excited to actually see the finished product.

Tutor Feedback. We were told similar to the student’s feedback, that our presentation was professional and excellently communicated as well as our slides were valuable to showcase our work. Our music fit the work correctly and even though we only showed one level, we were told that it was excellent and that the development should continue more and expand based on our ideas. As well to this we were congratulated on our work together as a group and that we should be very proud of our work ethic and attempts of working with the MA students as it took them a while to deliver anything for us to base the sound work for. An important part of the feedback which no one had said except the tutor was the idea of our perspective as sound art students, typically game design students will only think about the most obvious sounds. For example footsteps or breathing, shooting. But as sound artists and sound designers, it is our responsibility to think about everything that sound can encapsulate and how we can use sound to enhance motifs. Fear, strength, comfortability, a sense of place, all these things can be displayed with sound.

We took this all on board and did some final touches on our submission for the MA students, Jingya did some variations of the sound effects and typing keyboard noise. I and Will created some varied tracks for them to use across levels in the game. I also created a soundscape to go along with the levels as I felt even though the music was playing it needed a base layer of machinery noises.