BTS: Research Process

On this page I have summarized the initial research process, which includes a field trip to Amsterdam, research on stigmatization, storytelling as well as sound and visual design. Outcome of this phase was a number of initial porotypes in TouchDesigner as well as a self-written story in 3 acts to serve as foundation for the audio-visual story telling of my project.

Field Trip

Right at the start of this project, I did a little field trip to Amsterdam to visit the NXT museum and do the POPPI drugs tour.

The NXT museum is the “home of new media art” and exhibits many large scale, multi-sensory installations. I went to the museum to get some inspiration for my own installation, as I wanted to work towards creating an immersive 360 experience.

The POPPI drugs tour on the other hand is a self guided, digital tour through the streets of Amsterdam, that takes you through the rough eighties and nineties of Amsterdam’s drug history.

For instance, the tour takes you to Zeedijk, a street in the red light district just next to the central station, which was the hot spot for heroin trade, filled with street dealers, drug addicts and high corruption in the police. It was a very gripping tour, seeing where people hid their needles, created their safe spaces and where people overdosed. It was also interesting to learn that the famous jazz musician Chet Baker died here by falling out of a window in Zeedijk after an overdose.

A sign by the police stating that they can no longer warrant safety for area of the Zeedijk street due to drug crime.

Stigmatization

Next to my little field trip I also did some self study on stigmatization of drug users. One of the most interesting resources I found is the “The Power of Words”, which is a practical guide to support healthcare and other professionals working with people who use alcohol and other drugs to reduce stigma and improve health outcomes. Especially the background document was very insightful.

For instance, it explains what stigma is, differentiating between the different forms of stigma (social, structural and self-stigma); shows different groups of people that are affected by stigma; what impact stigmatization has on these groups; who contributes to the stigma; and finally discusses how we can work towards reducing the stigma.

Another source that would become very helpful in this project is the Story Powered Initiative. This site contains about a 100 stories of real persons and their personal experiences with drugs.

Story Telling

If you have read my research proposal, you have seen that initially I planned to interview stakeholders related to rehab organizations and safe use communities such as Tactus, Hengelo Narcotics Anonymous, Mainline or GGD Twente. Here, the goal was to learn more about people who use drugs as well as to conduct interviews to learn about their personal stories and use these to create a narrative that I could use for my installation.

However, the communication with these stakeholders was either non-existent or way too slow for the time plan of my project. Thus, I decided to create my own story. To do so, I filtered the Story Powered Initiative site for stories relating to stigma and analyzed about 20 different stories.

I then did some research on storytelling, specifically I was interested in creating a story that is very intimate in order to move the viewer. There are various elements one can consider for this, here are some strategies you can use to foster intimacy in your story telling:

Character Development: Invest time in developing well-rounded and relatable characters. Give them depth, vulnerabilities, and flaws that make them feel human and relatable to the audience. Show their inner thoughts, desires, and struggles to create an emotional bond.
Authentic Dialogue: Craft dialogue that feels natural and true to the characters. Use dialogue to reveal their personalities, emotions, and relationships. Realistic and heartfelt conversations can help the audience connect with the characters on a deeper level.
Point of View: Consider utilizing a point of view that allows the audience to experience the story through the eyes of a specific character. This can create a sense of intimacy by immersing the audience in their thoughts, emotions, and perceptions.
Inner Monologue: Incorporate inner monologues or voice-over narration to provide insights into a character's thoughts, fears, and desires. This technique allows the audience to intimately understand the character's internal world.
Vulnerability and Emotional Arcs: Explore moments of vulnerability and emotional growth for the characters. Show their struggles, fears, and personal transformations throughout the story. By witnessing their emotional journeys, the audience can forge a strong emotional connection.
Intimate Settings and Scenes: Create settings and scenes that foster intimacy. This can include private and personal spaces that allow for more intimate interactions between characters. Intimate settings can enhance the emotional depth of conversations or moments of connection.
Empathy and Universal Themes: Craft narratives that tap into universal themes and experiences that resonate with the audience. When the audience can empathize with the characters' struggles, joys, or conflicts, it deepens their emotional investment and connection.
Subtle Gestures and Expressions: Pay attention to subtle gestures, facial expressions, and body language to convey emotions and create intimacy. These non-verbal cues can often communicate more than words, allowing the audience to feel a deeper connection to the characters.
Emotional Stakes: Establish high emotional stakes within the story. Present conflicts or challenges that force the characters to confront their fears or make difficult choices. The greater the emotional investment, the stronger the potential for intimacy.
Honesty and Authenticity: Be true to the story and characters. Authenticity resonates with audiences and can create a genuine emotional connection. Avoid contrivances or manipulative tactics, and let the story unfold in a way that feels genuine and honest.

Using some of these strategies and my analysis from the 20 existing stories, I then created my own story in the typical 3 acts (setup, conflict, resolution), which you can find here.

This story follows the journey of a person named Elliot who grew up in a middle-class neighborhood, surrounded by addiction affecting various family members. The story tells about her own descent into substance abuse and how various traumatic experiences lead to her addiction. In particular, the story deals with our perception of cultural norms, how trauma can give rise to substance abuse and how stigmatization can spiral further into addiction, isolation and mental health problems.

Later in the project I held auditions to find a voice actor to narrate the story. There were many very good candidates and in the end I selected Juliana Zijlstra, who I saw earlier this year in the Addams Family Musical. Together with Juliana, we recorded three excerpts of the story which I then later used in my prototypes.

Audio and Visual Research

Next to research on stigmatization and storytelling, I also did some research on creating audio and visuals for my project.

Visuals

In this project I wanted to use TouchDesigner exclusively, for various reasons. First, there was not enough time in this project to explore different media (e.g. shooting scenes with actors, making a real film …); second, it creates a limitation to drive my creativity, and finally, I’ve been eyeing on learning TouchDesigner for some time, so this was the perfect opportunity to dive deeply into it.

If you don’t know TouchDesigner, it is a very powerful software that allows you to create all kinds of visuals, generative art and interactive installations. During my research, I watched a loooot of tutorials on TouchDesigner, from the basics to video manipulation, animation, typography art and even point cloud simulations. Next to watching tutorials, I also attended a couple of TouchDesign meetups by Music Hackspace, where you could exchange knowledge with other TouchDesigner artists.

You can find more details about what I learned in the respective prototype summaries, but if you want to learn TouchDesigner yourself, I can recommend you to check out Elekktronaut and PPPANIK.

Below you can find some of my earlier prototypes. Both of them are audio reactive, the first example switches between different input videos based on the kick drum of the song whereas in the second example, the geometry of the torus object is twisted in the X and Y direction based on low and high frequencies respectively,

Sound Design

I started with some self-study on general sound design like what kind of sound effects (e.g. risers, drones etc.) are typically used in film making, for what purposes they are used (e.g. transitions, ambience, etc.) and where to find high quality sound effects that are royalty-free (e.g. epidemic sounds).

I also did some research on sound design for creating music in the style of Disasterpeace and Lorn. Both of these artists have a very distinct sound that is very dark and atmospheric and I wanted to recreate this type of sound for my project. You can find more details on this on the prototype pages, specifically I talk about Lorn in “A Whole New World” and Disasterpeace in “The Last Drop”.

Lastly, I also had multiple sessions with an external expert, Rik Nieuwdorp, who is composer, sound designer and a teacher at the University of Twente for the course “Storytelling through Sound”.

One of the things I learned from Rik is that you can structure sound design in four layers: Dialog, Music, ambience and sound FX. All of these layers have different sounds and different purposes. For instance, dialog is about the character and the message you want to convey; music is about creating emotions whereas ambience (e.g. wind or forest sounds) are meant to create the sound of a location; lastly, sound FX are special effects to highlight certain things or events in a scene (e.g. glass breaking).

The most important message I took from Rik however was to always think about the purpose of the sounds I am using. What is the meaning behind certain sound or effects in relation to the visual and the story I am telling? How can I express this meaning in a way that is understandable for the audience?

Next to these general principles, I also got more specific tips and feedback from Rik on my sound design, like what kind of reverbs to use for emulating realistic spaces or vocal processing for the recordings of my voice actor. From my discussions with Rik I also got a lot of inspiration in general, through which I came up with the idea to use a vocoder for one of my prototypes.