Communication, Media, Sound and Interaction Design 2018
In the previous two weeks I covered an introduction to the ML-Agents Toolkit and explained examples that make use of it. Using that information, I applied the toolkit to a final project – a modified version of the Bouncer example, which I named Climber.
In this final blog, I report on my final project.
Last week I discovered and introduced the ML-Agents toolkit, which is a Unity utility that applies Machine Learning to Unity applications. A first look on the kart racing example project provided insights into this specific field. In summary, an RL agent receives some form of input and improves its behavior based on that input over time until its performance reaches a desired degree of quality. On the one hand, this utility fulfills my initial goal to test applications using AI while on the other hand, it even surpasses my intentions by providing extra features, like training NPCs.
In this blog, example projects of the ML-Agents toolkit are briefly shown and summarized.
When I covered libraries, frameworks and toolkits, I missed out on a potentially game-chaning toolkit, which I will dive into today.
In this blog, I take a first-hand look on the Unity Machine Learning-Agents Toolkit.
When I originally analyzed MarI/O and did my research for Metroid Learning, I skipped over the very basics of this field that is Goal Driven Systems. Thus, it is time to catch up on a bit of theory.
In this blog, I introduce goal-driven systems, reverse engineering and emulators.
Researching on the topic lead to the conclusion that I skipped one of the most important steps – assigning Reinforcement Learning (RL) bots their place in the field of Artificial Intelligence (AI). There is a lot of ground to cover, so it is now time to catch up on it.
In this blog, I properly introduce RL’s place in the field AI.
In this blog, I take the first steps in the practical development process, which include choosing the working environment, naming the project and setting up the GitHub repository.
Already in 1894 people started looking for a way to further support sound through imagery. One of the first was George Thomas, who tried to combine sound – of a live performance – with projected images. The first video ever created – or rather called a „sound film“ – was directed by Theodore Case made in New York in 1921. The film depicts Gus Visser singing the song „Ma, He’s Making Eyes at Me“ while holding a duck. The duck quacks each time the word „Ma“ is said, sounding as if she is saying „Ma“.
New blog series starting today; the final arc of my work: tool development.
In this diary-formatted blog, I summarize the progress made so far and try to create a small to do list for myself.
There are a lot of examples for good music videos produced in over the last years using Animation as their technique.
Here are just two examples, which I thing show really good, how music videos can support the message of a song/band. Both of them convey a different ambient by focusing on different color, animation style and storyline und thereby visually supporting the impact of the song.
For my master thesis I want to focus on the medium music video using animation/illustration technique. My aim is to look into the different aspects of storytelling used in music videos, its development over the past years and especially its emotional influence on the viewers. I want to explore how music as a fundamental channel of communication can, in combination with visuals, provide a medium by which people share emotions, intentions and meanings even though their spoken languages may be mutually incomprehensible.
The problem – nowadays, in the “digital-age” with its ever-growing media content, for designers, it’s getting harder and harder to reach and especially move people with a specific message. But not only the overflow of material is an obstacle that needs to be tackled, but also the way of communication needs to be adjusted to specific target groups – how can you deal with cultural differences or how to overcome language barriers. Is there a way to communicate a narrataive using an “universal language” such as sound and imagery? There are various studies on the topics of history and the developement of music videos also connected with emotional response of the viewers. As well as a few studies on how music videos can offer an educational value and influence a pro-active behaviour of the beholder.
Other Questions I want to tackle are: What role and relevance can be associated with music videos nowadays? In wich way can the correlation of music and visuals convey a narrative that moves the viewer emotionaly? Can a music video even offer an educational purpose? In what way are music videos replacing printed visual identity of a song/album? Can music videos be developed/ applied to different formats or even combined with different media – for example poster or webdesign – is there a way of creating a whole new way of communicating sound? Which different relationships between image and sound can be found in a music video and how are these interactions influencing the interpretation of a music video?
The thesis should stress the importance of evolving and adapting different media to the spirit of the age – and should make an example of how, in this case, a medium like a music video can be further explored and developed. The goal is to create a narrative based on a preexisting song, yet not defined, and create a visual identity for it – almost like a manifesto on an educational topic, yet defined. This experiment should show how a music video can be a powerful tool to convey a message and even provide educational qualities in an intercultural manner – by telling a story without using textual but visual language.
There are various books and articles on the topics:
Ferrandino, Matthew & Osborn, Brad. (2019). Seeing Stories, Hearing Stories in Narrative Music Video. The Society for Music Theory Videocast Journal. 5. 10.30535/smtv.5.5.
Burns, Gary. (1986). Dreams and Mediation in Music Video. Wide Angle. 10.2.
Antal, Silard. (2015). Visual analysis of music in function of music video. Zbornik Akademije umetnosti. 2015.47-56. 10.5937/ZbAkUm1503047A.
The Velvet Light Trap 56.1 (2005) 73-74 Vernallis, Carol. Experiencing Music Video: Aesthetics and Cultural Context. New York: Columbia UP, 2004. 480 pp.
Kleanthous, Styliani & Michael, Grigoris & Samaras, George & Dimitrova, Vania. (2017). Individual Differences in Music Video Interaction: An exploratory Analysis. 241-246. 10.1145/3099023.3099061.
Schmidt, Patricia. (2005). Thinking inside the box: In search of music -video culture.