In NomCatchWink: The Playformance, I am using gamification, body recognition and playful design with a dash of irony to address and criticise the performative behaviours on social media.
The project is a playable experience, connected to a database that acts as a gamified platform. It includes a set of three simple camera-based games, each of them targeting specific behaviours inspired by specific social media platforms that we use to represent us online.
The goal is to create an engaging experience, that brings fun to the player, yet allows them to critically reflect and question the performative nature and possible negative or toxic side of social media.
A concept that didn’t make it through. It was inspired by my research on how actors are perfecting their skills for stage, and the tutorials they create for people. Following that, I wanted to create a play-guide on how to create a perfect performance, divided into three categories that actors focus on when practising their skills. 01: Face and Expressions (WINK WINK), 02: Languege (NOM NOM) and 03: Body Movement: (CATCH CATCH). It was a fun concept, however, after all, I reflected and realised creating a guide/tutorial, takes the idea of a game and gamification away, also it was missing the moment of creating a platform, which better alligns with my enquiry focusing on social media.
A CONCEPT THAT DID MAKE IT 🙂
THE PLAYFORMANCE
In the concept of THE PLAYFORMANCE, I tried to find a way of adding motivation for all three games to frame them together. The motivation comes in the form of achievement badges that the player aims to reach through playing. The achievements were inspired by very simple motivation that is used in a lot of simple games, but also by achievements on social media you can get for engaging on social media – such as on Instagram)
The achievement badges as a motivation for a player.
Together, the games are framed into a playable platform with the collective gallery, that is connected into a database. The platform is sitting between a game – collecting achievements, playing the games & ”social” platform – Chats that generate random comments from a small database, each time the player plays, the comments will be different.
I was also playing with language in the games and in the platform – imagine there is a dialogue between the “quirky/cringy” tone and a grey suit man – example “OMG SO MUCH FUN! paired with something serious like “Playing can lead to addictive behaviour”
Feedback that helped me to take the whole concept further: Elements to spark criticality
During the Make a Thon session I had a great feedback session with Owen. He recommended me Implementing to add a few “bad” elements to each game that don’t belong to add criticality and unexpected moments – negative reactions, words and images, to reflect the toxicity of socials and emphasize that these spaces also bring negative things, that are happening while we ar performing our dances. The idea was to add just a few elements from now and then as a surprise that sparks the criticality about the social media spaces
of Coming up with concepts and inspiration for my games, I started putting together the first versions of the games. I started with creating the starting pages, each inspired by the developing visual language from the previous weeks. I was also brainstorming over the names of the games, even when recontextualising the games, I wanted to continue with the type of the names from projections 1, as I thought they had a nice ring and playfulness to them, reflecting perfectly the idea of the game, but also what the player is asked to do in the games. I decided to name the third game WINK WINK (as the player is asked to use their face to copy facial expressions – emotions). Also including NOM NOM and CATCH CATCH, together all three games created a collection NomCatchWink. 🙂
Creating the games, I continued using P5.js, which is a JavaScript library for creative coding. I started putting together the first chunks of codes, focusing on the mechanics of the games, reflecting the goals and ideas from my previous research and experiments.
THE GAMES DEVELOPMENT AS FOR 18 / 4 / 2024
NOM NOM
Using mouth to eat buzzwords/hashtags off the screen to collect points
CATCH CATCH
Similar to projections 1, using body movements- catching, to collect likes and points
WINK WINK
Using face and emotion recognition to copy facial expressions on the screen to gain points
After setting the base of the mechanics of games, I started working on adding the downloadable outcome. Same as in Projections one, CatchCatch outcome was a moving gif, a dancing performance from screengrabs taken throughout the game. For Wink Wink, I recycled the idea of image gallery, as it worked perfectly with the inspiration coming from capturing moments and still images that we share online. For nom nom, the inspiration for the game was the way we consume and produce information, that often is misleading or edited to the form so it gets the highest amount of likes and shares, often not paying attention to the content or reliability of the information. I created a list of hashtags and buzzwords that I outsourced from social media, and created a database of random sentences divided into 5 different types of thematical categories. Each of the sentences has a random word in them that is replaced by some of the hasthags the player has eaten, resulting into a collection of sentences.
this is a moving gif 🙂
BEHAVIOUR / PERFORMANCE / SOCIAL MEDIA / GAMIFICATION / BODY RECOGNITION
In my project, NOM CATCH WINK, I am using gamification, creative coding and face and body recognition tools to create a set of playable experiences, each inspired by specific behaviour on social media, that subvert the way we create and share content in digital spaces. Using a dash of humour, playfulness and irony, the set of games aims to prompt players to reflect on their actions and interactions within the game and in online world, fostering critical thinking about online performative behaviours and the influence of digital spaces.
“Rules of Play: Game Design Fundamentals” (Salen, Zimmerman, 2017)
A resource with rules and guidance whenmakinggames, inspiration for NomCatchWink tocreate an experience that is easy to interact with, well playable:
1. Game Design Core: – Understanding mechanics, dynamics, and systems. -> exploring different body recognition libraries, coding in connection to my motivation and enquiry – Clarifying rules and player interaction. -> clear instructions in the game, but also during the game – Creating immersive game worlds. -> Combining the physical world (camera) with funky visual elements
2. Compelling Mechanics: – Offering meaningful choices and interactivity. -> Interactivity in form of engaging with body movements to play through the game – Balancing skill and chance elements. -> the items in the games are random, but as active the player is, higher score they get – Encouraging creativity and exploration. -> Encouraging reflection and realisation through the games
3. Player Engagement: – Empowering player agency and control. -> physical embodiment of actions, using body to control the game – Providing feedback, rewards, and motivation. -> rewards in form of points, badges, feedback in a form of performance at the end – Implementing dynamic response mechanisms.
4. Dynamic Gameplay: – Allowing for evolving dynamics and player influence. – Incorporating strategic depth and adaptive systems. – Balancing between competition and cooperation.
5. Iterative Design Process: – Engaging in continuous iteration for improvement. – Prototyping, playtesting, and refinement. – Incorporating player feedback for enhancement
THE GAMES DEVELOPMENT AS FOR 26 / 4 / 2024
After having my 1:1 tutorial and after doing some research on games, all three games got to the point when. they could’ve been called the first “final” version (meaning they had a playable beginning, middle and ending :)). In the development I focused on better playability as I was encouraged in my tutorial, adding clear and easy-to-understand instructions, adding better languages to the endings of the games, and adding sounds for a more immersive experience. I was ready to bring these games to my production surgery with Oliver Smith, to talk about the possible framing and context of all three games, as I was still looking for a way to combine them into one, easy-to-circulate experience that would connect all three games nicely together without taking the meaning and motivations away. Talking to Oliver he helped me to realise what may be missing in the whole experience and what may be thought of when bringing the games together:
Moments from Oliver’s Feedback after showing him the games as they are displayed below:
As the games can be played separately, there is an opportunity to add something before / after or even in between the games, connect them to one
The physical embodiment of performance is 100% there
How can we add the social aspect? To create a feeling of a social space?
Using the website I am creating almost a platform for people to engage in, which could be a good concept to work around
I came up with the idea of the outcomes being shared in a collective gallery as presented in projections 1. -> How to frame the collective gallery?
Offering the player with metrics / social aspects in the form of chat, communication, and reactions?
NOM NOM
CATCH CATCH
WINK WINK
At this point, I am aware that I have a lot of work ahead. I can’t wait for the games to come together into one experience, as there was a lot of effort, energy, research, tears and hundreds of hours of work already, but I am also nervous as none of my ideas seems to feel right.. well it’s a process of course, and I believe eventually I will come up with something! I feel like at this point everything is more about making rather than more research, so I will just get to it and see how it goes haha.
Anyways, a little advice: if you’ve done a bare minimum to no coding and you are thinking about choosing that as a tool to work with for your project, please think twice so you don’t end up on the edge of mental breakdown as I did many times when working on this project :))) Yes, tutorials exist, chat GPT can also help, talented and smart friends and boyfriends as well, but after all, you will spend 8-16 hours pretty much everyday trying to figure out the smallest details haha. Well, it’s been fun!
During the workshop, we’ve run a lot of interesting conversations and discussions on the speculative term “Place-based Graphic Design Publishing”. What does it mean place-based publishing? How it can help us and our work as a designer, or what are the existing examples of works that utilise place-based publishing in their work? We also talked about the changes that are made to the understanding of the projects by taking them out of the space and publishing and printing collecting them into a book or an archive, What do they gain, what do they lose? These were a few of the many questions we were exploring, followed by a little trip outside to look for our own notions of place-based publishing.
Articulating the term into my words, place-based publishing in graphic design is a form of publishing, that utilises the surroundings and the place it is positioned in itself as a way to deepen the experience and the understanding of the audience while utilising and realising the connections between the place and the project.
One of the examples that I found was a public gallery with photographs and texts in Coal Drops Yard about Scientists. It was connected to the place, as the gallery was unfolding the Scientists’ daily work happening in the buildings surrounding the space. Placing the gallery in this space, allows the audience to realise a deeper connection to the place they are in, it gives them a new set of glasses to see through when looking at the surroundings and the big, quiet buildings around. It offers the audience a new information about the place, coming from it’ß cultural or societal background.
Overall the workshop was very interesting, especially as it gave me /and also others as well) a lot of space to think about how to deepen my graphic design practice in connection to the surroundings it is placed into, providing a few great examples and inspiration for positioning my design practice.
The action of entertaining other people by dancing, singing, acting or playing music
(OxfordLanguages, 2024)
PERFORMANCE DEFINITION 2:
How well a person or a machine does a piece of activity / how well a machine or a computer works / often measurable (OxfordLanguages, 2024) On Social media: engagement, impressions and reach shares
(Forbes, 2023)
Social media is performative.
How can I use my games with face and body recognition, to criticise the performative behaviours and the toxicity of social media?
After the feedback from Matthew and the user engagement feedback from my classmates, I decided to continue coding and pursuing the idea of playable experiences – games. To do so, however, I needed to narrow down more what the games will be focusing on and what they will be exploring. one of his ideas was to focus on specific behaviours connected to specific social platforms, which was a direction I really liked and decided to explore further.
THEORETICIAL FRAMEWORK – PERFORMANCE AS A PART OF HUMAN’S NATURE
As I was already engaged in my previous work’s psychological and behavioural research, I decided to focus on behavioural studies concerning social spaces and interaction. A great book I found, connected to performative behaviours and psychology was a book from 1959. Back then, an American sociologist and psychologist introduced a perspective on human behaviour called the dramaturgical approach. This approach, a dramaturgy Theory concept views social interaction as a theatrical performance. The World as a stage, people as actors who play their roles to fit in, as we, from the nature, are creatures who need social contact, validation and belonging.
That book became one of my main references to work around in my work. So what happens if this theory is applied to Social media, where the idea of performance has been expanded for measurable performance in the forms of likes, comments, and shares. With the fact these performances can be done from behind the screen, social media space offer an extensive and modern stage to perform in. My project in a sense is a contemporary, gamified lense to Goffman’s theories.
The emerging idea for the recontextualised set of games, coming from my first set.
FOCUSING ON: – Explore specific online behaviours on specific social media platform – Each game has differences and it’s own message – Creating three separate games, each will have a different place coming from, yet together they make one set – Using and expanding my emerging visual language and language: quirky, icky, funky, colourful, bold – Using still body and face recognition, each game to have different use corresponding with it’s motivations – Each of the games creates a downloadable performance
I started with exploring the specific behaviours on specific social media I could introduce to my games. Matthew gave me the first idea: The moving gif outcome reminded him of TikTok, how people create videos, dance and entertain, creating a form of performance to gain social engagement. With that in mind, I focused on two more very typical forms of posting content: VIDEOS, PHOTOS, TEXTUAL POSTS.
CATCH CATCH
Inspiration: Tik Tok – VIDEOS
Behaviours: Making videos, dancing and performing in order to gain social rewards. Mechanics: Body (hand) recognition, game progressing make the user to perform more and move faster. Outcome: screengrabs turned into a gif -> creating kind of a dancing performance
(disrupting the controlled behaviour people have when producing online videos)
NOM NOM
Inspiration: Twitter (X), Threads – TEXTUAL POSTS
Behaviours: Consuming, and producing (written) information online. Mechanics: Face (mouth) recognition to eat (consume) words on the screen. The words are divided into categories, at the beginning the player can choose, but loses control in the information overload. Outcome: A set of written performances, and messages that are randomly created based on the players activity
THIRD GAME: NAME? WINK?
Inspiration: Instagram – PHOTOS
Behaviours: Producing still images capturing chosen moment, mimicking others and behaviours online
Mechanics: Face expression recognition, Mimicking the gestures on the screen in a given time, screengrabs throughout the game
Outcome: A set of gallery stills/ shots capturing those expressions
Catch Catch Cover image from projections1
Nom Nom Cover image from projections1
INFLUENTAL PROCTICE – THE RODINA
Projects of The Rodina have been on my reference list quite often in my past projects and they’ve been a gret inspiration both in developing my project and my position as a designer. Diving into their approach in adapting gamification, coding, and creative technologies, using colourful, playful, inviting visuals to immerse and engage the audience, while fostering space for reflection. Influential was their use of performative design, to address wider societal issues. One their influential project “Playbour: Role of Reality” had a substantial impact on my work, also opening possible ways to expand my project above my MAGCD studies. Using the tools of graphic communication as I do, the „Role of reality“ offers a space where the player can reimagine the world of work, reflected in my project, that invites the player to reimagine the way we create online performances. (Studio The Rodina, 2024: https://www.therodina.com)
Summarising the feedback, the outcomes and the points of focus on coming from the engagement with the audience and mainly feedback from Matther from our 1:1 tutorial:
The games work well in terms of engaging the audience, communicating its intention, they are fun, easy to engage with
How to put the games and the outcome into a better context? – Something that may crystalise when contextualising the game/s better.
What am I making next? -> idea of focusing on specific behaviours online that make us perform
Narrow the enquiry and research to something more specific
The original two games are very similar in mechanics, more like two iterations, how to make each unique?
Developing/ updated enquiry to continue with in projections 2:
In my project, NOM CATCH WINK: The digital self-play, I am using gamification, creative coding and face and body recognition tools to create a set of playable experiences, each inspired by specific behaviour on social media, that subvert the way we create and share content in digital spaces. Using a dash of humour, playfulness and irony, the set of games aims to prompt players to reflect on their actions and interactions within the game and in online world, fostering critical thinking about online performative behaviours and the influence of digital spaces.
BEHAVIOUR / PERFORMANCE / SOCIAL MEDIA / GAMIFICATION
Enquiry: The project addresses the physical embodiment of reward-driven behaviour in digital spaces. It prompts the audience to question the extent to which we shape our behaviour to conform to the mechanisms of digital platforms. The goal is to create an engaging experience that brings a dash of humour and irony while creating space for reflection.
At this stage, the project contains two interconnected face and body recognition-based games: NOM NOM: Eat the points! & CATCH CATCH: Get the likes! Each game addresses the topic by creating different mechanics of rewards-driven actions.
In both games, the player is asked to use their body movement to perform simple actions in front of their camera to collect points while their time is running out. As the game progresses the game elements increase in intensity and speed, triggering the player to intensify the effort, resulting in almost absurd and funny movements. The player’s activity is then rewarded by two downloadable visual outcomes reflecting their performance: a moving gif and a gallery of pictures. Aside from a good laugh at how silly we look, the activity prompts a question: Did you play the game or did the game play you?
The integration of a real-time camera enhances the individualised experience of the game, allowing players to engage with their physical movements in a digital context. This feature encourages active participation, contributing to a more meaningful and personalised experience. The simple, playful mechanics invite the player to enter the experience without any expectations or burdens, to allow a space for free interpretation and reflection when receiving the visual performance at the end.
Game Elements:
Game mechanics and user behaviour: Reflection of the concept of performing actions in the digital space. As the game progresses, players are required to perform more, mirroring how individuals engage with digital spaces. Eating reflects consuming, catching collecting.
Face and body recognition: Enhances immersion by integrating real-world features into the digital realm, creating a more individualized and engaging experience.
Snapshot gallery: The gallery prompts players to reflect on their actions and interactions within the game, fostering critical thinking about their online behaviour and the influence of digital spaces while creating collective performance.
In week 5 of the process, I was focusing on refining my enquiry a little bit more while exploring the possibilities, the visual language of the game I was working on, while also articulating not only to myself, but also my classmates, the connections of this project stage to my whole process and studio practice development.
In the game, I was working around the idea of addressing the physical embodiment of reward-driven behaviour in digital space, which is often an outcome of the algorithms and mechanisms of digital platforms. Continuing my previous work and research, I aimed to create a playful, engaging experience that the audience can immerse into while creating a space for reflection. The way to do that could have been the game’s ending and the language used in the games, so I focused on that to create a complete experience.
CURRENT DEVELPMENT
THE GAME The player starts by setting their starting position in the window on the screen, “curating” their starting position into a square cut-out. They are encouraged to perform (eat the points of the screen) to gain a special reward! The reward is not mentioned, they are not aware what they are playing for, the game offers the motivation through the gamification.
In the game, the player is asked to ”eat the points of social validation” while the time is running out. The player needs to perform, and move to gain points and unlock a “special reward”.A progress bar shows how far the player is from reaching the “special reward”
As the game progresses and the time is running, more points appear and they move faster, encouraging the player to perform more The more points the player gets, the slower the progress bar fills and it can never be fully reached. The player gives energy, to reach points that will bring nothing. Only to start the whole process again. How long does it take the player to realise the bar can never be filled?
THE GAME OUTCOME:
There are 6 special “tokens” in the game, when these are eaten, it triggers a screenshot function (without the graphics over it). These screenshots are displayed at the end of the game as a gallery. For now, there is a download button to download the pictures.
FEEDBACK
It is an effective way to draw the audience in and get them play, it’s fun and communicates the intention quite well
How to refine the games even more? Is there a way to iterate on them to see what that does?
How to use the language in the games more effectively?
Maybe there is too many things happening /the unreachable progress bar/ – think about keeping it more simple to creating more suggestive experience
Try to avoid the semantics that tell away immediately what the game is about, using more neutral language could lead to more personalized and suggestive experience, that is open for interpretation
How to put the game and the outcome into a better context? Where does it go?
I’ve attended another cross-year studio workshop, this time with Michaela focusing on Gestaltism principles and how we can apply them to our work, and to a collective creation, resulting in producing a printed publication at the end of the workshop. We’ve not only had so much fun, because Michaela is a great person and an inspiring tutor / practitioner, but seeing the World through optics of Gestaltism, focusing on every detail in the space surrounding us was a refreshing, fun and unique way to work. (PS. mine are the light purplish pages after yellow :))
The Gestaltism rule I was working with was the rule of PROXIMITY, which says that things closer to each other appear more related than things farther apart. I took a little twist and added a bit of my own interpretation and exploration, and focused on the rule of proximity when it comes to perspective, and how the perspective we are looking at things from, can affect if the rule is or is not applied. Was a fun experience!
On the example above you can see that when the perspective shifts, the idea of the objects being part of one group/shape (left) breaks when moving just a few steps to the left.
In my research, I tried to finally narrow down my focus even deeper.
I started researching and jumping between different topics to find a connection to my previous work and a deeper enquiry: how the curation of digital identity reflects to a physical space -> different aspects of digital identity and of course touching social media -> labelling and social media, how we are adjusting our behaviour to fit the label/trend that has been given to us or that we’ve given to use -> And also the behavioural changes -> dopamine loops and reward system loop
Enquiry Update: EXPLORING HOW THE CURATION OF DIGITAL PERSONAS WITH A GOAL (that we sometimes don’t realise) TO ACTIVATE THE REWARD-DRIVEN LOOP REFLECT BACK TO OUR PHYSICAL REALITY
Graphic Design Lenses: Using programming, face recognition and in real-time “activity” to address the topic and to bring reflection / discussion
______
Iteration and process:
REFERENCE
Reference I was working with moving further is a project by the studio The Rodina. As a part of the exhibition, the studio created A non-violent exploratory video game addressing the creative labour and extending the visual identity of an exhibition. Players collect labour tokens while their time is mercilessly running out. Landscapes, sounds, and score mechanics reveal the dynamics of a creative worker’s life intertwined with the post-fordist labour performed under the constant pressure of social media and inhumane neoliberal competitiveness. It renders the virtual environment as both liberating and limiting space in which the player can reflect and reimagine the world of work.
_____
ITERATION AND PROJECT IDEA
In order to create my own interactive space, that is playable, I started experimenting and learning with Java programming, studying different resources, and online tutorials and using the help of AI, I created a playable simple game.
Using my skills, online resources, my previous work and references, I programmed an in-real-time interactive space working on a face-recognition system that addresses how the curation of digital personas with a goal to activate the reward-driven loop reflect back to our physical reality.
Players curate their persona in the digital space to collect social points while their time is running out. Only to start again and again.
Based on my research on how our identities are being shaped through digital spaces. It underscores the often irrational, looped nature of digital spaces we get caught in.
It illustrates how these processes mold our behaviour in pursuit of validation within a constructed digital realm while mirroring back the absurdity of our actions (camera on the right). This creates a perpetual cycle of action and reaction, as we continuously navigate and adapt the both realms – the physical and the digital.
I placed the game on a website, that can be accessed by the audience. This was a first sketch and a roughly created idea that I will be developing and shaping further, however I feel like I am finally on track and know where I am heading (aka I don’t feel TOTALLY lost and I am maybe seeing a little light somewhere in the distance :))
TRY IT YOURSELF ON THIS LINK (ONLY PC / LAPTOP): https://logical-reassurance-485194.framer.app/
