Numbered diagram of explorations in a mirror space showing 4 planes of interaction
Rachel Jacobs Research

Reflections in the Mirror Space

I am currently conducting a (virtual) artist residency as part of the TAS programme, working alongside the TAS for Health project, that is exploring the potential for diagnostic interactive mirrors that could be placed in the home for people with conditions such as alzheimers, MS and strokes.

Background: The work I am conducting follows on from the Invisible Project that took place in 2017 https://invisible.wp.horizon.ac.uk/. This was an international collaboration between myself, researchers at MRL, Primary Studios (an arts centre) in Nottingham, and the British Brazilian artist Silvia Leal and the Flor de Pequi collective (an arts and community centre) at Guaimbê, in Pirenópolis (Brazil). The project resulted in the creation of several prototype performative interactive mirror experiences that were developed to connect the two places, across the UK and Brazil. We wrote a research paper about our findings from this research ‘in the wild’, focusing on the context and experience of interacting with the mirrors, outlining a series of design strategies for the future development of performative interactive mirror experiences [5].

See image above: Setting the Stage, Jacobs et al. (2019)

TAS Artist Residency

I have started this new residency by conducting a process of research and exploration, whilst getting to know more about the TAS for Health project and how the two projects might inform each other. My previous work with interactive mirrors had left me with several unanswered questions, that I have been taking time to explore.

Firstly, Why use an interactive mirror? Whilst my previous work focused on what we might do with an interactive mirror within a performance or theatrical context I felt I had never completely understood the mirror as an object, or technology, in its own right. Similar types of interactions can take place using interactive projections with a Kinect or a simple camera and facial recognition software, as I will explain later, and I couldn’t really understand how much the overlaying of a semi-transparent mirrored surface changed the interactive experience – beyond the ability to create additional performative spaces around the mirror (as shown by our diagram above, for setting the stage with an interactive mirror). Even with our findings on the space in front of, within and behind the mirror I felt there was not that much new for my artistic practice that hadn’t been previously explored through previous research into presence, projection and performance by artists and researchers such as Paul Sermon [8], Janet Cardiff [6], Johannes Birringer [7] and Gabriella Gianacchi [4].

Image of Paul Sermon's Telematic Dreaming, a woman lying on a bed reaching out to another woman projected on the other side of the bed, both women are looking at each other

Telematic Dreaming, Paul Sermon 1992

Secondly, in response to the TAS for Health project I have become very interested in exploring whether interactive mirror experiences can become a meaningful and trustworthy addition to people’s lives? Specifically in response to the now well known concerns around the built in biases in the facial recognition software often used in interactive mirror experiences, and the inability for tracking software such as the Kinect to detect different bodies, mobility and movement, skin colour, hair and body shape.

These concerns have led me to go right back to the beginning of the history of mirrors and explore a series of questions:

What is unique, attractive and novel about the mirror object on its own terms?

How have different people and cultures around the world traditionally interacted with mirrors?

How does the analogue technology of a mirror make this different from other technologies (e.g a phone, a camera, a screen)?

What happens when we layer data capture and interactivity onto each of these existing ‘analogue’ technologies?

How does this make the interactions different – and therefore the trustworthiness of these interactions?

Finally, through a parallel artistic practice-led process I am hoping to explore how the physical, conceptual and material context of the mirror disrupts or gains our trust in what is reflected and how this impacts on our sense of self, through our interactions with our ‘mirror self’.

The Mirror Never Lies: A Review of In The Looking Glass, Shrum, 2017

Mirrors, as we think of them now, are relatively new technologies. Yet reflective surfaces have held an important role in human cultural and collective consciousness, referred to in myths, stories and drawings from the first humans, (as a) key to ritual, death rites and sacred activities, used by healers as a way of treating and diagnosing people, as well as providing a developmental tool for our ability to distinguish and recognise our public selves in relation to the ‘other’.

Shrum [11] has explored the evolution of mirror technology and the cultural context of the object, its materiality and use. The following summary hopefully provides a useful description of how people traditionally interacted with mirrors, in view of the work we are doing on interactive mirrors today.

It wasn’t until the sixteenth century in Europe that Venetian craftsmen developed an ‘accurately reflective looking glass’, what we now know as a ‘mirror’, by developing the technology where a surface could accurately and clearly reflect what was in front of it. Prior to this the surface of a mirror was often metal and not flat. In the twelfth century, the technology evolved to use glass surfaces, often convex and rarely flat. The first accurately reflective looking glasses were made using high levels of craftsmanship, creating a near perfect flat, clear glass surface with a quicksilver back. Further evolutions of the technology developed the reflective coating, size, shape and materials.

Mirrors and reflective surfaces had been used for health and divination, prior to the development of accurately reflective looking glasses. Different styles of mirrors – hand mirrors, pocket mirrors, mirrors with candles in front of them, and earlier sword blades and water had different uses and types of distortions to aid diagnostics or divinations. When mirrors were made of metal and other materials that distorted the reflection ‘practitioners frequently claimed to be able to see things in mirrors that were not physically present before them or to find a connection between this world and the world beyond in those veiled reflections’. In the 1600s ‘another meaning for a looking glass was a chamber pot or urinal, used by doctors at this time to search a patient’s urine for signs of disease’. Existing practices of divination and medicine, through reflective surfaces, have been seen to be held by Europeans, Native Americans and in Africa, whilst mirrors were also used in burials ‘with the powers of deflection’ of evil spirits.

It took a long time for the new ‘accurately reflective’ mirror technology to become accessible around the world and did so alongside colonisation, in Asia, Australasia, Africa and the Americas. Mirrors were traded for land, fur and influence. Once accurately reflective looking glasses became available around the world people gained the opportunity to fully discover their ‘mirror selves’ for the first time.

Shrum suggests that one of the liberating ways that people from all backgrounds were able to interact with mirrors were as ‘items that could be used without having to be touched and used more widely than just by their owners’. This was particularly relevant to African Americans in slavery who were able to ‘make use of mirrors in their master’s house, if they had access to them’ – providing an opportunity to come to know their mirror selves without intervention from the slave owners.

Mirrors were given as gifts to Kings, Chiefs and other powerful and wealthy people as Europeans travelled, providing opportunities to flatter and aggrandise through these gifts. An example of this is a description of the King of Igbo who was given a chair by a European. He then called for a looking glass (gifted by a previous visitor). The looking glass appeared to be critical for the king to observe himself in the chair, laugh, feel generally pleased and ‘incorporate the chair into his understanding of himself’.

Conversely, it appears that for many African American slaves it wasn’t until they were freed that they could have a relationship with their mirrored self, as Millie Manuel said ‘I didn’t knowd what I looked like ’til I was free’. Whilst possibly owning your own mirrored self was also seen as a symbol of freedom, rather than relying on opportunities to glimpse your mirrored self through the mirror owned by another, particularly a slave master.

The concept of a mirrored self is key to an understanding of ‘trustworthy vision’. In the 19th and 20th century, vision for Europeans was seen as the ‘noblest sense’, connected to the idea that the mirror self was an accurate depiction of their actual selves and the world. Wealthy Europeans relied on accurately reflective looking glasses to provide key information about their ‘public selves’ becoming ‘trusted visual tools in gaining knowledge about, and exerting mastery over, many aspects of the personal front’, this included mirrors that reflected the architecture and environments that they owned.

The concept of the mirrored self was also gendered. Wealthy European women often appeared to substitute mirrors for the ‘male gaze’, whilst providing an element of agency over the expectations of how they should look, their marriage prospects and how they should behave. Some women wrote about how they were able to use mirrors to ‘exorcise emotions that should not be exposed in public’. Whilst men’s relationships with their mirrored self tended to differ, not necessarily having a sense of judgement over the mirrored self against a standard, as with women. It appears that men tended to focus more on the practicalities of grooming and shaving, using the mirror as a tool to extend vision (for instance, attached to a gun) and as a tool for diagnosing injury, ill health and closeness to death – particularly when hand mirrors were taken onto a battlefield.

Despite the accurately reflective looking glasses providing a sense of an accurate representation of the self, beliefs about reflective powers have remained in superstition, folklore and divination rituals. People still believe that mirrors that are broken will bring about bad luck, and turn mirrors to face the wall or cover them at the time of a death.

Most interestingly for our research into trustworthiness, mirrors were often seen in Europe as a metaphor for truthfulness.

A Trustworthy Mirror

Is data capture the obvious next step towards a truly accurate reflective surface – or a step back to the occluded mirrors of the past?

How can people trust a mirror that is potentially capturing and analysing data about what is reflected? How does our relationship with a mirror change when we become aware of it not only reflecting back ourselves and the world but acting as a capture device, and potentially a diagnostic tool with another hidden ‘viewer’ intervening with the reflection?

Historically the mirror’s evolution is based on the search for an ‘accurate reflection’. The move away from the mythical, sacred, ritual, spiritual and medicinal uses came as reflections became clearer and more accurate.

The most recent developments in this technology are possibly harder to understand within Shrum’s historical context, moving away from the desire to create the most accurate reflection, an interactive mirror begins again to disrupt, hide and reveal.

When the reflection is occluded then there is room for imagination, for other entities and possibilities to exist within the frame, opportunities for disruption – for the mirror to tell us more than what is shown on the surface. When the reflection is accurate, the surface is flat, there is no longer room for other possibilities, we begin to believe the reflection – the mirror never lies.

So what happens to our trust in the mirror’s ‘accurate reflection’ if hidden within it is a capture device, and behind that another hidden viewer (possibly making diagnosis or judgements on your reflection). Yet none of this is present in the mirror’s reflection.

How does this impact on our trust in the image in the mirror?

You Talking to Me?

When we look directly in the mirror we see ourselves. What happens when the reflection of ourselves is disrupted… and we discover someone else is watching?

In 1967 Guy deBord described society as increasingly a spectacle of the self [3].

 

In 2002 the documentary film maker Adam Curtis created a series called ‘The Century of the Self’ [2] expanding on DeBord’s concept to suggest that twentieth century culture encouraged a belief in self actualisation that created new ways for governments and corporations to maintain political and economic control, through the emphasis of our individual desires over our personal and collective needs.

https://www.bbc.co.uk/iplayer/episodes/p00ghx6g/the-century-of-the-self

In 2008/2009 my artist collective Active Ingredient developed two interactive installations – The Fantasy A-List Generator and You Talkin’ To Me? These works responded to this hypothesis, as we also witnessed an increasing obsession with selfies and celebrity culture.

The Fantasy A-List Generator, Active Ingredient, 2008

In each installation a small photo booth was set up, one in an art gallery and one in the reception area of a media centre. Once inside people could sit on a seat facing a screen that acted as a mirror. The person’s image captured by a camera above the screen was embedded within an interface that played out once the participant pressed a button next to the screen.

In the first installation people started by choosing a celebrity name. They were then invited to dress up as this celebrity character, using the clothes on a rail that was placed within the booth. When they were ready they pressed the button. In this case the interface was designed to look like a TV interview, with the interviewee’s name appearing on the screen and a series of questions appearing in a ticker below their image. The questions were taken from real TV interviews with the celebrity. In this case the participant was invited to respond with their own answers. A timer appeared giving them a countdown to limit the time they had to answer each question. Once recorded the video was projected onto a large screen outside the video booth, in the public gallery, and was added to an online database that could be viewed via the project website.

In the second installation the screen was made to look like a distorted mirror, the video camera capturing the image had a fish eye lens and the mirror was high up so the person had to look up into the mirror. Again the participant was invited to dress up. In this case the question ‘you lookin’ at me?’ was posed, to encourage people to talk directly to the person they encountered in the mirror – that was at once themselves but not themselves.

Both artworks played with the concept of the ‘accurate reflection’ in the mirror, how a mirror not only reflects our image but also our sense of identity, disrupting the distinction between our private and public/performative selves. In both cases people were asked to enter the booth on their own or in small groups, creating a totally private space, the interface only reflected the person/people in the booth.

In both installations people appeared to forget that the videos would be made public. All kinds of people entered the booth; old ladies with their shopping bags; parents and their children; we even discovered the security guards at the gallery had done secret recordings after the gallery was closed. They dressed up and spoke to the mirror, seeming to lose their inhibitions once the door was shut and their costume was on. Creating a form of confessional.

The first installation allowed people to imagine themselves into a purely performative sense of self, taking on the character of another ‘famous’ person to become a member of society’s ‘A-List’.

The second installation played with the idea of losing yourself so much that you disassociate from your own reflection. This is a classic trope used by film directors to indicate a protagonists descent into madness. We based this second installation on the scene in Taxi Driver [9] when the character Travis Bickle (played by Robert De Nero) threatens himself with a gun in the mirror and asks himself ‘You talkin’ to me?’

We were also influenced by the film Donnie Darko [10], where the teenage Donnie sees Frank the terrifying rabbit in the mirror. In this case both the reflection and the surface of the bathroom mirror lose their trustworthiness. Donnie is able to create ripples on its surface with his hands and Frank responds in the same way. Similarly in the installation by using a fish eye lens and raising the mirror, we changed the sense of an accurate reflection.

This cinematic trope leads us back to the historical vision of a reflective surface as a portal to another world – the mirror in Alice Through the Looking Glass – and the potential for the occluded mirror to reveal that which we cannot see.

These two works explored both the positive and negative possibilities for an accurately reflective surface that hides an invisible presence or audience, as well as encouraging us to capture and reveal something that is hidden or unexpected within ourselves, below the surface of the spectacle of the self.

Prototypes

Can these types of interactive mirror experiences help us become more of ourselves, build a more accurate and healthy sense of self, teach us to recognise our healthy, actual selves and diagnose when there is a problem in order to fix and transform the self?

Or will our endless desire to capture data and be part of a spectacle, even in our most private moments, take us further into being controlled by algorithms, ‘all watched over by machines of loving grace’ as the poet Richard Braughtigan wrote [1], leading us into the madness of Travis Bickle and Donnie Darko – a world of disassociation, paranoia, narcissism and distrust?

I am now starting work on a series of small prototype mirrors, based on the early non-accurately reflective pocket mirrors and hand mirrors, exploring ancient, mythical, superstitious and instinctive uses of reflection alongside meditative experiences, linking back to my previous research in the Invisible Project on performative experiences, narrative and movement.

I will also start work on an app that could sit on the TAS for Health interactive mirror, that explores and disrupts our ideas of trustworthiness and designing a prototype for an interactive pocket mirror connected to a mobile phone, exploring how this links to wellbeing, trust and distrust. It is my hope that these prototypes will also act as an aid for TAS for Health workshop participants to consider trustworthiness and the significance of a mirror as a diagnostic tool, as well as exploring how an interactive mirror can support users wellbeing – beyond being a diagnostic tool.

Bibliography

[1] Braughtigan R., 1967. All Watched Over By Machines of Loving Grace

[2] Curtis A. 2002. Century of self. London: BBC Four;

[3] Debord G. 2021. The society of the spectacle. Good Press;

[4] Gabriela Giannachi, Nick Kaye, Michael Shanks (Eds.). 2012. Archaeologies of presence: Art, performance and the persistence of being. Routledge.

[5] Jacobs R, Schnädelbach H, Jäger N, Leal S, Shackford R, Benford S, Patel R. The performative mirror space. InProceedings of the 2019 CHI Conference on Human Factors in Computing Systems 2019 May 2 (pp. 1-14).

[6] Janet Cardiff. 1999. In Real Time. Retrieved 18 September 2017 from http://cardiffmiller.com/artworks/walks/realtime.html

[7] Johannes Birringer i-Map Retrieved 8 December 2021 https://interaktionslabor.de/lab06/preng06.htm

[8] Paul Sermon. 1992. Telematic Dreaming. Retrieved 18 September 2017 from http://www.paulsermon.org/dream/

[9] Martin S, Scorsese M. 1976. Taxi Driver. Columbia Pictures Corporation-Italo/Judeo Productions.

https://www.youtube.com/watch?v=4e9CkhBb18E

Retrieved 08/12/2021

[10] Kelly R, 2001 Donnie Darko, Flower Pictures.
https://www.youtube.com/watch?v=aAOthCk-QsI

Retrieved 08/12/2021

[11] Shrum. R. K, 2017. In the Looking Glass: Mirrors and Identity in Early America.