Song of the Machine: In Depth

Earlier this month, our new Lab Project, 'Song of the Machine' was premiered at Science Gallery's flagship exhibition 'HUMAN+'. The following week, it was featured on the Guardian website, alongside an article by our scientist on the project, Dr. Patrick Degeanaar. You can watch the 'Song of the Machine' film below.

Here's some of the project background taken from our exhibition blurb:

What if we could change our view of the world with the flick of a switch? The emerging field of optogenetics combines genetic engineering and electronics to manipulate individual nerve cells with light. With this technology, scientists are developing a new form of retinal prostheses. Using a virus to infect the degenerate eye with a light-sensitive protein, wearable optoelectronics can establish a direct optical link with the brain. Song of the Machine explores the possibilites of this new, modified – even enhanced – vision, where wearers adjust for a reduced resolution by tuning into streams of information and electromagnetic vistas, all inaccessible to the 'normally' sighted.

In this post, we'd like to share our thinking behind the project, the nature of our collaboration, our design process and how we move forwards.

• Our interest in optogenetics: A major focus of Superflux is to design for the 'imminently probable' - exploring the design possibilities and near-future implications of emerging technologies on people, culture and our environment. So when we met Dr. Degenaar and learnt of his pioneering work in Optogenetic Retinal Prothesis, we were immediately struck by both, the enormous possibilities as well as potential impact of his work. We knew it was something we wanted to explore further.

Opening of Human Plus, Science Gallery, Dublin

Our choice of medium:
The HUMAN+ exhibition at Science Gallery, Dublin set the initial frame for our first collaboration. Given a four-week window for production, film seemed like the most effective medium with which to translate our early conversations into a nuanced, poetic output. From a design perspective, the film serves as an early experience prototype. By probing the speculative potential of a new, modified/enhanced vision, we have created a space for the dialogue that will structure and spur our ongoing work with Dr. Degenaar.

The concept:
Dr. Degenaar's team is currently working on several different trials and prototypes, and they hope to have the technology prepped and ready for use on visually-impaired people in 2-3 years time. Early-stage vision will remained restricted, with low resolution. However, working with the unique combination of genetics and optoelectronics, we began to consider how a potential user might compensate for these limitations. At the core of this discussion was a conscious awareness of the substantial difference between this hybrid optoelectronic system and the 'closed' technology of bionic implants. Degenaar's work relies on modifying the body to better interface with a machine; a sharp contrast with the dominant model of visual prostheses, in which machines are designed to interface with the human body. With this distinction as our starting point, it was clear that the idea of the 'body-as-machine' would be integral to our final design.

 Far infra-red or thermal imaging stills  A white orchid, as captured in the ultraviolet spectrum

• It would have been easy to allow the film's narrative to mushroom, obscuring the affordances and possibilities of the technology. Instead, we tried to keep the story simple. A mundane narrative, ambient cityscape, and offbeat score helped set the scene for exploring the electromagnetic spectrums and visual 'channels' of our user's sensory perceptions. Ultraviolet, infrared, and augmented reality extended his world, with the digital artifacts and noise hinting at the technical limitations of low-res vision.

Stills from the film

Optogenetic retinal prostheses works by translating a visual input into pulses of light that mimic the eye's 'neural song', this gives the user the possibly to determine what – and how – they want to see. Developing the project, we were faced with several questions: How might you choose to 'compose' your vision of the world? How would that affect your sense of the world, and your place in it? What would it mean for your memories? Your dreams? How could you modify your environment to capitalize on these extended senses? In the first stage of the project, we kept our focus tight, concentrating on the potential and practicalities of the technology and the device, but as we move forward, more of the above questions will come into play. While we could have limited our exploration to 1-2 of the possible visual 'channels', instead, we adopted something of a 'kitchen sink' approach. By demonstrating the full range, even when it veered into the more overtly speculative, helped us frame the technology as something greater and more significant than an attempt to restore 'normal' sight.

• The machine; the device; the interface: In this instance, we created a relatively unobtrusive prop as the protagonist's main wearable, with an external LED suggesting the devices live image-sensing and processing capabilities.In the future we will work further with Dr. Degenaar in the design of the actual device, but this will be a full project in itself. This also applies to the handheld device. In the film a set of fiduciary markers allow for the headset to 'project' a visual interface over the device, removing the need for an actual screen but allowing simple gestural actions to transport the protagonist into different spectrums and vistas, as and when he chooses.

The wearable and handheld devices as props

• The photographs:

• Accompanying the film, we included three large photographic prints in the exhibition. The first shows the physical process of injecting the virus; the second represents the infecting of the users retinal cells with the light sensitive protein, made with a composite of images taken though a high-power microscope by Dr. Rolando Berlinguer-Palmini, who is a member of Degenaar's research team. And the last suggests a redesigned visual assessment, undertaken by the user once the device has been fitted.

The practicalities: From the beginning, we were determined to use the appropriate technical equipment to access the spectrums we were exploring in the film. No short-cuts. From hiring the thermal imaging camera equipment from an industrial warehouse in the midlands, hacking camera sensors for infrared access, right through to inviting the scholarly Bjørn, botanist-turned-UV-photographer, over from Norway. Amongst the kindness of friends and strangers, gallons of coffee and cyborg-specific humour, we stretched the possibilities afforded by time and budget. We may not have always made things easy for ourselves, but it has been a tremendous learning experience.

Team composition: Our working team was comprised of a designer-director, designer-techie, futurist, botanist-photographer, scientist, and ethicist. We felt that to effectively develop a project with this level of complexity, the insights and talents of such a multidisciplinary team would prove essential. And we were right.

• Moving ahead: This was just the start of what we hope will be a long a fruitful collaboration with Dr Degenaar and his team. Watch this space.

• 
  
• Credits: We are grateful to Science Gallery Dublin and Newcastle University for supporting the project. Massive thanks to George Grinsted, Bas Raijmakers, Geke Van Dijk, Carolina Vallejo, Tommaso Lanza, Martin Andersen, Sarah Dobbs, Pablo Milara, Liam Young, Dr. Rolando Berlinguer-Palmini and Advanced Camera Solutions, for their help and support.

Couple of screen grabs from the Guardian site where the project was featured:

And finally, some more photographs from our process: