Out of the blue, Vanessa Potter lost her sight. As she recovered, her senses mingled – hearing and touch changed the way she saw colours. Her quest to understand why introduced her to new tech that uses sound to help blind people see.
It took just 72 hours for me to lose my sight entirely, and for my hands and feet to feel like they were encased in ice. Just before my blindness hit, I had been laid up with an unknown virus that had left me suffering severe headaches and sweats. My body’s immune system had gone haywire, responding to the virus by attacking my own nerves, causing my loss of sight and mobility – you could say I had been struck down by biological friendly fire.
Going blind was devastating. I hadn’t just lost my primary sense – I had lost my livelihood too. As a television producer, my vision was my job, so I was desperate to see again. After a few weeks, I regained some movement and normal sensation in my legs and feet, but my vision was another matter entirely.
The morning my sight finally started to return, I opened my eyes to a strange, supernatural view. At first I could only make out subtle light shifts, everything was just a swirling grey fog with no perceptible shapes. I was momentarily elated that the world was no longer a suffocating black cloak wrapped around my head – but I realised quickly that I did not recognise anything around me. Over time, black lines started to appear, crudely constructing my visual landscape. These lines delineated windows and doorframes, but little else. Slowly the grey mist dissolved into a brown muddy haze that obscured anything more than a few feet away. Colour eluded me, and my family, padding softly around me at home, were hollow ghosts, skeletal figures with no solidity or humanness. Nothing looked like it should, and my children’s faces hovered agonisingly somewhere behind an opaque screen. I often had no idea where I was, and much of the time my heart was beating wildly in my chest. I wondered over and over if I would ever feel normal again.
© Paul Rousteau at Kiosk
As I recovered at home, colour started slowly creeping back into my life, whispering around corners. This was a very perplexing time, for often I only felt I was seeing a colour but was unable to identify it. I would stare endlessly at trees and lamp-posts, desperate to match the colour I believed was there with the strange sensory experience I was having. Bright primary colours were the first I could identify with any conviction. Red led the way, followed by blues and yellows – but cloudy and faded. I struggled enormously with greens, greys and any pale or muted colours. This was not the vibrant rainbow world I was used to.
Even though my visual world was still predominantly black and white, it felt like colours were talking to me – not literally, but as if my senses were communicating in ways I didn’t understand. Attempting to explain my new relationship with colour only provoked confused silences around me. It made no sense to my family, as they all had fully functioning sensory systems. And when I described it to the neurologists treating me, I was told that nobody knew what was causing this visual disturbance, but that perhaps my sensory system had become cross-wired.
I still had some channels of information transmitting the colours around me to my brain, but I was receiving only part of the message. My lifelong emotional associations with colours were still intact, even if my sight was not. I tried to use language to help myself recover. “You are green,” I would tell the grass. I believed the more I stimulated my brain by observing the world around me and reminding myself what colour was, the more the damaged circuitry in my brain would reconnect and bring my normal vision back online. I found the more I did this, the more it worked.
© Paul Rousteau at Kiosk
I began documenting my recovery daily and started using an audio recorder. I also experimented further with using language, when looking at an object whose colour I didn’t know. First of all I would stare hard and guess what colour I felt it might be. Then, if I was wrong, my husband Ed would tell me the correct colour. If I looked back at the object again and repeated that colour out loud myself, I would often temporarily see it, albeit muted. On one occasion I was able to see a row of different-coloured cable ties holding a gate together by repeating the colours Ed told me were there. Without hearing each colour expressed verbally, I had been unable to see them at all.
It was in this way that my home and neighbourhood became my own personal vision laboratory. My experiences were so outlandish that there were times when I couldn’t believe my eyes. Yet my eyes were undamaged – all of these strange occurrences were happening inside my brain.
My doctors told me I had experienced a rare form of monophasic neuromyelitis optica spectrum disorder, a condition that’s estimated to affect just one in 100,000 people in Europe. One of my primary symptoms had been optic neuritis, inflammation of the optic nerves, which caused my blindness. This inflammation was down to my immune system’s attack on its own body, and similarly led to sensory loss in my hands and feet. It also eventually led to my unusual and curious form of what I now know as synaesthesia – and to my desire to find out more about this condition.
Synaesthesia is a biological condition that causes some people’s senses to cross over, so one kind of sensory response is produced by stimulation of another sense. The name derives from the Ancient Greek words for ‘together’ and ‘feeling’. This sensory intermixing can occur in a multitude of different ways, involving various senses, but most synaesthetes are born with it. Estimates of its prevalence range from 1 to as much as 4 per cent of the population, at least in the UK.
Meet James Wannerton, one of the 4 per cent of people who have synaesthesia.
The condition was first reported over 200 years ago by an Austrian doctor named Georg Sachs, describing his own experience. In 1812 he documented in great detail what colours he associated with certain numbers and letters. During the 19th century it came to be accepted that some people experienced these colour associations, but there was debate over whether this was a condition of the brain or eye. We now know it is entirely neurological – of the brain – as even the thought of, say, a number can elicit its associated colour for a synaesthete. It’s not uncommon for a synaesthete struggling to recall a phone number to ‘see’ their associated colours before they recall the number itself.
One woman I met, Janet*, is what is known as a grapheme–colour synaesthete, as colours appear in her mind’s eye that correlate to individual letters or numbers (graphemes). Janet sees each letter of the alphabet as a particular colour, and for her each letter has a different personality – some sad, others jolly.
Janet offered me a vivid and authentic insight into how, to her surprise, she discovered that she sees the world differently to others. A university administrator, she had been unaware that anything was unusual until her early 40s (roughly the same age I lost my sight). One wintry afternoon in the office, Janet was discussing the arrival of a colleague’s baby. They were talking and laughing about having children, when she mentioned that choosing a name for a baby was so hard, particularly if your spouse chose a name that had a colour you didn’t like. The next thing she knew the room went silent, until one of her friends asked her what she meant.
As Janet explained that she saw individual letters as colours, her colleagues shot puzzled glances at each other. She baffled them further when she told them that the first letter of a word could even put a tint over the rest of the word. Something so obvious and woven into her life was, she found, an anomaly to everyone else. At the time, little was known about her condition, but eventually she found a name for it: synaesthesia.
Janet is a classic synaesthete, having been born with it and never knowing anything different. And while we’ve had many conversations about our shared experiences, I’ve discovered that my version of synaesthesia is much less common, and has quite a different personality.
My illness had been a terrifying ordeal. At the worst of it I had been unable to walk at all, but, a few months on, I was able to wobble unsteadily down the road using a stick.
One afternoon, I hobbled around the streets of south-east London on the arm of my husband. Taking small, tentative steps I forced my rubbery legs to move.
As Ed and I slowly trundled along the road we came across a row of recycling bins. With my burgeoning vision still very distorted I could barely identify the swirling shapes in the distance as bins, but I could identify their colour as blue.
The moment I became aware of the nearest bin, I stumbled forward on my own. Fixating on the lid, I could see what I can only describe as a firework display. I knew I was registering the colour blue, but it was spitting and fizzing like an erratic sparkler. The entire surface of the bin was an unstable, frothing mass. Mesmerised, I limped right up to the bin and reached out my right hand. I’m still not sure why I felt compelled to touch the bin, but I did. Feeling the hard plastic, I spoke the word “blue” out loud. The sparkling stopped immediately, and the bin became a flat, rather lifeless blue. Cautiously I stepped back, but to my astonishment, the blue fizzing started up again. I turned around to Ed, my jaw agape, unsure how to explain what had just happened.
We might have been able to put this down as a one-off hallucination (I was having lots of those as my vision returned) but it wasn’t. I found I could repeat the exercise and the bin would alternate between flat blue or erratic sparkling blue depending on whether I touched it or not.
I started hunting out other objects that might fizz and sparkle – but I quickly learned they had to be blue, and located in front of me. Reds and greens were sometimes flashing up interchangeably (which was baffling), but no other colour responded in this luminescent and explosive way. Blue gates, parking signs and even people’s coats took on a new fascination for me. The moment I looked at them those blues started glittering and erupting. For a time any bright blue object on the horizon would fizz and spit erratically with what was becoming a familiar glow. Through my touch, the effervescence would calm, become stable and inert, and when I stood back from it, the fizzing would light up again.
My walking companions often watched, mildly embarrassed, as I fumbled my way down strangers’ driveways to touch their front doors and softly whisper “blue” to objects in their gardens.
I came to call this brief time during my recovery my “blue period”. I was seeing something that, as far as I knew, nobody else had ever experienced. It became a deeply personal and emotional journey, and those blue sparks offered me a glimmer of hope that my visual system was repairing itself – albeit in a strange and unique way.
© Paul Rousteau at Kiosk
There is no absolute explanation for my blue period, other than that my brain had suffered a huge trauma and was continually adapting to stimuli around me. After a year of recovery I had restored my mobility and had sufficient sight to get by. My life resumed, and I was determined to tell my story – even if it meant typing in 28-point font. I reinvented myself as Patient H69 and launched a blog that would later become a book. And my fear about losing my sight was replaced by an unquenchable curiosity that drove me to seek out doctors, scientists and synaesthetes to understand what was happening to my brain. I accumulated hours and hours of recordings on my Dictaphone.
Acquired synaesthesia, as it is known, develops temporarily during a person’s lifetime, often provoked by hallucinogenic drugs like LSD or magic mushrooms, brain injury or, as in my case, sight loss. Researchers investigating nine cases of acquired auditory–visual synaesthesia, in people with optic neuritis, found that different sounds produced some kind of ‘seeing’ sensation such as simple flashes of white light and colourful flame-like hallucinations or wavy lines.
In my case, I had likely experienced two forms of acquired synaesthesia: one connecting colour and touch, and one connecting colour and spoken language. It is clear, however, that mine was an unusual case. In classic forms of colour synaesthesia there is no intelligible link between a certain stimulus (such as one of Janet’s letters) and the colour sensation it provokes. But for me, colour words were helping me see those same colours.
And while touch was stimulating a response for me, this was about stabilising the colour that was already there – not creating one.
The more I learned, the more questions I had. Had I unwittingly harnessed some kind of unconscious synaesthetic intuitions to boost my recovery? Had these coloured feelings helped me see again? Had synaesthesia helped anyone else to see too?
After I had recovered sufficiently, I was put in touch with Giles Hamilton-Fletcher, a researcher at Sussex University who specialises in synaesthesia and blindness. He is also a part of the UK Synaesthesia Association, a volunteer-run charity that connects synaesthetes, researchers and the media, with the aim of better understanding this extraordinary condition.
Giles invited me to the 2017 Open Senses Symposium at the University of London. There, shouting over the din of delegates, he handed me a piece of red fabric and a pair of headphones, explaining that these were part of a sensory substitution device (SSD), a technology that allows information from one sense, usually vision, to be re-routed through another sense. These clever devices can be used by people with visual impairments to produce some semblance of vision. Pointing at the red fabric, Giles explained that a sensor would translate its colour into an intuitive sound. I experimented moving the fabric around under the sensor, resulting in a high-pitched synthesised sound from the headphones. As I inched the fabric closer up towards the sensor, the sound became more staccato, offering me not just an auditory representation of the colour but awareness of where it was in space.
An SSD like this one converts a visual signal to a sound, which the brain can then learn to convert back into vision. Giles is interested in the relationship between this kind of sensory substitution and synaesthesia. After all, translating one sense into another is what synaesthesia is, in a nutshell. Because of this, SSDs are one of the first technologies to meaningfully demonstrate just what synaesthesia – or at least one kind of it – might feel like to those who have never had it.
Giles and his colleagues are investigating the practical applications of these devices, which are opening up new possibilities for people with permanent blindness.
Daniel Hajas is a physics undergraduate at Sussex and has been blind since he was 16. He first heard about Giles and the SSDs when Giles was looking for blind students to test the devices. Daniel found that one called the Creole could help him access vast swathes of visual, colour-coded data, and open a door back to colour that he had previously thought shut.
In physics, a crucially important image is the cosmic microwave background radiation map, a kind of speckled blue orb, sprinkled with patches of yellow, and red streaks. This colour map signifies the location and intensity of cosmic radiation in the universe resulting from the Big Bang.
© Paul Rousteau at Kiosk
Daniel can’t see it. But, using the Creole, he can hear it.
The Creole works as a stylus with a sensor at the end and a tablet connected to a laptop. On the laptop screen is the radiation map, and by placing the stylus somewhere on the tablet, Daniel can make the Creole produce sound that corresponds to the colour of the equivalent part of the map.
As he moves the stylus, there are erratic beeps and a low droning sound reminiscent of the dialing tone of old telephones. With occasional pauses to register the sound, he can accurately identify the colours – even nuanced shades. For instance, by blending two of the seven main sounds associated with each colour, he can recognise the colour purple. In the same way we would mix paint, Daniel can hear purple as a blend of red and blue sounds. By employing the principles of synaesthesia, Daniel is hearing the colours he needs to see.
Associations between sound and colour are not limited to synaesthetes but exist unconsciously in everyone. Studies by the Sussex researchers have revealed that the brain likes to ‘marry up’ certain visual phenomena with specific sounds. For example, playing notes on a piano, as the notes ascend in pitch, we expect any accompanying visuals to get progressively lighter in colour, spikier in shape, and higher in space. Some of these links even exist in blind people’s brains – and harnessing such intuitions and expectations can provide a gateway into the world of vision.
Giles and his colleagues have shown that SSDs that exploit these synaesthetic intuitions are much easier for people to learn to use. If you build these associations into devices – higher-pitched notes to signify brighter colours, for instance – it can increase the speeds at which people match these colours and sounds up. This way, the SSDs synchronise what we feel should be there with what is.
In our own ways, both Daniel and I learned to exploit an unconscious relationship with colour in a way that benefited us. Our stories are different, but you might say synaesthesia has opened both our eyes.
Several years on, I still have profound colour loss and some visual disturbances. I still think about my blue period – those strange weeks when the world burst into blue flames. While at times it was eerie inhabiting a twilight visual world, there was something magical about it. Few have seen anything like what I have seen. Most synaesthetes are born with the ability, but I have the unique perspective of having experienced a time before, during and after it, which makes the colour I see now all the more poignant.
I am certainly more consciously aware of colour now. I suppose in a way it still speaks to me. My brief time as a synaesthete showed me an innate association that I believe I unthinkingly tapped to boost my recovery. The extent to which my newly found synaesthesia helped my recovery is likely to remain shrouded in mystery, both to me and, for now, to science. However, I can’t shake the feeling that my synaesthesia acted like a magician, pulling back the curtain on the visual world. My brain knew what to do – it was showing me the colours I needed to see, even when the world went dark.
* Name has been changed.
With thanks to Professor Jamie Ward and colleagues at Sussex University and the RM Phillips Network.