Drawing out the visual richness of our memories

Thumb wilma
Jan 28, 2019
1
0

In daily life, we are constantly inundated by a flood of information to process, navigate, and remember. We actively seek out this new information, traveling to new countries, enjoying concerts and film, relishing deep exchanges with friends and families. Yet as time passes, our memories transform and we lose pieces of what was once a vivid sensory experience, while retaining others. It’s hard for us to predict what we will and won’t recall about specific places, and surprisingly little is known about the precise content of our memories. 

Some research suggests that visual memory is fallible and malleable – we can miss large changes in our environment, and our memories can be influenced by misdirection or emotion. At the same time, other research has shown that we can recognize thousands of new images with high accuracy. However, to truly understand the limits and powers of human memory, we have to understand what we are actually remembering: what is the specific content of our memories? 

Until now, studies looking at this question have asked people to verbally describe their memories, but such measures lose the rich visual information we may also remember – for example, the sizes, colors, or relations of objects. If we want to understand the content in visual memories, we need to have a visual task. So, Chris Baker (@Chris_I_Baker), Elizabeth Hall (@eheatherhall) and I (@WilmaBainbridge) decided to investigate whether we could use drawings to uncover people’s memories. When we started, we thought this study would serve as a first, quick, small pilot but when we saw the richness of these drawings, this grew into a much bigger project than originally anticipated. 

We had people come into the lab, and study 30 scene photos for later memory testing. They didn’t know how their memories would be tested, so when we asked them to draw the images from memory, a lot of them grumbled and complained, warning us of their lack of artistic abilities. But, in the end, they did a pretty good job showing us their memories. As an example, here is my attempt to draw a photo of an unfamiliar living room from memory.



You can see there’s a fair amount of accurate detail here that is hard to describe verbally – objects, colors, shapes, sizes, and objects in their correct places. There are some interesting errors here as well – a mix-up of the flower’s color, a misordering of the pillows on the couch. You can also see shortcuts in the drawing, like my very rough attempt at giving texture to the coffee table. It’s clear that this drawing reveals a lot about what’s in the memory, but it’s hard to measure through objective means. The critical challenge we faced was how to evaluate these sketches objectively.

So, we turned to citizen science – rather than us three scientists trying to determine what counts as a successfully remembered item or not, we gathered the intuitions of thousands of people (8,596) for thousands of drawings (2,682). These “drawing scorers” participated through online crowd-sourcing, and they collectively determined a wide range of details about the images: how well the drawing matched its photo, what objects it contained, what errors it showed, and where objects were.  

What we found, reported in our new paper in Nature Communications, was that people were recalling much more detail than we might have originally guessed. When we had people study and then draw 30 real-world images of scenes from memory, they drew over 150 objects across their drawings, with few errors, and with high precision for both the size and position of objects. This level of detail would have been impossible to measure using verbal tasks.  

In the future, drawings combined with crowd-sourced scoring could be used to reveal even further insights into the human mind. They could be used to characterize visual or memory disorders, such as Alzheimer’s Disease. We’ve also taken some initial steps to test computational models for predicting what objects people will recall, so we can better understand what sorts of information we remember and why. We hope more people will think about using drawings as a way to measure mental representations, and we have made all of the drawings and scoring code publicly available for research use.

Now, the flood of visual information in our memories is a little more clear.

Medium wilma

Wilma A. Bainbridge

Postdoctoral Fellow, National Institute of Mental Health

No comments yet.