Intentionality is the core of human cognition and movement, and Jose Luis “Pepe” Contreras-Vidal is intent on understanding, for all intents and purposes, the neural mechanisms of intention in the human brain. The UH Cullen College electrical and computer engineering professor is using electroencephalography, EEG, to noninvasively record electrical activity in the brains of thousands of human subjects across multiple research projects to develop brain-machine interfaces that benefit populations affected by various affective, cognitive and motor skill deficits.
Pepe’s passion for neuroscience, and consequently his prolific body of work, emerged from his intent to deal proactively with a harrowing experience that occurred when he was a young adult. During his final semester as an undergraduate electronics and communications student at the Monterrey Institute of Technology in Monterrey, Mexico, his mother slipped into a coma that left her unresponsive for an entire year.
General incomprehension among doctors about his mother’s prolonged state of unconsciousness left Pepe feeling powerless. He wondered whether or not she heard him when he talked to her, and desperate for signs that she was still somewhere inside her motionless body, he wondered whether her occasional eye movements were attempts to communicate or just involuntary tics.
He began reading as much literature about the brain and models of the brain as was available at the time, and his doggedness eventually led to academic pursuits. He earned his master’s degree in electrical engineering from the University of Colorado-Boulder and his doctoral degree in cognitive and neural systems from Boston University. He pursued postdoctoral research in computational clinical neuroscience at Arizona State University and the University of Fribourg in Switzerland.
“It’s been a long journey for me,” Pepe said. “I moved to Houston for the support UH is providing and for the medical resources Houston is providing – my colleagues in the medical center – so I can do this work.”
Neuroscientists would do well to study Pepe’s brain in similar ways considering the generous funding that pours into his laboratory and the abundant research that flows out. Since joining UH in 2011, Pepe has earned more than 12 individual and collaborative grants that amount to approximately $12 million. More than 30 undergraduate, graduate and post-doctoral students work in his Laboratory for Noninvasive Brain-Machine Interface Systems at the University of Houston. Students and faculty from around the world visit his lab to acquire training and to conduct collaborative research.
His teams are training brain-machine interfaces to control movement of next-generation upper and lower prosthetic limbs and to instruct exoskeletons to walk for paralyzed children. They are developing neural interfaces to induce neuroplasticity for restoration of the brain’s ability to learn cognitive-motor functions and to reveal mechanisms of imitation and social cognition in normally developing infant brains to better treat dysfunctional development.
In collaboration with visual and performing artists, their interfaces are capturing and analyzing massive amounts of natural brain activity in large populations of participants responding to aesthetic stimuli, creating art in public settings and moving expressively during dance performances.
“My work involves not only research and developing new methods, but also training students – the next generation of engineers – who will continue developing technology and providing specialized system maintenance and novel applications,” Pepe said. “So my grants support the research, but also the training of students and post-doc fellows.”
Balancing art and science for better brain research
Among other definitions, the Merriam-Webster Dictionary defines balance as, “equipoise between contrasting, opposing or interacting elements; a state in which different things occur in equal or proper amounts or have an equal or proper amount of importance; and the ability to move or to remain in a position without losing control or falling.” Perhaps balanced is the best way to describe the Renaissance, a period of immense innovation in world history that struck a balance between art and science.
To ensure a prosperous future for America, leaders in art, science and education fields are pushing for another period of innovation with efforts to convert STEM, science, technology, engineering and mathematics, into STEAM, with the addition of arts to education. In 2012, the National Science Foundation (NSF) granted almost $3 million to the Art of Science Learning initiative for a project aimed at fostering innovation through a combination of STEM and arts-based learning. The Rhode Island School of Design amassed support to launch the STEM to STEAM Initiative, which prompted the 2013 bipartisan Congressional caucus held in Washington, D.C., to discuss ways to implement STEAM education.
“I argue that art and science are on a continuum in which artists work with possible worlds whereas scientists are constrained to working in this world,” wrote physiologist Robert Root-Brownstein in “The International Handbook of Innovation.” “But sometimes perceiving this world differently is the key to making discoveries. Thus, arts and sciences are on a continuum in which artistic thinking produces possibilities that scientists can evaluate for efficacy here and now.”
Leonardo da Vinci, an artist, scientist, architect, engineer and inventor, best embodied the Renaissance movement, which began in Italy in the 14th century and spread throughout Europe for 400 years. His masterpiece, “The Last Supper,” a mural painted on an interior wall of a convent in Milan, depicts immediate reactions of the 12 apostles to Christ’s announcement about imminent betrayal by one of them. Da Vinci described those reactions as “motions of the mind,” according to the Metropolitan Museum of Art website. The genius intended to provide outward interpretations of his subjects’ inner minds with his artistry, which is echoed five centuries later by scientists intending to reveal relationships between behavior and brain activity with complex modeling systems.
Important inventions such as the printing press, compass and telescope emerged during the movement, as well as prestige and financial support for visual artists, musicians and writers. For the first time, philosophers made it acceptable to challenge longstanding beliefs, artists used mathematics and geometry to achieve perspective and proportion in their compositions, and scientists established mathematical relationships with the natural world.
Specialization during the last century has widened the schism between art and science so much that intellectuals have trouble straddling the divide, according to a 2005 Nature journal editorial by Alison Abbott and Adam Rutherford.
“It is hard to find today a true artist–scientist like Leonardo da Vinci, as noted for his science and engineering skills as his ‘Mona Lisa’ and ‘Last Supper’,” the authors wrote. “But in the past decade there has been an increasing awareness on the part of some artists of the heritage of scientists and vice versa.”
Albert Einstein did not credit logic or mathematics for his insights, but rather intuition and inspiration, according to a 2010 Psychology Today article by Root-Brownstein and his wife Michele Root-Brownstein.
“The greatest scientists are artists as well. When I examine myself and my methods of thought, I come close to the conclusion that the gift of imagination has meant more to me than any talent for absorbing absolute knowledge,” the authors quoted the musically-inclined Einstein saying. “All great achievements of science must start from intuitive knowledge … At times, I feel certain I am right while not knowing the reason.”
During the 19th century, Spaniard Santiago Ramón y Cajal, known as the father of neuroscience, also balanced artistry and logic for innovation. The scientist, who possessed a talent for illustration, used Golgi’s method, a silver staining technique for observing nerve tissue under a light microscope, to accurately draw the intricacies of thousands of neurons, which he compiled in a journal. He discovered that each neuron is an autonomous unit, and he established the law of dynamic polarization – information flows in one direction through a neuron, from the dendrites, through the cell body, to the axon.
At the University of Houston, Pepe is reestablishing the balance between art and science for exploration of the human brain. In collaborations with conceptual artist, Dario Robleto, and UH dance faculty member, Becky Valls, he is mapping activity in brains of both expert and amateur observers and creators of art to determine neural bases for their creativity and insight. The U.S. Food and Drug Administration is supporting their efforts.
“By recognizing signatures in the brain for creative processes, maybe we can help people reach those levels of innovation through assimilation training,” Pepe said. “Understanding what it takes to innovate and think outside the box could also be very useful for education and for engineering new technological innovations.”
Pepe’s research also can assist efforts to reverse engineer the human brain and to develop advanced therapies and medical applications. For example, art therapies are known to help patients with mental and movement disorders but researchers cannot explain the reasons. Patients with Parkinson’s disease, which causes poverty of movement, can move normally to rhythmic music of marching bands, yet the mechanisms that allow external music to bypass their internal motor pathways are inexplicable.
Furthermore, researchers do not understand additional positive effects they observe in Parkinson’s patients who dance with partners. Many believe reward and affective signals are important aspects of these benefits. Most likely, creativity shares commonalities in the brain across different domains such as visual art, music, language and dance, but to confirm this, researchers must map regions of the brain engaged during these aesthetic experiences.
“There are many ramifications of understanding this process that have been neglected for a long time because it was hard to quantify, hard to measure,” Pepe said. “But now we have technology, algorithms and partnerships, so it’s the right time to start asking questions that were unthinkable just a few years ago.”
Art museums moonlight as laboratories for brain research
Driving to an art exhibit, an individual might crank up the radio’s volume when an old song stimulates memories of someone from the past. Perusing the exhibit, that same person might stop at a particular sculpture that stimulates a sense of camaraderie with its creator. Engaging with an interactive piece, that individual might repeatedly navigate a virtual reality that stimulates motivation for some reason.
Graceful or jarring dance movements, understated or electrifying theatrical performances, and budding or shriveling landscapes can provoke both common and different responses among observers. The mechanisms in the brain behind these aesthetic experiences are not understood. To study human brains operating under innumerable influences such as food, stress, mood, medication, genetics and personal experiences, Pepe and his team must mine data from thousands of freely behaving individuals across wide ranges of demographics including gender, age, occupation and educational background.
“If we want to understand how the brain works, we need to capture all of those aesthetics, and that’s very difficult in a lab with just a few volunteers,” Pepe said. “So we need to go outside the lab to public places.”
Last fall, Pepe partnered with the Menil Collection to record electrical signals in the brains of 450 individuals as they engaged with the work of artist Dario Robleto in a public art installation. The demonstration led to a $300,000 grant from the NSF, in support of the BRAIN Initiative, to further explore individuality and variation in neural activity among large and diverse groups of people, including children, who are experiencing aesthetics at the UH Blaffer Art Museum and other venues.
“Having the opportunity to collaborate with our colleagues from the arts provides a very constructive way of understanding the process of creativity and uncovering in very large numbers of people any patterns associated with enjoyment, pleasure and beauty,” Pepe said. “For the first time, we have a way to capture activity in action and in context, in a real public setting, from many, many people from different backgrounds and demographics. That provides very valuable information that we can use to learn a little bit more about the brain and how the brain responds, in this case, to art.”
Robleto’s installation at the Menil Collection, “The Boundary of Life is Quietly Crossed,” combined sculpture, historical recordings of heartbeats and brainwaves, and objects belonging to the museum. The inspiration for the exhibit originated with an early version of an artificial heart that Robleto studied as an art-research fellow at the Smithsonian Institute in Washington, D.C, which led to early recordings of heart and brain activity and interesting associated stories. On his artistic journey into science, Robleto discovered the recordings of brain waves and heartbeats belonging to Ann Druyan, executive producer and writer for the television series “Cosmos: A Spacetime Odyssey,” just after she fell in love with Carl Sagan, famous cosmologist and author.
Sagan included the recordings on the Golden Records that were among other contents in a time capsule launched beyond Earth’s atmosphere aboard the Voyager 1 space probe in 1977. Robleto found the consummation of art and science in NASA’s project to reach unknown expanses of the universe with an encapsulation of humankind. Although technology to detect emotion in EEG and EKG recordings did not exist on Earth, Sagan and Druyan hoped that some form of extraterrestrial life might find the Golden Records and have the means to decipher human love in her brainwaves and heartbeats.
Almost 40 years later, the idea of extracting emotion from such recordings did not seem so outlandish to Robleto, and his search for technology led to Pepe. Although the large-scale study is still underway, Pepe and his team developed computer algorithms to analyze the brain recordings in a subset of the 450 participants who found one particular work of art most pleasing.
In unison with timing of their activities captured with local positioning devices, their answers to questions about their perceptions of the artwork and their demographics, the team mapped the neural networks engaged while the participants experienced aesthetically pleasing visual art. Appropriately, Robleto’s scientifically inspired artistic installation and Pepe’s artistically inspired scientific study crossed boundaries. Pepe recently submitted the first of a series of papers, which is currently in review.
“Beauty is in the eye of the observer, it’s very difficult to define that,” Pepe said. “We want to see if there is a pattern by looking at the brain activity generated at that moment – it will not lie to you, it’s very dynamic. It’s going to use sensory information, prior experience, and situational and emotional context as ways to generate these sensations, these judgments.”
Pepe’s groundbreaking Menil Collection collaboration led to funding to expand his research. With a recent NSF grant, he is partnering with the UH Blaffer Art Museum to map brain activity in thousands more subjects fitted with EEG skullcaps as they respond to still art. With an interactive exhibition, he also intends to explore patterns in the brain related to the process of creating art.
“We all create at some point, but we don’t under - stand the mechanisms that make some more crea - tive than others – the people who innovate and see things in different ways,” Pepe said. “We need a permanent collection where everyday people can go to contribute their data to science and be part of the exhibit.”
Pepe and his team are extending their neuroaesthetic studies to other venues like the Children’s Museum of Houston as well as to other artistic mediums, such as dancing.
Dancers open their minds for advanced understanding of expressive movement
Pepe is no stranger to visual and performance art. When the neuroscientist is not developing algorithms to analyze complex communication in the brain, he is designing pieces of contemporary stained glass and practicing the expressive movements associated with Spanish Flamenco dancing. He selected dance for his first dip into understanding brain mechanisms involved in aesthetic experiences.
In a 2014 collaborative paper that published in the Frontiers in Human Neuroscience journal, Pepe explored brain activity in dancers modulating functional movement to express messages. The partnership with dance professor Karen Bradley at the University of Maryland led to a collaborative demonstration with Becky Valls at the UH School of Theatre and Dance earlier this year. This semester, Pepe and Valls are expanding their partnership to study brain activity in dancers through monthly performances on the UH campus.
“Dancers want to communicate intent, an emotional state or a conceptual idea with expressive movement, so it goes beyond functional movement like walking, which has a purpose in terms of action,” Pepe said.
For the University of Maryland project, Pepe fitted professional and amateur dancers with EEG skullcaps to record brain activity while they danced, and he developed computer algorithms to determine patterns of brain signals associated with specific expressive movements. He and his team determined 16 patterns related to different dance efforts, such as feeling light or heavy and moving quickly or slowly. By the end of the study, they could successfully predict the dancers’ movements based on their brain activity alone.
“We were able to match the grammar of expressive dance with the grammar of emotional brain activity,” Pepe said.
In February, Valls sported an EEG skullcap when she performed her mesmerizing composition, “Red Square,” in the Jose Quintero Theatre on the UH campus. As Ravi Shankar’s music played, she danced against a backdrop featuring projections of her brainwaves. Pepe used patterns of Becky’s emotional brain activity to modulate the stage lighting in real time, so the environment evolved with the internal states of Valls’ brain.
This semester, Pepe and Valls are hosting dance performances each month in the atrium of the UH architecture building. Pepe is recording brain activity in three types of dancers – the choreographer, the professional dancer and the amateur dancer – as they perform the same composition. He and his team are developing computer algorithms to analyze the differences in brain activity between deliveries of the performance art by dancers with varying levels of skill and creative input. Neuroaesthetic studies of dance and visual art offer potential for understanding and developing therapeutic art treatments, but also provide potential for creating new types of media art installations and for teaching the arts.
“These are collaborations at the boundaries of art, neuroscience and engineering that could tell us a little more about the process of creating art – creativity – the process of understanding art, making judgments about art, and understanding how that processing, those judgments, are affected or modulated by your cultural background, your past training, your age and your life,” Pepe said. “This is still an open question, and we’re just starting, so we’re very excited about the initial results we’re getting.”