General Information

Mail: University of Houston
Cullen College of Engineering
E421 Engineering Bldg 2, 4722 Calhoun Rd, Houston, TX 77204-4007
Map & Driving Directions (includes parking information)
Email: info [at] egr [dot] uh [dot] edu

CULLEN COLLEGE OF ENGINEERING

University of Houston Cullen College of Engineering

News

Professor Joe Francis’ Mind is Teeming with Ways to Team with Robots

Printer-friendly versionSend by emailPDF version

By: 

Laurie Fickman
In a galaxy on UH campus, Professor Joe Francis is perfecting Jedi mind control
In a galaxy on UH campus, Professor Joe Francis is perfecting Jedi mind control

To the unfamiliar, it might seem a Jedi mind trick – the notion of controlling robots by simply thinking of what you wish for them to do. If so, then Joe Francis, associate professor of biomedical engineering, is Obi-Wan Kenobi. The force, along with a new grant from the National Science Foundation (NSF) for $308,077, is with him as he works to expand humans’ ability to work with robots.

To Francis, his project, “NRI: Collaborative Research: Multimodal Brain Computer Interface for Human-Robot Interaction,” is all about teamwork. “We’re teaming the human with the robotic systems,” he said. “The robots can do certain things really well, but they’re going to need some guidance, and rather than have the person guide them continuously with trying to inform them about everything, there’s this interplay between the two systems.”

The goal is to determine how well artificial agents and humans can work together instead of getting in each other’s way.

It’s personal

Once that teamwork is improved, Francis believes it and other research he is conducting will change people’s lives by improving sensory prosthetics. His devotion to this belief is a deeply-felt cause, one he’s probably had since first grade.

That was the year he lost the big toe on his left foot in a terrible accident, and he immediately found ways to overcome the loss, excelling at sports quickly and all the way through college. Sure enough, he helped himself profoundly with his determination. Now he’s set his mind on doing the same for others.

“If somebody is blind, I want them to be able to see. If somebody can’t hear, I want them to be able to hear. If they don’t have an arm, I want to give them an arm and be able to control it naturally and I want them to feel it as if it was a part of them so they don’t feel like they’re using this tool, but they feel whole again,” said Francis.

That’s where focus shifts to the brain. Current human robot interaction (HRI) centers around video, keyboards and, plainly, just speaking to them – telling them what you want. The world Francis envisions makes robots part of everyday life and our communication with them second nature. That’s also the idea of co-robotics, as expressed by the NSF, which acknowledges humans as part of a larger robotic control system.

Coordinating the team

Francis works to take information from humans and give it to robots so they can anticipate what humans want in the future. He does this by recording human brain waves through EEG and brain hemodynamics, or blood changes, through an imaging process called Functional Near-Infrared Spectroscopy (fNIRS).

“From that we can determine how active different brain regions are,” said Francis. He also collects data by watching the eyes and recording them with high-speed cameras since pupils actually change size when certain rewarding or punishing events happen.

Unexpectedly, the researchers can also determine a reward/expectation signal from a person’s brain.

“We can basically tell if they think something is leading to a reward or something good,” said Francis.

Once that human data is collected, Francis takes it to improve the software for his robots and improve their learning agent.

He’s robotic, too

In the old television series, “The Six Million Dollar Man,” the lead character had robotic and bionic limbs. Francis doesn’t have any of that, but you can call him the six million dollar man, too, because his robot research has brought in just about that amount of funding since arriving at the UH Cullen College of Engineering in March.

In the words of Obi-Wan, “I have the high ground."

Faculty: 

Department: 

Related News Stories

Cullen College Student Spreads International Compassion and Kindness to Needy

When he's not out in the world assisting those in need, Pietro Cicalese is usually in the M.D. Anderson library studying.

In the summer of 2015 UH biomedical engineering student Pietro Cicalese found himself surrounded by astonishing squalor in Haiti. The area was densely populated yet there was no water or medical resources for the sick. And the houses – if you could call them that – were shacks.

“I have never seen anything like that. It was very eye opening to see poverty of that scale,” said Cicalese.

William A. Brookshire Leaves Legacy of Generosity and Dedication to the Cullen College

On May 19 at 1:30 p.m. Dr. Brookshire’s family will host a memorial on the UH campus at the Student Center, Houston Room. Friends and family are invited.

UH alumnus William A. Brookshire, Ph.D. (BSChE '57), co-founder and chairman of the board of S&B Engineers and Constructors, died on April 21, 2017.

Joseph Tedesco, Elizabeth D. Rockwell Dean of the Cullen College of Engineering said, “Dr. Brookshire was a fine gentleman and one of our great philanthropists, always in search of new ways to help students and professors. We will forever feel his generosity and his loss in equal measures.”

Upcoming Events / Seminars