Toward Physical AI: Autonomy, Collaboration, and Language-Guided Behavior in Robotic Systems
This talk presents the technical foundations and system-level architectures in autonomous robotic systems operating in complex, GPS-denied environments. I will begin with our work on micro aerial vehicles capable of agile flight using only onboard sensing, mapping, and planning, enabling operations in forests, industrial sites, and subterranean spaces. Building on this, I will describe our framework for air-ground collaboration, where UAVs and UGVs use shared semantic maps and opportunistic communication to coordinate perception and action in the field. Finally, I will introduce SPINE, a system that enables robot teams to interpret and execute incomplete natural language commands, inferring missing context and replanning on the fly. This tightly coupled language, perception, and planning framework represents a significant step toward Physical AI - robotic systems that understand, collaborate, and adapt in the real world. Together, these efforts chart a path toward intelligent, resilient, and human-interpretable autonomy at scale.
Vijay Kumar is the Nemirovsky Family Dean of Penn Engineering with appointments in the Departments of Mechanical Engineering and Applied Mechanics, Computer and Information Science, and Electrical and Systems Engineering at the University of Pennsylvania. He received his Bachelor of Technology from the Indian Institute of Technology, Kanpur and his Ph.D. from The Ohio State University in 1987. He has been on the Faculty in the Department of Mechanical Engineering at the University of Pennsylvania since 1987.
In addition to holding many administrative positions at Penn, Kumar has served as the assistant director of robotics and cyber physical systems at the White House Office of Science and Technology Policy (2012 – 2013). His lab has spun off many startups in robotics, and he is the founder of Exyn Technologies. He is a Fellow of the American Society of Mechanical Engineers (ASME) and the Institute of Electrical and Electronic Engineers (IEEE).
Kumar has served on the editorial boards of the IEEE Transactions on Robotics and Automation, IEEE Transactions on Automation Science and Engineering, ASME Journal of Mechanical Design, and the Springer Tract in Advanced Robotics (STAR), and was the chief editor for the ASME Journal of Mechanisms and Robotics. He has won best paper awards at DARS 2002, ICRA 2004, ICRA 2011, RSS 2011, RSS 2013, ICRA 2014, BICT 2015, and MARSS 2016 and has advised doctoral students who have won Best Student Paper Awards at ICRA 2008, RSS 2009, and DARS 2010. He was recognized with the RSS Time of Test Award in 2025.
Additionally, Kumar is the recipient of the 2012 ASME Mechanisms and Robotics Award, the 2012 IEEE Robotics and Automation Society Distinguished Service Award, a 2012 World Technology Network (wtn.net) award, a 2013 Popular Mechanics Breakthrough Award, a 2014 Engelberger Robotics Award, the 2017 IEEE Robotics and Automation Society George Saridis Leadership Award, the 2017 ASME Robert E. Abbott Award, the 2018 IEEE Robotics and Automation Pioneer Award, and the 2020 IEEE Robotics and Automation Field Award. He was recognized with the John Scott Award in 2024 by City Trusts of the City of Philadelphia. He was elected to the National Academy of Engineering in 2013, the American Philosophical Society in 2018, and the American Academy of Arts and Sciences and the National Academy of Inventors in 2022.
About Elizabeth D. Rockwell
A fourth generation Houstonian, Mrs. Rockwell was an Executive Director, Private Client Division of CIBC Oppenheimer Corp. She was widely recognized as an expert in retirement, estate, investment, and tax planning. She was an early proponent of the Keogh and IRA plans, for which she has been nationally recognized.
In 1991, she qualified to be a member of the Million Dollar Round Table as well as the Texas Leaders Round Table. Since 1990 she had authored a monthly column for the Houston Chronicle.
Mrs. Rockwell served as President of the UH College of Business Administration Foundation Board, as a member of the Dean’s Advisory Board, and was an Executive Professor for the college. She also served as a member of the advisory board of the Health Law and Policy Institute and as a Trustee of the University of Houston System’s Foundation, as well as a member of the UH System’s Planned Giving Council.
Mrs. Rockwell served on the Board of Governors for the Houston Forum, and as a Board member of the American Red Cross, the Greater Houston Women’s Foundation, the University of Houston Alumni Organization, and the Greater Houston Preservation Alliance. She was a member of the River Oaks Business Women’s Exchange Club, the National Tax Sheltered Annuity Association, the Texas Association of College Teachers, and the Houston Association of Life Underwriters.
Among her numerous honors, she has received an Honorary Doctorate of Humane Letters from the University of Houston, the Distinguished Alumna Award from the Houston Alumni Organization and from the UH College of Business Administration. Throughout the years she has been recognized for her many achievements by the Education Foundation of Harris County, the Houston Community College System’s Television Station Advisory Council, and the Houston Mayor’s READ Commission.
The Houston Alumni Center is home to the Elizabeth D. Rockwell President’s Suite. In September 1997, the Elizabeth D. Rockwell Career Services Center was opened in the UH College of Business Administration. She endowed the Chair for the Dean of the M.D. Anderson Library.
Mrs. Rockwell was listed in the Who’s Who in the South and Southwest; Who’s Who in finance and Industry; Who’s Who of American Women; and Who’s Who in the World.