Dr. Jaime Fernández Fisac

Assistant Professor, Department of Electrical and Computer Engineering, Princeton University

Time: 11 am (coffee and cookies at 10:30 am in PA 367)

Date: Friday, December 8

Location: PA 466

Abstract: Autonomous robotic systems are promising to revolutionize our homes, cities and roads—but will we trust them with our lives? This talk will take stock of today’s safety-critical robot autonomy, highlight recent advances, and offer some reasons for optimism. We will first see that a broad family of safety schemes from the last decade can be understood under the common lens of a universal safety filter theorem, lighting the way for the systematic design of next-generation safety mechanisms. We will explore reinforcement learning (RL) as a general tool to synthesize global safety filters for previously intractable robotics domains, such as walking or driving through abrupt terrain, by departing from the traditional notion of accruing rewards in favor of a safety-specific Bellman equation. We will show that this safety-RL formulation naturally allows learning from near misses and boosting robustness through adversarial gameplay. Finally, we will turn our attention to dense urban driving, where safety hinges on the autonomous vehicle’s rapidly unfolding interactions with other road users. We will examine how robots can bolster safety by leaning on their future ability to seek out missing key information about other agents’ intent or even their location. We will conclude with an outlook on future autonomous systems and the role of transparent, real-time safety proofs in generating public trust.

Bio: Jaime Fernández Fisac is an Assistant Professor of Electrical and Computer Engineering at Princeton University, where he directs the Safe Robotics Laboratory and co-directs the Princeton AI4ALL outreach summer program. His research combines control theory, artificial intelligence, cognitive science, and game theory with the goal of enabling robots to operate safely in human-populated spaces in a way that is well understood, and thereby trusted, by their users and the public at large. Prior to joining the Princeton faculty, he was a Research Scientist at Waymo (formerly Google’s Self-Driving Car project) from 2019 to 2020, working on autonomous vehicle safety and interaction. He received an Engineering Diploma from the Universidad Politécnica de Madrid, Spain in 2012, a M.Sc. in Aeronautics from Cranfield University, U.K. in 2013, and a Ph.D. in Electrical Engineering and Computer Sciences from the University of California, Berkeley in 2019. He is a recipient of the La Caixa Foundation Fellowship, the Leon O. Chua Award, the Google Research Scholar Award, and the Sony Focused Research Award.

Read More

Kostas Daniilidis
Prof., CIS, University of Pennsylvania

Dec 1, 2023 (Friday), 12:00 – 1:00 pm
Room 220, Building C

Abstract: Scaling up data and computation is regarded today as the key to achieving unprecedented performance in many visual tasks. Given the lack of scale in real-world experience, robotics has turned to simulation as the vehicle for scaling up. Biological perception is characterized though by principles of efficiency implemented through symmetry and efficient sensing. By respecting the symmetries of the problem at hand, equivariant models can generalize better, often requiring fewer parameters and less data to learn effectively. Moreover, they provide insights into the underlying structures and symmetries of the data, which can be invaluable in developing more robust and interpretable models. In this talk, we will present a framework for achieving equivariance by design rather than data augmentation and using orders of magnitude lower capacity models than competing approaches. Then, we will show how the new paradigm of event-based vision can facilitate visual motion and reconstruction tasks with asynchronous low-bandwidth processing.

Bio: Kostas Daniilidis is the Ruth Yalom Stone Professor of Computer and Information Science at the University of Pennsylvania where he has been faculty since 1998. He is an IEEE Fellow. He was the director of the GRASP laboratory from 2008 to 2013, Associate Dean for Graduate Education from 2012-2016, and Faculty Director of Online Learning from 2013- 2017. He obtained his undergraduate degree in Electrical Engineering from the National Technical University of Athens, 1986, and his PhD in Computer Science from the University of Karlsruhe, 1992, under the supervision of Hans-Hellmut Nagel. He received the Best Conference Paper Award at ICRA 2017. He co-chaired ECCV 2010 and 3DPVT 2006. His most cited works have been on event-based vision, equivariant learning, 3D human pose, and hand-eye calibration.

Developing and Deploying Situational Awareness in Autonomous Robotic Systems

Dr. Philip Dames

Associate Professor, Department of Mechanical Engineering, Temple University

Bio: Philip Dames is an Associate Professor of Mechanical Engineering at Temple University, where he directs the Temple Robotics and Artificial Intelligence Lab (TRAIL). Prior to joining Temple, he was a Postdoctoral Researcher in Electrical and Systems Engineering at the University of Pennsylvania. He received his PhD Mechanical Engineering and Applied Mechanics from the University of Pennsylvania in 2015 and his BS and MS degrees in Mechanical Engineering from Northwestern University in 2010. He is the recipient of an NSF CAREER award. His research aims to improve robots’ ability to operate in complex, real-world environments to address societal needs.

Multi-Modal Mobility Morphobot (M4) with Appendage Repurposing for Locomotion Plasticity Enhancement

Alireza Ramezani,

Assist. Prof., ECE, Northeastern University

Nov. 3, 2023 (Friday), 12:00 – 1:00 pm

Room 115, Building C

Abstract: Robot designs can take many inspirations from nature, where there are many examples of highly resilient and fault-tolerant locomotion strategies to navigate complex terrains by recruiting multi-functional appendages. For example, birds such as Chukars and Hoatzins can repurpose wings for quadrupedal walking and wing-assisted incline running. These animals showcase impressive dexterity in employing the same appendages in different ways and generating multiple modes of locomotion, resulting in highly plastic locomotion traits which enable them to interact and navigate various environments and expand their habitat range. The robotic biomimicry of animals’ appendage repurposing can yield mobile robots with unparalleled capabilities. Taking inspiration from animals, we have designed a robot capable of negotiating unstructured, multi-substrate environments, including land and air, by employing its components in different ways as wheels, thrusters, and legs. This robot is called the Multi-Modal Mobility Morphobot, or M4 in short. M4 can employ its multi-functional components composed of several actuator types to (1) fly, (2) roll, (3) crawl, (4) crouch, (5) balance, (6) tumble, (7) scout, and (8) loco-manipulate. M4 can traverse steep slopes of up to 45 deg. and rough terrains with large obstacles when in balancing mode. M4 possesses onboard computers and sensors and can autonomously employ its modes to negotiate an unstructured environment. We present the design of M4 and several experiments showcasing its multi-modal capabilities.

Bio: Alireza Ramezani is an Assistant Professor at the Department of Electrical & Computer Engineering at Northeastern University (NU). Before joining NU in 2018, he served as a postdoctoral fellow at Caltech’s Division of Engineering and Applied Science. He received his Ph.D. in Mechanical Engineering from the University of Michigan, Ann Arbor, under the supervision of Jessy Grizzle. Alireza’s research interests lie in the design of bioinspired mobile robots with nontrivial morphologies, involving high degrees of freedom and dynamic interactions with the environment, as well as the analysis and nonlinear, closed-loop feedback design of locomotion systems. His designs have been featured in several high-impact journals, including two cover articles in Science Robotics Magazine, listed among the top 5% of all research outputs scored by Science Magazine, and one research article in Nature Communications. Alireza has been awarded the Breakthrough, Innovative, and Game-Changing (BIG) Idea Award from Space Technology Mission Directorate (STMD) Program twice (in 2020 and 2022) for designing bioinspired locomotion systems to explore the Moon and Mars craters. Furthermore, in 2022, Northeastern’s team, under his leadership, won NASA’s top award, the ARTEMIS Award, at the BIG Idea Challenge Competitions. He is also the recipient of JPL’s Faculty Research Program Position. Alireza’s research has received widespread attention and has been covered by over 200 news outlets, including IEEE Spectrum, Space Magazine, Independent, New York Times, Wall Street Journal, Associated Press, National Geographic, CNN, NBC, Euronews, etc., since 2018.

Machine Learning for Safety-Critical Systems

Navid Azizan

Esther & Harold E. Edgerton Career Development Assistant Professor, Massachusetts Institute of Technology

Oct. 27, 2023 (Friday), 12:00 – 1:00 pm

Room 220, Building C

Abstract: The integration of machine learning, particularly deep neural networks (DNNs) into autonomous systems has revolutionized their capabilities, enabling sophisticated interpretation of high-dimensional sensory data and informed decision-making. However, the deployment of these systems in safety-critical applications is hindered by the opaque nature of DNNs, especially their unpredictable behavior under out-of-distribution (OoD) or anomalous conditions. This talk presents recent results on enhancing the safety and reliability of machine learning models for autonomous systems. Specifically, we will discuss (1) run-time monitors for learning-enabled components, namely uncertainty estimation and anomaly detection mechanisms for pre-trained models as well as latent representations, mitigating risks associated with unforeseen operational deviations; (2) model adaptation techniques and continual learning algorithms, to ensure consistent integration of new data without the setback of “catastrophic forgetting,” thereby sustaining the model’s adaptiveness and relevance in dynamic environments; and (3) safety-assured, learning-based control and decision-making systems, focusing on controllers intrinsically designed with safety and stability guarantees.

Bio: Navid Azizan is the Esther & Harold E. Edgerton (1927) Assistant Professor at MIT, where he is a Principal Investigator in the Laboratory for Information & Decision Systems (LIDS) and holds dual appointments in the Department of Mechanical Engineering (Control, Instrumentation, & Robotics) and the Schwarzman College of Computing’s Institute for Data, Systems, & Society (IDSS). His research interests broadly lie in machine learning, systems and control, mathematical optimization, and network science. His research lab focuses on various aspects of enabling large-scale intelligent systems, with an emphasis on principled learning and optimization algorithms for autonomous systems and societal networks. He obtained his PhD in Computing and Mathematical Sciences (CMS) from the California Institute of Technology (Caltech), co-advised by Babak Hassibi and Adam Wierman, in 2020, his MSc in electrical engineering from the University of Southern California in 2015, and his BSc in electrical engineering with a minor in physics from Sharif University of Technology in 2013. Prior to joining MIT, he completed a postdoc in the Autonomous Systems Laboratory (ASL) at Stanford University in 2021. Additionally, he was a research scientist intern at Google DeepMind in 2019. His work has been recognized by several awards, including the 2020 Information Theory and Applications (ITA) Gold Graduation Award and the 2016 ACM GREENMETRICS Best Student Paper Award. He was named in the list of Leading Academic Data Leaders from the CDO Magazine in 2023, named an Amazon Fellow in Artificial Intelligence in 2017, and a PIMCO Fellow in Data Science in 2018. He was also the first-place winner and a gold medalist at the 2008 National Physics Olympiad in Iran.

Bounded Rational Intelligence and Metalearning in Autonomous Systems

Prof. Kyriakos G. Vamvoudakis

Dutton-Ducoffe Endowed Professor, The Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology

11:00 am – 12:00 pm, Friday October 20, 2023

Packard Lab 466

Abstract: Autonomous systems will be tasked with operating in complex, human-centric environments both in cooperation and in competition with humans and other agents. In this talk, issues of prediction in the context of game theory and multi-agent cyber-physical systems will be addressed. To account for the cognitive limitations of human and machine decision-makers, we introduce ideas and principles of bounded rationality for autonomy using tools from control theory and reinforcement learning. Specifically, we will formulate level-k thinking and cognitive hierarchy models in nonlinear and linear noncooperative differential games, where each agent is assigned an intelligence level corresponding to several thinking iterations. The applicability of this approach will be highlighted via the example of a pursuit evasion game between Unmanned Aerial Vehicles. The versatility of the proposed methods will be shown via results of level-k thinking in discrete stochastic games. Finally, to design more advanced decision-making algorithms that explicitly exploit the learning abilities of the other agents in the environment, a meta-learning framework in games will be presented, via which an autonomous agent can achieve learning manipulation and deception.

About Dr. Kyriakos G. Vamvoudakis: Dr. Kyriakos G. Vamvoudakis was born in Athens, Greece. He received the Diploma (a 5-year degree, equivalent to a Master of Science) in Electronic and Computer Engineering from the Technical University of Crete, Greece in 2006 with highest honors. After moving to the United States of America, he studied at The University of Texas at Arlington with Frank L. Lewis as his advisor, and he received his M.S. and Ph.D. in Electrical Engineering in 2008 and 2011 respectively. From May 2011 to January 2012, he was working as an Adjunct Professor and Faculty Research Associate at the University of Texas at Arlington and at the Automation and Robotics Research Institute. During the period from 2012 to 2016 he was a project research scientist at the Center for Control, Dynamical Systems and Computation at the University of California, Santa Barbara. He was an assistant professor at the Kevin T. Crofton Department of Aerospace and Ocean Engineering at Virginia Tech until 2018. He currently serves as the Dutton-Ducoffe Endowed Professor at The Daniel Guggenheim School of Aerospace Engineering at Georgia Tech. He holds a secondary appointment in the School of Electrical and Computer Engineering. His research interests include reinforcement learning, control theory, cyber-physical security, bounded rationality, and safe/assured autonomy. Dr. Vamvoudakis is the recipient of a 2019 ARO YIP award, a 2018 NSF CAREER award, a 2021 GT Chapter Sigma Xi Young Faculty Award, and of several international awards including the 2016 International Neural Network Society Young Investigator (INNS) Award, the Best Paper Award for Autonomous/Unmanned Vehicles at the 27th Army Science Conference in 2010, and the Best Researcher Award from the Automation and Robotics Research Institute in 2011. He has also served on various international program committees and has organized special sessions, workshops, and tutorials for several international conferences. He currently is a member of the Technical Committee on Intelligent Control of the IEEE Control Systems Society, a member of the Technical Committee on Adaptive Dynamic Programming and Reinforcement Learning of the IEEE Computational Intelligence Society, a member of the IEEE Control Systems Society Conference Editorial Board, an Associate Editor of: Automatica; IEEE Transactions on Automatic Control; IEEE Computational Intelligence Magazine; IEEE Transactions on Systems, Man, and Cybernetics: Systems; IEEE Transactions on Artificial Intelligence; Neurocomputing; Journal of Optimization Theory and Applications; IEEE Control Systems Letters; and of Frontiers in Control Engineering-Adaptive, Robust and Fault Tolerant Control. He is also a registered Electrical/Computer engineer (PE), a member of the Technical Chamber of Greece, and a Senior Member of both IEEE and AIAA.

A Picture of the Prediction Space of Deep Networks

Prof. Pratik Chaudhari
Electrical and Systems Engineering and Computer and Information Science, University of Pennsylvania

Oct. 13, 2023 (Friday), 1:30 – 2:30 pm
Room 220, Building C

Abstract: Deep networks have many more parameters than the number of training data and can therefore overfit—and yet, they predict remarkably accurately in practice. Training such networks is a high-dimensional, large-scale and non-convex optimization problem and should be prohibitively difficult—and yet, it is quite tractable. This talk aims to illuminate these puzzling contradictions.
We will argue that deep networks generalize well because of a characteristic structure in the space of learnable tasks. The input correlation matrix for typical tasks has a “sloppy” eigenspectrum where, in addition to a few large eigenvalues, there is a large number of small eigenvalues that are distributed uniformly over a very large range. As a consequence, the Hessian and the Fisher Information Matrix of a trained network also have a sloppy eigenspectrum. Using these ideas, we will demonstrate an analytical non-vacuous PAC Bayes generalization bound for general deep networks.
We will next develop information-geometric techniques to analyze the trajectories of the predictions of deep networks during training. By examining the underlying high-dimensional probabilistic models, we will reveal that the training process explores an effectively low dimensional manifold. Networks with a wide range of architectures, sizes, trained using different optimization methods, regularization techniques, data augmentation techniques, and weight initializations lie on the same manifold in the prediction space. We will also show that predictions of networks being trained on different tasks (e.g., different subsets of ImageNet) using different representation learning methods (e.g., supervised, meta-, semi supervised and contrastive learning) also lie on a low-dimensional manifold.

Bio: Pratik Chaudhari is an Assistant Professor in Electrical and Systems Engineering and Computer and Information Science at the University of Pennsylvania. He is a core member of the GRASP Laboratory. From 2018-19, he was a Senior Applied Scientist at Amazon Web Services and a Postdoctoral Scholar in Computing and Mathematical Sciences at Caltech. Pratik received his PhD (2018) in Computer Science from UCLA, and his Master’s (2012) and Engineer’s (2014) degrees in Aeronautics and Astronautics from MIT. He was a part of NuTonomy Inc. (now Hyundai-Aptiv Motional) from 2014-16. He is the recipient of the Amazon Machine Learning Research Award (2020), NSF CAREER award (2022) and the Intel Rising Star Faculty Award (2022).

Search-based Planning for Higher-dimensional Robotic Systems

Prof. Maxim Likhachev
Robotics Institute, Carnegie Mellon University

April. 28, 2023 (Friday), 12:00 – 1:00 pm
In-person: Room 220, Building C

Abstract: Search-based Planning refers to planning by constructing a graph from systematic discretization of the state- and action-space of a robot and then employing a heuristic search to find an optimal path from the start to the goal vertex in this graph. This paradigm works well for low-dimensional robotic systems such as mobile robots and provides rigorous guarantees on solution quality. However, when it comes to planning for higher-dimensional robotic systems such as mobile manipulators, humanoids and ground and aerial vehicles navigating at high-speed, Search-based Planning has been typically thought of as infeasible. In this talk, I will describe some of the research that my group has done into changing this thinking.  In particular, I will focus on two different principles. First, constructing multiple lower-dimensional abstractions of robotic systems, solutions to which can effectively guide the overall planning process using Multi-Heuristic A*. Second, using offline preprocessing to provide online planning algorithms that provably guarantee to return solutions within a (small) constant time for repetitive planning tasks. I will present algorithmic frameworks that utilize these principles, describe their theoretical properties, and demonstrate their applications to a wide range of physical high-dimensional robotic systems.

Bio: Maxim Likhachev is an Associate Professor at Carnegie Mellon University, directing Search-based Planning Laboratory (SBPL), and a Senior Staff Software Engineer at Waymo. His group at CMU researches heuristic search, decision-making and planning algorithms, all with applications to the control of robotic systems including unmanned ground and aerial vehicles, mobile manipulation platforms, humanoids, and multi-robot systems. Maxim obtained his Ph.D. in Computer Science from Carnegie Mellon University with a thesis called “Search-based Planning for Large Dynamic Environments.” He has over 150 publications in top journals and conferences on AI and Robotics and numerous paper awards. His work on Anytime D* algorithm, an anytime planning algorithm for dynamic environments, has been awarded the title of Influential 10-year Paper at International Conference on Automated Planning and Scheduling (ICAPS) 2017, the top venue for research on planning and scheduling. Other awards include selection for 2010 DARPA Computer Science Study Panel that recognizes promising faculty in Computer Science and being on a team that won 2007 DARPA Urban Challenge and on a team that won the Gold Edison award in 2013.  Maxim founded RobotWits, a company devoted to developing advanced planning and decision-making technologies for self-driving vehicles and recently acquired by Waymo, and co-founded TravelWits, an online travel tech company that brings AI to make travel logistics easier. Finally, Maxim is an executive co-producer of regional Emmy-nominated The Robot Doctor TV series aimed at showing the use of mathematics in Robotics and inspiring high-school students to pursue careers in science and technology.

Formal methods for Cyber Physical Systems: State of the Art and Future Challenges

Prof. Necmiye Ozay

Electrical Engineering and Computer Science, University of Michigan

April. 14, 2023 (Friday), 12:00 – 1:00 pm

In-person: Room 220, Building C

Abstract: Modern cyber-physical systems, like high-end passenger vehicles, aircraft, or robots, are equipped with advanced sensing, learning, and decision making modules. On one hand these modules render the overall system more informed, possibly providing predictions into the future. On the other hand, they can be unreliable due to problems in information processing pipelines or decision making software. Formal methods, from verification and falsification to correct-by-construction synthesis hold the promise to detect and possibly eliminate such problems at design-time and to provide formal guarantees on systems’ correct operation. In this talk, I will discuss several recent advances in control synthesis and corner case generation for cyber-physical systems with a focus on scalability, and what role data and learning can play in this process. I will conclude the talk with some thoughts on challenges and interesting future directions.   Bio: Necmiye Ozay received her B.S. degree from Bogazici University, Istanbul in 2004, her M.S. degree from the Pennsylvania State University, University Park in 2006 and her Ph.D. degree from Northeastern University, Boston in 2010, all in electrical engineering. She was a postdoctoral scholar at the California Institute of Technology, Pasadena between 2010 and 2013. She joined the University of Michigan, Ann Arbor in 2013, where she is currently an associate professor of Electrical Engineering and Computer Science, and Robotics. Dr. Ozay’s research interests include hybrid dynamical systems, control, optimization and formal methods with applications in cyber-physical systems, system identification, verification & validation, autonomy and dynamic data analysis. Her papers received several awards. She has received the 1938E Award and a Henry Russel Award from the University of Michigan for her contributions to teaching and research, and five young investigator awards, including NSF CAREER, DARPA Young Faculty Award, ONR Young Investigator Award, and NASA Early Career Faculty Award. She is also a recent recipient of the Antonio Ruberti Young Researcher Prize from the IEEE Control Systems Society for her fundamental contributions to the control and identification of hybrid and cyber-physical systems.

Overview of the Dynamics, Control, and Systems Diagnosis Program at NSF

Prof. Eva Adnan Kanso

NSF Program Manager, Zohrab A. Kaprielian Fellow in Engineering, and Professor of Aerospace and Mechanical Engineering, University of Southern California

March. 3, 2023 (Friday), 1:15 – 2:00 pm

In-person: Room 220, Building C

Abstract: Prof. Kanso will present a brief overview of the Dynamics, Control, and Systems Diagnosis Program in the CMMI division at NSF, point out funding opportunities such as the EAGER, RAISE, and other mechanisms, and bring to your attention strategic priorities across the engineering directorate at NSF.

Bio: Eva Kanso joined the Division of Civil, Mechanical & Manufacturing Innovation (CMMI) of NSF in Fall 2021 as an IPA rotator from the University of Southern California, where she is a professor and the Z.H. Kaprielian Fellow in Aerospace and Mechanical Engineering. Prior to joining USC in 2005, Kanso held a two-year postdoctoral position in Computing and Mathematical Sciences at Caltech. She received a Ph.D. degree in 2003 and an M.S. degree in 1999 in Mechanical Engineering, as well as an M.A. degree in 2002 in Mathematics, all from the University of California at Berkeley. She obtained her Bachelor of Engineering degree from the American University of Beirut with distinction. Kanso held visiting positions at Princeton University in 2004, the Laboratoire LadHyX at the Ecole Polytechnique in 2015, the Courant Institute of Mathematical Sciences in 2016-2017, the Simons Foundation in 2016-2017, and the Ecole Supérieure de Physique et de Chimie Industrielles in 2021. Her research interests concern fundamental problems in the biophysics of cellular and subcellular processes and the physics of animal behavior, both at the individual and collection levels. A central theme in her work is the role of the mechanical environment, specifically the fluid medium and fluid-structure interactions, in shaping and driving biological functions.