AIRLab Invited Talks


Date and Time: Friday, Dec 03, 2021, at 12 pm
Speaker: Prof. Kaidi Xu  (Drexel University)
Talk Title: Neural Networks with Provable Robustness Guarantees

Abstract ⬇️

Neural networks have become a crucial element in modern artificial intelligence. However, they are often black-boxes and can sometimes behave unpredictably and produce surprisingly wrong results. When applying neural networks to mission-critical systems such as autonomous driving and aircraft control, it is often desirable to formally verify their trustworthiness such as safety and robustness. Unfortunately, the complexity of neural networks has made the task of formally verifying their properties very challenging. To tackle this challenge, we first propose an efficient verification algorithm based on linear relaxations of neural networks, which produces guaranteed output bounds given bounded input perturbations. The algorithm propagates linear inequalities through the network efficiently in a backward manner and can be applied to arbitrary network architectures using our auto_LiRPA library. To reduce relaxation errors, we further develop an efficient optimization procedure that can tighten verification bounds rapidly on GPUs. Lastly, I discuss how to further empower the verifier with branch and bound by incorporating the additional branching constraints into the bound propagation procedure. The combination of these advanced neural network verification techniques leads to α,β-CROWN (alpha-beta-CROWN), a scalable, powerful and GPU-based neural network verifier that won the 2nd International Verification of Neural Networks Competition (VNN-COMP’21) with the highest total score.

Short Bio ⬇️

Kaidi Xu is an assistant professor in the Department of Computer Science at Drexel University. He obtained his Ph.D. from Northeastern University in 2021. Kaidi’s primary research interest is the robustness of machine learning, including physical adversarial attacks and rigorous robustness verification. Kaidi has published in various top international conferences and his work ‘Adversarial T-shirt’ has received more than 200 media coverage. Kaidi with his team is also the global winner of the 2nd International Verification of Neural Networks Competition.

 


Date and Time: Friday, Nov 19, 2021, at 12 pm
Speaker: Xianyi Cheng  (Carnegie Mellon University)
Talk Title: Contact Mode Guided Motion Planning for General Nonprehensile Dexterous Manipulation

Abstract ⬇️

Human beings are highly creative in manipulating objects: grasping, pushing, pivoting, manipulating with limbs, using the environment as extra support, and so on. In contrast, robots lack such intelligence and capabilities for dexterous manipulation. How do we bring human-level dexterity into robotic manipulation? Our current work focuses on dexterous manipulation motion generation, the first step towards general manipulation intelligence. We considered the dexterity from planning through both the robot hand contacts and environment contacts. The contact-rich nature makes this problem challenging and unsolved. To address it, we first studied the mechanics of contacts. We proved that the complexity of enumerating contact modes for one object is polynomial, instead of exponential, to the number of contacts. This exciting observation inspired us to use instantly enumerated contact modes as guidance to an RRT-based planner. Contact modes guide the motion generation like automatically generated motion primitives, capturing all possible transitions of contacts in the system. As a result, our planner can generate dexterous motions for many different nonprehensile manipulation tasks only given the start and goal of object poses. We also observed that some generated strategies resemble human manipulation behaviors, such as the “simultaneous levering out and grasp formation” in human grasping. To the best of our knowledge, our planner is the first method capable of solving diverse dexterous manipulation tasks without any pre-designed skill or pre-specified contact modes.

Short Bio ⬇️

Xianyi Cheng is a third-year Ph.D. student at Carnegie Mellon University, advised by Professor Matthew T. Mason. She received a master’s degree in Robotics from Carnegie Mellon University and a bachelor’s degree in Astronautical Engineering from Harbin Institute of Technology. Her primary research interests include mechanics, planning, and optimization in robotic manipulation. Specifically, her current work focuses on generating versatile and robust dexterous manipulation skills. She is a recipient of a Foxconn Graduate Fellowship.

Video ⬇️

 


Date and Time: Friday, Oct 29, 2021, at 12 pm
Speaker: Prof. Chengyu Li  (Villanova University)
Talk Title: Learning from nature: Odor-guided navigation of flying insects

Abstract ⬇️

Insects rely on their olfactory system for detecting food sources, prey, and mates. They can sense odors emitting from sources of their interest, use their highly efficient flapping-wing mechanism to follow odor trails, and track down odor sources. During such odor-guided navigation, flapping wings not only serve as propulsors for generating lift and maneuvering, but also actively draw odors to the antennae via wing-induced flow. This helps enhance olfactory detection, mimicking “sniffing” in mammals. However, the flow physics underlying this odor-tracking behavior is still unclear due to insects’ small wing size, fast flapping motion, and the unpredictability of their flying trajectories. Limited success has been achieved in evaluating the impact of wing-induced flow on odorant transport during odor-guided navigation. Utilizing an in-house computational fluid dynamics solver, we investigate the unsteady aerodynamics and olfactory sensitivities of insects in upwind surge motion. This study aims to advance our understanding of the odor-tracking capability of animal navigation and leads to transformative advancements in unmanned aerial devices that will have the potential to greatly impact national security equipment and industrial applications for chemical disaster, drug trafficking detection, and GPS-denied indoor environment.

Short Bio ⬇️

Dr. Chengyu Li is an Assistant Professor in the Department of Mechanical Engineering at Villanova University. He received his Ph.D. in Mechanical and Aerospace Engineering from the University of Virginia in 2016 and was a Postdoctoral Researcher at Ohio State University from 2016 to 2018. His research is situated at the intersection of fluid dynamics and computation with an emphasis on engineering and healthcare applications. In particular, he focuses on developing state-of-the-art computational methods that leverage mathematical models and numerical simulations to improve understanding of biological and physiological flows. Dr. Li’s research interests are in bio-inspired flow, unsteady aerodynamics, and odorant transport phenomena of animal olfaction. His interdisciplinary research was recognized with Polak Young Investigator Award by the Association for Chemoreception Sciences in 2017 and Ralph E. Powe Junior Faculty Enhancement Award by Oak Ridge Associated Universities in 2019. He also received an NSF CAREER Award in 2021 for examining the fluid dynamic mechanisms of odor-guided flapping flight in nature.

 


Date and Time: Friday, Oct 22, 2021, at 12 pm
Speaker: Dr. Armon Shariati  (Amazon Robotics)
Talk Title: From Academia to Amazon – Life of a Roboticist

Abstract ⬇️

Amazon Robotics builds the world’s largest mobile robotic fleet where hundreds of thousands of robots help deliver hundreds of millions of customer orders per year. In order to support this massive system, Amazon Robotics relies on cutting edge technologies within the fields of robotic movement, machine learning, computer vision, manipulation, simulation, cloud computing, and data science. In this talk, I will provide an overview of some of the largest software challenges facing our scientists and engineers within each of these domains today. In addition, as a first year scientist fresh out of academia, I’ll also be sharing my own background and experiences which ultimately drove me to pursue my career at Amazon, in an effort to help guide those interested in following a similar path.

Short Bio ⬇️

Armon Shariati is a Research Scientist in the Applied Machine Learning team and the Research and Advanced Development team at Amazon Robotics. His research interests include large-scale optimization and machine learning. Armon joined Amazon after defending his doctoral thesis titled “Online Synthesis of Speculative Building Information Models for Robot Motion Planning” under the advisement of Prof. Camillo Jose Taylor at the University of Pennsylvania’s GRASP Laboratory. He earned his B.S. degree in Computer Engineering from Lehigh University in 2015, where he also worked as an undergraduate research assistant in the VADER Laboratory under the advisement of Prof. John Spletzer.

 


Date and Time: Friday, Sep 24, 2021, at 12 pm
Speaker: Jingxi Xu  (Columbia University)
Talk Title: Dynamic Grasping with Reachability and Motion Awareness

Abstract ⬇️

Grasping in dynamic environments presents a unique set of challenges. A stable and reachable grasp can become unreachable and unstable as the target object moves, motion planning needs to be adaptive, and in real-time, the delay in computation makes prediction necessary. In this paper, we present a dynamic grasping framework that is reachability-aware and motion-aware. Specifically, we model the reachability space of the robot using a signed distance field which enables us to quickly screen unreachable grasps. Also, we train a neural network to predict the grasp quality conditioned on the current motion of the target. Using these as ranking functions, we quickly filter a large grasp database to a few grasps in real-time. In addition, we present a seeding approach for arm motion generation that utilizes a solution from the previous time step. This quickly generates a new arm trajectory that is close to the previous plan and prevents fluctuation. We implement a recurrent neural network (RNN) for modeling and predicting object motion. Our extensive experiments demonstrate the importance of each of these components, and we validate our pipeline on a real robot.

Short Bio ⬇️

Jingxi is a second-year Ph.D. student in Computer Science at Columbia University, co-advised by Professor Matei Ciocarlie and Professor Shuran Song. He also works closely with Dinesh Jayaraman and Nikolai Matni at the University of Pennsylvania. Before joining Columbia, Jingxi also spent some time at MIT with Leslie Pack Kaelbling and Tomás Lozano-Pérez (2018), and at Columbia with Peter Allen (2018-2019). He received a Bachelor’s degree from Edinburgh. His research interests are machine learning and vision-based methods for better planning, perception, and control abilities in autonomous robots.

Video ⬇️

 


Date and Time: Friday, June 18, 2021, at 11 am
Speaker: Prof. Stephanie Gil  (Harvard University)
Talk Title: Trust in Multi-Robot Systems and its Role in Achieving Resilient Coordination

Abstract ⬇️

Our understanding of multi-robot coordination and control has experienced great advances to the point where deploying multi-robot systems in the near future seems to be a feasible reality. However, many of these algorithms are vulnerable to non-cooperation and/or malicious attacks that limit their practicality in real-world settings. An example is the consensus problem where classical results hold that agreement cannot be reached when malicious agents make up more than half of the network connectivity; this quickly leads to limitations in the practicality of many multi-robot coordination tasks. However, with the growing prevalence of cyber-physical systems comes novel opportunities for detecting attacks by using cross-validation with physical channels of information. In this talk we consider the class of problems where the probability of a particular (i,j) link being trustworthy is available as a random variable. We refer to these as “stochastic observations of trust.” We show that under this model, strong performance guarantees such as convergence for the consensus problem can be recovered, even in the case were the number of malicious agents is greater than ½ of the network connectivity and consensus would otherwise fail. Moreover, under this model we can reason about the deviation from the nominal (no attack) consensus value and the rate of achieving consensus. Finally, we make the case for the importance of deriving such stochastic observations of trust for cyber-physical systems and we demonstrate one such example for the Sybil Attack that uses wireless communication channels to arrive at the desired observations of trust. In this way our results demonstrate the promise of exploiting trust in multi-robot systems to provide a novel perspective on achieving resilient coordination in multi-robot systems.

Short Bio ⬇️

Stephanie is an Assistant Professor in the John A. Paulson School of Engineering and Applied Sciences (SEAS) at Harvard University. Her work centers around trust and coordination in multi-robot systems. Her work has received recognition including the National Science Foundation CAREER award (2019), and selection as a 2020 Sloan Research Fellow for her contributions at the intersection of robotics and communication. She has held a Visiting Assistant Professor position at Stanford University during the summer of 2019, and an Assistant Professorship at Arizona State University from 2018-2020. She completed her Ph.D. work (2014) on multi-robot coordination and control and her M.S. work (2009) on system identification and model learning. At MIT she collaborated extensively with the wireless communications group NetMIT, the result of which were two U.S. patents recently awarded in adaptive heterogeneous networks for multi-robot systems and accurate indoor positioning using Wi-Fi. She completed her B.S. at Cornell University in 2006.

 


Date and Time: Friday, June 4, 2021, at 11 am
Speaker: Prof. Guilherme Pereira  (West Virginia University)
Talk Title: Contributions to Autonomous Field Robotics

Abstract ⬇️

In this talk I will present the current research projects at the Field and Aerial Robotics Laboratory at West Virginia University. These projects aim the development of mobile robotic systems that work for a long time in challenging real-world environments that include forests and underground mines. I will show our solutions for hardware design and localization but will focus the talk on motion planning. Our proposed motion planning approaches are mostly based on vector-fields associated with optimal planners. In these combination, global vector-fields encode the main robotic task, but do not have a complete knowledge of the environment, while optimal planners are used by the robot to track the vector-field and avoid previously unknown obstacles in the environment. Results that show ground and aerial robots operating using this methodology will be presented along the talk.

Short Bio ⬇️

Guilherme A. S. Pereira received his bachelor and master’s degrees in electrical engineering from the Federal University of Minas Gerais, Brazil, in 1998 and 2000, respectively, and his PhD degree in computer science from the same university in 2003. He has received the Gold Medal Award from the Engineering School of UFMG for the first place among the electrical engineer students in 1998. From 11/2000 to 05/2003, he was a visiting scientist at the GRASP Laboratory of the University of Pennsylvania. He was also a visiting scholar at The Robotics Institute of Carnegie Mellon University from 08/2015 to 07/2016. From July/2004 to July/2018, he was a full-time professor at the electrical engineering department of the Federal University of Minas Gerais. He joined WVU in October 2018 as an associate professor of the Department of Mechanical and Aerospace Engineering. He also holds an adjunct appointment at the Lane Department of Computer Science and Electrical Engineering. At WVU he directs the Field and Aerial Robotics (FARO) Laboratory. His main research areas are field robotics, robot motion planning, aerial robotics, state estimation, cooperative robotics, and computer vision.

Video ⬇️

 


Date and Time: Friday, May 14, 2021, at 11 am
Speaker: Prof. Rafael Fierro  (The University of New Mexico)
Talk Title: Towards Stable Interstellar Flight: Modeling and Stability of a Laser-Propelled Sailcraft

Abstract ⬇️

Traveling to distant stars has long fascinated humanity, but vast distances limited space exploration to our solar system. The Breakthrough Starshot Program aims at eliminating this limitation by traveling to Alpha Centauri, which is 4.37 light-years away. The idea is to accelerate a sail to relativistic speeds using a laser beam aimed at the sail. Currently, sail dynamic stability of the sailcraft is not well understood and there is no agreement on the proper shape of the beam-driven sail. In this seminar, I will present some results on dynamic stability of a beam-driven sail modeled as a rigid body whose shape is parameterized by a sweep function. We analyze the stability of the beam-driven sail and estimate its region of attraction (ROA) using Lyapunov theory and Sum-Of-Square (SOS) programming. I will also summarize our recent efforts on aerial robotics including monitoring of extreme environments, a testbed and methods for counter small UAS, and airborne manipulation.

Short Bio ⬇️

Rafael Fierro is a Professor of the Department of Electrical and Computer Engineering, the University of New Mexico, where he has been since 2007. He received his Bachelors of Science from the Escuela Politécnica Nacional, Quito-Ecuador, his MSc degree in control engineering from the University of Bradford, England, and his Ph.D. in electrical engineering from the University of Texas at Arlington in 1997. Before joining UNM, he held a postdoctoral appointment with the General Robotics, Automation, Sensing & Perception (GRASP) Laboratory at the University of Pennsylvania and a faculty position with the Department of Electrical and Computer Engineering at Oklahoma State University. His areas of interests include cyber-physical systems; heterogeneous robotic networks; and uncrewed aerial vehicles (UAV). The National Science Foundation (NSF), US Army Research Laboratory (ARL) and Air Force Research Laboratory (AFRL), Department of Energy (DOE), and Sandia National Laboratories have funded his research. He directs the AFRL-UNM Agile Manufacturing Center and the Multi-Agent, Robotics, and Heterogeneous Systems (MARHES) Laboratory. Dr. Fierro was the recipient of a Fulbright Scholarship, a 2004 National Science Foundation CAREER Award, and the 2008 International Society of Automation (ISA) Transactions Best Paper Award. He has served on the editorial boards of the Journal of Intelligent & Robotic Systems, IEEE Control Systems Magazine, IEEE Transactions on Control of Network Systems T-CNS, and IEEE Transactions on Automation Science and Engineering T-ASE.

 


Date and Time: Friday, April 2, 2021, at 11 am
Speaker: Cherie Ho  (Carnegie Mellon University)
Talk Title: Adaptive Safety Margins for Safe Replanning under Time-Varying Disturbances

Abstract ⬇️

Safe real-time navigation is a considerable challenge because engineers often need to work with uncertain vehicle dynamics, variable external disturbances, and imperfect controllers. A common strategy used to address safety is to employ hand-defined margins for obstacle inflation. However, arbitrary static margins often fail in more dynamic scenarios, and using worst-case assumptions proves to be overly conservative in most real-world settings where disturbance varies.
In this work, we propose a middle ground: safety margins that adapt on-the-fly using online measurements. In an offline phase, we use Monte Carlo simulations to pre-compute a library of safety margins for multiple levels of disturbance uncertainty. At runtime, our system estimates the current disturbance level and queries the associated safety margins for real-time replanning that present the best trade-off between safety and performance. We validate using extensive simulated and real-world flight tests that our approach better balances safety and performance than baseline static margin methods.

Video ⬇️

 


Date and Time: Friday, Feb 22, 2021, at 11 am
Speaker: Dr. Juan D. Hernández Vega  (Cardiff University)
Talk Title: Motion Planning for Multipurpose Autonomous Systems

Abstract ⬇️

Once limited to highly controlled industrial environments only, today robots are continuously evolving towards becoming autonomous entities, capable of operating in changing and unstructured settings. This evolution is only possible due to multi-disciplinary efforts, which endow autonomous systems with the required capabilities to deal with such changing and uncertain conditions. One of these disciplines, commonly known as motion planning, consists in computational techniques that calculate collision-free and feasible motions of autonomous systems. In this talk, we will discuss how motion planning is being used to improve the decision-making capabilities of different types of autonomous systems, such as autonomous underwater vehicles (AUVs), autonomous/automated cars and service robots.

 


Date and Time: Friday, Dec 18, at 11 am
Speaker: Bing Song (Columbia University)
Talk Title: What can robot learning bring to control?

Video ⬇️

 


Date and Time: Friday, Dec 11, at 11 am
Speaker: Juan David Pabon (Universidad Nacional de Colombia)
Talk Title: Event-Triggered Control for Weight-Unbalanced Directed Robot Networks

Video ⬇️

 


Date and Time: Friday, Sep 25th, at 11 am
Speaker: Bowen Wen (Rutgers University)
Talk Title: Robust, Real-time 6D Pose Estimation and Tracking for Robot Manipulation

Abstract ⬇️

Many manipulation tasks, such as picking, placing or within-hand manipulation, require the object’s pose relative to the manipulating agent, robot or human. These tasks frequently involve significant occlusions, which complicate the estimation and tracking process. This work presents a framework based on RGB-D data. It aims towards robust pose estimation under severe occlusions, such as those arising due to a hand occluding a manipulated object. It also aims for short response times so as to allow for responsive decision making in dynamic setups. In particular, the proposed frameworks leverage the complementary attributes of: i) novel deep neural network architectures via domain disentanglement, and ii) 3D geometric reasoning by using the end-effector’s state to guide the pose estimation of severely occluded objects. Additionally, only synthetic data are required for training, making the proposed approach applicable to new tasks, circumventing the overhead of collecting labeled real world data. Extensive evaluation on multiple real world benchmarks, including public ones and some developed as part of this effort, demonstrates superior performance when compared with state-of-the-art approaches and generalization to different robotic manipulation scenarios. This talk will conclude with ongoing effort for generalizing this line of work in the case of perceiving and manipulating novel objects without access to models.

Short Bio ⬇️

Bowen Wen is currently a 3rd-year PhD student in the Computer Science Department at Rutgers working with Prof Kostas Bekris. He got his BS in Energy and Automation from Xi’an Jiaotong University in 2016 and MS in ECE from Ohio State University in 2018. His research interests include robotics, computer vision and artificial intelligence. He has published papers at ICRA, IROS, RA-L, Conference on Robotic Learning etc. He worked as a research intern with Facebook Reality Lab, Amazon Lab126 and SenseTime in summer 2020, 2019 and 2018 respectively.

Video ⬇️

 

Previous Talks:

Sep 18th, 2020, Towards Scalable Algorithms for Distributed Optimization and Learning, Dr. Cesar Uribe (MIT)

Aug 28th, 2020, Tightly Coupled Vision-based Perception and Control for Autonomous Aerial Robots, Dr. Eric Cristofalo (Stanford)

Aug 14th, 2020, Introduction to F1tenth race: 1/10 the size, 10 times the fun, Prof. Rosa Zheng (Lehigh)