Research Projects

Robots are rapidly moving beyond structured industrial settings into unstructured, human-centric environments such as homes, hospitals, and offices. In such spaces, robots must interact with and manipulate objects in cluttered scenes where movable obstacles are common. These scenarios introduce geometric constraints, making it increasingly difficult to compute collision-free object manipulation. As a result, motion planning becomes a major computational bottleneck. A key class of problems that captures this complexity is Task and Motion Planning (TAMP), which integrates discrete-level planning (e.g., pick-and-place actions) with continuous motion-level feasibility. However, not all of the discrete actions may be geometrically feasible in a given scene. For example, a target object may be partially occluded by clutter. Determining whether an action is feasible typically requires invoking a sampling-based motion planner. These planners are probabilistically complete but cannot certify infeasibility; they must run until timeout to declare failure. Consequently, traditional TAMP systems incur high computational cost for infeasible actions, especially in cluttered scenes or long-horizon plans. This project proposes LEAP (Learned Estimation of Action Plausibility), a learning-based framework designed to estimate the feasibility of symbolic actions in real-world environments. LEAP aims to serve as a fast feasibility oracle that can be integrated into TAMP pipelines to eliminate expensive motion planning attempts for clearly infeasible actions. The central hypothesis of this project is that it is possible to learn a reliable predictor of geometric feasibility for symbolic actions directly from scene context , thereby bridging the gap between symbolic planning and motion-level reasoning. Scientific Objectives: To develop a Transformer-based model that predicts the geometric feasibility of symbolic actions based on the spatial configuration of objects in the scene. To design and validate a scalable pipeline for automated dataset generation and labeling in simulation, capturing a wide variety of scene layouts and object configurations. To test whether the learned model can generalize across object geometries and long-horizon task plans. To evaluate the performance of the learned model in real-world robotic settings, identifying limitations and addressing sim-to-real gaps. Methodology and Main Experiments: In the second phase, we will expand the dataset to include more diverse object geometries, including geometric primitives (e.g., spheres, cuboids, cylinders) and complex 3D shapes represented via point clouds. The final phase will involve real-world validation using physical robot platforms. We will assess the model’s robustness to perception noise, shape variance, and kinematic inaccuracies. Where needed, we will explore fine-tuning or domain adaptation using real-world data. Expected Significance: LEAP will provide a novel, efficient solution to a fundamental bottleneck in robotic manipulation—identifying infeasible actions early without resorting to costly motion planning. The approach will enable faster, more scalable planning in complex environments. The project’s contributions lie in advancing our scientific understanding of learned feasibility prediction and its integration into hybrid planning pipelines. From an application standpoint, this work has the potential to enhance autonomy in assistive robots, service robots in homes and hospitals, and logistics automation in warehouses—domains where long-horizon manipulation in unstructured environments is a critical challenge.