Learning Robot–Environment Interaction for Legged Autonomy

Legged robots operating in unstructured environments must reason about more than terrain geometry. Their behavior depends on terrain, commanded motion, robot dynamics, payload, and the locomotion controller.

Our research develops robot-centric representations that connect perception, physical interaction, locomotion adaptation, uncertainty estimation, and risk-aware navigation.

Rather than treating terrain as inherently safe or unsafe, we study how the robot and its controller respond while interacting with the environment.

Robot-Centric
Perception
→
Interaction
Context
→
Uncertainty and
Representation
→
Safe Planning
and Adaptation

Context-Aware Terrain Understanding

CART · Context-Aware Terrain Adaptation for Stable Locomotion

42th International Symposium of Robotics Research (ISRR) 2026

CART context-aware terrain adaptation

Visual observations are interpreted together with recent physical interaction.

Terrain appearance does not always correspond to physical properties. Visually similar surfaces can produce different robot responses, while visually different surfaces can behave similarly.

CART combines visual observations with recent proprioceptive history. A temporal sequence-selection mechanism identifies relevant context and adapts velocity and body-height commands.

Highlights

  • Combines exteroception and proprioception for adaptation.
  • Handles mismatches between appearance and physical response.
  • Uses temporal sequence selection to identify relevant context.
  • Adapts commands without changing the low-level controller.
Paper Video Code

Task-Conditioned Uncertainty for Safe Navigation

Task-Conditioned Uncertainty Costmaps for Legged Locomotion

29th conference on walking and climbing robots (CLAWAR) 2026

Task-conditioned uncertainty costmaps

Terrain and commanded motion jointly determine predicted locomotion uncertainty.

Terrain uncertainty depends on the action the robot intends to execute. A terrain patch may be manageable under a slow command but unsafe under a fast or turning command.

This work predicts future footholds and their epistemic uncertainty using terrain observations, robot state, and commanded motion. The uncertainty is projected into spatial costmaps for MPPI and Nav2.

Highlights

  • Predicts future footholds and epistemic uncertainty.
  • Conditions uncertainty on terrain, state, and commands.
  • Detects out-of-distribution terrain-command combinations.
  • Generates planner-compatible uncertainty costmaps.
  • Integrates with MPPI and Nav2.
Paper Video Code

Controller-Aware Terrain Representations

TERRA-Nav · Terrain-Encoded Locomotion with Risk-Aware Navigation

Work in Progress

TERRA-Nav controller-aware terrain representation

A shared terrain-response representation connects locomotion learning and risk-aware navigation.

Terrain categories alone do not explain how a locomotion controller will respond. The relevant quantity is the controller-dependent behavior produced during terrain transitions.

TERRA-Nav learns a terrain-response representation from exteroception and proprioceptive history. It guides curriculum allocation during training and risk-aware command selection during deployment.

Current Direction

  • Learning controller-aware terrain-response representations.
  • Detecting terrain transitions without manual terrain categories.
  • Prioritizing informative terrain during curriculum learning.
  • Supporting risk-aware command selection during navigation.
  • Reusing one representation across training and deployment.
Work in Progress

Autonomous Legged-Robot Field Deployment

INSPECTOR · Autonomous Culvert Inspection

ICRA 2025 Workshop on Field Robotics

Autonomous culvert inspection using Boston Dynamics Spot

Autonomous traversal and inspection in challenging canal environments.

Real-world deployment introduces challenges that are difficult to reproduce in simulation, including steep slopes, poor illumination, vegetation, narrow spaces, water, and motion blur.

INSPECTOR demonstrates autonomous canal-culvert inspection using Boston Dynamics Spot, integrating bank traversal, navigation, active illumination, and visual inspection.

Highlights

  • Demonstrates autonomous operation in canal environments.
  • Integrates steep-bank traversal and culvert inspection.
  • Uses active illumination for low-light feature tracking.
  • Improves data quality by reducing motion blur.
Project Video Paper

Payload-Aware Terrain Adaptation

PANOS · Payload-Aware Navigation in Off-Road Scenarios

Accepted at NERC 2024

PANOS payload-aware terrain adaptation

Terrain observations and proprioceptive response are combined for payload-aware locomotion.

Changes in payload alter the dynamics of a legged robot and can increase vibration, tracking error, and instability even when the terrain remains unchanged.

PANOS combines terrain observations with proprioceptive feedback to select stable locomotion commands across terrain and payload conditions without requiring explicit terrain-property labels.

Highlights

  • Connects terrain perception with payload-dependent response.
  • Combines exteroception and proprioception without terrain labels.
  • Reduces locomotion instability and vibration.
  • Adapts across different terrain and payload conditions.
Paper Video Code

Role-Specialized Terrain Adaptation

TRACER · Hierarchical Decomposition of Terrain Adaptation for Quadrupedal Locomotion

IROS 2026 Workshop

TRACER hierarchical terrain-adaptation framework

Terrain adaptation is decomposed into Objective, Interaction, Strategy, and Execution roles.

TRACER studies which component of a quadruped locomotion system should adapt when terrain-induced failures arise.

The framework explicitly separates task preference, robot–terrain interaction, high-level locomotion strategy, and low-level execution.

Highlights

  • Hierarchical decomposition of terrain adaptation.
  • Explicit preference-to-strategy interface semantics.
  • Ongoing extension toward role-specialized adaptation.
Project Paper Poster

Long-Term Research Vision

The long-term goal is to develop legged robots that understand both their environment and the limits of their own perception, controller, and physical capabilities.

These systems should estimate interaction context, quantify uncertainty, predict future behavior, and adapt planning and whole-body motion while maintaining safety.

People

This research is led by Kartikeya Singh under the supervision of Prof. Karthik Dantu at the DRONES Lab , University at Buffalo.

ksingh35@buffalo.edu