Research

We design sensors that encode touch, shape and spectra as measurable patterns of light. For scientific instruments, we use physics-informed AI to predict signals and interpret measurements.

Imaging and Perception Systems for RoboticsPhysics-Informed AI for Scientific Instrumentation

Imaging and Perception Systems for Robotics

We start with what a robot needs to measure, then design the sensor and perception algorithm together.

Sensor design and prototyping

FringeTact

A transparent tactile sensor reads patterns of light to locate contact and estimate normal and shear forces.

PolarTact3D

A polarization camera captures both the 3D shape and color of a contact surface in a single shot.

Snapshot hyperspectral imaging

We design diffractive optical elements for compact, fast, light-efficient hyperspectral cameras that capture spectral information in one exposure.

Perception for robotic manipulation

3PT and 3PTv2

With Google Intrinsic, we developed the 3D-Object Perception Transformer (3PT) to estimate a part’s position and orientation from its 3D model and a color image. Its successor, 3PTv2, estimates poses for parts it never saw during training. In this zero-shot setting, 3PTv2 substantially reduces inference time and improves pose accuracy.

Agricultural robotics

Marine robotics and perception

FishEye

We’re developing a low-cost way to estimate a fish’s length and volume from a photo or video.

FishPod

FishPod uses a stereo camera mounted on a remotely operated underwater vehicle to estimate fish length from paired video frames.

Human–robot interaction

Robot–tablet interaction in older adults

We use a robot and tablet in brief, standardized sessions with older adults to measure behavioral markers of cognitive concern that are difficult to capture with paper-based, tablet-only, or static voice-based screening.

Physics-Informed AI for Scientific Instrumentation

We model how moving atoms scatter X-rays and how a laser cavity’s geometry changes its output. We also work backward from measurements, testing how well models can reconstruct a scene from optical data or trace a shoreline in a satellite image.

Predicting instrument response

Learning ultrafast X-ray dynamics

We use simulated atomic motion to predict how X-ray scattering changes over time, then compare those predictions with experimental measurements.

Modeling free-electron lasers

We build surrogate models to speed up simulations of free-electron laser oscillators and help researchers tune the laser cavity.

Evaluating frontier models for imaging applications

ImagingBench

ImagingBench evaluates frontier multimodal AI models on 22 tasks, from predicting what a camera will record to reconstructing a scene from sensor measurements.

Satellite-derived shorelines

We compare shorelines estimated from satellite images with reference shorelines to test how well mapping methods transfer from one coast to another.