Samuel Clarke

DexSkin: High-Coverage Conformable Robotic Skin for Learning Contact-Rich Manipulation

Suzannah Wistreich*, Baiyu Shi*, Stephen Tian*, Samuel Clarke, Michael Nath, Chengyi Xu, Zhenan Bao, and Jiajun Wu

CoRL 2025

@inproceedings{wistreich2025dexskin,
  title={DexSkin: High-Coverage Conformable Robotic Skin for Learning Contact-Rich Manipulation},
  author={Suzannah Wistreich and Baiyu Shi and Stephen Tian and Samuel Clarke and Michael Nath and Chengyi Xu and Zhenan Bao and Jiajun Wu},
  booktitle={Conference on Robot Learning},
  year={2025}
}

We introduce a soft, capacitive electronic skin that conforms to curved robotic surfaces and provides localized tactile sensing with high coverage. We show that it enables manipulation tasks requiring sensing across the whole finger, to support learning contact-rich manipulation in the real world.

X-Capture: An Open-Source Portable Device for Multi-Sensory Learning

Samuel Clarke, Suzannah Wistreich, Yanjie Ze, and Jiajun Wu

ICCV 2025

@inproceedings{clarke2025xcapture,
  title={X-Capture: An Open-Source Portable Device for Multi-Sensory Learning},
  author={Samuel Clarke and Suzannah Wistreich and Yanjie Ze and Jiajun Wu},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2025}
}

We design an open-source, portable, and cost-effective device for real-world multi-sensory data collection, capturing correlated RGBD images, tactile readings, and impact audio for under $1,000 in parts. We use this device to curate a dataset of 3,000 points on 500 everyday objects from diverse real-world environments, and show that both the quantity and sensory breadth of this data help in pretraining and fine-tuning multi-modal representations for object-centric tasks.

Hearing Anything Anywhere

Mason Wang*, Ryosuke Sawata*, Samuel Clarke, Ruohan Gao, Shangzhe Wu, and Jiajun Wu

CVPR 2024

@inproceedings{wang2024hearing,
  title={Hearing anything anywhere},
  author={Wang, Mason Long and Sawata, Ryosuke and Clarke, Samuel and Gao, Ruohan and Wu, Shangzhe and Wu, Jiajun},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={11790--11799},
  year={2024}
}

We collect a dataset of real Room Impulse Responses (RIRs) from four rooms, then introduce a new Novel Viewpoint Acoustic Synthesis method based on differentiable audio rendering. Our method uses physics-based biases to achieve practical sample efficiency, only requiring RIRs to be measured at roughly 12 distinct locations in the room.

SoundCam: A Dataset for Finding Humans Using Room Acoustics

Mason Wang*, Samuel Clarke*, Jui-Hsien Wang, Ruohan Gao, and Jiajun Wu

NeurIPS Datasets & Benchmarks 2023

@inproceedings{wang2023soundcam,
  title={SoundCam: A Dataset for Finding Humans Using Room Acoustics},
  author={Wang, Mason Long and Clarke, Samuel and Wang, Jui-Hsien and Gao, Ruohan and Wu, Jiajun},
  booktitle={Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
  year={2023}
}

We record thousands of room impulse responses and music clips in different real rooms with humans standing in different positions in the room. Learning-based models can use these minute differences in the room's acoustics to track, identify, or detect humans in the room. Our data can be used to develop more robust and sample-efficient methods, with applications in home assistants, security, and robotics.

RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools

Haochen Shi*, Huazhe Xu*, Samuel Clarke, Yunzhu Li, and Jiajun Wu

CoRL 2023 Best Systems Paper

@inproceedings{shi2023robocook,
  title={RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools},
  author={Haochen Shi and Huazhe Xu and Samuel Clarke and Yunzhu Li and Jiajun Wu},
  booktitle={7th Annual Conference on Robot Learning},
  year={2023}
}

We introduce a framework for accomplishing long-horizon tasks in soft-body manipulation and show that it can learn to make dumplings with a variety of tools, with very little training data for each tool. We also show that our framework can learn to use tools to achieve other tasks in soft-body manipulation, such as shaping dough into target shapes, autonomously selecting tools for each step of the task.

RealImpact: A Dataset of Impact Sound Fields for Real Objects

Samuel Clarke, Ruohan Gao, Mason Wang, Mark Rau, Julia Xu, Jui-Hsien Wang, Doug James, and Jiajun Wu

CVPR 2023 Highlight

@inproceedings{clarke2023realimpact,
  title={RealImpact: A Dataset of Impact Sound Fields for Real Objects},
  author={Samuel Clarke and Ruohan Gao and Mason Wang and Mark Rau and Julia Xu and Jui-Hsien Wang and Doug L. James and Jiajun Wu},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition},
  year={2023}
}

We collect 150,000 annotated recordings of impacts of 50 everyday objects, recorded from 600 distinct microphone locations. We show how our data can be used to tune and validate acoustic simulations, or used directly in interesting downstream audio and audiovisual tasks.

DiffImpact: Differentiable Rendering and Identification of Impact Sounds

Samuel Clarke, Negin Heravi, Mark Rau, Ruohan Gao, Jiajun Wu, Doug James, and Jeannette Bohg

CoRL 2021 Oral Presentation

@inproceedings{clarke2021diffimpact,
  title={DiffImpact: Differentiable Rendering and Identification of Impact Sounds},
  author={Samuel Clarke and Negin Heravi and Mark Rau and Ruohan Gao and Jiajun Wu and Doug James and Jeannette Bohg},
  booktitle={5th Annual Conference on Robot Learning},
  year={2021},
  url={https://openreview.net/forum?id=wVIqlSqKu2D}
}

Differentiable physics-based models provide a useful bias for learning from impact sounds to solve both forward and backward problems on impact audio. We show we can both infer models from data in the wild, and then use these models to perform source separation better than generic learning-based alternatives.

Robot Learning for Manipulation of Granular Materials Using Vision and Sound

Samuel Clarke (in collaboration with Travers Rhodes and advisors Christopher G. Atkeson and Oliver Kroemer)

CMU Masters in Robotics Thesis 2019

@mastersthesis{clarke2019robot,
  title={Robot learning for manipulation of granular materials using vision and sound},
  author={Clarke, Samuel},
  year={2019},
  school={Carnegie Mellon University}
}

Deep learning-based data-driven models can both predict the effects of a scooping operation on a granular material using vision and can learn to use audio for feedback on scooping and pouring granular materials.

Learning Audio Feedback for Estimating Amount and Flow of Granular Material

Samuel Clarke, Travers Rhodes, Christopher G. Atkeson, and Oliver Kroemer

CoRL 2018

@inproceedings{clarke2018learning,
  title={Learning Audio Feedback for Estimating Amount and Flow of Granular Material},
  author={Clarke, Samuel and Rhodes, Travers and Atkeson, Christopher G and Kroemer, Oliver},
  booktitle={Conference on Robot Learning},
  pages={529--550},
  year={2018}
}

Deep learning-based data-driven models can accurately predict the amount of granular materials a robot pours or shakes, based only on audio recordings. With machine learning, we can use recordings from a $3 microphone to outperform the measurement resolution of a $3,000 wrist force-torque sensor.

NaturalMotion: Exploring Gesture Controls for Visualizing Time-Evolving Graphs

Samuel Clarke, Nathan Dass, and Polo Chau

VIS 2016

@article{clarke2016naturalmotion,
  title={NaturalMotion: Exploring gesture controls for visualizing time-evolving graphs},
  author={Clarke, Samuel and Dass, Nathan and Chau, Duen Horng Polo},
  journal={Proceedings of IEEE VIS (poster session)},
  year={2016}
}

We took the Matrix Cube, a tool for visualizing time-evolving graphs, and developed a new 3D interface controlled by hand gestures. Gestures were captured with a Leap Motion device.

LatentGesture: Active User Authentication through Background Touch Analysis

Premkumar Saravanan, Samuel Clarke, Polo Chau, and Hongyuan Zha

Chinese CHI 2014

@inproceedings{saravanan2014latentgesture,
  title={LatentGesture: Active user authentication through background touch analysis},
  author={Saravanan, Premkumar and Clarke, Samuel and Chau, Duen Horng Polo and Zha, Hongyuan},
  booktitle={Proceedings of the Second International Symposium of Chinese CHI},
  pages={110--113},
  year={2014},
  organization={ACM}
}

With data collected from a user's brief interactions with common UI elements, such as checkboxes and sliders, machine learning models can uniquely identify the user. Such a system could authenticate a mobile device's user continuously and seamlessly, without many of the vulnerabilities common to traditional authentication methods.

CPR+ Mask

Finalist for Georgia Tech's 2017 Inventure Prize

We designed a mask to walk an untrained rescuer through performing standard-of-care CPR to a cardiac arrest victim, in real time. The mask is equipped with sensors for monitoring the state of the victim and the quality of the CPR and uses a speaker and LEDs to give instructions and cues to the rescuer.