THE TACMIND BENCHMARK

TacMind: Benchmarking and Modeling Tactile Interaction States for Robotic Manipulation

Jiaxin Shi1,4Zibin Wu2,4Yichi Zhang3,4Jianghua Duan4Siyu Zhu5Weihao Yuan4,6
A CLOSER LOOK AT CONTACTONE EPISODE. EIGHT VIEWS.
LANGUAGE INSTRUCTION

“Grasp the felt eraser, erase the TacMind lettering from the rectangular whiteboard, then lift the eraser clear.”

128Manipulation tasksStandard (28) + Extra (100)
4Task suitesPrecision · Recog
Dynamic · Extra
287
Object assets

Unique object assets, including targets and cut-food pieces. Scene fixtures are excluded.

12 evaluation perturbationsLighting (4) · Camera viewpoint (6) · Object scale (2)Applied to Standard tasksExplore

Robotic manipulation tasks

128 tasks across insertion, recognition, dynamic interaction, and more. Standard tasks include GSmini and Daimon sensor recordings.

01 / PRECISION

Precision insertion

Insert tubes, keys, connectors, gears, power plugs, and bulbs into their target fixtures.

Fine manipulationContact-rich insertion
PRECISION 04 Power plug insertion

Loading task catalog…

EVALUATION CONDITIONS

Evaluation perturbations

Evaluate trained policies under changes in lighting, camera viewpoint, and object scale.

Warm lighting

2700 K lighting preset.

Drag a divider to compare all three scenes.

Loading visual comparisons…

Left: nominal baseline. Right: selected perturbation. Each preview changes one condition at a time, across three Standard tasks. Static scenes · seed 0.

TACMIND-BENCH · GELSIGHT MINI

Leaderboard

Performance across precision, recognition, and dynamic manipulation.

GSmini

Ranked by average OOD · Higher is better

Rank by
Performance on TacMind-Bench (GelSight Mini)
ModelPreciseRecognitionDynamicAvg
InDLightCamSizeInDLightCamSizeInDLightCamSizeInDOOD ↓
Loading benchmark results…

InD In-distributionOOD Out-of-distributionLight / Cam / Size Lighting, camera, and object-size perturbations

ACT+Tactile and DP use single-task training: a separate policy is trained for each task.

Results from the table. Bold values mark the best score in each column. Scroll horizontally to explore all metrics.

TODO

We will continue developing TacMind-Bench:

  • Support a wider range of tactile sensors.
  • Design more tasks centered on tactile interaction.

News

Updates from TacMind.

Acknowledgements

We thank the contributors to the following open-source projects for sharing their tools, datasets, models, and research with the robotics community.

Perturbation comparison

Drag the divider, or use the arrow keys, to compare baseline and perturbation.

Contact behaviors

Grouped by the actions in the task definitions. A task can involve several behaviors.

BENCHMARK STATUS

Benchmark availability

TacMind-Bench includes 128 manipulation tasks: 28 Standard tasks and 100 Extra tasks. Explore recorded demonstrations and evaluation perturbations on this website.

Find dataset releases on Hugging Face and ModelScope.

The benchmark is under active development. Code will be released progressively, with support for more tactile sensors and tasks. Follow the project on GitHub for updates.