betaFeedback

WorksComputer Science

Flip Anything with Agentic Robotics Self-Learning

Abstract

Flipping is a fundamental robotic manipulation skill that requires reorienting a target object in a stable pose on a planar support surface by rotating it 180∘^\circ about an axis parallel to the support surface – think of flipping a coin from heads to tails. Although flipping is a common objective in industrial tasks such as parts inspection, mechanical assembly, and tool use, coding a bimanual robot with grippers to flip diverse objects is a significant challenge. In this paper, we show that a coding agent can autonomously develop and refine object-flipping strategies through iterative learning in physics simulations across diverse object geometries. Progressive skill acquisition, from basic handover to toppling and pushing, robust grasping and sensing, and thin-object manipulation with recovery, improves task success from 24.7% to 95.0%. The resulting graph-based code integrates high-level strategy selection with reactive control, enabling multi-stage flipping on physical robots.

Agentic self-learning for flipping skill in simulation and flip skill real world deployment

Figure 1.

Introduction

Flipping is a fundamental robotic manipulation skill that requires reorienting a target object in a stable pose on a planar support surface by rotating it 180∘180^\circ about an axis parallel to the support surface – think of flipping a coin from heads to tails. Although flipping is a common objective in industrial tasks such as parts inspection, mechanical assembly, and tool use, coding a bimanual robot with grippers to flip diverse objects is a significant challenge. Flipping may require diverse strategies depending on object geometry, robot kinematics, and contact constraints [1, 2, 3, 4, 5]. For example, flipping a hammer may require a bimanual handover, whereas flipping a wide multimeter may require coordinated pushing and toppling. Such diversity introduces distinct action modes and coordination challenges for visuomotor policies and Vision Language Action models.

In this short paper, we present FlipAnything, an Agentic Robotics benchmark and case study. We use RoboSimStudio [6] and our Graph-as-Policy (GaP) harness [7] with Claude Fable 5.1 [8], to iteratively develop and refine an algorithmic solution through simulation-based self-learning without demonstration data. The resulting graph-based code uses RGB-D observations and robot proprioception to control bimanual YAM arms. We find that structured code provides several distinct multimodal manipulation behaviors, enabling a single policy to select and compose strategies such as bimanual handover, repositioning, pushing, and toppling based on perceived geometry and robot constraints. Iterative simulation-based self-learning enables the agent to identify failure modes, refine its control logic, and progressively improve coverage across diverse task instances. Results across 360 evaluation episodes involving 24 objects show that the graph-based code’s success rate improves from 24.7% to 95.0%. The code learned in simulation can also transfer zero-shot to real-world manipulation of unseen objects, including a clamp, blue tape, and a foldable helix screwdriver.

FlipAnything benchmark showing 24 objects and their evaluation success counts

Figure 1 (PDF p. 2). FlipAnything benchmark: 24 objects, with successes out of 15 evaluation episodes for the best tuned controller.

Method

Overview We describe the flipping task in English and provide CAD models of 24 objects and the URDF of the YAM arms as inputs to an iterative code-generation and evaluation loop, where the coding agent improves the executable policy [9] through interaction with a physics simulator [10, 7]. We use RoboSimStudio [6] and Newton for rigid-body physics simulation, with a MuJoCo-based solver [11] and CoACD [12] for convex decomposition of object meshes. We use the GaP harness [7] with Claude Fable 5.1 [8] as the coding agent to generate execution graphs. The agent has access to the simulator’s execution context, including spatial information and execution results.

Agentic Self-Learning. Given the flipping task and benchmark objects, the agent generates an initial graph that attempts flipping through bimanual handover. Then, it evaluates the generated flipping techniques across a set of objects with randomized positions for each object. After each evaluation, the agent analyzes success rates, execution traces, and failure modes, including infeasible grasps, lost contacts, failed handovers, and incorrect orientation estimates. Based on this feedback, the agent revises the code by refining perception, grasp selection, motion primitives, strategy selection, and recovery logic. The updated controller is then re-evaluated across the object set to measure improvements and detect regressions. Unlike gradient-based policy training, the learned artifact is a graph that can be converted into an executable program whose conditional logic and control parameters are progressively refined to cover the 24 object geometries and manipulation strategies.

Self-learning loop

Figure 2. Self-learning loop. Each version is frozen before it is evaluated. The data on the right is used to revise the code and is not an input to the policy.

Results

Self-Learning Improves FlipAnything Performance. We evaluate the agent-generated graph across 24 objects, with 15 evaluation episodes per object, totaling 360 episodes per iteration. Starting from an initial handover-only controller, the agent achieves a success rate of 24.7% (89/360). After self-learning additional manipulation strategies, including stand-and-topple and push-based flipping, the success rate increases to 80.6% (290/360). Subsequent self-improvement iterations refine grasp stability, geometric reasoning, contact handling, and recovery behaviors, progressively increasing performance to 95.0% (342/360) in the best tuned controller. These results suggest that agentic robotics can improve manipulation performance by identifying failure modes and modifying its executable control logic, rather than training separate policies for different objects.

Figure 3. Graph policy. The generated Graph-as-Policy graph routes each observation through plan selection to the strategy components, then re-observes and updates its belief. Blue nodes call the camera or robot; green nodes are agent-written scripts.

Code Representation Can Generalize Across Manipulation Strategies. Qualitative results show that the agent progressively develops distinct manipulation strategies to accommodate different object geometries and constraints, instead of a single grasp-and-rotate motion. For long-handled objects such as the claw hammer, the graph-based code performs an in-air bimanual handover, achieving ing 14/15 successes. For compact objects such as lock nuts, where a second grasp is difficult, the graph-based code instead uses table-supported manipulation, first standing the object on its edge and then pushing it beyond the tipping point, achieving 15/15 successes on both lock nuts. Wide objects such as multimeters are flipped through coordinated fingertip pushing without requiring a stable grasp, achieving 15/15 successes. When workspace constraints prevent a feasible handover, the graph-based code first repositions the object before attempting the flip, as demonstrated by tape measures (14/15) and pipe wrenches (13/15). Thin objects such as surgical scissors require guarded-contact grasping and slower rotation to prevent slippage, improving from 0/15 successes in earlier graph-based code versions to 13/15 in the final controller. Beyond selecting these strategies, the final code incorporates closed-loop recovery behaviors, including in-hand pose correction, grip verification, and additional pushing when an object only partially flips over.

Grasp selection for all 24 objects

Figure 4. Grasp selection for all 24 objects. Offline run of the policy on the saved episode-0 camera frames. Top-down view of the perceived height map, darker cells higher. Everything shown is computed from the cameras; the object mesh is never read.

Real-world trials

Figure 5. Real-world trials. Keyframes of one real trial per object, with successes out of three trials. The screwdriver trial shown failed. Curves are schematic.

Future Work. We evaluate the FlipAnything code in real-world experiments on three objects whose geometries were not encountered during simulation-based self-learning: a clamp, blue tape, and a foldable helix screwdriver. Without object-specific tuning, the code succeeds in 2 of 3 trials for the clamp, 3 of 3 trials for the blue tape, and 1 of 3 trials for the foldable helix screwdriver. We will study generalization to unseen objects in future work.

References

  1. [1]N. Chavan-Dafle, A. Rodriguez, R. Paolini, B. Tang, S. S. Srinivasa, M. Erdmann, M. T. Mason, I. Lundberg, H. Staab, and T. Fuhlbrügge. Extrinsic Dexterity: In-Hand Manipulation with External Forces. In IEEE International Conference on Robotics and Automation (ICRA), 2014. URL https://personalrobotics.cs.washington.edu/publications/chavandafle2014extrinsic/.
  2. [2]Y. Hou, Z. Jia, and M. T. Mason. Fast Planning for 3D Any-Pose-Reorienting using Pivoting. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 1631–1638, 2018. URL https://www.ri.cmu.edu/app/uploads/2018/03/ICRA2018_Reorienting_by_Pivoting.pdf.DOI
  3. [3]W. Zhou and D. Held. Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity. In Proceedings of the 6th Conference on Robot Learning, volume 205 of Proceedings of Machine Learning Research, pages 150–160. PMLR, 2023. URL https://proceedings.mlr.press/v205/zhou23a.html.arxiv.org/abs/2211.01500
  4. [4]S.-M. Yang, M. Magnusson, J. A. Stork, and T. Stoyanov. Learning Extrinsic Dexterity with Parameterized Manipulation Primitives. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 5404–5410. IEEE, 2024. URL https://arxiv.org/abs/2310.17785.
  5. [5]J. Yamada, A. L. Mitchell, J. Collins, and I. Posner. COMBO-Grasp: Learning Constraint-Based Manipulation for Bimanual Occluded Grasping. In Proceedings of the 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, pages 1102–1119. PMLR, 2025. URL https://proceedings.mlr.press/v305/yamada25a.html.arxiv.org/abs/2502.08054
  6. [6]K. Chen, S. Xie, U. Yoo, X. Zhu, C. Magner, A. Zhu, D. Kim, E. Heiden, E. Coumans, M. Macklin, S. Huang, and K. Goldberg. RoboSimStudio: An Interactive Toolkit for Human and Agent Rapid Real2Sim of Robot Manipulation Tasks, 2026. URL https://arxiv.org/abs/2610.18835.
  7. [7]K. Chen, S. Xie, L. Fu, J. Yu, W. Pacini, S. Bajamahal, H. Kim, J. Drake, D. Kim, H. Xue, J. Francis, C. Juette, P. Schdaldenbrand, M. Y. Seker, R. Wickramarachchi, U. Yoo, G. Wang, A. Murali, B. Sundaralingam, S. S. Sastry, Y. Zhu, L. J. Fan, S. Huang, and K. Goldberg. GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks, 2026. URL https://arxiv.org/abs/2607.05369.
  8. [8]Anthropic. Introducing Claude Fable 5.1 and Claude Mythos 5.1, September 2026. URL https://www.anthropic.com/claude-fable-and-mythos-5-1. Accessed October 26, 2026.
  9. [9]J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng. Code as Policies: Language Model Programs for Embodied Control. In IEEE International Conference on Robotics and Automation (ICRA), 2023. URL https://arxiv.org/abs/2209.07753.
  10. [10]M. Fu, J. Yu, K. El-Refai, E. Kou, H. Xue, H. Huang, W. Xiao, G. Wang, F.-F. Li, G. Shi, et al. Cap-X: A framework for benchmarking and improving coding agents for robot manipulation. arXiv preprint arXiv:2603.22435, 2026.arxiv.org/abs/2603.22435
  11. [11]E. Todorov, T. Erez, and Y. Tassa. MuJoCo: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5026–5033. IEEE, Oct. 2012. doi:10.1109/IROS.2012.6386109.DOI
  12. [12]X. Wei, M. Liu, Z. Ling, and H. Su. Approximate Convex Decomposition for 3D Meshes with Collision-Aware Concavity and Tree Search. ACM Transactions on Graphics, 41(4): 1–18, 2022. doi:10.1145/3528223.3530103. URL https://doi.org/10.1145/3528223.3530103.

Paper details

vlnc:W_vdpzsvvx
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents