What ManiFlow-tb does
ManiFlow-tb is a low-level robotics policy designed for ToyBlocks, a simulated environment where a Franka arm manipulates blocks on a table. It receives camera images, the robot’s state, a task description, and a spatial target signal, then produces actions for carrying out the manipulation. Its role is to execute movements, not independently decide the full strategy for a complex task.
That distinction matters when assessing its capabilities. The model was evaluated with a reference high-level planner that supplied the plan for each recorded task. The published result therefore primarily evaluates the ManiFlow-tb controller in that setup; it does not measure how well an integrated system can perceive a new scene, choose a plan, and execute every step without assistance.
The model was released with weights and training code under Apache 2.0. That license allows those components to be used and modified subject to its terms, but it does not mean that running the system is cost-free. Simulation, data, accelerators, and dependencies have their own requirements; some ToyBlocks dependencies and assets may have different licenses.
ToyBlocks is built on Isaac Sim and Isaac Lab and provides demonstration generation, scenes, and closed-loop evaluation. The availability of ManiFlow-tb weights is not the same as a hosted API or a ready-to-operate robotics service. The documented usage path relies on ToyBlocks code and a policy server, which require the necessary infrastructure to be configured.
Inputs, outputs, and intended tasks
The model card specifies two RGB images: one from the front camera and one from the gripper camera, each at 224 × 224 pixels. It also receives a nine-dimensional robot state, the task name, and a target heatmap supplied by the high-level policy. The model combines these signals to determine how to continue the manipulation within ToyBlocks’ intended format.
The policy was trained on simulated tabletop block-manipulation data, including pick-and-place movements and in-place rotations. The described domain focuses on three blocks and stacking tasks; the model card does not present this checkpoint as a general-purpose controller for arbitrary robots, objects, or environments.
Its output is a segment of absolute actions: it predicts a horizon of 50 actions and executes 25 before generating or receiving the next segment. The final dimension represents an execution mode: continue, pause, or indicate that a skill has finished. The system must feed the execution state back when required; it is not interchangeable with a movement command.
The architecture uses a 12-layer, 8-head DiTX action head with width 768, 322.5 million parameters, and weights stored in BF16. These details describe the released checkpoint; they do not guarantee that a different configuration, precision conversion, or external control system will preserve the same behavior.
ToyBlocks can connect the low-level policy to a high-level agent. This design supports studying how target selection and execution interact. However, the target heatmap and communication protocol are part of the setup: replacing the planner, cameras, or robot requires checking that the inputs and actions still have the intended meaning.
Input and output contract at a glance
| Component | Published specification | Practical implication |
|---|---|---|
| Images | Front and gripper cameras; RGB, 224 × 224 | The policy depends on these views and their preprocessing; other cameras or resolutions require adaptation and evaluation. |
| State | Nine-dimensional robot-state vector | This is not a policy that acts from images alone. |
| Target | Task name and heatmap provided by the high-level component | The model assumes a compatible target interface. |
| Actions | Segments of 25 executed actions, with a prediction horizon of 50 | The controller must handle the continue, pause, and skill-finished modes. |
What the published results mean
The official model card reports four simulation evaluation runs on the same 128 validation episodes. In each run, the high-level component uses the plan from the recorded task; the goal is to measure the low-level component. The reported success rates were 57%, 59%, 59%, and 54%. The 54–59% range applies to those four runs, not to an independent measure across all ToyBlocks tasks.
The evaluation allows up to 194 seconds, calculated as three times the longest expert demonstration in that setup. The same model card warns that the default 30-second limit prevents most episodes from reaching completion: under that limit, between 26 and 28 of the 128 episodes succeeded. The time limit is therefore part of the result’s interpretation and should not be omitted when comparing scores.
The published success definition requires all goal predicates to hold, the blocks to be settled, and the gripper to be open. An episode ends when the plan is complete, a collision occurs, or the time limit is reached. The percentage is therefore not simply a visual judgment that a stack looks correct; it depends on the stated conditions and formal criterion.
The four runs do not produce identical results. The model card attributes variation to simulation and batched inference not being bitwise deterministic; the same scene can have different outcomes across runs. Two runs agreed on between 82 and 97 of the 128 episodes. A single summary number hides some of that variability.
The figures come from an evaluation reported by the team releasing the model. No independent reproduction of these numbers is available. Their practical value lies in the stated methodology and the published model, configuration, and statistics files—not in assuming that another lab has already confirmed the result.
ToyBlocks evaluation runs
All four runs used the same episodes and the published configuration. The success range summarizes those runs; they are not four independent datasets.
| Run | Successful episodes | Success rate | Success at episode end | Collisions |
|---|---|---|---|---|
| 1 | 73 of 128 | 57% | 64 | 32 |
| 2 | 75 of 128 | 59% | 67 | 32 |
| 3 | 76 of 128 | 59% | 66 | 30 |
| 4 | 69 of 128 | 54% | 63 | 38 |
Deployment costs and requirements
No official per-call price or commercial API service for this checkpoint is documented. The model card provides downloadable weights and training code, so usage costs depend on the infrastructure supplied by whoever runs it. Downloading the weights does not remove the costs of storage, processing, accelerators, electricity, or operations.
The figure of 322.5 million parameters describes model size, but does not by itself determine the hardware needed. BF16 format, batch size, image processing, inference frequency, and simulation also matter. The dossier provides no verified GPU-memory requirement for ManiFlow-tb, so estimates published for another ToyBlocks component should not be applied to it.
The published training run used three ToyBlocks-generated datasets, 200,000 steps, and eight GPUs, with a batch size of 128 per device. Those values describe training for the checkpoint; they are not minimum inference requirements or a promise that any system can reproduce the result exactly. The repository provides training scripts and configurations, but running them requires preparing compatible data and dependencies.
The evaluation workflow also involves the simulator, policy server, virtual cameras, and episode dataset. Simply loading the model does not necessarily reproduce the published test. For comparable results, the episode conditions and time limit must be held constant, and the model version and evaluation configuration must be recorded.
Practical planning before running the model
There is no single cost figure that applies to every deployment. Before planning a test, separate the model’s resources from those required for simulation and evaluation.
- 01Check that the system can load the BF16 weights and process visual inputs at the intended inference frequency.
- 02Budget separately for Isaac Sim, Isaac Lab, the policy server, and evaluation data.
- 03Decide whether the goal is inference only, additional training, or reproducing the evaluation; each has different requirements.
- 04Record memory use, runtime, completed episodes, and configuration; do not extrapolate the cost of one run to different infrastructure.
Limitations, safety, and alternatives
The checkpoint was trained on simulated data and is described for tabletop block manipulation with a Franka arm. The results do not demonstrate transfer to a physical robot, other cameras, everyday objects, or tasks outside the stated domain. Even within simulation, the score depends on the reference plan and the time allowed.
The model card states that the model has not undergone safety benchmarking. This means there is no published safety benchmark for the model; by itself, it does not establish that the system is safe or unsafe. The policy should be treated as a research component, with appropriate operating limits, supervision, and evaluation before connection to hardware.
The model is not an autonomous solution for an end-to-end task. A mistaken high-level decision may become a coherent but unsuitable action; conversely, a correct instruction may fail during execution. ToyBlocks allows components to be replaced with oracle-style references to investigate where a failure occurs, but that comparison does not make the remaining components infallible.
The right alternative depends first on the experimental question. For studying low-level visual control on the specific ToyBlocks task, ManiFlow-tb provides weights and code alongside a documented evaluation. A complete strategy also requires evaluating the high-level planner. If the robot, camera, or objects differ, compatibility must be validated and performance measured in that configuration rather than assumed to transfer.
The primary ManiFlow-tb and ToyBlocks materials are released under Apache 2.0, but the ToyBlocks repository notes that submodules and downloaded assets may have their own licenses. Open weights do not automatically make every dependency reusable under the same terms. Teams should review the license for each component they incorporate.
Choosing based on intended use
| Goal | When ManiFlow-tb may fit | What to check first |
|---|---|---|
| Study low-level execution in ToyBlocks | When the task, inputs, and simulated arm match the intended environment | Use the published time limit and success criteria; keep track of versions and episodes. |
| Evaluate a complete robotic system | Only as one component in the evaluation | Measure perception, plan selection, execution, and failure recovery separately. |
| Use a different robot or environment | As a research starting point, not an already validated controller | Check state, actions, cameras, actuators, physics, safety, and performance in the new setup. |
| Estimate operating costs | When suitable infrastructure is available and dependencies can be installed | Measure model and simulator resources separately; do not assume a hosted API or published price. |
Keep exploring
Sources consulted
Corrections and transparency
If you spot incorrect or outdated information, send us a correction with the page and source we should review.
Submit a correction