Agents · In progress
Milo: Embodied Robot Lab
Milo is a visual simulator for exploring what it takes for an agent to act in a physical space. A local model makes fast operational decisions while a larger model supervises tasks, with explicit request and token budgets. The operator cockpit shows a full 3D world, camera and map previews, live telemetry, and a console for conversation and traces. Scenarios range from apartments and kitchens to warehouses and parks, and every change is checked against a fixed regression suite.
- My role
- Creator and builder
- Started
- Updated
- Visibility
- Open Source
- Built with
- Python · React · 3D simulation · Local and cloud models · Mapping and navigation · Regression evaluation
- reproducible test scenarios
- 20+
- local model plus supervising model
- 2-tier
- request and token limits per task
- Budgeted
Problem
Agents that act in the physical world fail in very different ways than chat agents: collisions, lost localization, timeouts, and runaway inference cost. Those failures are hard to study without a reproducible environment.
Solution
A simulator with repeatable scenarios, scripted capability tests, real-model qualification runs, and a performance ledger that keeps requirements, evidence, and dated outcomes separate.
Architecture
A simulation backend with physics, sensors, and mapping exposes the robot to a two-tier agent setup: a local model for operational decisions and a supervising model for task-level guidance. A React cockpit and a test archive track every run, with power states and budgets to control inference cost.
Lessons learned
Embodied agents make evaluation discipline unavoidable. Separating scripted tests from real-model runs, and tracking regressions honestly, matters more than any single impressive demo run.
embodied-agents · robotics · simulation · evaluation