Meta Built a 120-Task Test for Glasses Assistants Where the Only Passing Grade Is the State of the Room

Meta researchers pushed a repository called InteractiveWorld to the facebookresearch GitHub organisation on October 24. It is the code for a paper titled InteractiveWorld: A Verifiable Human-Centric Environment for Building Better Interactive MLLM Assistants, and it targets a category of assistant that most benchmarks handle badly: the kind that sits on someone's face and talks while the person moves around a house.

The mismatch is easy to state. A standard multimodal benchmark shows a model a fixed image or clip and grades the text it writes back. A wearable assistant has a body to talk to, and that body changes the scene. If the assistant says the keys are on the shelf behind you, the person turns, moves a jacket, and the next frame is different. Whether the advice worked depends on what happened in the room, not on how fluent the sentence was.

The simulator is built on Isaac Sim 5.1 and OmniGibson 3.9.0, and ships 44 BEHAVIOR-1K household and commercial scenes plus 6,831 baked objects, a dataset of roughly 31 gigabytes. The human is an SMPL+H body driven by short commands such as walk, grab, put and sit, with planning for where to stand, how to approach and how to grip, drawing walking motion from the ACCAD capture set. It covers 36 action families, from opening doors and cutting to stirring, pouring, wiping, using a computer and answering a phone. Runs need a Linux host with an NVIDIA GPU, tested on an A100 with 80 gigabytes, and about 100 gigabytes of disk.

InteractiveBench sits on top: 120 frozen multi-turn tasks, 20 in each of six categories. Guidance walks a user through a task step by step. Planning asks for a plan inside a time and resource budget and then a replan when it breaks. Accessibility withholds images from the simulated human, so the assistant has to do the seeing. Decision requires combining what is visible with world knowledge. Prevention is scored on restraint, warning before a mistake without interrupting needlessly. Finding tests memory, asking where an object went after a prologue the assistant watched but was not asked about.

An episode runs as a loop. The simulator replays a prologue of what the person did before asking, then the human speaks the request, and turns alternate: the assistant sees only frames and dialogue and can reply by voice, stay silent, or declare the task done, while the human responds with speech and simulator commands. A run ends when the assistant declares completion, the human finishes, or the tick budget runs out. Twenty-two tasks, all of Prevention plus one Decision and one Finding task, keep the human acting continuously so the assistant is watching rather than being asked.

Scoring is where the design gets interesting. A native task score is available immediately: strict success requires the final world state to satisfy every goal predicate plus any process requirements, and a completion score records the fraction of goals met. The official numbers come from a separate release rubric of 464 criteria across six protocols, which requires a human to read transcripts and watch video before a judgment file is written and the run is re-scored. Runs that break because the simulator, the human model or the assistant transport failed are marked invalid and reported separately rather than counted as failures.

The gaps are worth stating plainly. The repository publishes no model scores and no leaderboard, and the paper itself was not available at release. The code carries a non-commercial CC BY-NC 4.0 licence, the scenes are indoor only, and each task takes five to twenty minutes, so a full sweep of 120 tasks needs multiple GPUs and hours. The simulated human is fixed at Gemini 3.1 Pro for comparability, which means a score is a score against that particular user model, not a general claim about a pair of glasses.

What survives all that is the evaluation pattern, and it is the reason to read the repository even without numbers. Separate the assistant from a fixed simulated user, give the assistant only the input the device would actually have, treat silence as a legal action, and grade on the end state of the world rather than on a model reading a transcript. That pattern can be rebuilt in a much lighter environment than Isaac Sim, and it catches the failure mode every assistant on a face will hit, which is sounding helpful while the person ends up with the wrong snack.