Performance · Published 26 August 2026
Benchmarking an AI 3D Director from prompt to stable frame
A reproducible protocol for planning latency, tool validity, scene mutation, save time, frame budget, and recovery.
Split the prompt pipeline
Measure first token, first valid tool call, plan completion, mutation validation, asset resolution, durable save, first rendered frame, and interactive frame stability separately.
Publish scene size, asset mix, browser, GPU, network, model configuration, cache state, and revision state.
Validity is a performance metric
Count malformed SSE events, rejected tool calls, missing targets, asset failures, collisions, unsupported exports, and recovery attempts. A quick unusable scene is not a successful run.
Track the fraction of prompts that reach an editable, durably saved scene without manual repair.
Large scenes need bounded UI work
Measure draw calls, triangle count, texture memory, scene-tree rendering, selection latency, and frame time as object count grows. Virtualize lists and decouple editor panels from the render loop.
Test reduced-motion and low-memory conditions as well as a flagship desktop GPU.
Compare jobs, not demos
A mesh generator, visual 3D editor, game engine, and AI scene composer overlap but optimise different steps. Compare editability, hierarchy, collaboration, export, runtime interaction, and failure recovery independently.
Use current vendor docs for mutable limits and pricing.
Production checklist
- Verify Revision safety against the deployed environment, not a screenshot.
- Verify Owner isolation against the deployed environment, not a screenshot.
- Verify Strict streaming against the deployed environment, not a screenshot.
- Verify State-preserving recovery against the deployed environment, not a screenshot.