# RIFT-1 — Frontier Repository Engineering Benchmark You ar...

Prompt

# RIFT-1 — Frontier Repository Engineering Benchmark You are the sole senior engineer working inside an existing robotics / semantic-exploration repository. Your task is to inspect the repository, understand its architecture, and implement a robust, deterministic, crash-recoverable semantic exploration subsystem. Do not create a disconnected toy demo. Integrate with the existing project and preserve useful functionality. ## Operating rules 1. Inspect the repository before modifying it. 2. Work autonomously; do not ask the user questions. 3. Preserve existing public interfaces where reasonable. 4. Do not use hosted APIs, cloud models, or network services. 5. Do not use simulator ground-truth object IDs for tracking, graph construction, planning, or actions. 6. Do not hard-code seeds, positions, labels, or expected test results. 7. Do not delete or weaken existing tests to obtain passing results. 8. Run commands yourself before claiming success. 9. Clearly disclose failures and incomplete work. 10. CPU execution is required. GPU acceleration is optional. 11. Avoid unbounded memory growth. 12. Prefer correctness, integration, and testing over UI polish. Maintain `EVAL_STATE.md` throughout the task with: - discovered architecture; - important invariants; - decisions; - completed work; - unresolved problems; - commands run; - next task. ## Goal The robot must explore an initially unknown environment, process noisy asynchronous multimodal observations, maintain stable object identities, build a temporally valid semantic scene graph, execute actions with enforced preconditions, persist state across crashes, and deterministically replay complete runs. --- # Required Python API Create: ```text frontier_eval/ __init__.py api.py ``` Expose: ```python class ExplorationSystem: def submit_frame(self, frame): ... def step_exploration(self): ... def execute_action(self, request): ... def get_graph(self): ... def get_metrics(self): ... def export_replay(self, destination): ... def close(self): ... def create_system(config, seed, storage_dir): ... def resume_system(storage_dir): ... def replay_system(replay_path, storage_dir): ... ``` Configuration must: - deeply merge with defaults; - reject unknown or invalid keys; - validate ranges and covariance; - support partial nested configuration. Example: ```python create_system( {"sensor": {"position_noise": 0.05}}, seed=42, storage_dir="./run" ) ``` must work. --- # Observation processing `submit_frame()` receives frames containing: - frame ID; - event time; - arrival time; - robot pose; - zero or more detections. Detections may contain: - modality; - label probability distribution; - position; - covariance; - embedding; - confidence; - attributes; - state. Frames may: - arrive out of order; - contain false positives; - contain multiple modalities observing the same object; - omit modalities; - contain duplicate IDs; - arrive too late. Duplicate frame submission must be idempotent. --- # Multimodal tracking Support at least: - RGB; - depth; - fused observations; - one configurable additional modality. Modality reliability must come from configuration and materially affect association and fusion. Implement global one-to-one tracking with explicit unassignment. Association must consider: - predicted position; - uncertainty/covariance; - label compatibility; - embedding similarity; - elapsed event time; - modality reliability; - observation confidence; - track lifecycle. Requirements: 1. Use Hungarian/global assignment or equivalent. 2. Invalid/gated pairs must remain unmatched. 3. Same-frame detections cannot update the same track unless fused beforehand. 4. Distinct same-label objects must remain distinct. 5. Same-class crossing objects should preserve identity when motion/embedding evidence supports it. 6. Temporary occlusion should preserve identity. 7. Lost tracks must eventually retire. 8. Retired objects must not advertise executable actions. 9. Use a motion model with uncertainty, e.g. Kalman filter. 10. Process using event time, not arrival order. 11. Track history must remain bounded. Tracks must expose: - stable ID; - tentative/confirmed/lost/retired lifecycle; - label probabilities; - position; - velocity; - covariance; - fused embedding; - first/last seen; - evidence count; - missed count; - modalities; - attributes/state; - association confidence. Document association cost and gates. --- # Out-of-order events Support configurable maximum lateness. Frames inside the lateness window must produce the same final canonical state as event-time-ordered submission. Frames older than the watermark must be rejected or quarantined, never silently applied. Behavior must be deterministic. --- # Semantic scene graph Maintain typed nodes: - robot; - room; - object; - surface; - container; - action; - frontier; - observation source. Required edges: - located-in; - near; - on; - inside; - observed-by; - connected-to; - supports-action; - targets; - carried-by. Nodes/edges must contain: - stable ID; - type; - confidence; - valid-from; - valid-to; - provenance; - revision. Requirements: 1. Graph results must not depend on insertion order. 2. Relations must update/invalidate after movement/state changes. 3. Historical relations may remain but must not appear active. 4. Retired tracks must expose no active action support. 5. Tracker IDs must never be treated as simulator IDs. 6. `get_graph()` must return deterministic canonical JSON. 7. Validate graph invariants automatically. Examples: - an object cannot be actively inside two exclusive containers; - a carried object cannot simultaneously be on a surface; - `inside` requires a container; - active edges must reference existing nodes. --- # Actions Use a configurable affordance registry. Support: - navigate-to; - inspect; - open; - close; - pick; - place-on; - place-inside; - release. Action failure should return structured information such as: ```json { "success": false, "code": "PRECONDITION_FAILED", "failed_preconditions": ["target.state.open == true"] } ``` Requirements: 1. Availability and execution must use the same preconditions. 2. Unavailable actions and reasons must be inspectable. 3. Cannot place inside a closed/non-container target. 4. Cannot place on a non-surface. 5. Cannot pick a fixed object. 6. Placement requires a carried object. 7. Successful actions must update authoritative state. 8. Effects must survive observations, restart, export, and replay. 9. Failed actions must not partially mutate state. 10. Duplicate action request IDs must be idempotent. --- # Autonomous exploration Implement real frontier-based exploration. The planner must: - detect reachable frontiers; - choose a frontier using a documented utility; - plan collision-free paths; - move incrementally; - rescan; - update graph/tracks; - replan after dynamic obstacles; - terminate only when no reachable frontier remains. Use A*, D* Lite, LPA*, or comparable path planning. A direct line plus hand-picked waypoints is insufficient. Expose: - status; - target frontier; - planned path; - explored percentage; - reachable unexplored cells; - distance travelled; - replanning count; - observations; - track counts; - graph revision; - termination reason. Do not use hidden map knowledge unavailable to the robot. Planner state must survive restart. --- # Persistence and replay Use an append-only event log as authoritative history. Persist: - config/seed; - robot movement; - observations; - graph-changing events; - actions/results; - planner decisions; - snapshots; - metrics. Requirements: 1. Restart must preserve the run. 2. `resume_system()` must recover current state. 3. Replay must reconstruct state by replaying events, not trusting a final serialized graph. 4. Replayed graph, metrics, robot pose, actions, tracks, and planner state must match. 5. Records must use checksums or corruption detection. 6. A corrupted final record must not destroy earlier valid state. 7. Recovery must stop at the last valid event and report corruption. 8. Writes must be atomic/transactionally safe. 9. Separate storage directories must remain isolated. 10. Do not use unsafe pickle-based replay import. --- # CLI Implement: ```bash python -m frontier_eval doctor --json python -m frontier_eval simulate \ --seed 42 --steps 500 \ --storage-dir ./runs/42 --json python -m frontier_eval replay \ --input ./runs/42/replay.jsonl \ --storage-dir ./runs/replayed --json python -m frontier_eval validate \ --storage-dir ./runs/42 --json python -m frontier_eval benchmark \ --seeds 11,22,33 \ --frames 1000 \ --objects 100 --json ``` Commands must: - use non-zero exit codes on failure; - provide machine-readable JSON; - avoid fabricated passing output; - work headlessly. `validate` must check log integrity, graph invariants, and deterministic replay. --- # Required tests Add tests for at least: 1. deterministic seed/config; 2. repeated observations deduplicate; 3. distant same-label objects remain separate; 4. incompatible labels remain unmatched; 5. one-to-one assignment; 6. crossing identities; 7. occlusion/re-identification; 8. multimodal fusion; 9. modality reliability changes fusion; 10. out-of-order frames; 11. too-late frame handling; 12. duplicate frame/action idempotency; 13. graph insertion-order independence; 14. stale relation invalidation; 15. retired objects lose actions; 16. action preconditions; 17. action effects survive future observations; 18. replay equivalence; 19. corrupted-tail recovery; 20. autonomous exploration completion; 21. dynamic-obstacle replanning; 22. unreachable-region termination; 23. multiple independent runs; 24. malformed replay rejection; 25. partial nested configuration; 26. bounded history; 27. existing repository regressions. Use property-based testing where useful. Do not duplicate implementation logic inside tests. --- # Performance Benchmark at least: - 1,000 frames; - 100 relevant tracked objects; - 500 graph edges. Requirements: - no full deep-copy of growing run state every frame; - no unbounded per-frame history growth; - no operation cubic in total historical observations; - report runtime and peak memory when possible. Correctness has priority over benchmark-specific optimization. --- # Documentation Document: - original and updated architecture; - tracker lifecycle; - association function; - uncertainty model; - event ordering policy; - graph invariants; - affordance system; - persistence/recovery; - planner; - simulator/real-robot boundary; - concurrency assumptions; - performance limitations; - remaining technical debt; - path toward ROS 2, RGB/depth sensors, and real odometry. Do not claim real-robot readiness unless tested. --- # Before stopping You must: 1. inspect changed files; 2. run existing tests; 3. run new tests; 4. run `doctor`; 5. run one deterministic simulation; 6. export and replay it; 7. compare canonical state; 8. corrupt the last event record and verify recovery; 9. run the benchmark; 10. inspect for duplicate tracks/stale relations; 11. review the diff for secrets/generated artifacts; 12. update `EVAL_STATE.md`; 13. create `EVAL_REPORT.md`. `EVAL_REPORT.md` must include: - implementation summary; - architecture changes; - exact commands executed; - exact test results; - benchmark timings; - replay result; - corruption-recovery result; - known failures; - incomplete requirements; - changed files; - major trade-offs. Never claim a command passed unless you actually ran it. Continue implementing and testing until the execution budget is exhausted. Do not stop after only producing a plan.