feat: improve operator progress and recording accuracy

artemis/agents/operator/operator.json

@@ -1,3 +1,3 @@
 {
-  "main_template": "# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.\n  2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidirectional; never reverse direction merely to prove completion. If the task declares an earlier exit condition (for example, find one verified match or process N items), that explicit condition may end the traversal without reaching the list boundary.\n  3. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.\n  4. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed with confirmed anchor continuity and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for \"double-checking\", even if older execution history has been pruned. Resume from the last recorded anchor and direction. Only the single bounded recovery above, or a refreshed/mutated list, permits limited re-verification; otherwise continue the single pass over new content.\n- **Dynamic Monitoring & Reactive Loop Protocol (Monitor-Trigger-Action)**: When executing an active milestone matching `[Loop] <Goal> into note <TargetNote> (Exit: <ExitCondition>; Interval: <Interval>)` or `[Loop:continuous] <Goal> into note <TargetNote> (Interval: <Interval>)` (the milestone grammar itself is defined under Task Plan Grammar below), run this 5-step cyclic state machine:\n  - **Step 1 (Baseline Sync)**: On the initial cycle, observe all visible entries/metrics in the primary monitoring view and save the full snaLine truncated
+  "main_template": "# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.\n  2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidirectional; never reverse direction merely to prove completion. If the task declares an earlier exit condition (for example, find one verified match or process N items), that explicit condition may end the traversal without reaching the list boundary.\n  3. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.\n  4. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed with confirmed anchor continuity and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for \"double-checking\", even if older execution history has been pruned. Resume from the last recorded anchor and direction. Only the single bounded recovery above, or a refreshed/mutated list, permits limited re-verification; otherwise continue the single pass over new content.\n- **Dynamic Monitoring & Reactive Loop Protocol (Monitor-Trigger-Action)**: When executing an active milestone matching `[Loop] <Goal> into note <TargetNote> (Exit: <ExitCondition>; Interval: <Interval>)` or `[Loop:continuous] <Goal> into note <TargetNote> (Interval: <Interval>)` (the milestone grammar itself is defined under Task Plan Grammar below), run this 5-step cyclic state machine:\n  - **Step 1 (Baseline Sync)**: On the initial cycle, observe all visible entries/metrics in the primary monitoring view and save the full snaLine truncated
 }

artemis/agents/operator/operator.py

@@ -46,6 +46,7 @@ from artemis.utils.decorators import wrap_with_callbacks
 from artemis.utils.element_hit_test import find_element_at_point
 from artemis.utils.logger import get_logger
 from artemis.utils.notes import get_note_file_path
+from artemis.utils.plan_grammar import parse_plan
 from artemis.memory.context_policy import build_history_for
 from artemis.utils.task_tree import get_active_subgoal_hashes
 from artemis.utils.visualization import format_minimal_list_with_elements
@@ -77,7 +78,8 @@ from artemis.agents.operator.prompts import (
     ExecutionIncidentPromptComponent,
     CheckItemsExplainerPromptComponent,
     BackgroundTasksPromptComponent,
-    TaskPlanWarningPromptComponent,
+    render_plan_ledger_bounce,
+    unwritten_action_streak,
     ToolLimitWarningPromptComponent,
     InjectedInstructionPromptComponent,
     render_transcript_static_system,
@@ -120,6 +122,8 @@ class OperatorNode:
         # Index into the rendered message list where this turn's tail begins;
         # set by the transcript build and consumed after the tool loop.
         self._transcript_turn_base: int | None = None
+        self._turn_plan_baseline: str | None = None
+        self._turn_ledger_streak: int = 0
 
     def _available_device_actions(self) -> frozenset[str]:
         """Device actions the installed actuator backend provides.
@@ -216,7 +220,6 @@ class OperatorNode:
                 (InjectedInstructionPromptComponent(), {}),
                 (BackgroundTasksPromptComponent(), {}),
                 (FeedbackPromptComponent(), {}),
-                (TaskPlanWarningPromptComponent(), {}),
                 (ToolLimitWarningPromptComponent(), {}),
             ]
         else:
@@ -230,6 +233,7 @@ class OperatorNode:
             kwargs_to_pass = {
                 "prompts": self.prompts,
                 "plan_and_history": plan_and_history,
+                "task_plan": task_plan,
                 "latest_screenshot_b64": latest_screenshot_b64,
                 "minimal_list": minimal_list,
                 "current_step_num": current_step_num,
@@ -378,7 +382,6 @@ class OperatorNode:
             (InjectedInstructionPromptComponent(), {}),
             (BackgroundTasksPromptComponent(), {}),
             (FeedbackPromptComponent(), {}),
-            (TaskPlanWarningPromptComponent(), {}),
             (ToolLimitWarningPromptComponent(), {}),
         ]
         for component, extra_kwargs in components:
@@ -433,6 +436,7 @@ class OperatorNode:
         raw_thoughts = []
         native_thoughts = []
         tool_limit_exceeded = False
+        plan_gate_bounced = False
 
         for iteration in range(max_iterations):
             if iteration == max_iterations - 1:
@@ -709,7 +713,25 @@ class OperatorNode:
                 )
                 actions_list = []
                 burst_cap = self._max_burst_actions()
-                if len(action_calls) > burst_cap:
+                bounce = None
+                if (
+                    not plan_gate_bounced
+                    and iteration < max_iterations - 1
+                    and len(action_calls) == 1
+                    and not getattr(state, "open_incident", None)
+                    and self._plan_changed_this_turn() is False
+                ):
+                    bounce = self._plan_ledger_bounce_reason()
+                if bounce:
+                    plan_gate_bounced = True
+                    validation_errors = True
+                    logger.info(f"Plan ledger gate: bouncing action. {bounce}")
+                    for tc in action_calls:
+                        tool_outputs.append(
+                            ToolMessage(tool_call_id=tc["id"], content=bounce, status="error")
+                        )
+                    action_calls_to_translate = []
+                elif len(action_calls) > burst_cap:
                     # A multi-action turn is a fast-action burst that the Validator
                     # fires without the safety net; cap its length before it runs.
                     validation_errors = True
@@ -730,6 +752,11 @@ class OperatorNode:
                         )
                     action_calls_to_translate = []
                 else:
+                    if plan_gate_bounced and self._plan_changed_this_turn() is False:
+                        logger.warning(
+                            "Plan ledger gate: executing actions after one bounce without"
+                            " a task_plan change (ledger gap this turn)."
+                        )
                     action_calls_to_translate = action_calls
                 for tc in action_calls_to_translate:
                     translated_actions, error = self._translate_and_validate_tool(tc, state)
@@ -838,6 +865,8 @@ class OperatorNode:
 
         # 3. Get history and plan
         steps, task_plan = self._get_history_and_plan()
+        self._turn_plan_baseline = self._plan_ledger_baseline(task_plan)
+        self._turn_ledger_streak = unwritten_action_streak(steps)
 
         # 4. Format minimal list
 
@@ -940,6 +969,63 @@ class OperatorNode:
             "operator_tool_limit_exceeded": tool_limit_exceeded,
         }
 
+    def _plan_ledger_gate_enabled(self) -> bool:
+        try:
+            from artemis.config import load_agent_config
+
+            return bool(load_agent_config().pro.execution.plan_ledger_gate)
+        except (AttributeError, OSError, TypeError, ValueError):
+            return True
+
+    def _read_task_plan_text(self) -> str | None:
+        """Current ``task_plan`` note text, or ``None`` when it cannot be read."""
+        engine = getattr(self.ctx, "data_engine", None)
+        if not engine:
+            return None
+        try:
+            path = get_note_file_path(engine.base_dir, "task_plan")
+            if not path.exists():
+                return None
+            return path.read_text(encoding="utf-8")
+        except (AttributeError, OSError, TypeError) as e:
+            logger.warning(f"Plan ledger gate: failed to read task_plan: {e}")
+            return None
+
+    def _plan_ledger_baseline(self, task_plan: str) -> str | None:
+        """Return the plan text used as the current turn's baseline."""
+        if not self._plan_ledger_gate_enabled() or not getattr(self.ctx, "data_engine", None):
+            return None
+        if not parse_plan(task_plan).has_top_level:
+            return None
+        return task_plan
+
+    def _plan_ledger_stale_turns(self) -> int:
+        try:
+            from artemis.config import load_agent_config
+
+            return int(load_agent_config().pro.execution.plan_ledger_stale_turns)
+        except (AttributeError, OSError, TypeError, ValueError):
+            return 4
+
+    def _plan_ledger_bounce_reason(self) -> str | None:
+        """Bounce message when the (unchanged) ledger is broken or stale, else ``None``."""
+        current = self._read_task_plan_text()
+        if current is None:
+            return None
+        return render_plan_ledger_bounce(
+            current, self._turn_ledger_streak, self._plan_ledger_stale_turns()
+        )
+
+    def _plan_changed_this_turn(self) -> bool | None:
+        """Whether ``task_plan`` differs from the turn's baseline (``None``: gate inactive)."""
+        baseline = self._turn_plan_baseline
+        if baseline is None:
+            return None
+        current = self._read_task_plan_text()
+        if current is None:
+            return None
+        return current != baseline
+
     def _get_active_subgoal_hash(self) -> str:
         """Parses task_plan.md to find the active top-level subgoal hash."""
 

artemis/agents/operator/prompts.py

@@ -652,41 +652,69 @@ class BackgroundTasksPromptComponent(PromptComponent):
             builder.add_human_content("\n".join(lines) + "\n")
 
 
-class TaskPlanWarningPromptComponent(PromptComponent):
-    """Suggests a task_plan update after an action turn that did not touch it.
+_PLAN_NOTE_TOOLS = ("update_note", "save_note", "append_note")
 
-    Only the most recent step is inspected: if it executed a Turn-Ending
-    Action and no note tool wrote to ``task_plan``, a soft suggestion is
-    injected into the next turn. Pure diagnosis / waiting / observation turns
-    legitimately leave the plan alone and get no reminder.
-    """
 
-    NOTE_TOOLS = ("update_note", "save_note", "append_note")
+def step_updated_task_plan(step: dict) -> bool:
+    """Whether a recorded step's tool calls wrote the ``task_plan`` note."""
+    for tc in step.get("tool_calls", []):
+        if tc.get("name") not in _PLAN_NOTE_TOOLS:
+            continue
+        args = tc.get("args", {})
+        note_key = args.get("key") or args.get("name")
+        if note_key == "task_plan":
+            return True
+    return False
 
-    @classmethod
-    def _step_updated_task_plan(cls, step: dict) -> bool:
-        for tc in step.get("tool_calls", []):
-            if tc.get("name") not in cls.NOTE_TOOLS:
-                continue
-            args = tc.get("args", {})
-            note_key = args.get("key") or args.get("name")
-            if note_key == "task_plan":
-                return True
-        return False
 
-    async def __call__(self, builder: PromptBuilder, state: State, ctx: ArtemisContext, **kwargs):
-        steps = kwargs.get("steps", [])
-        if not steps:
-            return
-        last = steps[-1]
-        if not last.get("action_taken") or self._step_updated_task_plan(last):
-            return
-        builder.add_human_content(
-            "\nReminder: your last turn executed a Turn-Ending Action without"
-            " updating task_plan. If progress was made, record it now with"
-            " surgical `update_note` edits: mark what completed and add the"
-            " subtasks you are now pursuing."
+def unwritten_action_streak(steps: list[dict]) -> int:
+    """Count trailing action steps since the last ``task_plan`` write."""
+    streak = 0
+    for step in reversed(steps or []):
+        if step_updated_task_plan(step):
+            break
+        if step.get("action_taken"):
+            streak += 1
+    return streak
+
+
+def _clip(text: str, limit: int = 80) -> str:
+    text = " ".join(text.split())
+    return text if len(text) <= limit else text[: limit - 1] + "\u2026"
+
+
+_LEDGER_BOUNCE_TAIL = (
+    " with `update_note` (key `task_plan`), then re-issue the action in the same"
+    " tool-call list (Live Sub-goal Ledger)."
+)
+
+
+def render_plan_ledger_bounce(task_plan: str, streak: int, stale_turns: int) -> str | None:
+    """Return a plan-ledger validation error, if any."""
+    snapshot = parse_plan(task_plan or "")
+    if not snapshot.has_top_level or snapshot.all_top_level_done:
+        return None
+    milestone = snapshot.active_milestone()
+    if milestone is None:
+        return (
+            "Not executed: no milestone is marked in progress. Mark the one you are"
+            " executing `[/]` and open its `[/]` sub-goal" + _LEDGER_BOUNCE_TAIL
         )
+    leaf = snapshot.active_leaf(milestone)
+    if leaf is None:
+        return (
+            f'Not executed: the active milestone "{_clip(milestone.text)}" has no'
+            " in-progress `[/]` sub-goal beneath it. Open one for what you are doing now"
+            + _LEDGER_BOUNCE_TAIL
+        )
+    turns = streak + 1
+    if stale_turns > 0 and turns >= stale_turns:
+        return (
+            f'Not executed: the sub-goal "{_clip(leaf.text)}" has stayed in progress for'
+            f" {turns} action turns without any plan change. Settle it (`[x]` / `[!]`) or"
+            " record its progress and open the next leaf" + _LEDGER_BOUNCE_TAIL
+        )
+    return None
 
 
 class ToolLimitWarningPromptComponent(PromptComponent):

artemis/config/agent.py

@@ -663,6 +663,21 @@ class ExecutionConfig(BaseModel):
             " rejected before execution and fed back to the Operator."
         ),
     )
+    plan_ledger_gate: bool = Field(
+        default=True,
+        description=(
+            "Require active plan sub-goals and periodically prompt the Operator to"
+            " update stale plan progress."
+        ),
+    )
+    plan_ledger_stale_turns: int = Field(
+        default=4,
+        ge=0,
+        description=(
+            "Action turns allowed without a task_plan change before the ledger gate"
+            " prompts for an update. Set to 0 to disable staleness checks."
+        ),
+    )
 
 
 class ProProfileConfig(BaseModel):

artemis/controllers/unified_controller.py

@@ -39,6 +39,7 @@ from artemis.utils.video import (
     DEFAULT_MAX_DURATION_SECONDS,
     RecordingSession,
     VideoRecordingResult,
+    await_scrcpy_first_frame,
     build_scrcpy_record_command,
     cleanup_video_segments,
     concatenate_videos,
@@ -346,7 +347,8 @@ class UnifiedMobileController:
 
             # Strict timeline alignment:
             # DataEngine session start time is the reference T0 (0.0s).
-            # Video recording start time is session.start_time.
+            # session.start_time is the wall-clock time of the first frame in
+            # the recording (the file's t=0), estimated at recording start.
             offset = 0.0
             if session.data_engine_start_time is not None:
                 offset = max(0.0, session.start_time - session.data_engine_start_time)
@@ -517,14 +519,19 @@ class UnifiedMobileController:
         process = await self._spawn_scrcpy(
             build_scrcpy_record_command("scrcpy", session.device_id, new_video_path)
         )
-        await asyncio.sleep(0.8)
+        spawned_at = time.time()
+        first_frame_at = await await_scrcpy_first_frame(process, spawned_at)
         if process.returncode is not None:
             stderr = await process.stderr.read()
             logger.error(f"Failed to start scrcpy segment: {stderr.decode(errors='replace')}")
             return False
         session.process = process
         session.local_video_path = new_video_path
-        session.android_segment_started_at = time.time()
+        session.android_segment_started_at = first_frame_at
+        logger.info(
+            f"scrcpy segment {session.android_segment_index} first frame estimated "
+            f"{first_frame_at - spawned_at:.2f}s after spawn"
+        )
         if display_state:
             session.android_rotation, session.capture_width, session.capture_height = display_state
         return True
@@ -621,7 +628,52 @@ class UnifiedMobileController:
             if display_state:
                 session.capture_width, session.capture_height = display_state[1:]
 
-            # Persist to local database if Data Engine is active
+            # Start scrcpy in background
+            cmd = build_scrcpy_record_command("scrcpy", device_id, local_video_path)
+
+            process = await self._spawn_scrcpy(cmd)
+            spawned_at = time.time()
+
+            session.process = process
+            set_active_session(device_id, session)
+
+            logger.info(f"Started scrcpy recording on {device_id}, saving to {local_video_path}")
+
+            # The file's t=0 is the first captured frame, about a second after
+            # spawn. Anchor the recording timeline there: segment offsets and
+            # analyzer clip trimming all treat session.start_time as that t=0.
+            first_frame_at = await await_scrcpy_first_frame(process, spawned_at)
+            if process.returncode is not None:
+                stderr = await process.stderr.read()
+                err_msg = stderr.decode()
+                logger.error(f"scrcpy failed to start on {device_id}: {err_msg}")
+                if self.ctx and self.ctx.data_engine:
+                    self.ctx.data_engine.record_video_start(
+                        video_id=video_id,
+                        device_id=device_id,
+                        local_video_path=local_video_path,
+                        start_time=session.start_time,
+                    )
+                self._record_recording_failure(session, f"scrcpy failed to start: {err_msg}")
+                remove_active_session(device_id)
+                return VideoRecordingResult(
+                    success=False,
+                    message=f"scrcpy failed to start: {err_msg}",
+                )
+
+            session.start_time = first_frame_at
+            session.android_segment_started_at = first_frame_at
+            timeline_note = ""
+            if session.data_engine_start_time is not None:
+                timeline_note = (
+                    "; recording timeline offset from session start: "
+                    f"{first_frame_at - session.data_engine_start_time:.2f}s"
+                )
+            logger.info(
+                f"scrcpy first frame estimated {first_frame_at - spawned_at:.2f}s after spawn"
+                f"{timeline_note}"
+            )
+
             if self.ctx and self.ctx.data_engine:
                 self.ctx.data_engine.record_video_start(
                     video_id=video_id,
@@ -630,30 +682,6 @@ class UnifiedMobileController:
                     start_time=session.start_time,
                 )
 
-            # Start scrcpy in background
-            cmd = build_scrcpy_record_command("scrcpy", device_id, local_video_path)
-
-            process = await self._spawn_scrcpy(cmd)
-
-            session.process = process
-            set_active_session(device_id, session)
-
-            logger.info(f"Started scrcpy recording on {device_id}, saving to {local_video_path}")
-
-            # Check if process failed immediately
-            await asyncio.sleep(0.8)
-            if process.returncode is not None:
-                stderr = await process.stderr.read()
-                err_msg = stderr.decode()
-                logger.error(f"scrcpy failed to start on {device_id}: {err_msg}")
-                self._record_recording_failure(session, f"scrcpy failed to start: {err_msg}")
-                remove_active_session(device_id)
-                return VideoRecordingResult(
-                    success=False,
-                    message=f"scrcpy failed to start: {err_msg}",
-                )
-
-            # Start background watchdog to auto-recover if scrcpy terminates
             session.watchdog_task = asyncio.create_task(self._recording_watchdog(device_id))
 
             return VideoRecordingResult(

artemis/utils/plan_grammar.py

@@ -185,6 +185,45 @@ class PlanSnapshot:
                 parent = candidate
         return parent
 
+    def children_of(self, item: PlanItem) -> tuple[PlanItem, ...]:
+        """Nested items under a top-level item (up to the next top-level line)."""
+        if not item.is_top_level:
+            return ()
+        children: list[PlanItem] = []
+        for candidate in self.items:
+            if candidate.line_no <= item.line_no:
+                continue
+            if candidate.is_top_level:
+                break
+            children.append(candidate)
+        return tuple(children)
+
+    def active_milestone(self) -> PlanItem | None:
+        """The milestone currently being executed.
+
+        An explicitly active top-level item wins; otherwise the parent of the
+        bottom-most active nested item; otherwise, when every item is still
+        untouched, the first pending milestone (the same fallback the step
+        stamping uses). ``None`` when nothing is in progress.
+        """
+        for item in self.top_level:
+            if item.is_active:
+                return item
+        nested_active = self.last_active()
+        if nested_active is not None:
+            return self.parent_of(nested_active)
+        if self.top_level and all(item.is_pending for item in self.top_level):
+            return self.top_level[0]
+        return None
+
+    def active_leaf(self, milestone: PlanItem) -> PlanItem | None:
+        """Bottom-most ``[/]`` nested item under ``milestone`` (the current sub-goal)."""
+        leaf = None
+        for child in self.children_of(milestone):
+            if child.is_active:
+                leaf = child
+        return leaf
+
     def check_items_of(self, item: PlanItem) -> tuple[CheckItem, ...]:
         """Check lines anchored to the given top-level subgoal."""
         return tuple(ci for ci in self.check_items if ci.parent_key == item.key)
@@ -394,7 +433,7 @@ def unintended_milestone_edits(before: PlanSnapshot, after: PlanSnapshot) -> lis
 _PLAN_GRAMMAR_BASE = f"""### Task Plan Grammar (machine-enforced contract)
 Every checklist line MUST match `<indent>- [<status>] <text>`:
 - Status characters: `[{STATUS_PENDING}]` pending, `[{STATUS_ACTIVE}]` in progress, `[{STATUS_DONE}]` completed, `[{STATUS_BLOCKED}]` blocked.
-- Top-level lines (no indentation) are strategic milestones; their sub-tasks are indented by 2 spaces. All other text in the note is free-form context.
+- Top-level lines (no indentation) are strategic milestones. Nested lines are their sub-goals: 2 spaces for a sub-goal, 4 spaces for a sub-sub-goal beneath it (deeper nesting is allowed). Only zero-indent lines count as milestones for termination and validation; nested lines are the executor's live ledger. All other text in the note is free-form context.
 - `{LOOP_TAG}` tags a BOUNDED iterative milestone: declare its exit boundary (e.g., `(Exit: <condition>; Interval: <cadence>)`) and mark it `[{STATUS_DONE}]` only once that exit condition is verifiably met.
 - `{CONTINUOUS_LOOP_TAG}` tags an UNBOUNDED continuous-monitoring milestone: it must stay `[{STATUS_ACTIVE}]` and can never be marked `[{STATUS_DONE}]`, deleted, or untagged by you — the system mechanically rejects such edits. Only an explicit external stop signal injected by the user unlocks its completion; that signal is an external interruption delivered by the system, never something inferred from the screen or the plan.
 - The task terminates only when every top-level milestone is `[{STATUS_DONE}]`."""

artemis/utils/task_tree.py

@@ -825,11 +825,13 @@ def build_plan_and_history(
 
 
 def get_active_subgoal_hashes(task_plan: str) -> tuple[str, str | None]:
-    """Parses task_plan content to find the active top-level subgoal hash.
+    """Parses task_plan content to find the active (top-level, nested) subgoal hashes.
 
-    Consolidates all active subtasks under their parent Level 1 subgoal,
-    returning (parent_hash, None). Falls back to the first pending subgoal if no
-    active subgoal is found and all are pending.
+    Returns ``(parent_hash, sub_hash)``: the parent is the active top-level
+    milestone (all active nested items consolidate under it); the sub hash is
+    the bottom-most active nested item — the live sub-goal ledger leaf — or
+    ``None`` when the milestone has no in-progress sub-goal. Falls back to the
+    first pending subgoal if no active subgoal is found and all are pending.
     """
     if not task_plan:
         return "default", None
@@ -843,7 +845,7 @@ def get_active_subgoal_hashes(task_plan: str) -> tuple[str, str | None]:
             if not active_item.is_top_level:
                 parent = snapshot.parent_of(active_item)
                 if parent is not None:
-                    return parent.key, None
+                    return parent.key, active_item.key
             return active_item.key, None
 
         # No active subgoal found. Check if all are completed.

artemis/utils/video.py

@@ -24,6 +24,7 @@ from pathlib import Path
 import platform
 import re
 import shutil
+import time
 import subprocess
 from typing import Any
 from uuid import UUID
@@ -37,6 +38,11 @@ logger = get_logger(__name__)
 
 DEFAULT_MAX_DURATION_SECONDS = 900  # 15 minutes
 ANDROID_RECORDING_SEGMENT_SECONDS = 1800
+# The recording marker closely tracks the first frame; startup measurements
+# put the fallback near three quarters of a second after process creation.
+SCRCPY_RECORDING_STARTED_MARKER = "Recording started"
+SCRCPY_STARTUP_FALLBACK_SECONDS = 0.75
+SCRCPY_STARTUP_TIMEOUT_SECONDS = 6.0
 VIDEO_READY_DELAY_SECONDS = 1
 ANDROID_DEVICE_VIDEO_PATH = "/sdcard/screen_recording.mp4"
 ANDROID_MAX_RECORDING_DURATION_SECONDS = 180  # Android screenrecord limit
@@ -78,6 +84,43 @@ def build_scrcpy_record_command(
     return command
 
 
+async def await_scrcpy_first_frame(
+    process: Any,
+    spawned_at: float,
+    *,
+    timeout: float = SCRCPY_STARTUP_TIMEOUT_SECONDS,
+) -> float:
+    """Estimate when the first frame of a scrcpy recording reached the host."""
+    fallback = spawned_at + SCRCPY_STARTUP_FALLBACK_SECONDS
+    reader = getattr(process, "stdout", None)
+    if not isinstance(reader, asyncio.StreamReader):
+        await asyncio.sleep(0.8)
+        return fallback
+
+    deadline = time.monotonic() + timeout
+    while True:
+        remaining = deadline - time.monotonic()
+        if remaining <= 0:
+            logger.warning(
+                f"scrcpy did not report '{SCRCPY_RECORDING_STARTED_MARKER}' within "
+                f"{timeout:.1f}s; assuming the first frame at spawn + "
+                f"{SCRCPY_STARTUP_FALLBACK_SECONDS:.2f}s"
+            )
+            return fallback
+        try:
+            line = await asyncio.wait_for(reader.readline(), timeout=remaining)
+        except TimeoutError:
+            continue
+        if not line:
+            try:
+                await asyncio.wait_for(process.wait(), timeout=1.0)
+            except (AttributeError, ProcessLookupError, TimeoutError):
+                pass
+            return fallback
+        if SCRCPY_RECORDING_STARTED_MARKER in line.decode(errors="replace"):
+            return max(spawned_at, time.time())
+
+
 class _CyFunctionDetectorMeta(type):
     def __instancecheck__(self, instance):
         name = type(instance).__name__