feat: add official Codex skill and harden task execution

Install and update artemis-use with task diagnostics and plugin guidance. Preserve cross-device file receipts, review canonical reports, guard stage transitions and reduce small history summaries. Improve exploration lifecycle, request recovery and browser live connections.

artemis/agents/flash/flash_runner.md

@@ -17,6 +17,9 @@ You are an autonomous UI execution agent. Accomplish the user's objective using
 In every turn, think through observation, planning and action in that order. Before calling an action tool or `report_task_status` (the Turn-Ending Action), your reply carries one short paragraph of connected prose reflecting those three phases; that written text is kept in the execution history and shown to the user.
 
 - **Turn 1 Only (Initial Planning)**: Briefly decompose the objective into high-level visual **Milestones** in your text output. Do not repeat this macro breakdown in subsequent turns.
+{% if "update_task_progress" in available_tools %}
+- For cross-resource work, call `update_task_progress` once with short ordered stages drawn from the actual task, including each stage's remaining checks and exit condition. Follow explicit user ordering; there is no universal Android/PC order. Only the active stage's resources may receive actions. Read-only availability checks on another resource do not activate that resource's testing stage. Keep menu/footer/final-scroll checks in their original stage instead of returning later to fill omissions. Update at stage boundaries (can share a call list with the next action), never on every scroll. Record actual step/file references before marking done; a detected product defect can still mean the inspection stage is done. Replan only for new guidance or observed blockers, preserving outstanding scope. Do not broaden a stage just to bypass its ordering guard.
+{% endif %}
 
 ---
 
@@ -31,6 +34,7 @@ Keep in mind that the mobile device is an asynchronous, constantly evolving envi
 ---
 
 # 3. READING YOUR HISTORY & THE SESSION CLOCK
+{% if "save_note" in available_tools %}- **Task text files (`save_note`)**: Save requested reports or other text deliverables directly to the current task trace with a plain key, e.g. "report" or "reusable_case"; the result supplies `notes/<key>.md`, size, SHA-256 and read-back verification. Use `read_note` only when content needs checking. Never use Android file writing or desktop UI editors to create host trace reports. Do not maintain extra notes on every scroll; preserve only requested deliverables or important findings.{% endif %}
 {% if "android_file" in available_tools %}- **Android files (`android_file`)**: To save the current phone screenshot, call `action="screenshot"`; do not use PC screenshot shortcuts or the computer plugin. Files default to this task's trace, with an additional desktop copy only for `destination="desktop"`. The list/read_text/write_text/pull/push actions operate on the selected Android device; device_path is an Android path, never a Windows path. Check the returned verified path/hash before claiming a file was saved. Saving a screenshot does not verify that its page meets the goal; inspect the screen too. Use a new filename unless the user explicitly authorized overwriting.{% endif %}
 - **Conversation = execution history**: The earlier turns of this conversation are your raw execution history. Each observation opens with `# CURRENT OBSERVATION [T+mm:ss]`. Every device action runs immediately and its tool result states what the device did (e.g. `Swiped up.`); whether it had the intended effect is yours to judge from the observation that follows. Only a turn whose action failed ends with an `--- Action Execution Result (T+mm:ss) ---` message (`Status: failed` with the `Error:` line). Every timestamp is a session-relative `T+mm:ss` offset from the start of this session; it is the only clock you have, for pacing waits as well as for reasoning about elapsed time.
 - **Compression**: Screenshots older than a few turns are replaced by short `--- Historical Visual Transition ---` summaries and old `--- Visible UI Elements ---` lists are dropped. Long spans render as `[Era N | Steps a–b | T+.. → T+..]` blocks (a merged synopsis, segment titles, and a per-step action ledger). Because of this, if you spot critical text that you will need later (e.g., verification codes, specific titles, search lists), **you must explicitly verbalize and write those details into your text reasoning** so they persist across turns.
@@ -42,6 +46,7 @@ Keep in mind that the mobile device is an asynchronous, constantly evolving envi
 ---
 
 # 4. STAGNATION & LOOP PREVENTION
+- **视觉问题复核**：报告遮挡/布局缺陷前，对照同来源、同内容的前后截图及相邻步骤，引用触发动作与实际影响，遵守用户明确允许的行为。正文滚入固定导航后方或出现背景模糊不单独证明缺陷；之前可读、可到达的内容不能仅因后续滚动而判为不可访问。持续阻碍阅读/操作或违反用户明确视觉标准时才确认相应问题。低对比度观感不能直接推断点击失效或不满足未指定的量化阈值；区分已确认问题、待确认风险和未测项。优先复用已有证据，仅补缺少的观察，不为消除疑点无限重跑。
 - **Retrying is not the only option.**: If you perform the same action on the same target for two consecutive turns and the screen or the UI element list shows **no substantial change**, further retries are unlikely to be useful; inspect the actual tool result, target, coordinate space, focus, overlays and scroll container before choosing a different strategy. Do not assume delivery proves an effect or dismiss a tool error.
 - **Flexibly Adjust Strategies:** Always prioritize resolving the immediate issue. If you encounter overlays or pop-ups blocking input, or if an element is unresponsive, actively consider alternative ways to complete the task. If an action fails to proceed as expected, systematically consider whether the issue lies with the device itself, the operational path, or a change in state occurring between your actual interaction and the screenshot (Time-Sensitive Tasks).
 
@@ -56,8 +61,12 @@ Keep in mind that the mobile device is an asynchronous, constantly evolving envi
 ---
 
 # 6. TERMINATION PROTOCOL (`report_task_status`)
+- **Continue feasible work**: A requirement that has not been attempted is pending work, not a blocker. If authorized steps remain and the tools are available, execute the next missing step before writing the final report. Do not invent a short execution window or remaining-time budget from the session clock, history compression, the Flash name, or the amount of work. Honor actual runtime stop/limit notices and explicit user instructions to stop; cite the observed obstacle when further execution is genuinely blocked.
 - **Completion (`status="completed"`)**: ONLY call `report_task_status(status="completed", ...)` after the latest screenshot and UI elements support the claimed result. If those sources disagree during a transition, wait and observe again before concluding. Tie each claimed page title or value to an observed screenshot or UI element; mark any missing evidence as unverified.
 - **Deliver the requested output**: Completing device actions does not complete a requested report. Before finishing, check the original goal and user guidance for required deliverables. Put the actual report/answer in the explanation field, covering the requested sections with observed findings, source-specific evidence and untested or blocked scope. Do not replace it with "completed" or a promise to provide it later. The short-paragraph rule above applies to turn commentary, not to this final deliverable. Use the existing record; retrieve only missing evidence instead of repeating the whole test. Do not invent findings or treat finding a product defect as execution failure when the goal is to investigate and report it.
+- **Report storage**: ARTEMIS saves the complete explanation field to this task's `notes/output.md` before final review and supplies the real save result to the reviewer. You do not need a separate file tool call to save that final report. {% if "save_note" in available_tools %}An additional requested text file can be saved with `save_note`; the key "output" is reserved for the final response. {% endif %}A report not yet written is pending work, not an environmental dead end. Finish the report from the existing evidence. If a file receipt/path was compressed out of context, consult history before claiming it cannot be verified.
+- If the full requested report was already saved as a note, use `report_note_key` when ending; that exact document becomes the displayed final report. Do not independently rewrite its completion status. Keep execution coverage, file delivery and business pass/fail separate. For visual claims (theme, occlusion, layout), cite the actual resource and screenshot step, never infer from a summary of the other device. Runtime file receipts verify saving, not visual correctness; use them instead of claiming missing files because observation text was clipped.
+- **Recorded screenshots**: Step pre/post screenshots are already saved trace evidence. Equal image hashes can represent separate captures of an unchanged page; use the step/action/timestamp and visible anchors to establish the sequence. Do not require another copy merely to obtain a different filename, unless the user explicitly asks for separate exports or a particular destination/format.
 - **Failure (`status="failed"`)**: If the task is genuinely impossible due to paywalls, unrecoverable crashes, or hitting an absolute dead end after sensible fallbacks, call `report_task_status(status="failed", ...)` with the exact blocking reason.
 
 

artemis/agents/flash/progress.py

@@ -0,0 +1,109 @@
+"""Small execution-stage ledger; no extra planner/model or scheduler."""
+
+import json
+from typing import Literal
+
+from pydantic import BaseModel, Field, model_validator
+
+from artemis.core.tool_declaration import ToolDeclaration
+from artemis.utils.notes import save_verified_note
+
+
+class Stage(BaseModel):
+    id: str = Field(min_length=1, max_length=80)
+    scope: str = Field(
+        min_length=1,
+        max_length=600,
+        description="Requirements covered by this stage, including its exit condition.",
+    )
+    resources: list[str] = Field(
+        min_length=1,
+        max_length=8,
+        description="android or actual plugin IDs; several only when the stage requires interleaving.",
+    )
+    status: Literal["pending", "active", "done", "blocked"]
+    evidence: str = Field(
+        default="",
+        max_length=1200,
+        description="Observed facts and step/file references; required for done/blocked. done means scope checked, not test passed.",
+    )
+
+
+class ProgressUpdate(BaseModel):
+    stages: list[Stage] = Field(min_length=1, max_length=16)
+    reason: str = Field(
+        min_length=1,
+        max_length=600,
+        description="Why the stage changed; when replanning, explain changed requirements or blocking evidence.",
+    )
+
+    @model_validator(mode="after")
+    def ordered(self):
+        if len({s.id for s in self.stages}) != len(self.stages):
+            raise ValueError("阶段 id 不能重复")
+        if sum(s.status == "active" for s in self.stages) > 1:
+            raise ValueError("只能有一个当前阶段")
+        outstanding = False
+        for stage in self.stages:
+            if stage.status in {"done", "blocked"} and not stage.evidence.strip():
+                raise ValueError("完成或阻塞阶段必须记录事实与步骤/文件依据")
+            if stage.status in {"active", "done"} and outstanding:
+                raise ValueError("前面的阶段尚未检查完，不能提前推进；需先完成或注明阻塞")
+            outstanding = outstanding or stage.status in {"active", "pending"}
+        return self
+
+
+PROGRESS_TOOL = ToolDeclaration(
+    "update_task_progress",
+    "Record ordered stages for a cross-resource task before acting. Follow the user's sequence; "
+    "read-only preflight is allowed on other resources. Update only at stage boundaries or genuine "
+    "replanning, not every turn. Can precede the next action in the same tool-call list. "
+    "Keep outstanding scope; done requires recorded evidence. This ledger does not verify business success.",
+    ProgressUpdate.model_json_schema(),
+)
+
+
+class TaskProgress:
+    def __init__(self):
+        self.value: ProgressUpdate | None = None
+        self.last_resource: str | None = None
+
+    def update(self, args: dict, *, resources: set[str], directory=None) -> str:
+        update = ProgressUpdate.model_validate(args)
+        if any(set(stage.resources) - resources for stage in update.stages):
+            raise ValueError(f"阶段引用了不可用资源；可用：{sorted(resources)}")
+        if self.value:
+            missing = {s.id for s in self.value.stages} - {s.id for s in update.stages}
+            if missing:
+                raise ValueError(
+                    f"不能删除既有阶段 {sorted(missing)}；保留范围并记录完成依据或阻塞原因"
+                )
+        if directory:
+            save_verified_note(directory, "task_progress", update.model_dump_json(indent=2))
+        self.value = update
+        return json.dumps(
+            {
+                "ok": True,
+                "active": next((s.id for s in update.stages if s.status == "active"), None),
+                "saved": bool(directory),
+            },
+            ensure_ascii=False,
+        )
+
+    def guard(self, resource: str | None, *, completed: bool = False) -> str | None:
+        if completed and self.value:
+            missing = [s.scope for s in self.value.stages if s.status in {"active", "pending"}]
+            if missing:
+                return (
+                    "仍有未检查完的阶段：" + "; ".join(missing) + "。先核对并更新进度，再结束任务。"
+                )
+        if resource is None:
+            return None
+        if self.value:
+            active = next((s for s in self.value.stages if s.status == "active"), None)
+            if active is None or resource not in active.resources:
+                scope = active.scope if active else "尚未选择"
+                return f"当前阶段：{scope}；暂未执行 {resource} 操作。先完成当前阶段或根据真实阻塞调整 update_task_progress；其他端只读核验仍可进行。"
+        elif self.last_resource and resource != self.last_resource:
+            return "跨端操作前需用 update_task_progress 记录已完成与待做范围，遵循用户指定顺序；只读预检不能直接扩展为另一端整段操作。"
+        return None

artemis/agents/flash/runner.py

@@ -56,6 +56,7 @@ from langchain_core.messages import (
 )
 
 from artemis.agents.flash.summarizer import VisualStepSummarizer, build_focus_context
+from artemis.agents.flash.progress import PROGRESS_TOOL, TaskProgress
 from artemis.agents.operator.prompts import REASONING_REMINDER, UserGuidance, render_user_guidance
 from artemis.agents.validator.tool_declarations import (
     ASK_EXPLORER_TOOL,
@@ -207,8 +208,7 @@ class FlashRunner:
         return getattr(setup, "video_recording_tools_enabled", False) is True
 
     def _get_tools(self) -> list:
-        # Flash has no tool that writes notes and its prompt never teaches
-        # them, so the Validator set's note readers are left out here.
+        # Task-local deliverables use the same notes and history as Pro.
         tools = [
             t
             for t in VALIDATOR_TOOLS_DECLARATION
@@ -216,6 +216,24 @@ class FlashRunner:
         ]
         tools.insert(1, CLICK_SEQUENCE_TOOL)
         tools.append(ASK_EXPLORER_TOOL)
+        if self.ctx.data_engine is not None:
+            from artemis.core.tool_declaration import ToolDeclaration
+            from artemis.tools.scratchpad import SaveNoteArgs
+
+            tools.extend([READ_NOTE_TOOL, LIST_NOTES_TOOL])
+            tools.append(
+                ToolDeclaration(
+                    "save_note",
+                    "Save a requested text deliverable to this task's trace/notes/<key>.md. "
+                    "Use a plain key such as report or reusable_case, never an absolute path. "
+                    "Returns the actual path, size, SHA-256 and read-back verification. Existing "
+                    "notes with this key are overwritten. The output key is reserved for the "
+                    "automatically saved final response. Use read_note to inspect saved content. "
+                    "Do not save/rewrite a plan every turn; only persist requested deliverables "
+                    "or important findings not already retained in history.",
+                    SaveNoteArgs.model_json_schema(),
+                )
+            )
         # Helper tools shared with the Pro operator: the history tools are
         # declared from the very same args schemas the LangChain tools bind
         # (same availability gates: a DataEngine session, recall config).
@@ -246,6 +264,7 @@ class FlashRunner:
             tools = filter_declarations(tools, actuator, "flash")
         if getattr(self.ctx, "plugins", None):
             tools.extend(self.ctx.plugins.declarations())
+            tools.append(PROGRESS_TOOL)
         return tools
 
     def _prune_intermediate_screenshots(self, messages: list[BaseMessage]) -> None:
@@ -594,9 +613,31 @@ class FlashRunner:
         xml_list,
         messages: list[BaseMessage],
         native_text: str | None = None,
-    ) -> dict:
+    ) -> dict | None:
         """Records the final step, acknowledges the tool call, and flushes."""
-        final_report = args
+        final_report = dict(args)
+        if args.get("report_note_key"):
+            from artemis.utils.notes import read_note_content
+
+            try:
+                key = args["report_note_key"]
+                if not isinstance(key, str) or not re.fullmatch(r"[\w-]{1,128}", key):
+                    raise ValueError("report_note_key 必须是已保存报告的普通笔记名称")
+                body = read_note_content(self.ctx.data_engine.base_dir, key)
+                if not body.strip():
+                    raise ValueError("报告正文为空")
+                final_report["explanation"] = body
+            except (OSError, ValueError, AttributeError) as exc:
+                messages.append(
+                    ToolMessage(
+                        tool_call_id=tc_id,
+                        name=name,
+                        status="error",
+                        content=f"最终报告未提交：{exc}。修正报告名称或直接提供完整 explanation。",
+                    )
+                )
+                return None
+        args = final_report
         self.final_report_received = True
         if self.ctx.data_engine:
             try:
@@ -645,7 +686,7 @@ class FlashRunner:
             return post_img_bytes
         # File operations have their own verified receipts. Do not add a UI
         # observation/visual summary that could obscure that non-visual evidence.
-        if post_img_bytes or name not in action_names or name == "android_file":
+        if post_img_bytes or name not in action_names or name in {"android_file", "save_note"}:
             return post_img_bytes
         if state is not None:
             state.indexed_points = []
@@ -764,7 +805,7 @@ class FlashRunner:
             )
             action_dict.pop("coordinates", None)
             action_dict.pop(COORDINATE_SPACE_KEY, None)
-        if name == "android_file":
+        if name in {"android_file", "save_note"}:
             action_dict.pop("coordinates", None)
             action_dict.pop(COORDINATE_SPACE_KEY, None)
         return action_dict
@@ -863,7 +904,7 @@ class FlashRunner:
         # extension tools, so extension steps are recorded like actions.
         # File helpers perform real work too. Persist their receipts as steps,
         # otherwise helper-only turns vanish from the final checker's evidence.
-        action_names = self.executor.action_tool_names | {"android_file"}
+        action_names = self.executor.action_tool_names | {"android_file", "save_note"}
         plugin_action = False
         try:
             operation = self.executor.execute(
@@ -929,8 +970,8 @@ class FlashRunner:
                     exec_result,
                     raw_text,
                     step_token_usage,
-                    pre_screenshot_bytes,
-                    xml_list,
+                    None if name == "save_note" else pre_screenshot_bytes,
+                    "" if name == "save_note" else xml_list,
                     post_img_bytes,
                     injected=injected,
                     native_text=native_text,
@@ -1104,6 +1145,8 @@ class FlashRunner:
         from artemis.plugins.runtime import get_plugin_runtime
 
         plugins = get_plugin_runtime(self.ctx)
+        if not hasattr(self, "_progress"):
+            self._progress = TaskProgress()
         android_tools = action_names | {"ask_explorer"}
         if plugins:
             android_tools -= plugins.tool_names
@@ -1116,6 +1159,63 @@ class FlashRunner:
             args = tc.get("args") or {}
             tc_id = tc.get("id") or str(uuid.uuid4())
 
+            if name == "update_task_progress":
+                try:
+                    resources = (
+                        {"android"}
+                        if getattr(self.ctx, "execution_target", None) != "computer"
+                        else set()
+                    )
+                    resources.update(plugins.packages if plugins else [])
+                    directory = getattr(self.ctx.data_engine, "base_dir", None)
+                    text = self._progress.update(args, resources=resources, directory=directory)
+                    status = "success"
+                except (ValueError, OSError) as exc:
+                    text, status = str(exc), "error"
+                messages.append(
+                    ToolMessage(tool_call_id=tc_id, name=name, status=status, content=text)
+                )
+                if self.ctx.data_engine:
+                    result = ToolExecutionResult(
+                        tool_name=name, tool_call_id=tc_id, status=status, text_summary=text
+                    )
+                    step_id = self._record_action_step(
+                        name,
+                        args,
+                        result,
+                        raw_text,
+                        step_token_usage,
+                        None,
+                        "",
+                        None,
+                        injected=injected,
+                        native_text=native_text,
+                    )
+                    turn.step_keys.append(str(step_id or tc_id))
+                if status == "error":
+                    self._defer_tool_calls(
+                        messages,
+                        tool_calls[call_index + 1 :],
+                        reason="进度更新失败，先修正后再执行关联动作。",
+                    )
+                    return None, pre_screenshot_bytes, xml_list, action_sequence
+                continue
+
+            resource = None
+            if plugins and name in plugins.bindings:
+                if not plugins.is_read_only(name):
+                    resource = plugins.bindings[name][0]
+            elif name in android_tools and name not in {"wait_for_delay", "ask_explorer"}:
+                if name != "android_file" or args.get("action") in {"write_text", "push"}:
+                    resource = "android"
+            blocked = self._progress.guard(
+                resource,
+                completed=name == "report_task_status" and args.get("status") == "completed",
+            )
+            if blocked:
+                self._defer_tool_calls(messages, tool_calls[call_index:], reason=blocked)
+                return None, pre_screenshot_bytes, xml_list, action_sequence
+
             # A positive wait on the PC means elapsed time, not a phone refresh.
             if (
                 getattr(self, "_android_observation_stale", False)
@@ -1204,6 +1304,8 @@ class FlashRunner:
                 native_text=native_text,
                 index_elements=index_elements,
             )
+            if resource:
+                self._progress.last_resource = resource
             if returning_to_android:
                 # wait_for_delay has no stale target and its normal post-action
                 # observation is fresh. The model must see it before any further