feat: add official Codex skill and harden task execution

Install and update artemis-use with task diagnostics and plugin guidance. Preserve cross-device file receipts, review canonical reports, guard stage transitions and reduce small history summaries. Improve exploration lifecycle, request recovery and browser live connections.

artemis/agents/operator/operator.json

@@ -1,3 +1,3 @@
 {
-  "main_template": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。\n\n每次调用设备操作工具前，用一小段简体中文说明当前观察、即将执行的动作和目的；操作后仅在获得结果或新观察时说明实际变化。不要只在最终答复中使用中文。工具参数中的自由描述文本也使用简体中文，精确引用的界面文字仍保留原文。\n\n# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **要求与证据**：沿用现有任务计划，将用户明确要求的前置条件、开关状态、检查动作和交付物列为必要项，区分待检查、已有证据、受阻。仅在状态变化时简短记下要求、来源/步骤和实际观察，不每轮重抄清单。跨端切换前和结束前核对必要项及最新用户指导，只补尚缺且可安全取得的证据；无法补齐时明确列出缺项。用户希望发生、动作已发出或计划已勾选都不证明验收通过。\n- **证据冲突**：关键文字、数值或开关状态有冲突/不清晰时，回查对应步骤原图或控件树，保留来源和不确定性，不按常识或多数摘要猜测。手机与电脑证据分别归属，仅在同条件观察支持时报告跨端差异。保持已有可靠证据，只回查缺失或矛盾部分。\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.\n  2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading statLine truncated
+  "main_template": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。\n\n每次调用设备操作工具前，用一小段简体中文说明当前观察、即将执行的动作和目的；操作后仅在获得结果或新观察时说明实际变化。不要只在最终答复中使用中文。工具参数中的自由描述文本也使用简体中文，精确引用的界面文字仍保留原文。\n\n# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **视觉问题复核**：报告遮挡/布局缺陷前，对照同来源、同内容的前后截图及相邻步骤，引用触发动作与实际影响，遵守用户明确允许的行为。正文滚入固定导航后方或出现背景模糊不单独证明缺陷；之前可读、可到达的内容不能仅因后续滚动而判为不可访问。持续阻碍阅读/操作或违反用户明确视觉标准时才确认相应问题。低对比度观感不能直接推断点击失效或不满足未指定的量化阈值；区分已确认问题、待确认风险和未测项。优先复用已有证据，仅补缺少的观察，不为消除疑点无限重跑。\n- **要求与证据**：沿用现有任务计划，将用户明确要求的前置条件、开关状态、检查动作和交付物列为必要项，区分待检查、已有证据、受阻。仅在状态变化时简短记下要求、来源/步骤和实际观察，不每轮重抄清单。跨端切换前和结束前核对必要项及最新用户指导，只补尚缺且可安全取得的证据；无法补齐时明确列出缺项。用户希望发生、动作已发出或计划已勾选都不证明验收通过。\n- **证据冲突**：关键文字、数值或开关状态有冲突/不清晰时，回查对应步骤原图或控件树，保留来源和不确定性，不按常识或多数摘要猜测。手机与电脑证据分别归属，仅在同条件观察支持时报告跨端差异。保持已有可靠证据，只回查缺失或矛盾部分。\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, soLine truncated
 }

artemis/agents/operator/operator.py

@@ -33,6 +33,7 @@ from artemis.graph.state import State
 from artemis.graph.visibility import strict_state
 from artemis.mcp.action_specs import OPERATOR_SHELL_ORDER, operator_shell_tool
 from artemis.llm.usage import usage_from_message
+from artemis.llm.structured import content_to_text
 from artemis.runtime.task_control import TaskControl, has_pending_instruction
 from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
 from artemis.tools.command_tool import (
@@ -571,9 +572,9 @@ class OperatorNode:
         fallback_limit = settings.ARTEMIS_TOOL_FALLBACK_MAX_NO_PROGRESS_ATTEMPTS
         active_llm = base_llm
         using_tool_fallback = False
-        exploration_plan_before = (
-            self._read_task_plan_text() if getattr(self.ctx, "exploration_id", None) else None
-        )
+        state.execution_report = None
+        report_requested = False
+        other_tool_failed = False
 
         for iteration in range(max_iterations):
             control = getattr(self.ctx, "task_control", None)
@@ -583,7 +584,25 @@ class OperatorNode:
             ):
                 # The device may have changed while another task held its lease.
                 return None, None, None, False
-            if iteration == max_iterations - 1:
+            if (
+                not report_requested
+                and not other_tool_failed
+                and self._exploration_ready_for_report(state)
+            ):
+                current_messages.append(
+                    SystemMessage(
+                        content=(
+                            "探索计划已完成。现在交付最终答复，不要为了结束再操作设备。"
+                            "沿用本次上下文，按用户要求给出完整报告正文，而不是仅引用笔记、"
+                            "承诺稍后提供或重复计划。需要时可读取已保存报告再原样交付，"
+                            "不重复调查或重写已有完整报告。明确区分实际发现、未测范围与"
+                            "调查完成度，不能将执行完成等同于测试通过。"
+                            "最后以不含工具调用的正文回复；系统会保存并交给独立验收。"
+                        )
+                    )
+                )
+                report_requested = True
+            if iteration == max_iterations - 1 and not report_requested:
                 logger.warning(
                     "Operator has reached the maximum number of tool calls. Adding reminder."
                 )
@@ -614,13 +633,14 @@ class OperatorNode:
             if interrupted:
                 return None, None, None, False
             assert response is not None
-            if getattr(self.ctx, "exploration_id", None):
-                from artemis.exploration.runtime import publish_response
-
-                publish_response(self.ctx, response)
             if isinstance(control, TaskControl) and has_pending_instruction(control.directory):
                 # The model answered a prompt that predates the user's new direction.
                 return None, None, None, False
+            if getattr(self.ctx, "exploration_id", None):
+                from artemis.exploration.runtime import needs_exploration_turn
+
+                if needs_exploration_turn(self.ctx):
+                    return None, None, None, False
 
             usage = usage_from_message(response)
             # The ledger's compaction thresholds read the operator's own
@@ -680,12 +700,29 @@ class OperatorNode:
                 except Exception as e:
                     logger.error(f"Failed to parse function_call fallback: {e}")
 
+            if not response.tool_calls and not getattr(response, "invalid_tool_calls", None):
+                report_text = content_to_text(response.content).strip()
+                if report_requested and report_text and self._exploration_ready_for_report(state):
+                    state.execution_report = report_text
+                    current_messages.append(response)
+                    logger.info("Exploration final report prepared; returning for final review.")
+                    break
+
+            if getattr(self.ctx, "exploration_id", None):
+                from artemis.exploration.runtime import publish_response
+
+                publish_response(self.ctx, response)
+
             if not response.tool_calls:
                 no_progress_attempts += 1
                 if no_progress_attempts >= (
                     fallback_limit if using_tool_fallback else no_progress_limit
                 ):
                     if using_tool_fallback:
+                        if report_requested:
+                            raise OperatorNoProgressError(
+                                "模型未生成可用的最终报告正文，已停止空转。"
+                            )
                         raise OperatorNoProgressError(
                             f"首选模型连续 {no_progress_limit} 次、兜底模型连续 "
                             f"{no_progress_attempts} 次未提供可执行的工具调用；"
@@ -697,16 +734,23 @@ class OperatorNode:
                     logger.warning("Operator 首选模型未调用工具，当前步骤切换到兜底模型")
                     current_messages.append(
                         SystemMessage(
-                            content="本步骤已切换兜底模型。请使用工具完成操作或明确报告阻碍。"
+                            content=(
+                                "已切换兜底模型。请直接交付完整报告正文，不要再次操作设备。"
+                                if report_requested
+                                else "本步骤已切换兜底模型。请使用工具完成操作或明确报告阻碍。"
+                            )
                         )
                     )
                     continue
-                logger.warning("LLM returned no tool call; requesting a concrete tool call.")
+                logger.warning("LLM returned no usable result; requesting bounded recovery.")
                 current_messages.append(response)
                 current_messages.append(
                     SystemMessage(
                         content=(
-                            "No tool call was returned, so no device action occurred. "
+                            "尚未收到可用的最终报告正文。请用文字交付实际结果；"
+                            "缺少证据时如实说明，不要为了结束再操作设备。"
+                            if report_requested
+                            else "No tool call was returned, so no device action occurred. "
                             "Use the available tool interface now, not a prose promise or JSON "
                             "inside text. Use an index from the current element list or "
                             "normalized coordinates with target_description. If blocked, "
@@ -1040,32 +1084,6 @@ class OperatorNode:
             for tm in tool_outputs:
                 current_messages.append(tm)
 
-            if (
-                getattr(self.ctx, "exploration_id", None)
-                and other_calls
-                and not has_terminal_calls
-                and not other_tool_failed
-            ):
-                from artemis.exploration.runtime import needs_exploration_turn
-
-                plan_after = self._read_task_plan_text()
-                snapshot = parse_plan(plan_after or "")
-                if (
-                    plan_after != exploration_plan_before
-                    and snapshot.has_top_level
-                    and snapshot.all_top_level_done
-                    and (
-                        not snapshot.continuous_top_level
-                        or getattr(state, "user_stop_requested", False)
-                    )
-                    and not needs_exploration_turn(self.ctx)
-                ):
-                    # Completing the plan is useful progress. Return to the
-                    # convergence gate and final review instead of demanding
-                    # another device action just to end the operator turn.
-                    logger.info("Exploration plan completed; returning for final review.")
-                    break
-
             if action_calls and not validation_errors:
                 break
 
@@ -1244,6 +1262,7 @@ class OperatorNode:
 
         return {
             "structured_decisions": structured_decisions,
+            "execution_report": state.execution_report,
             "operator_raw_thinking": raw_thinking,
             "operator_native_thinking": native_thinking,
             "indexed_points": state.indexed_points,
@@ -1253,6 +1272,19 @@ class OperatorNode:
             "operator_tool_limit_exceeded": tool_limit_exceeded,
         }
 
+    def _exploration_ready_for_report(self, state: State) -> bool:
+        if not getattr(self.ctx, "exploration_id", None):
+            return False
+        from artemis.exploration.runtime import needs_exploration_turn
+
+        snapshot = parse_plan(self._read_task_plan_text() or "")
+        return bool(
+            snapshot.has_top_level
+            and snapshot.all_top_level_done
+            and (not snapshot.continuous_top_level or getattr(state, "user_stop_requested", False))
+            and not needs_exploration_turn(self.ctx)
+        )
+
     def _plan_ledger_gate_enabled(self) -> bool:
         try:
             from artemis.config import load_agent_config