Compare commits

...
4 Commits
29 changed files with 641 additions and 54 deletions

No files matched your search

+12 -2
View File
@@ -21,7 +21,7 @@
| 探索、示范、用例提炼与复跑 | [探索说明](docs/exploration-mode.md) |
| 插件协议、适配或市场协同 | [插件开发规范](docs/plugin-development.md)及插件仓库自身的 AGENTS.md、DESIGN.md |
| 远程 Python SDK | [SDK 设计](packages/artemis-client/DESIGN.md),保持标准库薄客户端边界 |
| 继续历史开发、确认版本或发布 | [迁入时的开发快照](docs/development-handoff.md)、[2.1.0 发行说明](docs/releases/2.1.0.md),再核对实时 Git 状态;`docs/plans/` 保留各自阶段的状态 |
| 继续历史开发、确认版本或发布 | [README 当前发行入口](README.md)、对应发行说明及实时 Git 状态;[开发快照](docs/development-handoff.md)与 `docs/plans/` 仅作为历史线索,不替代当前源码与发布包核验 |
## Quick Commands
@@ -60,7 +60,7 @@ Windows 首次安装使用 `start.bat`,国内镜像使用 `start-cn.bat`。启
## Conventions
- Python 使用绝对导入、4 空格缩进和双引号,由 Ruff 检查;不要放宽质量基线来掩盖新增问题。
- 新增可调参数集中进入配置层,并更新环境示例、网页/MCP 入口和必要文档;默认值同步 `config/artemis.jsonc` 与 `artemis/resources/config/artemis.jsonc`。
- 新增可调参数集中进入配置层,按作用域更新环境示例、网页/MCP 入口和文档;明确仅供环境配置的参数不增加前端入口。JSON 默认值同步 `config/artemis.jsonc` 与 `artemis/resources/config/artemis.jsonc`。
- MCP 工具增删时同步工具注册、CLI 安装白名单、测试、Skill 和能力文档。
- 同步 Codex MCP/Skill 时保留已有工具白名单、禁用状态和审批策略;异常时保留原配置并告警,不生成非法 TOML 或阻断网页启动。
- Windows PowerShell 启动脚本保持兼容 Windows PowerShell 5.1,避免编码导致解析失败。升级只做 Git 快进,保护已有配置和本地修改。
@@ -75,6 +75,16 @@ Windows 首次安装使用 `start.bat`,国内镜像使用 `start-cn.bat`。启
- 设备验证沿用用户指定设备;多设备且未指定时先确认。编写应用交互测试前,先通过 ARTEMIS 探索实际页面和操作路径。
- 提交前检查 `git diff --check` 和暂存范围,只暂存本次具体路径;报告实际执行的验证及未完成项,不将未运行的检查写成通过。
## Release Gates
- 主程序、SDK、插件和第三方依赖独立版本化。主程序升版只修改 `pyproject.toml` 的项目版本、`uv.lock` 的 artemis 条目及前端应用的根版本字段,发行文档另行更新。禁止对仓库或整个锁文件全局替换版本字符串。
- 单独审查锁文件差异。第三方条目的版本、下载地址、完整性校验和依赖声明必须相互匹配;主程序升版不应改变这些条目。真正升级依赖时由包管理器生成并核对上游声明;发生 EINTEGRITY 先核验来源,禁止关闭校验或只照错误提示改哈希。
- 发布前必须在隔离目录解压最终 ZIP,使用独立空 npm 缓存完成 `npm ci` 和生产构建,覆盖本次发布的官方/国内安装路径。包内安装或构建输入变化后必须重跑;不得清空用户全局缓存或改动用户运行目录来制造干净环境。
- 记录实际 Node/npm 版本、镜像、源码提交、ZIP SHA256 和验证范围。已有 node_modules、构建指纹、缓存命中或大量单元测试通过,均不能证明全新安装成功;某个 Node 版本通过也不能冒充其他版本实测。测试和构建命令见前端局部指南。
- 发布顺序:固定提交并打包 → 验证最终包 → 推送标签 → 创建草稿并上传包及校验文件 → 回读校验 → 正式发布 → 无认证下载核验。标签必须指向打包提交;推送标签后检查 Release 状态,本仓库曾出现 Gitea 将已有草稿自动发布的情况。失败时修复并重新验证,不提前宣布完成。
- 候选包与正式包目录明确区分。候选包失效后移出可分发路径并标明替代版本,不能留下同名旧 ZIP 供用户误取;对外只给已核验的正式发布链接或准确文件路径。正式版本不覆盖原标签或附件,后续修复发补丁版。
- 安装或市场版本不符时,先核对用户实际 ZIP 的 SHA256、运行源码提交及生效市场地址;不凭界面版本号判断,也不把未经验证的缓存假设当成原因。主程序与插件版本、索引兼容性分别验证。
## Boundaries
- 始终保护用户任务、历史库、环境配置和凭据。不要把个人设备序列号、绝对目录或 API Key 写入源码。
+1 -1
View File
@@ -2,7 +2,7 @@
基于 [ARTEMIS](https://github.com/google/artemis) 扩展的 Android AI 自动化测试平台。用自然语言描述任务,即可通过简体中文网页或 MCP 驱动真机、模拟器执行操作并收集结果。模型统一使用 **OpenAI Responses API**。
**2.4.0**:人工评审支持原始用例、自动保存与上一条/下一条;执行详情默认折叠长字段。升级 OpenAI/LangChain 及安全依赖,修复 Windows 资源锁、报告证据与历史用量。配套「任务人工评审 1.1.0」独立安装。见 [2.4.0 发行说明](docs/releases/2.4.0.md);上一版本见 [2.3.0](docs/releases/2.3.0.md)。
**2.4.3**:异常模型输出耗尽重试后只失败当前任务,避免触发全局暂停;补充请求关联和有界参数摘要,便于定位流式异常。见 [2.4.3 发行说明](docs/releases/2.4.3.md)。人工评审插件保持 1.1.0;列表优化见 [2.4.2](docs/releases/2.4.2.md),安装修复见 [2.4.1](docs/releases/2.4.1.md)。
## 主要功能
+3 -1
View File
@@ -12,7 +12,7 @@ from apps.admin_console.services.exploration_listing import EXPLORATION_RUNS_CTE
from apps.admin_console.services.automation_store import identifier, store
from apps.admin_console.services import automation_service as service
from apps.admin_console.services import batch_archive as archive_service
from apps.admin_console.services import report_export, task_import
from apps.admin_console.services import report_export, task_extensions, task_import
from artemis.config.settings import settings
from apps.admin_console.services.case_review import read_reviewed_result
@@ -131,7 +131,9 @@ def batch_by_request(request_id: str):
@router.get("/batches/{batch_id}")
def batch_detail(batch_id: str):
jobs = store.jobs(batch_id)
reviewed = task_extensions.reviewed_batch_sessions(batch_id, store)
for job in jobs:
job["human_reviewed"] = job["session_id"] in reviewed
job["test_result"] = read_reviewed_result(
job["session_id"], job["status"], job.get("error") or "", storage=store
)
@@ -36,6 +36,32 @@ def installation_signature() -> str:
)
def reviewed_batch_sessions(batch_id: str, store: AutomationStore) -> set[str]:
"""Read current human review values once per batch, without loading history."""
manifest = next((item for item in active_manifests() if item.id == "task-review"), None)
if manifest is None or manifest.workspace is None:
return set()
with store.connection() as conn:
rows = conn.execute(
"""SELECT j.session_id, json_extract(r.payload, '$."task-review".values') AS review_values
FROM automation_jobs j JOIN automation_records r
ON r.kind='task_extensions' AND r.id=j.session_id
WHERE j.batch_id=?""",
(batch_id,),
).fetchall()
reviewed = set()
for row in rows:
values = json.loads(row["review_values"] or "{}")
if any(
isinstance(value := values.get(field.id), str)
and value.strip()
and (field.kind == "text" or value in field.options)
for field in manifest.workspace.fields
):
reviewed.add(row["session_id"])
return reviewed
def task_context(session_id: str, store: AutomationStore) -> dict:
identifier(session_id)
session = SessionRepository(store.path).get_session_by_id(session_id) or {}
+3
View File
@@ -30,6 +30,9 @@ npm run build
## Local Conventions
- 发版遵守根指南的 Release Gates。前端应用版本只改 package.json 顶层 `version` 和 package-lock.json 顶层及 `packages[""].version`;禁止把相同数字的第三方依赖一并替换。
- Node 支持范围以锁定的 Angular CLI/构建依赖实际 `engines` 为准,核对启动脚本的主、次、补丁版本边界和提示;不要只判断“Node 22/24/26”。
- 安装或依赖变更需在隔离目录、独立空 npm 缓存下运行 `npm ci`,再运行上述测试和构建;国内入口同时核验镜像安装。不得以关闭完整性校验、强制安装或删除锁文件掩盖不一致,也不得用已装依赖的构建结果宣称全新安装通过。
- 页面组件在 src/app/components,服务在 src/app/services;先复用已有组件、样式及 API 服务。
- 保持现有简体中文文案、颜色、间距和控件样式,菜单、弹窗和表单必须清晰可读。
- 修改任务或配置交互时覆盖加载、保存、失败和空状态;不要将执行中误标成录屏中或把完成误标成验证通过。
+8 -8
View File
@@ -1,12 +1,12 @@
{
"name": "artemis-showcase-ui",
"version": "2.4.0",
"version": "2.4.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "artemis-showcase-ui",
"version": "2.4.0",
"version": "2.4.3",
"dependencies": {
"@angular/cdk": "^22.2.4",
"@angular/common": "^22.2.3",
@@ -253,7 +253,7 @@
"@angular-devkit/schematics": "22.2.4",
"@inquirer/prompts": "8.7.2",
"@listr2/prompt-adapter-inquirer": "4.2.7",
"@modelcontextprotocol/server": "2.4.0",
"@modelcontextprotocol/server": "2.3.1",
"@schematics/angular": "22.2.4",
"jsonc-parser": "3.3.1",
"listr2": "11.1.0",
@@ -2043,8 +2043,8 @@
]
},
"node_modules/@modelcontextprotocol/core": {
"version": "2.4.0",
"resolved": "https://registry.npmjs.org/@modelcontextprotocol/core/-/core-2.4.0.tgz",
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/@modelcontextprotocol/core/-/core-2.3.1.tgz",
"integrity": "sha512-laVmIhPGpWi7rG9q+0UigblRN0Hws/Yjlb6qLFtVqryXVLkc4h+yN8z731POD2LnAGQlFe7/tXhUm7Bi0YB91g==",
"dev": true,
"license": "Apache-2.0",
@@ -2056,13 +2056,13 @@
}
},
"node_modules/@modelcontextprotocol/server": {
"version": "2.4.0",
"resolved": "https://registry.npmjs.org/@modelcontextprotocol/server/-/server-2.4.0.tgz",
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/@modelcontextprotocol/server/-/server-2.3.1.tgz",
"integrity": "sha512-e59MfWuSssj6DoQ96JCTqdW1AXT5/XtnTl0WWd6MU1Ka434fFUGGosCLJbGNySDhHGvxQWoRWK137FXYFkYNWw==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"@modelcontextprotocol/core": "2.4.0",
"@modelcontextprotocol/core": "2.3.1",
"zod": "^4.2.0"
},
"engines": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "artemis-showcase-ui",
"version": "2.4.0",
"version": "2.4.3",
"scripts": {
"ng": "ng",
"start": "ng serve --proxy-config proxy.conf.json",
@@ -108,6 +108,19 @@
</div>
}
@if(b.report_error){<p class="error">报告生成失败:{{b.report_error}}</p>}
@for(j of b.jobs; track j.session_id){<div class="job" [class.selected]="agent.currentSessionId() === j.session_id"><app-favorite-button [sourceId]="j.session_id" /><button (click)="openBatchTask(j.session_id)" [title]="j.case.goal" [attr.aria-pressed]="agent.currentSessionId() === j.session_id">{{j.case.case_id ? j.case.case_id + ' ' : ''}}{{j.case.name}}@if(j.repeat_count > 1){ · 第 {{j.iteration}} / {{j.repeat_count}} 轮}</button><span>{{label(j.status)}}</span><app-test-result-badge [result]="j.test_result" [executionStatus]="j.status" [compact]="true" />@if(j.error){<small>{{j.error}}</small>}</div>}
@for(j of b.jobs; track j.session_id){
<div class="job" [class.selected]="agent.currentSessionId() === j.session_id">
<div class="job-heading">
<app-favorite-button [sourceId]="j.session_id" />
<button (click)="openBatchTask(j.session_id)" [title]="j.case.goal" [attr.aria-pressed]="agent.currentSessionId() === j.session_id">{{j.case.case_id ? j.case.case_id + ' ' : ''}}{{j.case.name}}@if(j.repeat_count > 1){ · 第 {{j.iteration}} / {{j.repeat_count}} 轮}</button>
</div>
<div class="job-statuses">
<span>{{label(j.status)}}</span>
<app-test-result-badge [result]="j.test_result" [executionStatus]="j.status" [compact]="true" />
@if(j.human_reviewed){<span class="human-review-badge" title="人工评审已有保存内容,不代表测试通过">人工已审</span>}
</div>
@if(j.error){<small>{{j.error}}</small>}
</div>
}
</section>}
}
@@ -37,7 +37,15 @@ input[type=checkbox]{width:15px;height:15px;accent-color:var(--app-primary-text,
@keyframes archive-spin{to{transform:rotate(360deg);}}
.code{font-family:Consolas,monospace;font-size:12px;line-height:1.6;resize:vertical;background:var(--app-surface-soft, #f8fafc);}.table-wrap{max-height:260px;overflow:auto;border:1px solid var(--app-border, #e2e8f0);border-radius:8px;margin-bottom:14px;}table{width:100%;border-collapse:collapse;}th,td{padding:9px 10px;border-bottom:1px solid var(--app-border, #e2e8f0);text-align:left;overflow-wrap:anywhere;}th{background:var(--app-surface-soft, #f8fafc);color:var(--app-text-secondary, #475569);font-weight:500;}
.preview{white-space:pre-wrap;max-height:400px;overflow:auto;font:inherit;color:var(--app-text-control, #334155);background:var(--app-surface-soft, #f8fafc);border-radius:8px;padding:12px;border:1px solid var(--app-border, #e2e8f0);}
.job{display:flex;flex-wrap:wrap;gap:8px;align-items:center;padding:10px 0;border-bottom:1px solid var(--app-border, #f1f5f9);}.job:last-child{border-bottom:0;}.job button{flex:1;justify-content:flex-start;text-align:left;min-width:150px;}.job span{font-size:12px;background:var(--app-surface-soft, #f1f5f9);color:var(--app-text-secondary, #475569);border-radius:5px;padding:3px 7px;}.job small{width:100%;}details{margin:14px 0;}summary{color:var(--app-text-secondary, #475569);cursor:pointer;padding:8px 0;}
.job{display:flex;flex-direction:column;gap:8px;padding:12px 0;border-bottom:1px solid var(--app-border, #f1f5f9);}
.job:last-child{border-bottom:0;}
.job-heading{position:relative;min-width:0;}
.job-heading>app-favorite-button{position:absolute;left:6px;top:4px;z-index:1;}
.job-heading>button{box-sizing:border-box;width:100%;min-width:0;padding-left:40px;justify-content:flex-start;text-align:left;overflow-wrap:anywhere;}
.job-statuses{display:flex;flex-wrap:wrap;align-items:center;gap:8px;}
.job-statuses>span{display:inline-flex;align-items:center;box-sizing:border-box;height:20px;padding:0 6px;border:1px solid transparent;border-radius:5px;font-size:11px;line-height:14px;background:var(--app-surface-soft, #f1f5f9);color:var(--app-text-secondary, #475569);white-space:nowrap;}
.job small{overflow-wrap:anywhere;}
details{margin:14px 0;}summary{color:var(--app-text-secondary, #475569);cursor:pointer;padding:8px 0;}
.task-guidance{margin-top:16px;padding:14px;border:1px solid var(--app-primary-border, #bfdbfe);border-radius:10px;background:var(--app-primary-soft, #f8fbff);}
.task-guidance label{margin:8px 0 4px;}
.report-overview{padding:14px;border:1px solid var(--app-primary-border, #dbeafe);border-radius:10px;background:var(--app-primary-soft, #f8fbff);}
@@ -58,3 +66,5 @@ input[type=checkbox]{width:15px;height:15px;accent-color:var(--app-primary-text,
table{width:100%;border-collapse:collapse;font-size:12px;}th,td{padding:7px 9px;border:1px solid var(--app-border, #e2e8f0);vertical-align:top;}th{background:var(--app-primary-soft, #eff6ff);color:var(--app-primary-text, #1e3a8a);}
.md-para-gap{height:8px;}
}
.job .human-review-badge{white-space:nowrap;border:1px solid var(--app-primary-border, #bfdbfe);background:var(--app-primary-soft, #eff6ff);color:var(--app-primary-text, #1d4ed8);font-weight:500;}
@@ -90,6 +90,30 @@ describe('AutomationPanelComponent batch deletion', () => {
expect(http.post).not.toHaveBeenCalled();
});
it('shows only saved human review badges and removes them on a batch refresh', async () => {
const fixture = TestBed.createComponent(AutomationPanelComponent);
const component = fixture.componentInstance;
await new Promise<void>(resolve => setTimeout(resolve, 0));
component.tab.set('batches');
const batch = { id: 'review-batch', name: '评审批次', status: 'completed', jobs: [
{ session_id: 'reviewed', status: 'completed', human_reviewed: true, case: { name: '已评用例' } },
{ session_id: 'empty', status: 'completed', human_reviewed: false, case: { name: '空白用例' } },
{ session_id: 'legacy', status: 'completed', case: { name: '旧版任务' } }
] };
component.batch.set(batch);
fixture.detectChanges();
const rows = fixture.nativeElement.querySelectorAll('.job');
expect(rows[0].querySelector('.human-review-badge').textContent).toBe('人工已审');
expect(rows[1].querySelector('.human-review-badge')).toBeNull();
expect(rows[2].querySelector('.human-review-badge')).toBeNull();
expect(fixture.nativeElement.textContent).not.toContain('人工未审');
(http.get.and as any).callFake((url: string) => of(url.endsWith('/review-batch')
? { ...batch, jobs: batch.jobs.map(job => ({ ...job, human_reviewed: false })) } : []));
await component.refresh();
fixture.detectChanges();
expect(fixture.nativeElement.querySelector('.human-review-badge')).toBeNull();
});
it('confirms deletion and removes the batch from the visible list', async () => {
const fixture = TestBed.createComponent(AutomationPanelComponent);
fixture.componentInstance.tab.set('batches');
@@ -129,7 +153,7 @@ describe('AutomationPanelComponent batch deletion', () => {
{session_id:'task-b',status:'completed',case:{name:'用例 B',goal:'观察主页'}}
]});
fixture.detectChanges();
const jobs: HTMLButtonElement[] = [...fixture.nativeElement.querySelectorAll('.job > button')];
const jobs: HTMLButtonElement[] = [...fixture.nativeElement.querySelectorAll('.job-heading > button')];
jobs[0].click(); jobs[1].click();
expect(agent.selectSession.calls.allArgs()).toEqual([['task-a',true],['task-b',true]]);
expect(component.tab()).toBe('batches');
+53
View File
@@ -0,0 +1,53 @@
"""Bounded fingerprints of received argument deltas, without retaining text."""
import hashlib
PREFIX_BYTES = 4096
MAX_PREFIXES = 16
MAX_TRACKED_TOOLS = 64
class ArgumentFingerprint:
def __init__(self):
self.characters = 0
self.bytes = 0
self.deltas = 0
self.encoding_valid = True
self.digest = hashlib.sha256()
self.prefixes: list[dict] = []
def observe(self, arguments: str) -> None:
self.characters += len(arguments)
if not arguments:
return
self.deltas += 1
if not self.encoding_valid:
return
try:
encoded = arguments.encode("utf-8")
except UnicodeEncodeError:
# Diagnostics must not repair, reject or otherwise alter the stream.
self.encoding_valid = False
return
offset = 0
while offset < len(encoded) and len(self.prefixes) < MAX_PREFIXES:
size = min(PREFIX_BYTES - self.bytes % PREFIX_BYTES, len(encoded) - offset)
self.digest.update(encoded[offset : offset + size])
self.bytes += size
offset += size
if self.bytes % PREFIX_BYTES == 0:
self.prefixes.append({"bytes": self.bytes, "sha256": self.digest.hexdigest()})
if offset < len(encoded):
self.digest.update(encoded[offset:])
self.bytes += len(encoded) - offset
def snapshot(self) -> dict:
return {
"argument_chars": self.characters,
"argument_bytes": self.bytes if self.encoding_valid else None,
"argument_sha256": self.digest.hexdigest() if self.encoding_valid else None,
"argument_deltas": self.deltas,
"argument_prefixes": [dict(prefix) for prefix in self.prefixes],
"argument_encoding_valid": self.encoding_valid,
}
+35 -10
View File
@@ -13,6 +13,7 @@ from uuid import uuid4
from PIL import Image
from artemis.data_engine.context_vars import CURRENT_NODE_NAME, CURRENT_TRACE_ID
from artemis.llm.argument_fingerprint import ArgumentFingerprint, MAX_TRACKED_TOOLS
from artemis.llm.final_review import ACTIVE_FINAL_REVIEW, FinalReviewBudgetExceeded
from artemis.llm.reliability import LLMExhaustedError
from artemis.llm.usage import usage_from_message
@@ -107,6 +108,7 @@ class RequestAttempt:
self.clock = time.monotonic()
self.last_publish = self.clock
self.tools: dict[int | str, dict] = {}
self.tool_fingerprints: dict[int | str, ArgumentFingerprint] = {}
self.review = ACTIVE_FINAL_REVIEW.get()
if self.review:
self.review.request_started(request_id, attempt)
@@ -126,6 +128,9 @@ class RequestAttempt:
"usage": None,
"output_chunks": 0,
"tool_argument_chars": 0,
"tool_observation_layer": "langchain_tool_call_chunks",
"argument_fingerprint_version": 1,
"tool_stats_truncated": False,
"tools": [],
"step_id": str(self.step_id) if self.step_id else None,
"parent_trace_id": str(self.parent_id) if self.parent_id else None,
@@ -167,9 +172,10 @@ class RequestAttempt:
self.data.update(
headers_at=time.time(),
http_status=response.status_code,
server_request_id=_identifier(
response.headers.get("x-request-id") or response.headers.get("request-id")
),
server_request_id=_identifier(response.headers.get("x-oneapi-request-id"))
or _identifier(response.headers.get("x-request-id"))
or _identifier(response.headers.get("request-id")),
response_x_request_id=_identifier(response.headers.get("x-request-id")),
server_trace_id=_identifier(response.headers.get("x-trace-id")),
phase="waiting_output",
)
@@ -196,18 +202,31 @@ class RequestAttempt:
self.data["usage"] = usage
tool_chunks = getattr(chunk, "tool_call_chunks", None) or []
# Responses exposes the same argument delta in content and tool_call_chunks.
# Count only the latter; retain lengths/names, never argument bodies.
# Count only the latter; retain bounded fingerprints, never argument bodies.
for tool in tool_chunks:
arguments = tool.get("args")
if isinstance(arguments, str):
self.data["tool_argument_chars"] += len(arguments)
index = tool.get("index")
key = index if index is not None else tool.get("id") or "unknown"
stats = self.tools.setdefault(key, {"name": None, "argument_chars": 0})
call_id = _identifier(tool.get("id"))
key = index if index is not None else call_id or "unknown"
if key not in self.tools and len(self.tools) >= MAX_TRACKED_TOOLS:
self.data["tool_stats_truncated"] = True
continue
stats = self.tools.setdefault(key, {"name": None})
if call_id:
stats["call_id"] = call_id
if index is not None:
stats["chunk_index"] = index
name = tool.get("name")
if isinstance(name, str) and re.fullmatch(r"[\w.:-]{1,128}", name):
stats["name"] = name
arguments = tool.get("args")
fingerprint = self.tool_fingerprints.get(key)
if fingerprint is None:
fingerprint = self.tool_fingerprints[key] = ArgumentFingerprint()
if isinstance(arguments, str):
stats["argument_chars"] += len(arguments)
self.data["tool_argument_chars"] += len(arguments)
fingerprint.observe(arguments)
stats.update(fingerprint.snapshot())
self.data["tools"] = list(self.tools.values())
content = getattr(chunk, "content", None)
substantive = isinstance(content, str) and bool(content)
@@ -252,7 +271,9 @@ class RequestAttempt:
if isinstance(status, int):
self.data["http_status"] = status
if request_id := _identifier(getattr(error, "request_id", None)):
self.data["server_request_id"] = request_id
self.data["error_request_id"] = request_id
if not self.data.get("server_request_id"):
self.data["server_request_id"] = request_id
if category == "timeout":
self.data.setdefault("timeout_scope", "transport")
if response is not None:
@@ -301,6 +322,7 @@ async def observe_request_headers(request):
if attempt is None:
return
request.headers.setdefault("X-Client-Request-Id", attempt.data["request_id"])
request.headers.setdefault("X-Client-Attempt-Id", attempt.data["attempt_id"])
routing = {}
for name in ("thread-id", "thread_id", "session-id", "session_id"):
value = request.headers.get(name)
@@ -309,9 +331,12 @@ async def observe_request_headers(request):
attempt.data.update(
request_sent_at=time.time(),
client_request_id=_identifier(request.headers.get("X-Client-Request-Id")),
client_attempt_id=_identifier(request.headers.get("X-Client-Attempt-Id")),
routing_header_fingerprints=routing,
client_request_id_matches=request.headers.get("X-Client-Request-Id")
== attempt.data["request_id"],
client_attempt_id_matches=request.headers.get("X-Client-Attempt-Id")
== attempt.data["attempt_id"],
)
attempt.publish()
+1 -1
View File
@@ -153,7 +153,7 @@ class LLMPermanentError(LLMCallError):
class LLMExhaustedError(LLMCallError):
"""Retryable failures exhausted every attempt and the pause deadline."""
"""Retryable failures exhausted attempts and any applicable pause deadline."""
# ---------------------------------------------------------------------------
+1 -1
View File
@@ -23,7 +23,7 @@ class ToolArgumentOverflow(ValueError):
"limit_chars": SMALL_ACTION_ARGUMENT_LIMIT,
}
super().__init__(
f"模型生成的 {name} 参数异常膨胀({characters} 字符,"
f"收到的 {name} 参数流超出安全限制({characters} 字符,"
f"保护上限 {SMALL_ACTION_ARGUMENT_LIMIT});已关闭输出流,未执行本次动作。"
)
+13 -5
View File
@@ -381,7 +381,7 @@ async def _run_with_recovery[T](
immediately: retrying cannot help and pausing would hang the task.
- When retryable attempts are exhausted: if a fallback model is waiting
(with_fallback), raise LLMExhaustedError so it takes over immediately;
final acceptance also raises instead of pausing an already-finished task.
final acceptance and invalid output also raise instead of globally pausing.
Other calls pause (bounded by settings.LLM_PAUSE_TIMEOUT_SECONDS) and
retry the same request on resume.
"""
@@ -470,7 +470,7 @@ async def _run_with_recovery[T](
)
if failure.category is FailureCategory.INVALID_OUTPUT:
# One fresh response can recover a generation loop. Repeated
# runaway output must reach the existing fallback/pause path,
# runaway output must reach fallback or fail this request,
# even when ordinary network retries are configured higher.
max_attempts = min(
max_attempts, total_attempts + max(0, policy.max_attempts - attempt)
@@ -505,7 +505,11 @@ async def _run_with_recovery[T](
ACTIVE_ATTEMPT.reset(observation_token)
# Retryable attempts exhausted.
if _FALLBACK_AVAILABLE.get() or ACTIVE_FINAL_REVIEW.get() is not None:
if (
_FALLBACK_AVAILABLE.get()
or ACTIVE_FINAL_REVIEW.get() is not None
or last_failure.category is FailureCategory.INVALID_OUTPUT
):
if failed_endpoints is not None and endpoint_key:
failed_endpoints[endpoint_key] = last_error
_record_llm_event(
@@ -514,7 +518,11 @@ async def _run_with_recovery[T](
"error": str(last_error)[:1000],
"category": last_failure.category.value,
"retryable": True,
"handover": "fallback" if _FALLBACK_AVAILABLE.get() else "final_review_end",
"handover": "fallback"
if _FALLBACK_AVAILABLE.get()
else "final_review_end"
if ACTIVE_FINAL_REVIEW.get() is not None
else "task_failed",
},
status="failed",
)
@@ -1253,7 +1261,7 @@ async def _with_fallback[T](
)
# The contextvar must be reset BEFORE fallback_call runs: the fallback has
# no further fallback behind it, so its own exhaustion should pause.
# no further fallback behind it, so apply its category's terminal policy.
fallback_token = _FALLBACK_AVAILABLE.set(True)
try:
result = await main_call()
+39
View File
@@ -0,0 +1,39 @@
# 模型流式参数诊断
小型内置动作的参数流仍受 16384 字符保护。超限时关闭输出流、丢弃不完整参数,不执行该次动作;一次新响应重试和可用备用端点耗尽后,只结束当前任务,不进入共享的模型暂停。批次仍遵循“失败即停止”、人工暂停和取消设置。网络故障、认证错误和最终验收保留各自的恢复规则。
## 请求关联
每个模型请求尝试沿用现有 `llm_diagnostic` 记录,不另建日志数据库。
| 字段 / 请求头 | 含义 |
| --- | --- |
| `request_id` / `X-Client-Request-Id` | 同一次恢复循环内保持稳定的逻辑请求编号;切换备用调用或恢复暂停后可能建立新的逻辑请求 |
| `attempt_id` / `X-Client-Attempt-Id` | 本次客户端尝试编号;重试时变化,`previous_attempt_id` 关联前次尝试 |
| `client_*_id_matches` | 实际发送的请求头是否与本地编号一致;调用方已指定的头保留原值 |
| `server_request_id` | 首个有效的 `X-Oneapi-Request-Id`、`X-Request-Id` 或 `Request-Id`,按此前后顺序选择 |
| `response_x_request_id` | 响应中的原始 `X-Request-Id`,不推定它来自模型上游 |
| `error_request_id` | SDK 异常带回的编号,不覆盖已经从响应头读取的编号 |
| `response_id` | SDK 实际返回的 Responses 响应编号 |
编号只接收有长度限制的可打印白名单字符。请求头不控制会话亲和性,不替换 `thread-id` / `session-id`。网关自身的重试应使用独立编号或序号,不能假装是一次新的客户端尝试。
## 参数指纹 v1
`tool_observation_layer=langchain_tool_call_chunks` 明确表示客户端观测的是 SDK 转换后的参数片段,不是原始 SSE 帧。每工具记录实际可得的 `call_id`、`chunk_index`、工具名,以及:
- `argument_chars`:参数片段的 Unicode 字符数之和。
- `argument_bytes` / `argument_sha256`:按原顺序连接片段后,其 UTF-8 字节长度和 SHA256;不解析、重排参数 JSON,也不规范化空白或 Unicode。
- `argument_deltas`:非空参数片段数,不能直接与网关原始事件数比较。
- `argument_prefixes`:每 4096 字节的累计 SHA256,最多保留前 16 个。可以在分片大小不同、上游预读或中途取消时比较相同字节位置。
- `argument_encoding_valid`:遇到不能编码为 UTF-8 的片段时为 false,当前总字节数和总哈希留空,避免伪造可比结果;诊断本身不改变执行流程。
最多记录 64 个工具;超过时设置 `tool_stats_truncated=true`,总参数字符计数仍继续。诊断不保留参数正文。哈希仅证明已收到的字节前缀,不能证明输出完整或动作成功;完整性仍由响应终止状态和解析校验决定。
跨三端优先使用 `response_id + call_id` 关联。LangChain 的 `chunk_index` 可能因文本或推理子块不同于原始 `output_index`,缺失编号时不能按数组位置猜测。正常的 `arguments.done`、`output_item.done`、`response.completed` 快照不重复计入参数;自定义工具转换、非空初始参数或协议异常需要先核对 SDK 映射,不能仅凭跨层哈希不同认定某一端重复发送。
历史日志没有保存的原始帧无法由新增诊断恢复。HTTP 200 只说明响应头已到达;客户端保护后出现的 499 或 Broken pipe 也不能证明最初原因。网关写入本机下游 writer 成功不等于客户端已消费或执行。
## 离线验证
`tests/unit/test_llm_stream_guard.py` 验证保护、有限重试、相同/不同备用端点,以及不会创建或清除其他请求的全局暂停。`tests/unit/test_request_diagnostics.py` 验证请求编号、超时重试、响应头及单次参数计数。`tests/unit/test_argument_fingerprint.py` 验证不同分片、中文和 emoji、固定字节前缀、有界统计与异常编码。这些测试不调用真实模型或设备。
+2
View File
@@ -1,5 +1,7 @@
# ARTEMIS 2.4.0
> 全新安装的 npm 校验失败已在 [2.4.1](2.4.1.md) 修复,请使用补丁版源码包或通过更新脚本升级。
本版改进批量用例人工评审与执行详情阅读,并包含 OpenAI SDK 适配、安全依赖更新、Windows 资源锁和报告证据修复。配套「任务人工评审」独立发布 **1.1.0**。
## 人工评审与原始用例
+26
View File
@@ -0,0 +1,26 @@
# ARTEMIS 2.4.1
本补丁修复 2.4.0 全新安装时的 `npm ci / EINTEGRITY`,适用于 `start.bat` 和 `start-cn.bat`。人工评审插件版本保持 **1.1.0**。
## 修复
- 2.4.0 发版时误将锁文件中的 MCP 间接依赖版本从 2.3.1 改为 2.4.0,仍保留 2.3.1 的 SHA-512。现在恢复 Angular CLI 22.2.4 官方声明的 `@modelcontextprotocol/server@2.3.1` 及其 `core@2.3.1`,使依赖关系、下载地址和校验值一致。未关闭 npm 完整性验证,也不要求用户清空全局缓存。
- Windows 启动脚本准确检查 Angular 支持的 Node.js 范围:22.x 至少 22.22.3,24.x 至少 24.15.0,或 26.0.0 及以上;提示文案同步修正。
- 保留最终 2.4.0 中的独立 v4 市场索引,提供人工评审 1.1.0;旧版宿主继续使用旧索引。曾用于发版准备的早期 2.4.0 候选包仍使用 v3 索引,会显示人工评审 1.0.0,不应继续分发。
## 验证
- Windows、Node.js 24.22.1、npm 11.19.1,独立源码目录及空 npm 缓存,使用国内镜像执行 `npm ci`:448 个包安装成功。实际依赖树与 Angular CLI 官方声明一致。
- 同一全新环境下,前端 ChromeHeadless 347 项通过,生产构建通过;官方 npm 审计未发现已知漏洞。
- 默认后端回归 3268 通过、2 跳过、8 按标记排除;包含 Node 22/24/26 兼容边界与启动脚本回归。Ruff、质量基线和核心类型检查通过。
- 未启动用户服务,未执行设备或真实业务验收;以上为安装、构建和确定性测试结果。
## 升级
已有安装请先等待任务结束,再运行 `update-cn.bat`(国内镜像)或 `update.bat`,随后重新启动。建议使用更新脚本保留已有配置与数据,不要直接用源码 ZIP 覆盖安装目录。
如果此前已手动修改 `package-lock.json`,更新脚本可能因本地修改而停止;先备份这份文件,确认只包含该安装问题的临时修正后再处理冲突,不要对整个仓库执行重置或清理。
新安装请下载本版 `artemis-2.4.1-source.zip`,按 `SHA256SUMS.txt` 校验后使用。插件仍需在市场中单独安装或更新并启用,升级主程序不会自动授权插件。
本次不构建 Docker,不启动或重启用户服务,不操作真实设备或业务任务。
+24
View File
@@ -0,0 +1,24 @@
# ARTEMIS 2.4.2
本版为批次任务增加人工评审标记,并调整任务列表布局。人工评审插件保持 **1.1.0**,第三方依赖版本不变。
## 更新
- 安装并启用人工评审插件后,任务存在已保存的有效评审内容时,批次列表显示“人工已审”。评审结论或非空备注均计入;清空全部内容后隐藏,禁用或卸载插件后也不显示。
- 标记仅说明已有人工评审内容,不代表测试通过,不覆盖 AI 判定或改变任务执行流程。没有安装插件的用户不显示该标记。
- 批次任务标题与状态分行显示:收藏星标位于标题框内,执行状态、AI 判定和人工已审标签在下方等高、左对齐排列。此布局适用于所有用户。
- 按批次一次读取当前评审内容,沿用列表自动刷新,不逐条增加网络请求;重跑任务不继承旧任务的已审标记。
- 补充发版约束:应用与第三方依赖独立版本化,检查锁文件差异,并验证最终源码包的全新安装。
## 验证
- 默认 Python 回归:3269 通过,2 跳过,8 按标记排除;Ruff、质量基线和核心类型检查通过。
- 前端 ChromeHeadless:348 项通过;生产构建通过。
- 内置浏览器使用隔离数据检查收藏星标、两行布局及有评审/无评审标签显示。未写入真实任务评审数据。
- 最终源码包另行执行独立目录、空 npm 缓存的官方源和国内镜像安装及生产构建,结果随发布附件提供。
## 升级
已有安装请等待任务结束,运行 `update-cn.bat` 或 `update.bat` 后重启。新安装下载 `artemis-2.4.2-source.zip`,按 `SHA256SUMS.txt` 校验后启动。请勿直接覆盖旧安装目录。
人工评审插件仍需单独安装并启用;本次无需升级已安装的 1.1.0。发布过程不构建 Docker,不重启用户服务,不执行设备或真实业务任务。
+25
View File
@@ -0,0 +1,25 @@
# ARTEMIS 2.4.3
本版修复异常模型输出影响整个批次的问题,并增加流式参数诊断。人工评审插件保持 **1.1.0**,SDK、其他插件及第三方依赖版本不变。
## 更新
- 已识别的异常输出(包括工具参数溢出、响应完整性校验失败)在有限重试和可用备用端点耗尽后,只结束当前任务,不再创建全局模型暂停。批次未开启“失败即停止”时可继续下一条;人工暂停、失败即停止、网络恢复及最终验收的原有规则保留。
- 保留 16384 字符工具参数保护,关闭异常输出流,丢弃未完成内容,不执行本次异常动作。报错改为描述“收到的参数流”,避免在证据不足时直接归责模型。
- 增加单次尝试请求头,区分逻辑请求、客户端尝试、网关响应与 SDK 异常中的请求编号,便于跨服务核对。
- 对收到的工具参数片段记录字符数、UTF-8 字节数及有界 SHA256 前缀摘要,默认不新增参数正文记录。最多跟踪 64 个工具,每工具最多保留 16 个固定位置前缀。SDK 的 chunk_index 与原始协议 output_index 分开命名。
诊断字段及边界见 [流式参数诊断](../llm-stream-diagnostics.md)。本次主程序安装不会部署或升级 NEWAPI、CodexManager;服务端修复独立交付。
## 验证
- 默认 Python 回归:3284 通过,6 跳过,8 按标记排除;Ruff 格式和静态检查、质量基线、核心类型检查通过。
- 五类合成输入经过实际 CodexManager、NEWAPI 转发函数及原 OpenAI/LangChain SDK 的离线文件串接:正常输入有效;超长、累计全文、重复及半关闭输入被拒绝;各层参数摘要一致,设备动作执行数为 0。
- 最终源码 ZIP 的隔离解压、独立空 npm 缓存、官方源与国内镜像安装和生产构建结果随发布附件 `INSTALL-VERIFICATION.md` 提供。
- 未重跑用户原业务批次、未做高并发压力测试或真实模型端到端验证。历史原始参数帧未保存,不能据此认定原事故参数膨胀首先发生在哪一层。
## 升级
已有安装请等待任务结束,运行 `update-cn.bat` 或 `update.bat` 后重启。新安装下载 `artemis-2.4.3-source.zip`,按 `SHA256SUMS.txt` 校验后启动。请勿直接覆盖旧安装目录。
人工评审插件 1.1.0 无需更新。本次发布不构建 Docker,不重启用户服务,不执行设备或真实业务任务。
+1 -1
View File
@@ -14,7 +14,7 @@
[project]
name = "artemis"
version = "2.4.0"
version = "2.4.3"
description = "Autonomous AI Assistant for Mobile Automation"
authors = [
+6 -10
View File
@@ -218,14 +218,10 @@ function Test-NodeCompatible {
$verRaw = (& node -v 2>$null)
if (-not $verRaw) { return $false }
$verStr = $verRaw.ToString().Trim().TrimStart('v')
$parts = $verStr.Split('.')
if ($parts.Count -ge 2) {
$major = [int]$parts[0]
$minor = [int]$parts[1]
if ($major -ge 26) { return $true }
if ($major -eq 24 -and $minor -ge 15) { return $true }
if ($major -eq 22 -and $minor -ge 22) { return $true }
}
$version = [version]$verStr
if ($version.Major -ge 26) { return $true }
if ($version.Major -eq 24 -and $version -ge [version]'24.15.0') { return $true }
if ($version.Major -eq 22 -and $version -ge [version]'22.22.3') { return $true }
} catch {
return $false
}
@@ -383,7 +379,7 @@ if ($frontendChanged -or ((-not (Test-Path $ShowcaseIndex)) -and (-not (Test-Pat
if (-not (Test-NodeCompatible)) {
if (Test-CommandExists "node") {
$curVer = (& node -v 2>$null)
Write-Host (Get-LaunchText " [INFO] Detected Node.js $curVer, but Angular CLI requires Node.js >= v22.22.0." " [检查] 检测到 Node.js $curVer;前端需要 v22.22.0 或更新版本。") -ForegroundColor Yellow
Write-Host (Get-LaunchText " [INFO] Detected Node.js $curVer; Angular CLI requires 22.x >= 22.22.3, 24.x >= 24.15.0, or >= 26.0.0." " [检查] 检测到 Node.js $curVer;前端需要 22.x >= 22.22.3、24.x >= 24.15.0 或 >= 26.0.0。") -ForegroundColor Yellow
} else {
Write-Host (Get-LaunchText " [INFO] Node.js/npm not found (required for Showcase UI)." " [检查] 未找到 Node.js/npm,网页构建需要安装。") -ForegroundColor Cyan
}
@@ -401,7 +397,7 @@ if ($frontendChanged -or ((-not (Test-Path $ShowcaseIndex)) -and (-not (Test-Pat
$useWinGetNode = $false
if ($ArtemisDownloads.mode -ne 'China' -and (Test-CommandExists "winget")) {
if ([Console]::IsInputRedirected -eq $false) {
Write-Host (Get-LaunchText " [INFO] Node.js >= v22.22.0 required for Showcase UI." " [检查] 网页需要 Node.js v22.22.0 或更新版本。") -ForegroundColor Cyan
Write-Host (Get-LaunchText " [INFO] Showcase UI requires Node.js 22.x >= 22.22.3, 24.x >= 24.15.0, or >= 26.0.0." " [检查] 网页需要 Node.js 22.x >= 22.22.3、24.x >= 24.15.0 或 >= 26.0.0。") -ForegroundColor Cyan
$ans = Read-Host (Get-LaunchText " Install/Upgrade Node.js via WinGet (may require Administrator)? [Y/n]" " 通过 WinGet 安装/升级 Node.js(可能需要管理员权限)?[Y/n]")
if ($ans -eq "" -or $ans -match "^[Yy]") {
$useWinGetNode = $true
@@ -57,15 +57,16 @@ def manifest_payload():
}
def install(plugins, original_case=False):
def install(plugins, original_case=False, plugin_id="review"):
package = BytesIO()
manifest = manifest_payload()
manifest["id"] = plugin_id
if original_case:
manifest["workspace"]["panels"][0]["kind"] = "original_case"
with zipfile.ZipFile(package, "w") as archive:
archive.writestr("plugin.json", json.dumps(manifest))
install_archive(plugins, package.getvalue())
plugins.set_enabled("review", True)
plugins.set_enabled(plugin_id, True)
@pytest.fixture
@@ -108,6 +109,71 @@ def save(client, values, revision=0, request_id=None):
)
def test_batch_review_badge_tracks_current_values_and_plugin_lifecycle(context, monkeypatch):
from apps.admin_console.routers import automation
store, plugins, client = context
monkeypatch.setattr(automation, "store", store)
result = {"status": "inconclusive", "source": "execution"}
monkeypatch.setattr(automation, "read_reviewed_result", lambda *args, **kwargs: result)
client.app.include_router(automation.router)
store.create_batch({"id": "review-batch", "status": "completed"}, [], {})
store.save_job(
{"session_id": "batch-task", "batch_id": "review-batch", "status": "completed", "case": {}}
)
store.save_job(
{"session_id": "new-run", "batch_id": "review-batch", "status": "completed", "case": {}}
)
def badge():
response = client.get("/api/automation/batches/review-batch")
assert response.status_code == 200
jobs = {job["session_id"]: job for job in response.json()["jobs"]}
assert "batch-task" in jobs, response.json()
assert jobs["batch-task"]["test_result"] == result
assert jobs["new-run"]["human_reviewed"] is False
return jobs["batch-task"]["human_reviewed"]
assert badge() is False
install(plugins)
assert (
client.put(
"/api/task-extensions/batch-task/review",
json={
"values": {"accuracy": "Pass"},
"expected_revision": 0,
"request_id": str(uuid4()),
},
).status_code
== 200
)
assert badge() is False # Other workspace plugins are not the human review plugin.
install(plugins, plugin_id="task-review")
assert badge() is False
for revision, (values, expected) in enumerate(
[
({}, False),
({"accuracy_note": " \n"}, False),
({"accuracy": "Pass"}, True),
({"accuracy": "Failed"}, True),
({"accuracy": "", "accuracy_note": ""}, False),
({"accuracy_note": "人工核对了截图"}, True),
]
):
response = client.put(
"/api/task-extensions/batch-task/task-review",
json={"values": values, "expected_revision": revision, "request_id": str(uuid4())},
)
assert response.status_code == 200
assert badge() is expected
plugins.set_enabled("task-review", False)
assert badge() is False
plugins.set_enabled("task-review", True)
assert badge() is True
plugins.uninstall("task-review")
assert badge() is False
def test_original_case_import_requires_enabled_plugin_and_keeps_execution_prompt(context):
from apps.admin_console.services import task_import
+59
View File
@@ -0,0 +1,59 @@
import hashlib
import json
import pytest
from langchain_core.messages import AIMessageChunk
from artemis.llm.argument_fingerprint import ArgumentFingerprint, MAX_PREFIXES, PREFIX_BYTES
from artemis.llm.diagnostics import RequestAttempt
@pytest.mark.parametrize("fragment_size", [1, 37, 4096, 100000])
def test_fingerprints_match_at_byte_offsets_despite_unicode_and_different_fragmentation(
fragment_size,
):
arguments = '{"target":"' + "私人内容🙂" * 8000 + '"}'
expected = arguments.encode("utf-8")
fingerprint = ArgumentFingerprint()
for offset in range(0, len(arguments), fragment_size):
fingerprint.observe(arguments[offset : offset + fragment_size])
result = fingerprint.snapshot()
assert result["argument_chars"] == len(arguments)
assert result["argument_bytes"] == len(expected)
assert result["argument_sha256"] == hashlib.sha256(expected).hexdigest()
assert result["argument_encoding_valid"] is True
assert result["argument_prefixes"] == [
{"bytes": size, "sha256": hashlib.sha256(expected[:size]).hexdigest()}
for size in range(PREFIX_BYTES, PREFIX_BYTES * MAX_PREFIXES + 1, PREFIX_BYTES)
]
assert "私人" not in json.dumps(result, ensure_ascii=False)
def test_invalid_utf8_is_marked_without_failing_or_forging_a_digest():
fingerprint = ArgumentFingerprint()
for value in ["safe", "\ud800", "later"]:
fingerprint.observe(value)
assert fingerprint.snapshot() == {
"argument_chars": 10,
"argument_bytes": None,
"argument_sha256": None,
"argument_deltas": 3,
"argument_prefixes": [],
"argument_encoding_valid": False,
}
def test_tool_tracking_is_bounded_but_total_count_keeps_all_received_arguments():
attempt = RequestAttempt(
None, request_id="bounded", attempt=1, timeout=300, max_retries=0, metadata={}
)
for index in range(65):
attempt.chunk(
AIMessageChunk(
content=[],
tool_call_chunks=[{"name": "click", "args": "{}", "index": index}],
)
)
assert len(attempt.data["tools"]) == len(attempt.tool_fingerprints) == 64
assert attempt.data["tool_stats_truncated"] is True
assert attempt.data["tool_argument_chars"] == 130
+28
View File
@@ -75,6 +75,34 @@ if ($env:npm_config_registry) { throw 'Official mode selected a China registry'
assert result.returncode == 0, result.stdout + result.stderr
def test_node_compatibility_matches_angular_supported_release_lines(tmp_path):
result = powershell(
tmp_path,
"""
$tokens = $null; $errors = $null
$ast = [System.Management.Automation.Language.Parser]::ParseFile($env:LAUNCH_SCRIPT, [ref]$tokens, [ref]$errors)
if ($errors) { throw 'Launcher parse error' }
$checker = $ast.Find({ param($node) $node -is [System.Management.Automation.Language.FunctionDefinitionAst] -and $node.Name -eq 'Test-NodeCompatible' }, $true)
. ([scriptblock]::Create($checker.Extent.Text))
function Test-CommandExists { return $true }
function node { return $script:nodeVersion }
$cases = @{
'v20.19.0' = $false; 'v22.22.0' = $false; 'v22.22.2' = $false
'v22.22.3' = $true; 'v22.23.2' = $true; 'v23.0.0' = $false
'v24.14.0' = $false; 'v24.15.0' = $true; 'v24.22.1' = $true
'v24.15.0-rc.1' = $false; 'v25.0.0' = $false; 'v26.0.0' = $true
'not-a-version' = $false
}
foreach ($case in $cases.GetEnumerator()) {
$script:nodeVersion = $case.Key
if ((Test-NodeCompatible) -ne $case.Value) { throw "Wrong compatibility for $($case.Key)" }
}
""",
LAUNCH_SCRIPT=str(ROOT / "scripts/start.ps1"),
)
assert result.returncode == 0, result.stdout + result.stderr
@pytest.mark.parametrize("first_failure", ["http", "timeout", "hash", "all"])
def test_scrcpy_source_failover_checks_hash_before_installing(tmp_path, first_failure):
archive = tmp_path / "fixture.zip"
+29
View File
@@ -2,6 +2,7 @@
import asyncio
import json
import hashlib
from types import SimpleNamespace
from unittest.mock import AsyncMock
@@ -15,6 +16,7 @@ import pytest_asyncio
from artemis.llm.reliability import FailureCategory, LLMExhaustedError
from artemis.llm.response_guard import ResponseIntegrityError, ensure_response_complete
from artemis.llm.router import ModelEndpoint, ModelFactory
from artemis.llm.diagnostics import RequestAttempt
from artemis.services import llm as llm_service
from artemis.services.llm import RobustChatModelWrapper, acomplete
@@ -195,6 +197,33 @@ async def test_completed_stream_returns_original_valid_arguments(transport_case)
assert result.tool_calls[0]["args"] == {"passed": True}
@pytest.mark.asyncio
@pytest.mark.parametrize("transport_case", ["fragmented"], indirect=True)
async def test_real_sdk_fingerprint_counts_received_arguments_once(transport_case):
case = transport_case
arguments = json.dumps({"passed": True, "evidence": "中文🙂" * 1000}, ensure_ascii=False)
case.outputs.append(("completed", arguments))
observation = RequestAttempt(
None, request_id="sdk-test", attempt=1, timeout=300, max_retries=0, metadata={}
)
merged = None
async for chunk in case.model.astream([]):
observation.chunk(chunk)
merged = chunk if merged is None else merged + chunk
observation.finish("completed", response=merged)
assert len(observation.data["tools"]) == 1
tool = observation.data["tools"][0]
assert tool["call_id"] == "call_test"
assert tool["argument_chars"] == len(arguments)
assert tool["argument_bytes"] == len(arguments.encode())
assert tool["argument_sha256"] == hashlib.sha256(arguments.encode()).hexdigest()
assert tool["argument_prefixes"][0] == {
"bytes": 4096,
"sha256": hashlib.sha256(arguments.encode()[:4096]).hexdigest(),
}
assert tool["argument_deltas"] == 1
@pytest.mark.asyncio
@pytest.mark.parametrize("transport_case", ["fragmented"], indirect=True)
@pytest.mark.parametrize("status", ["completed", None])
+47 -1
View File
@@ -124,7 +124,9 @@ async def test_gateway_closes_bad_stream_discards_partial_and_bounds_retry(
with pytest.raises(LLMExhaustedError) as caught:
await wrapper.complete([])
assert caught.value.failure.category is FailureCategory.INVALID_OUTPUT
llm._wait_for_resume.assert_awaited_once()
llm._wait_for_resume.assert_not_awaited()
assert not llm.PAUSE_FILE.exists()
assert any(name == "llm_gave_up" and p["handover"] == "task_failed" for name, p in resets)
assert model.calls == 2
assert closed == [1, 2]
failure = next(
@@ -139,3 +141,47 @@ async def test_gateway_closes_bad_stream_discards_partial_and_bounds_retry(
assert failure["usage"] is None
assert "private" not in json.dumps(records + resets, default=str)
assert any(name == "llm_stream_reset" and p["action"] == "discard" for name, p in resets)
@pytest.mark.asyncio
@pytest.mark.parametrize("fallback_kind", ["same", "failing", "working"])
@pytest.mark.parametrize("already_paused", [False, True])
async def test_invalid_output_fallback_does_not_pause_or_clear_another_request(
fallback_kind, already_paused
):
calls = []
if already_paused:
llm.PAUSE_FILE.write_text("another request or user pause", encoding="utf-8")
async def main():
calls.append("main")
raise ToolArgumentOverflow("click", SMALL_ACTION_ARGUMENT_LIMIT + 1)
async def fallback():
calls.append("fallback")
if fallback_kind == "working":
return "valid response"
raise ToolArgumentOverflow("click", SMALL_ACTION_ARGUMENT_LIMIT + 2)
async def run():
return await llm.with_fallback(
lambda: llm._run_with_recovery(main, endpoint_key="main", max_retries=5),
lambda: llm._run_with_recovery(
fallback,
endpoint_key="main" if fallback_kind == "same" else "fallback",
max_retries=5,
),
)
if fallback_kind == "working":
assert await run() == "valid response"
else:
with pytest.raises(LLMExhaustedError) as caught:
await run()
assert caught.value.failure.category is FailureCategory.INVALID_OUTPUT
assert calls.count("main") == 2
assert calls.count("fallback") == {"same": 0, "failing": 2, "working": 1}[fallback_kind]
llm._wait_for_resume.assert_not_awaited()
assert llm.PAUSE_FILE.exists() is already_paused
if already_paused:
assert llm.PAUSE_FILE.read_text(encoding="utf-8") == "another request or user pause"
+79 -6
View File
@@ -1,6 +1,7 @@
import asyncio
from copy import deepcopy
import json
import hashlib
from types import SimpleNamespace
from uuid import uuid4
@@ -36,6 +37,46 @@ class Engine:
pass
@pytest.mark.parametrize(
("headers", "expected"),
[
({"X-Oneapi-Request-Id": "gateway-request"}, "gateway-request"),
(
{"X-Oneapi-Request-Id": "gateway-request", "X-Request-Id": "provider-request"},
"gateway-request",
),
({"X-Request-Id": "provider-request"}, "provider-request"),
({"Request-Id": "legacy-request"}, "legacy-request"),
(
{"X-Oneapi-Request-Id": "invalid header value", "X-Request-Id": "standard-request"},
"standard-request",
),
],
)
def test_gateway_request_id_preserves_the_standard_header_without_assuming_its_source(
headers, expected
):
attempt = RequestAttempt(
Engine(), request_id="local-request", attempt=1, timeout=300, max_retries=1, metadata={}
)
attempt.headers(httpx.Response(200, headers=headers))
assert attempt.data["server_request_id"] == expected
assert attempt.data["response_x_request_id"] == headers.get("X-Request-Id")
def test_sdk_error_cannot_overwrite_the_observed_gateway_request_id():
class RequestError(Exception):
request_id = "sdk-request"
attempt = RequestAttempt(
Engine(), request_id="local-request", attempt=1, timeout=300, max_retries=1, metadata={}
)
attempt.headers(httpx.Response(400, headers={"X-Oneapi-Request-Id": "gateway-request"}))
attempt.finish("failed", error=RequestError("failed"))
assert attempt.data["server_request_id"] == "gateway-request"
assert attempt.data["error_request_id"] == "sdk-request"
@pytest.mark.asyncio
async def test_request_hook_correlates_api_calls_without_logging_header_values():
engine = Engine()
@@ -55,13 +96,18 @@ async def test_request_hook_correlates_api_calls_without_logging_header_values()
)
await observe_request_headers(request)
assert request.headers["x-client-request-id"] == "local-request"
assert request.headers["x-client-attempt-id"] == str(attempt.trace_id)
assert attempt.data["client_attempt_id_matches"] is True
assert attempt.data["client_request_id_matches"] is True
assert len(attempt.data["routing_header_fingerprints"]) == 2
assert "private" not in json.dumps(engine.records, default=str)
request.headers["X-Client-Request-Id"] = "caller-id"
request.headers["X-Client-Attempt-Id"] = "caller-attempt"
await observe_request_headers(request)
assert request.headers["x-client-request-id"] == "caller-id"
assert attempt.data["client_request_id_matches"] is False
assert request.headers["X-Client-Attempt-Id"] == "caller-attempt"
assert attempt.data["client_attempt_id_matches"] is False
finally:
ACTIVE_ATTEMPT.reset(token)
@@ -75,6 +121,7 @@ async def test_headers_without_body_timeout_then_retry_keeps_evidence(monkeypatc
)
llm._ENDPOINT_BREAKER._states.clear()
calls = 0
wire_ids = []
class StalledBody(httpx.AsyncByteStream):
async def __aiter__(self):
@@ -84,6 +131,9 @@ async def test_headers_without_body_timeout_then_retry_keeps_evidence(monkeypatc
async def transport(request):
nonlocal calls
calls += 1
wire_ids.append(
(request.headers["X-Client-Request-Id"], request.headers["X-Client-Attempt-Id"])
)
headers = {"x-request-id": f"server-{calls}", "set-cookie": "secret-cookie"}
return httpx.Response(
200, headers=headers, stream=StalledBody() if calls == 1 else httpx.ByteStream(b"ok")
@@ -91,7 +141,7 @@ async def test_headers_without_body_timeout_then_retry_keeps_evidence(monkeypatc
async with httpx.AsyncClient(
transport=httpx.MockTransport(transport),
event_hooks={"response": [observe_response_headers]},
event_hooks={"request": [observe_request_headers], "response": [observe_response_headers]},
) as client:
class Model:
@@ -123,6 +173,10 @@ async def test_headers_without_body_timeout_then_retry_keeps_evidence(monkeypatc
assert final["request_id"] == failed["request_id"]
assert final["attempt_id"] != failed["attempt_id"] and final["attempt"] == 2
assert final["previous_attempt_id"] == failed["attempt_id"]
assert wire_ids == [
(failed["request_id"], failed["attempt_id"]),
(final["request_id"], final["attempt_id"]),
]
retry = next(r["payload"] for r in engine.records if r["name"] == "llm_retry")
assert retry["request_id"] == failed["request_id"]
assert retry["attempt_id"] == failed["attempt_id"]
@@ -163,10 +217,26 @@ def test_tool_deltas_are_counted_once_without_retaining_arguments():
attempt.finish("completed", response=combined)
assert attempt.data["output_chunks"] == 4
assert attempt.data["tool_argument_chars"] == len('{"secret":"private"}{"seconds":1}')
assert attempt.data["tools"] == [
{"name": "click", "argument_chars": len('{"secret":"private"}')},
{"name": "wait", "argument_chars": len('{"seconds":1}')},
]
for tool, name, arguments, call_id, index, count in zip(
attempt.data["tools"],
["click", "wait"],
['{"secret":"private"}', '{"seconds":1}'],
["c1", "c2"],
[0, 1],
[2, 1],
strict=True,
):
assert tool == {
"name": name,
"call_id": call_id,
"chunk_index": index,
"argument_chars": len(arguments),
"argument_bytes": len(arguments.encode()),
"argument_sha256": hashlib.sha256(arguments.encode()).hexdigest(),
"argument_deltas": count,
"argument_prefixes": [],
"argument_encoding_valid": True,
}
assert attempt.data["response_status"] == "completed"
assert "private" not in json.dumps(engine.records, default=str)
assert "secret" not in json.dumps(engine.records, default=str)
@@ -205,7 +275,10 @@ async def test_timeout_during_tool_generation_keeps_last_output_and_tool_name(mo
)
assert failed["failure_phase"] == "generating_tool_arguments"
assert failed["last_output_at"] >= failed["first_output_at"]
assert failed["tools"] == [{"name": "click", "argument_chars": 5}]
assert len(failed["tools"]) == 1
tool = failed["tools"][0]
assert tool["name"] == "click" and tool["argument_chars"] == tool["argument_bytes"] == 5
assert tool["argument_sha256"] == hashlib.sha256(b'{"x":').hexdigest()
assert failed["usage"] is None
assert failed["max_output_tokens"] == 12000
Generated
+1 -1
View File
@@ -248,7 +248,7 @@ wheels = [
[[package]]
name = "artemis"
version = "2.4.0"
version = "2.4.3"
source = { editable = "." }
dependencies = [
{ name = "adbutils" },