Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0c3ca69c1d | ||
|
|
486e6d550d | ||
|
|
b53ca18595 |
No files matched your search
@@ -21,7 +21,7 @@
|
|||||||
| 探索、示范、用例提炼与复跑 | [探索说明](docs/exploration-mode.md) |
|
| 探索、示范、用例提炼与复跑 | [探索说明](docs/exploration-mode.md) |
|
||||||
| 插件协议、适配或市场协同 | [插件开发规范](docs/plugin-development.md)及插件仓库自身的 AGENTS.md、DESIGN.md |
|
| 插件协议、适配或市场协同 | [插件开发规范](docs/plugin-development.md)及插件仓库自身的 AGENTS.md、DESIGN.md |
|
||||||
| 远程 Python SDK | [SDK 设计](packages/artemis-client/DESIGN.md),保持标准库薄客户端边界 |
|
| 远程 Python SDK | [SDK 设计](packages/artemis-client/DESIGN.md),保持标准库薄客户端边界 |
|
||||||
| 继续历史开发、确认版本或发布 | [迁入时的开发快照](docs/development-handoff.md)、[2.1.0 发行说明](docs/releases/2.1.0.md),再核对实时 Git 状态;`docs/plans/` 保留各自阶段的状态 |
|
| 继续历史开发、确认版本或发布 | [README 当前发行入口](README.md)、对应发行说明及实时 Git 状态;[开发快照](docs/development-handoff.md)与 `docs/plans/` 仅作为历史线索,不替代当前源码与发布包核验 |
|
||||||
|
|
||||||
## Quick Commands
|
## Quick Commands
|
||||||
|
|
||||||
@@ -60,7 +60,7 @@ Windows 首次安装使用 `start.bat`,国内镜像使用 `start-cn.bat`。启
|
|||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
- Python 使用绝对导入、4 空格缩进和双引号,由 Ruff 检查;不要放宽质量基线来掩盖新增问题。
|
- Python 使用绝对导入、4 空格缩进和双引号,由 Ruff 检查;不要放宽质量基线来掩盖新增问题。
|
||||||
- 新增可调参数集中进入配置层,并更新环境示例、网页/MCP 入口和必要文档;默认值同步 `config/artemis.jsonc` 与 `artemis/resources/config/artemis.jsonc`。
|
- 新增可调参数集中进入配置层,按作用域更新环境示例、网页/MCP 入口和文档;明确仅供环境配置的参数不增加前端入口。JSON 默认值同步 `config/artemis.jsonc` 与 `artemis/resources/config/artemis.jsonc`。
|
||||||
- MCP 工具增删时同步工具注册、CLI 安装白名单、测试、Skill 和能力文档。
|
- MCP 工具增删时同步工具注册、CLI 安装白名单、测试、Skill 和能力文档。
|
||||||
- 同步 Codex MCP/Skill 时保留已有工具白名单、禁用状态和审批策略;异常时保留原配置并告警,不生成非法 TOML 或阻断网页启动。
|
- 同步 Codex MCP/Skill 时保留已有工具白名单、禁用状态和审批策略;异常时保留原配置并告警,不生成非法 TOML 或阻断网页启动。
|
||||||
- Windows PowerShell 启动脚本保持兼容 Windows PowerShell 5.1,避免编码导致解析失败。升级只做 Git 快进,保护已有配置和本地修改。
|
- Windows PowerShell 启动脚本保持兼容 Windows PowerShell 5.1,避免编码导致解析失败。升级只做 Git 快进,保护已有配置和本地修改。
|
||||||
@@ -75,6 +75,16 @@ Windows 首次安装使用 `start.bat`,国内镜像使用 `start-cn.bat`。启
|
|||||||
- 设备验证沿用用户指定设备;多设备且未指定时先确认。编写应用交互测试前,先通过 ARTEMIS 探索实际页面和操作路径。
|
- 设备验证沿用用户指定设备;多设备且未指定时先确认。编写应用交互测试前,先通过 ARTEMIS 探索实际页面和操作路径。
|
||||||
- 提交前检查 `git diff --check` 和暂存范围,只暂存本次具体路径;报告实际执行的验证及未完成项,不将未运行的检查写成通过。
|
- 提交前检查 `git diff --check` 和暂存范围,只暂存本次具体路径;报告实际执行的验证及未完成项,不将未运行的检查写成通过。
|
||||||
|
|
||||||
|
## Release Gates
|
||||||
|
|
||||||
|
- 主程序、SDK、插件和第三方依赖独立版本化。主程序升版只修改 `pyproject.toml` 的项目版本、`uv.lock` 的 artemis 条目及前端应用的根版本字段,发行文档另行更新。禁止对仓库或整个锁文件全局替换版本字符串。
|
||||||
|
- 单独审查锁文件差异。第三方条目的版本、下载地址、完整性校验和依赖声明必须相互匹配;主程序升版不应改变这些条目。真正升级依赖时由包管理器生成并核对上游声明;发生 EINTEGRITY 先核验来源,禁止关闭校验或只照错误提示改哈希。
|
||||||
|
- 发布前必须在隔离目录解压最终 ZIP,使用独立空 npm 缓存完成 `npm ci` 和生产构建,覆盖本次发布的官方/国内安装路径。包内安装或构建输入变化后必须重跑;不得清空用户全局缓存或改动用户运行目录来制造干净环境。
|
||||||
|
- 记录实际 Node/npm 版本、镜像、源码提交、ZIP SHA256 和验证范围。已有 node_modules、构建指纹、缓存命中或大量单元测试通过,均不能证明全新安装成功;某个 Node 版本通过也不能冒充其他版本实测。测试和构建命令见前端局部指南。
|
||||||
|
- 发布顺序:固定提交并打包 → 验证最终包 → 推送标签 → 创建草稿并上传包及校验文件 → 回读校验 → 正式发布 → 无认证下载核验。标签必须指向打包提交;推送标签后检查 Release 状态,本仓库曾出现 Gitea 将已有草稿自动发布的情况。失败时修复并重新验证,不提前宣布完成。
|
||||||
|
- 候选包与正式包目录明确区分。候选包失效后移出可分发路径并标明替代版本,不能留下同名旧 ZIP 供用户误取;对外只给已核验的正式发布链接或准确文件路径。正式版本不覆盖原标签或附件,后续修复发补丁版。
|
||||||
|
- 安装或市场版本不符时,先核对用户实际 ZIP 的 SHA256、运行源码提交及生效市场地址;不凭界面版本号判断,也不把未经验证的缓存假设当成原因。主程序与插件版本、索引兼容性分别验证。
|
||||||
|
|
||||||
## Boundaries
|
## Boundaries
|
||||||
|
|
||||||
- 始终保护用户任务、历史库、环境配置和凭据。不要把个人设备序列号、绝对目录或 API Key 写入源码。
|
- 始终保护用户任务、历史库、环境配置和凭据。不要把个人设备序列号、绝对目录或 API Key 写入源码。
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
基于 [ARTEMIS](https://github.com/google/artemis) 扩展的 Android AI 自动化测试平台。用自然语言描述任务,即可通过简体中文网页或 MCP 驱动真机、模拟器执行操作并收集结果。模型统一使用 **OpenAI Responses API**。
|
基于 [ARTEMIS](https://github.com/google/artemis) 扩展的 Android AI 自动化测试平台。用自然语言描述任务,即可通过简体中文网页或 MCP 驱动真机、模拟器执行操作并收集结果。模型统一使用 **OpenAI Responses API**。
|
||||||
|
|
||||||
**2.4.1**:修复全新安装时的 npm 校验失败,准确检查 Node.js 版本要求。人工评审插件继续使用 1.1.0,市场使用兼容旧宿主的独立索引。见 [2.4.1 发行说明](docs/releases/2.4.1.md);功能更新见 [2.4.0](docs/releases/2.4.0.md)。
|
**2.4.3**:异常模型输出耗尽重试后只失败当前任务,避免触发全局暂停;补充请求关联和有界参数摘要,便于定位流式异常。见 [2.4.3 发行说明](docs/releases/2.4.3.md)。人工评审插件保持 1.1.0;列表优化见 [2.4.2](docs/releases/2.4.2.md),安装修复见 [2.4.1](docs/releases/2.4.1.md)。
|
||||||
|
|
||||||
## 主要功能
|
## 主要功能
|
||||||
|
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ from apps.admin_console.services.exploration_listing import EXPLORATION_RUNS_CTE
|
|||||||
from apps.admin_console.services.automation_store import identifier, store
|
from apps.admin_console.services.automation_store import identifier, store
|
||||||
from apps.admin_console.services import automation_service as service
|
from apps.admin_console.services import automation_service as service
|
||||||
from apps.admin_console.services import batch_archive as archive_service
|
from apps.admin_console.services import batch_archive as archive_service
|
||||||
from apps.admin_console.services import report_export, task_import
|
from apps.admin_console.services import report_export, task_extensions, task_import
|
||||||
from artemis.config.settings import settings
|
from artemis.config.settings import settings
|
||||||
from apps.admin_console.services.case_review import read_reviewed_result
|
from apps.admin_console.services.case_review import read_reviewed_result
|
||||||
|
|
||||||
@@ -131,7 +131,9 @@ def batch_by_request(request_id: str):
|
|||||||
@router.get("/batches/{batch_id}")
|
@router.get("/batches/{batch_id}")
|
||||||
def batch_detail(batch_id: str):
|
def batch_detail(batch_id: str):
|
||||||
jobs = store.jobs(batch_id)
|
jobs = store.jobs(batch_id)
|
||||||
|
reviewed = task_extensions.reviewed_batch_sessions(batch_id, store)
|
||||||
for job in jobs:
|
for job in jobs:
|
||||||
|
job["human_reviewed"] = job["session_id"] in reviewed
|
||||||
job["test_result"] = read_reviewed_result(
|
job["test_result"] = read_reviewed_result(
|
||||||
job["session_id"], job["status"], job.get("error") or "", storage=store
|
job["session_id"], job["status"], job.get("error") or "", storage=store
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -36,6 +36,32 @@ def installation_signature() -> str:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def reviewed_batch_sessions(batch_id: str, store: AutomationStore) -> set[str]:
|
||||||
|
"""Read current human review values once per batch, without loading history."""
|
||||||
|
manifest = next((item for item in active_manifests() if item.id == "task-review"), None)
|
||||||
|
if manifest is None or manifest.workspace is None:
|
||||||
|
return set()
|
||||||
|
with store.connection() as conn:
|
||||||
|
rows = conn.execute(
|
||||||
|
"""SELECT j.session_id, json_extract(r.payload, '$."task-review".values') AS review_values
|
||||||
|
FROM automation_jobs j JOIN automation_records r
|
||||||
|
ON r.kind='task_extensions' AND r.id=j.session_id
|
||||||
|
WHERE j.batch_id=?""",
|
||||||
|
(batch_id,),
|
||||||
|
).fetchall()
|
||||||
|
reviewed = set()
|
||||||
|
for row in rows:
|
||||||
|
values = json.loads(row["review_values"] or "{}")
|
||||||
|
if any(
|
||||||
|
isinstance(value := values.get(field.id), str)
|
||||||
|
and value.strip()
|
||||||
|
and (field.kind == "text" or value in field.options)
|
||||||
|
for field in manifest.workspace.fields
|
||||||
|
):
|
||||||
|
reviewed.add(row["session_id"])
|
||||||
|
return reviewed
|
||||||
|
|
||||||
|
|
||||||
def task_context(session_id: str, store: AutomationStore) -> dict:
|
def task_context(session_id: str, store: AutomationStore) -> dict:
|
||||||
identifier(session_id)
|
identifier(session_id)
|
||||||
session = SessionRepository(store.path).get_session_by_id(session_id) or {}
|
session = SessionRepository(store.path).get_session_by_id(session_id) or {}
|
||||||
|
|||||||
@@ -30,6 +30,9 @@ npm run build
|
|||||||
|
|
||||||
## Local Conventions
|
## Local Conventions
|
||||||
|
|
||||||
|
- 发版遵守根指南的 Release Gates。前端应用版本只改 package.json 顶层 `version` 和 package-lock.json 顶层及 `packages[""].version`;禁止把相同数字的第三方依赖一并替换。
|
||||||
|
- Node 支持范围以锁定的 Angular CLI/构建依赖实际 `engines` 为准,核对启动脚本的主、次、补丁版本边界和提示;不要只判断“Node 22/24/26”。
|
||||||
|
- 安装或依赖变更需在隔离目录、独立空 npm 缓存下运行 `npm ci`,再运行上述测试和构建;国内入口同时核验镜像安装。不得以关闭完整性校验、强制安装或删除锁文件掩盖不一致,也不得用已装依赖的构建结果宣称全新安装通过。
|
||||||
- 页面组件在 src/app/components,服务在 src/app/services;先复用已有组件、样式及 API 服务。
|
- 页面组件在 src/app/components,服务在 src/app/services;先复用已有组件、样式及 API 服务。
|
||||||
- 保持现有简体中文文案、颜色、间距和控件样式,菜单、弹窗和表单必须清晰可读。
|
- 保持现有简体中文文案、颜色、间距和控件样式,菜单、弹窗和表单必须清晰可读。
|
||||||
- 修改任务或配置交互时覆盖加载、保存、失败和空状态;不要将执行中误标成录屏中或把完成误标成验证通过。
|
- 修改任务或配置交互时覆盖加载、保存、失败和空状态;不要将执行中误标成录屏中或把完成误标成验证通过。
|
||||||
|
|||||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
|||||||
{
|
{
|
||||||
"name": "artemis-showcase-ui",
|
"name": "artemis-showcase-ui",
|
||||||
"version": "2.4.1",
|
"version": "2.4.3",
|
||||||
"lockfileVersion": 3,
|
"lockfileVersion": 3,
|
||||||
"requires": true,
|
"requires": true,
|
||||||
"packages": {
|
"packages": {
|
||||||
"": {
|
"": {
|
||||||
"name": "artemis-showcase-ui",
|
"name": "artemis-showcase-ui",
|
||||||
"version": "2.4.1",
|
"version": "2.4.3",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@angular/cdk": "^22.2.4",
|
"@angular/cdk": "^22.2.4",
|
||||||
"@angular/common": "^22.2.3",
|
"@angular/common": "^22.2.3",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "artemis-showcase-ui",
|
"name": "artemis-showcase-ui",
|
||||||
"version": "2.4.1",
|
"version": "2.4.3",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"ng": "ng",
|
"ng": "ng",
|
||||||
"start": "ng serve --proxy-config proxy.conf.json",
|
"start": "ng serve --proxy-config proxy.conf.json",
|
||||||
|
|||||||
+14
-1
@@ -108,6 +108,19 @@
|
|||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if(b.report_error){<p class="error">报告生成失败:{{b.report_error}}</p>}
|
@if(b.report_error){<p class="error">报告生成失败:{{b.report_error}}</p>}
|
||||||
@for(j of b.jobs; track j.session_id){<div class="job" [class.selected]="agent.currentSessionId() === j.session_id"><app-favorite-button [sourceId]="j.session_id" /><button (click)="openBatchTask(j.session_id)" [title]="j.case.goal" [attr.aria-pressed]="agent.currentSessionId() === j.session_id">{{j.case.case_id ? j.case.case_id + ' ' : ''}}{{j.case.name}}@if(j.repeat_count > 1){ · 第 {{j.iteration}} / {{j.repeat_count}} 轮}</button><span>{{label(j.status)}}</span><app-test-result-badge [result]="j.test_result" [executionStatus]="j.status" [compact]="true" />@if(j.error){<small>{{j.error}}</small>}</div>}
|
@for(j of b.jobs; track j.session_id){
|
||||||
|
<div class="job" [class.selected]="agent.currentSessionId() === j.session_id">
|
||||||
|
<div class="job-heading">
|
||||||
|
<app-favorite-button [sourceId]="j.session_id" />
|
||||||
|
<button (click)="openBatchTask(j.session_id)" [title]="j.case.goal" [attr.aria-pressed]="agent.currentSessionId() === j.session_id">{{j.case.case_id ? j.case.case_id + ' ' : ''}}{{j.case.name}}@if(j.repeat_count > 1){ · 第 {{j.iteration}} / {{j.repeat_count}} 轮}</button>
|
||||||
|
</div>
|
||||||
|
<div class="job-statuses">
|
||||||
|
<span>{{label(j.status)}}</span>
|
||||||
|
<app-test-result-badge [result]="j.test_result" [executionStatus]="j.status" [compact]="true" />
|
||||||
|
@if(j.human_reviewed){<span class="human-review-badge" title="人工评审已有保存内容,不代表测试通过">人工已审</span>}
|
||||||
|
</div>
|
||||||
|
@if(j.error){<small>{{j.error}}</small>}
|
||||||
|
</div>
|
||||||
|
}
|
||||||
</section>}
|
</section>}
|
||||||
}
|
}
|
||||||
+11
-1
@@ -37,7 +37,15 @@ input[type=checkbox]{width:15px;height:15px;accent-color:var(--app-primary-text,
|
|||||||
@keyframes archive-spin{to{transform:rotate(360deg);}}
|
@keyframes archive-spin{to{transform:rotate(360deg);}}
|
||||||
.code{font-family:Consolas,monospace;font-size:12px;line-height:1.6;resize:vertical;background:var(--app-surface-soft, #f8fafc);}.table-wrap{max-height:260px;overflow:auto;border:1px solid var(--app-border, #e2e8f0);border-radius:8px;margin-bottom:14px;}table{width:100%;border-collapse:collapse;}th,td{padding:9px 10px;border-bottom:1px solid var(--app-border, #e2e8f0);text-align:left;overflow-wrap:anywhere;}th{background:var(--app-surface-soft, #f8fafc);color:var(--app-text-secondary, #475569);font-weight:500;}
|
.code{font-family:Consolas,monospace;font-size:12px;line-height:1.6;resize:vertical;background:var(--app-surface-soft, #f8fafc);}.table-wrap{max-height:260px;overflow:auto;border:1px solid var(--app-border, #e2e8f0);border-radius:8px;margin-bottom:14px;}table{width:100%;border-collapse:collapse;}th,td{padding:9px 10px;border-bottom:1px solid var(--app-border, #e2e8f0);text-align:left;overflow-wrap:anywhere;}th{background:var(--app-surface-soft, #f8fafc);color:var(--app-text-secondary, #475569);font-weight:500;}
|
||||||
.preview{white-space:pre-wrap;max-height:400px;overflow:auto;font:inherit;color:var(--app-text-control, #334155);background:var(--app-surface-soft, #f8fafc);border-radius:8px;padding:12px;border:1px solid var(--app-border, #e2e8f0);}
|
.preview{white-space:pre-wrap;max-height:400px;overflow:auto;font:inherit;color:var(--app-text-control, #334155);background:var(--app-surface-soft, #f8fafc);border-radius:8px;padding:12px;border:1px solid var(--app-border, #e2e8f0);}
|
||||||
.job{display:flex;flex-wrap:wrap;gap:8px;align-items:center;padding:10px 0;border-bottom:1px solid var(--app-border, #f1f5f9);}.job:last-child{border-bottom:0;}.job button{flex:1;justify-content:flex-start;text-align:left;min-width:150px;}.job span{font-size:12px;background:var(--app-surface-soft, #f1f5f9);color:var(--app-text-secondary, #475569);border-radius:5px;padding:3px 7px;}.job small{width:100%;}details{margin:14px 0;}summary{color:var(--app-text-secondary, #475569);cursor:pointer;padding:8px 0;}
|
.job{display:flex;flex-direction:column;gap:8px;padding:12px 0;border-bottom:1px solid var(--app-border, #f1f5f9);}
|
||||||
|
.job:last-child{border-bottom:0;}
|
||||||
|
.job-heading{position:relative;min-width:0;}
|
||||||
|
.job-heading>app-favorite-button{position:absolute;left:6px;top:4px;z-index:1;}
|
||||||
|
.job-heading>button{box-sizing:border-box;width:100%;min-width:0;padding-left:40px;justify-content:flex-start;text-align:left;overflow-wrap:anywhere;}
|
||||||
|
.job-statuses{display:flex;flex-wrap:wrap;align-items:center;gap:8px;}
|
||||||
|
.job-statuses>span{display:inline-flex;align-items:center;box-sizing:border-box;height:20px;padding:0 6px;border:1px solid transparent;border-radius:5px;font-size:11px;line-height:14px;background:var(--app-surface-soft, #f1f5f9);color:var(--app-text-secondary, #475569);white-space:nowrap;}
|
||||||
|
.job small{overflow-wrap:anywhere;}
|
||||||
|
details{margin:14px 0;}summary{color:var(--app-text-secondary, #475569);cursor:pointer;padding:8px 0;}
|
||||||
.task-guidance{margin-top:16px;padding:14px;border:1px solid var(--app-primary-border, #bfdbfe);border-radius:10px;background:var(--app-primary-soft, #f8fbff);}
|
.task-guidance{margin-top:16px;padding:14px;border:1px solid var(--app-primary-border, #bfdbfe);border-radius:10px;background:var(--app-primary-soft, #f8fbff);}
|
||||||
.task-guidance label{margin:8px 0 4px;}
|
.task-guidance label{margin:8px 0 4px;}
|
||||||
.report-overview{padding:14px;border:1px solid var(--app-primary-border, #dbeafe);border-radius:10px;background:var(--app-primary-soft, #f8fbff);}
|
.report-overview{padding:14px;border:1px solid var(--app-primary-border, #dbeafe);border-radius:10px;background:var(--app-primary-soft, #f8fbff);}
|
||||||
@@ -58,3 +66,5 @@ input[type=checkbox]{width:15px;height:15px;accent-color:var(--app-primary-text,
|
|||||||
table{width:100%;border-collapse:collapse;font-size:12px;}th,td{padding:7px 9px;border:1px solid var(--app-border, #e2e8f0);vertical-align:top;}th{background:var(--app-primary-soft, #eff6ff);color:var(--app-primary-text, #1e3a8a);}
|
table{width:100%;border-collapse:collapse;font-size:12px;}th,td{padding:7px 9px;border:1px solid var(--app-border, #e2e8f0);vertical-align:top;}th{background:var(--app-primary-soft, #eff6ff);color:var(--app-primary-text, #1e3a8a);}
|
||||||
.md-para-gap{height:8px;}
|
.md-para-gap{height:8px;}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.job .human-review-badge{white-space:nowrap;border:1px solid var(--app-primary-border, #bfdbfe);background:var(--app-primary-soft, #eff6ff);color:var(--app-primary-text, #1d4ed8);font-weight:500;}
|
||||||
+25
-1
@@ -90,6 +90,30 @@ describe('AutomationPanelComponent batch deletion', () => {
|
|||||||
expect(http.post).not.toHaveBeenCalled();
|
expect(http.post).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('shows only saved human review badges and removes them on a batch refresh', async () => {
|
||||||
|
const fixture = TestBed.createComponent(AutomationPanelComponent);
|
||||||
|
const component = fixture.componentInstance;
|
||||||
|
await new Promise<void>(resolve => setTimeout(resolve, 0));
|
||||||
|
component.tab.set('batches');
|
||||||
|
const batch = { id: 'review-batch', name: '评审批次', status: 'completed', jobs: [
|
||||||
|
{ session_id: 'reviewed', status: 'completed', human_reviewed: true, case: { name: '已评用例' } },
|
||||||
|
{ session_id: 'empty', status: 'completed', human_reviewed: false, case: { name: '空白用例' } },
|
||||||
|
{ session_id: 'legacy', status: 'completed', case: { name: '旧版任务' } }
|
||||||
|
] };
|
||||||
|
component.batch.set(batch);
|
||||||
|
fixture.detectChanges();
|
||||||
|
const rows = fixture.nativeElement.querySelectorAll('.job');
|
||||||
|
expect(rows[0].querySelector('.human-review-badge').textContent).toBe('人工已审');
|
||||||
|
expect(rows[1].querySelector('.human-review-badge')).toBeNull();
|
||||||
|
expect(rows[2].querySelector('.human-review-badge')).toBeNull();
|
||||||
|
expect(fixture.nativeElement.textContent).not.toContain('人工未审');
|
||||||
|
(http.get.and as any).callFake((url: string) => of(url.endsWith('/review-batch')
|
||||||
|
? { ...batch, jobs: batch.jobs.map(job => ({ ...job, human_reviewed: false })) } : []));
|
||||||
|
await component.refresh();
|
||||||
|
fixture.detectChanges();
|
||||||
|
expect(fixture.nativeElement.querySelector('.human-review-badge')).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
it('confirms deletion and removes the batch from the visible list', async () => {
|
it('confirms deletion and removes the batch from the visible list', async () => {
|
||||||
const fixture = TestBed.createComponent(AutomationPanelComponent);
|
const fixture = TestBed.createComponent(AutomationPanelComponent);
|
||||||
fixture.componentInstance.tab.set('batches');
|
fixture.componentInstance.tab.set('batches');
|
||||||
@@ -129,7 +153,7 @@ describe('AutomationPanelComponent batch deletion', () => {
|
|||||||
{session_id:'task-b',status:'completed',case:{name:'用例 B',goal:'观察主页'}}
|
{session_id:'task-b',status:'completed',case:{name:'用例 B',goal:'观察主页'}}
|
||||||
]});
|
]});
|
||||||
fixture.detectChanges();
|
fixture.detectChanges();
|
||||||
const jobs: HTMLButtonElement[] = [...fixture.nativeElement.querySelectorAll('.job > button')];
|
const jobs: HTMLButtonElement[] = [...fixture.nativeElement.querySelectorAll('.job-heading > button')];
|
||||||
jobs[0].click(); jobs[1].click();
|
jobs[0].click(); jobs[1].click();
|
||||||
expect(agent.selectSession.calls.allArgs()).toEqual([['task-a',true],['task-b',true]]);
|
expect(agent.selectSession.calls.allArgs()).toEqual([['task-a',true],['task-b',true]]);
|
||||||
expect(component.tab()).toBe('batches');
|
expect(component.tab()).toBe('batches');
|
||||||
|
|||||||
@@ -0,0 +1,53 @@
|
|||||||
|
"""Bounded fingerprints of received argument deltas, without retaining text."""
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
|
||||||
|
|
||||||
|
PREFIX_BYTES = 4096
|
||||||
|
MAX_PREFIXES = 16
|
||||||
|
MAX_TRACKED_TOOLS = 64
|
||||||
|
|
||||||
|
|
||||||
|
class ArgumentFingerprint:
|
||||||
|
def __init__(self):
|
||||||
|
self.characters = 0
|
||||||
|
self.bytes = 0
|
||||||
|
self.deltas = 0
|
||||||
|
self.encoding_valid = True
|
||||||
|
self.digest = hashlib.sha256()
|
||||||
|
self.prefixes: list[dict] = []
|
||||||
|
|
||||||
|
def observe(self, arguments: str) -> None:
|
||||||
|
self.characters += len(arguments)
|
||||||
|
if not arguments:
|
||||||
|
return
|
||||||
|
self.deltas += 1
|
||||||
|
if not self.encoding_valid:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
encoded = arguments.encode("utf-8")
|
||||||
|
except UnicodeEncodeError:
|
||||||
|
# Diagnostics must not repair, reject or otherwise alter the stream.
|
||||||
|
self.encoding_valid = False
|
||||||
|
return
|
||||||
|
offset = 0
|
||||||
|
while offset < len(encoded) and len(self.prefixes) < MAX_PREFIXES:
|
||||||
|
size = min(PREFIX_BYTES - self.bytes % PREFIX_BYTES, len(encoded) - offset)
|
||||||
|
self.digest.update(encoded[offset : offset + size])
|
||||||
|
self.bytes += size
|
||||||
|
offset += size
|
||||||
|
if self.bytes % PREFIX_BYTES == 0:
|
||||||
|
self.prefixes.append({"bytes": self.bytes, "sha256": self.digest.hexdigest()})
|
||||||
|
if offset < len(encoded):
|
||||||
|
self.digest.update(encoded[offset:])
|
||||||
|
self.bytes += len(encoded) - offset
|
||||||
|
|
||||||
|
def snapshot(self) -> dict:
|
||||||
|
return {
|
||||||
|
"argument_chars": self.characters,
|
||||||
|
"argument_bytes": self.bytes if self.encoding_valid else None,
|
||||||
|
"argument_sha256": self.digest.hexdigest() if self.encoding_valid else None,
|
||||||
|
"argument_deltas": self.deltas,
|
||||||
|
"argument_prefixes": [dict(prefix) for prefix in self.prefixes],
|
||||||
|
"argument_encoding_valid": self.encoding_valid,
|
||||||
|
}
|
||||||
+35
-10
@@ -13,6 +13,7 @@ from uuid import uuid4
|
|||||||
from PIL import Image
|
from PIL import Image
|
||||||
|
|
||||||
from artemis.data_engine.context_vars import CURRENT_NODE_NAME, CURRENT_TRACE_ID
|
from artemis.data_engine.context_vars import CURRENT_NODE_NAME, CURRENT_TRACE_ID
|
||||||
|
from artemis.llm.argument_fingerprint import ArgumentFingerprint, MAX_TRACKED_TOOLS
|
||||||
from artemis.llm.final_review import ACTIVE_FINAL_REVIEW, FinalReviewBudgetExceeded
|
from artemis.llm.final_review import ACTIVE_FINAL_REVIEW, FinalReviewBudgetExceeded
|
||||||
from artemis.llm.reliability import LLMExhaustedError
|
from artemis.llm.reliability import LLMExhaustedError
|
||||||
from artemis.llm.usage import usage_from_message
|
from artemis.llm.usage import usage_from_message
|
||||||
@@ -107,6 +108,7 @@ class RequestAttempt:
|
|||||||
self.clock = time.monotonic()
|
self.clock = time.monotonic()
|
||||||
self.last_publish = self.clock
|
self.last_publish = self.clock
|
||||||
self.tools: dict[int | str, dict] = {}
|
self.tools: dict[int | str, dict] = {}
|
||||||
|
self.tool_fingerprints: dict[int | str, ArgumentFingerprint] = {}
|
||||||
self.review = ACTIVE_FINAL_REVIEW.get()
|
self.review = ACTIVE_FINAL_REVIEW.get()
|
||||||
if self.review:
|
if self.review:
|
||||||
self.review.request_started(request_id, attempt)
|
self.review.request_started(request_id, attempt)
|
||||||
@@ -126,6 +128,9 @@ class RequestAttempt:
|
|||||||
"usage": None,
|
"usage": None,
|
||||||
"output_chunks": 0,
|
"output_chunks": 0,
|
||||||
"tool_argument_chars": 0,
|
"tool_argument_chars": 0,
|
||||||
|
"tool_observation_layer": "langchain_tool_call_chunks",
|
||||||
|
"argument_fingerprint_version": 1,
|
||||||
|
"tool_stats_truncated": False,
|
||||||
"tools": [],
|
"tools": [],
|
||||||
"step_id": str(self.step_id) if self.step_id else None,
|
"step_id": str(self.step_id) if self.step_id else None,
|
||||||
"parent_trace_id": str(self.parent_id) if self.parent_id else None,
|
"parent_trace_id": str(self.parent_id) if self.parent_id else None,
|
||||||
@@ -167,9 +172,10 @@ class RequestAttempt:
|
|||||||
self.data.update(
|
self.data.update(
|
||||||
headers_at=time.time(),
|
headers_at=time.time(),
|
||||||
http_status=response.status_code,
|
http_status=response.status_code,
|
||||||
server_request_id=_identifier(
|
server_request_id=_identifier(response.headers.get("x-oneapi-request-id"))
|
||||||
response.headers.get("x-request-id") or response.headers.get("request-id")
|
or _identifier(response.headers.get("x-request-id"))
|
||||||
),
|
or _identifier(response.headers.get("request-id")),
|
||||||
|
response_x_request_id=_identifier(response.headers.get("x-request-id")),
|
||||||
server_trace_id=_identifier(response.headers.get("x-trace-id")),
|
server_trace_id=_identifier(response.headers.get("x-trace-id")),
|
||||||
phase="waiting_output",
|
phase="waiting_output",
|
||||||
)
|
)
|
||||||
@@ -196,18 +202,31 @@ class RequestAttempt:
|
|||||||
self.data["usage"] = usage
|
self.data["usage"] = usage
|
||||||
tool_chunks = getattr(chunk, "tool_call_chunks", None) or []
|
tool_chunks = getattr(chunk, "tool_call_chunks", None) or []
|
||||||
# Responses exposes the same argument delta in content and tool_call_chunks.
|
# Responses exposes the same argument delta in content and tool_call_chunks.
|
||||||
# Count only the latter; retain lengths/names, never argument bodies.
|
# Count only the latter; retain bounded fingerprints, never argument bodies.
|
||||||
for tool in tool_chunks:
|
for tool in tool_chunks:
|
||||||
|
arguments = tool.get("args")
|
||||||
|
if isinstance(arguments, str):
|
||||||
|
self.data["tool_argument_chars"] += len(arguments)
|
||||||
index = tool.get("index")
|
index = tool.get("index")
|
||||||
key = index if index is not None else tool.get("id") or "unknown"
|
call_id = _identifier(tool.get("id"))
|
||||||
stats = self.tools.setdefault(key, {"name": None, "argument_chars": 0})
|
key = index if index is not None else call_id or "unknown"
|
||||||
|
if key not in self.tools and len(self.tools) >= MAX_TRACKED_TOOLS:
|
||||||
|
self.data["tool_stats_truncated"] = True
|
||||||
|
continue
|
||||||
|
stats = self.tools.setdefault(key, {"name": None})
|
||||||
|
if call_id:
|
||||||
|
stats["call_id"] = call_id
|
||||||
|
if index is not None:
|
||||||
|
stats["chunk_index"] = index
|
||||||
name = tool.get("name")
|
name = tool.get("name")
|
||||||
if isinstance(name, str) and re.fullmatch(r"[\w.:-]{1,128}", name):
|
if isinstance(name, str) and re.fullmatch(r"[\w.:-]{1,128}", name):
|
||||||
stats["name"] = name
|
stats["name"] = name
|
||||||
arguments = tool.get("args")
|
fingerprint = self.tool_fingerprints.get(key)
|
||||||
|
if fingerprint is None:
|
||||||
|
fingerprint = self.tool_fingerprints[key] = ArgumentFingerprint()
|
||||||
if isinstance(arguments, str):
|
if isinstance(arguments, str):
|
||||||
stats["argument_chars"] += len(arguments)
|
fingerprint.observe(arguments)
|
||||||
self.data["tool_argument_chars"] += len(arguments)
|
stats.update(fingerprint.snapshot())
|
||||||
self.data["tools"] = list(self.tools.values())
|
self.data["tools"] = list(self.tools.values())
|
||||||
content = getattr(chunk, "content", None)
|
content = getattr(chunk, "content", None)
|
||||||
substantive = isinstance(content, str) and bool(content)
|
substantive = isinstance(content, str) and bool(content)
|
||||||
@@ -252,7 +271,9 @@ class RequestAttempt:
|
|||||||
if isinstance(status, int):
|
if isinstance(status, int):
|
||||||
self.data["http_status"] = status
|
self.data["http_status"] = status
|
||||||
if request_id := _identifier(getattr(error, "request_id", None)):
|
if request_id := _identifier(getattr(error, "request_id", None)):
|
||||||
self.data["server_request_id"] = request_id
|
self.data["error_request_id"] = request_id
|
||||||
|
if not self.data.get("server_request_id"):
|
||||||
|
self.data["server_request_id"] = request_id
|
||||||
if category == "timeout":
|
if category == "timeout":
|
||||||
self.data.setdefault("timeout_scope", "transport")
|
self.data.setdefault("timeout_scope", "transport")
|
||||||
if response is not None:
|
if response is not None:
|
||||||
@@ -301,6 +322,7 @@ async def observe_request_headers(request):
|
|||||||
if attempt is None:
|
if attempt is None:
|
||||||
return
|
return
|
||||||
request.headers.setdefault("X-Client-Request-Id", attempt.data["request_id"])
|
request.headers.setdefault("X-Client-Request-Id", attempt.data["request_id"])
|
||||||
|
request.headers.setdefault("X-Client-Attempt-Id", attempt.data["attempt_id"])
|
||||||
routing = {}
|
routing = {}
|
||||||
for name in ("thread-id", "thread_id", "session-id", "session_id"):
|
for name in ("thread-id", "thread_id", "session-id", "session_id"):
|
||||||
value = request.headers.get(name)
|
value = request.headers.get(name)
|
||||||
@@ -309,9 +331,12 @@ async def observe_request_headers(request):
|
|||||||
attempt.data.update(
|
attempt.data.update(
|
||||||
request_sent_at=time.time(),
|
request_sent_at=time.time(),
|
||||||
client_request_id=_identifier(request.headers.get("X-Client-Request-Id")),
|
client_request_id=_identifier(request.headers.get("X-Client-Request-Id")),
|
||||||
|
client_attempt_id=_identifier(request.headers.get("X-Client-Attempt-Id")),
|
||||||
routing_header_fingerprints=routing,
|
routing_header_fingerprints=routing,
|
||||||
client_request_id_matches=request.headers.get("X-Client-Request-Id")
|
client_request_id_matches=request.headers.get("X-Client-Request-Id")
|
||||||
== attempt.data["request_id"],
|
== attempt.data["request_id"],
|
||||||
|
client_attempt_id_matches=request.headers.get("X-Client-Attempt-Id")
|
||||||
|
== attempt.data["attempt_id"],
|
||||||
)
|
)
|
||||||
attempt.publish()
|
attempt.publish()
|
||||||
|
|
||||||
|
|||||||
@@ -153,7 +153,7 @@ class LLMPermanentError(LLMCallError):
|
|||||||
|
|
||||||
|
|
||||||
class LLMExhaustedError(LLMCallError):
|
class LLMExhaustedError(LLMCallError):
|
||||||
"""Retryable failures exhausted every attempt and the pause deadline."""
|
"""Retryable failures exhausted attempts and any applicable pause deadline."""
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ class ToolArgumentOverflow(ValueError):
|
|||||||
"limit_chars": SMALL_ACTION_ARGUMENT_LIMIT,
|
"limit_chars": SMALL_ACTION_ARGUMENT_LIMIT,
|
||||||
}
|
}
|
||||||
super().__init__(
|
super().__init__(
|
||||||
f"模型生成的 {name} 参数异常膨胀({characters} 字符,"
|
f"收到的 {name} 参数流超出安全限制({characters} 字符,"
|
||||||
f"保护上限 {SMALL_ACTION_ARGUMENT_LIMIT});已关闭输出流,未执行本次动作。"
|
f"保护上限 {SMALL_ACTION_ARGUMENT_LIMIT});已关闭输出流,未执行本次动作。"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
+13
-5
@@ -381,7 +381,7 @@ async def _run_with_recovery[T](
|
|||||||
immediately: retrying cannot help and pausing would hang the task.
|
immediately: retrying cannot help and pausing would hang the task.
|
||||||
- When retryable attempts are exhausted: if a fallback model is waiting
|
- When retryable attempts are exhausted: if a fallback model is waiting
|
||||||
(with_fallback), raise LLMExhaustedError so it takes over immediately;
|
(with_fallback), raise LLMExhaustedError so it takes over immediately;
|
||||||
final acceptance also raises instead of pausing an already-finished task.
|
final acceptance and invalid output also raise instead of globally pausing.
|
||||||
Other calls pause (bounded by settings.LLM_PAUSE_TIMEOUT_SECONDS) and
|
Other calls pause (bounded by settings.LLM_PAUSE_TIMEOUT_SECONDS) and
|
||||||
retry the same request on resume.
|
retry the same request on resume.
|
||||||
"""
|
"""
|
||||||
@@ -470,7 +470,7 @@ async def _run_with_recovery[T](
|
|||||||
)
|
)
|
||||||
if failure.category is FailureCategory.INVALID_OUTPUT:
|
if failure.category is FailureCategory.INVALID_OUTPUT:
|
||||||
# One fresh response can recover a generation loop. Repeated
|
# One fresh response can recover a generation loop. Repeated
|
||||||
# runaway output must reach the existing fallback/pause path,
|
# runaway output must reach fallback or fail this request,
|
||||||
# even when ordinary network retries are configured higher.
|
# even when ordinary network retries are configured higher.
|
||||||
max_attempts = min(
|
max_attempts = min(
|
||||||
max_attempts, total_attempts + max(0, policy.max_attempts - attempt)
|
max_attempts, total_attempts + max(0, policy.max_attempts - attempt)
|
||||||
@@ -505,7 +505,11 @@ async def _run_with_recovery[T](
|
|||||||
ACTIVE_ATTEMPT.reset(observation_token)
|
ACTIVE_ATTEMPT.reset(observation_token)
|
||||||
|
|
||||||
# Retryable attempts exhausted.
|
# Retryable attempts exhausted.
|
||||||
if _FALLBACK_AVAILABLE.get() or ACTIVE_FINAL_REVIEW.get() is not None:
|
if (
|
||||||
|
_FALLBACK_AVAILABLE.get()
|
||||||
|
or ACTIVE_FINAL_REVIEW.get() is not None
|
||||||
|
or last_failure.category is FailureCategory.INVALID_OUTPUT
|
||||||
|
):
|
||||||
if failed_endpoints is not None and endpoint_key:
|
if failed_endpoints is not None and endpoint_key:
|
||||||
failed_endpoints[endpoint_key] = last_error
|
failed_endpoints[endpoint_key] = last_error
|
||||||
_record_llm_event(
|
_record_llm_event(
|
||||||
@@ -514,7 +518,11 @@ async def _run_with_recovery[T](
|
|||||||
"error": str(last_error)[:1000],
|
"error": str(last_error)[:1000],
|
||||||
"category": last_failure.category.value,
|
"category": last_failure.category.value,
|
||||||
"retryable": True,
|
"retryable": True,
|
||||||
"handover": "fallback" if _FALLBACK_AVAILABLE.get() else "final_review_end",
|
"handover": "fallback"
|
||||||
|
if _FALLBACK_AVAILABLE.get()
|
||||||
|
else "final_review_end"
|
||||||
|
if ACTIVE_FINAL_REVIEW.get() is not None
|
||||||
|
else "task_failed",
|
||||||
},
|
},
|
||||||
status="failed",
|
status="failed",
|
||||||
)
|
)
|
||||||
@@ -1253,7 +1261,7 @@ async def _with_fallback[T](
|
|||||||
)
|
)
|
||||||
|
|
||||||
# The contextvar must be reset BEFORE fallback_call runs: the fallback has
|
# The contextvar must be reset BEFORE fallback_call runs: the fallback has
|
||||||
# no further fallback behind it, so its own exhaustion should pause.
|
# no further fallback behind it, so apply its category's terminal policy.
|
||||||
fallback_token = _FALLBACK_AVAILABLE.set(True)
|
fallback_token = _FALLBACK_AVAILABLE.set(True)
|
||||||
try:
|
try:
|
||||||
result = await main_call()
|
result = await main_call()
|
||||||
|
|||||||
@@ -0,0 +1,39 @@
|
|||||||
|
# 模型流式参数诊断
|
||||||
|
|
||||||
|
小型内置动作的参数流仍受 16384 字符保护。超限时关闭输出流、丢弃不完整参数,不执行该次动作;一次新响应重试和可用备用端点耗尽后,只结束当前任务,不进入共享的模型暂停。批次仍遵循“失败即停止”、人工暂停和取消设置。网络故障、认证错误和最终验收保留各自的恢复规则。
|
||||||
|
|
||||||
|
## 请求关联
|
||||||
|
|
||||||
|
每个模型请求尝试沿用现有 `llm_diagnostic` 记录,不另建日志数据库。
|
||||||
|
|
||||||
|
| 字段 / 请求头 | 含义 |
|
||||||
|
| --- | --- |
|
||||||
|
| `request_id` / `X-Client-Request-Id` | 同一次恢复循环内保持稳定的逻辑请求编号;切换备用调用或恢复暂停后可能建立新的逻辑请求 |
|
||||||
|
| `attempt_id` / `X-Client-Attempt-Id` | 本次客户端尝试编号;重试时变化,`previous_attempt_id` 关联前次尝试 |
|
||||||
|
| `client_*_id_matches` | 实际发送的请求头是否与本地编号一致;调用方已指定的头保留原值 |
|
||||||
|
| `server_request_id` | 首个有效的 `X-Oneapi-Request-Id`、`X-Request-Id` 或 `Request-Id`,按此前后顺序选择 |
|
||||||
|
| `response_x_request_id` | 响应中的原始 `X-Request-Id`,不推定它来自模型上游 |
|
||||||
|
| `error_request_id` | SDK 异常带回的编号,不覆盖已经从响应头读取的编号 |
|
||||||
|
| `response_id` | SDK 实际返回的 Responses 响应编号 |
|
||||||
|
|
||||||
|
编号只接收有长度限制的可打印白名单字符。请求头不控制会话亲和性,不替换 `thread-id` / `session-id`。网关自身的重试应使用独立编号或序号,不能假装是一次新的客户端尝试。
|
||||||
|
|
||||||
|
## 参数指纹 v1
|
||||||
|
|
||||||
|
`tool_observation_layer=langchain_tool_call_chunks` 明确表示客户端观测的是 SDK 转换后的参数片段,不是原始 SSE 帧。每工具记录实际可得的 `call_id`、`chunk_index`、工具名,以及:
|
||||||
|
|
||||||
|
- `argument_chars`:参数片段的 Unicode 字符数之和。
|
||||||
|
- `argument_bytes` / `argument_sha256`:按原顺序连接片段后,其 UTF-8 字节长度和 SHA256;不解析、重排参数 JSON,也不规范化空白或 Unicode。
|
||||||
|
- `argument_deltas`:非空参数片段数,不能直接与网关原始事件数比较。
|
||||||
|
- `argument_prefixes`:每 4096 字节的累计 SHA256,最多保留前 16 个。可以在分片大小不同、上游预读或中途取消时比较相同字节位置。
|
||||||
|
- `argument_encoding_valid`:遇到不能编码为 UTF-8 的片段时为 false,当前总字节数和总哈希留空,避免伪造可比结果;诊断本身不改变执行流程。
|
||||||
|
|
||||||
|
最多记录 64 个工具;超过时设置 `tool_stats_truncated=true`,总参数字符计数仍继续。诊断不保留参数正文。哈希仅证明已收到的字节前缀,不能证明输出完整或动作成功;完整性仍由响应终止状态和解析校验决定。
|
||||||
|
|
||||||
|
跨三端优先使用 `response_id + call_id` 关联。LangChain 的 `chunk_index` 可能因文本或推理子块不同于原始 `output_index`,缺失编号时不能按数组位置猜测。正常的 `arguments.done`、`output_item.done`、`response.completed` 快照不重复计入参数;自定义工具转换、非空初始参数或协议异常需要先核对 SDK 映射,不能仅凭跨层哈希不同认定某一端重复发送。
|
||||||
|
|
||||||
|
历史日志没有保存的原始帧无法由新增诊断恢复。HTTP 200 只说明响应头已到达;客户端保护后出现的 499 或 Broken pipe 也不能证明最初原因。网关写入本机下游 writer 成功不等于客户端已消费或执行。
|
||||||
|
|
||||||
|
## 离线验证
|
||||||
|
|
||||||
|
`tests/unit/test_llm_stream_guard.py` 验证保护、有限重试、相同/不同备用端点,以及不会创建或清除其他请求的全局暂停。`tests/unit/test_request_diagnostics.py` 验证请求编号、超时重试、响应头及单次参数计数。`tests/unit/test_argument_fingerprint.py` 验证不同分片、中文和 emoji、固定字节前缀、有界统计与异常编码。这些测试不调用真实模型或设备。
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
# ARTEMIS 2.4.2
|
||||||
|
|
||||||
|
本版为批次任务增加人工评审标记,并调整任务列表布局。人工评审插件保持 **1.1.0**,第三方依赖版本不变。
|
||||||
|
|
||||||
|
## 更新
|
||||||
|
|
||||||
|
- 安装并启用人工评审插件后,任务存在已保存的有效评审内容时,批次列表显示“人工已审”。评审结论或非空备注均计入;清空全部内容后隐藏,禁用或卸载插件后也不显示。
|
||||||
|
- 标记仅说明已有人工评审内容,不代表测试通过,不覆盖 AI 判定或改变任务执行流程。没有安装插件的用户不显示该标记。
|
||||||
|
- 批次任务标题与状态分行显示:收藏星标位于标题框内,执行状态、AI 判定和人工已审标签在下方等高、左对齐排列。此布局适用于所有用户。
|
||||||
|
- 按批次一次读取当前评审内容,沿用列表自动刷新,不逐条增加网络请求;重跑任务不继承旧任务的已审标记。
|
||||||
|
- 补充发版约束:应用与第三方依赖独立版本化,检查锁文件差异,并验证最终源码包的全新安装。
|
||||||
|
|
||||||
|
## 验证
|
||||||
|
|
||||||
|
- 默认 Python 回归:3269 通过,2 跳过,8 按标记排除;Ruff、质量基线和核心类型检查通过。
|
||||||
|
- 前端 ChromeHeadless:348 项通过;生产构建通过。
|
||||||
|
- 内置浏览器使用隔离数据检查收藏星标、两行布局及有评审/无评审标签显示。未写入真实任务评审数据。
|
||||||
|
- 最终源码包另行执行独立目录、空 npm 缓存的官方源和国内镜像安装及生产构建,结果随发布附件提供。
|
||||||
|
|
||||||
|
## 升级
|
||||||
|
|
||||||
|
已有安装请等待任务结束,运行 `update-cn.bat` 或 `update.bat` 后重启。新安装下载 `artemis-2.4.2-source.zip`,按 `SHA256SUMS.txt` 校验后启动。请勿直接覆盖旧安装目录。
|
||||||
|
|
||||||
|
人工评审插件仍需单独安装并启用;本次无需升级已安装的 1.1.0。发布过程不构建 Docker,不重启用户服务,不执行设备或真实业务任务。
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
# ARTEMIS 2.4.3
|
||||||
|
|
||||||
|
本版修复异常模型输出影响整个批次的问题,并增加流式参数诊断。人工评审插件保持 **1.1.0**,SDK、其他插件及第三方依赖版本不变。
|
||||||
|
|
||||||
|
## 更新
|
||||||
|
|
||||||
|
- 已识别的异常输出(包括工具参数溢出、响应完整性校验失败)在有限重试和可用备用端点耗尽后,只结束当前任务,不再创建全局模型暂停。批次未开启“失败即停止”时可继续下一条;人工暂停、失败即停止、网络恢复及最终验收的原有规则保留。
|
||||||
|
- 保留 16384 字符工具参数保护,关闭异常输出流,丢弃未完成内容,不执行本次异常动作。报错改为描述“收到的参数流”,避免在证据不足时直接归责模型。
|
||||||
|
- 增加单次尝试请求头,区分逻辑请求、客户端尝试、网关响应与 SDK 异常中的请求编号,便于跨服务核对。
|
||||||
|
- 对收到的工具参数片段记录字符数、UTF-8 字节数及有界 SHA256 前缀摘要,默认不新增参数正文记录。最多跟踪 64 个工具,每工具最多保留 16 个固定位置前缀。SDK 的 chunk_index 与原始协议 output_index 分开命名。
|
||||||
|
|
||||||
|
诊断字段及边界见 [流式参数诊断](../llm-stream-diagnostics.md)。本次主程序安装不会部署或升级 NEWAPI、CodexManager;服务端修复独立交付。
|
||||||
|
|
||||||
|
## 验证
|
||||||
|
|
||||||
|
- 默认 Python 回归:3284 通过,6 跳过,8 按标记排除;Ruff 格式和静态检查、质量基线、核心类型检查通过。
|
||||||
|
- 五类合成输入经过实际 CodexManager、NEWAPI 转发函数及原 OpenAI/LangChain SDK 的离线文件串接:正常输入有效;超长、累计全文、重复及半关闭输入被拒绝;各层参数摘要一致,设备动作执行数为 0。
|
||||||
|
- 最终源码 ZIP 的隔离解压、独立空 npm 缓存、官方源与国内镜像安装和生产构建结果随发布附件 `INSTALL-VERIFICATION.md` 提供。
|
||||||
|
- 未重跑用户原业务批次、未做高并发压力测试或真实模型端到端验证。历史原始参数帧未保存,不能据此认定原事故参数膨胀首先发生在哪一层。
|
||||||
|
|
||||||
|
## 升级
|
||||||
|
|
||||||
|
已有安装请等待任务结束,运行 `update-cn.bat` 或 `update.bat` 后重启。新安装下载 `artemis-2.4.3-source.zip`,按 `SHA256SUMS.txt` 校验后启动。请勿直接覆盖旧安装目录。
|
||||||
|
|
||||||
|
人工评审插件 1.1.0 无需更新。本次发布不构建 Docker,不重启用户服务,不执行设备或真实业务任务。
|
||||||
+1
-1
@@ -14,7 +14,7 @@
|
|||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "artemis"
|
name = "artemis"
|
||||||
version = "2.4.1"
|
version = "2.4.3"
|
||||||
description = "Autonomous AI Assistant for Mobile Automation"
|
description = "Autonomous AI Assistant for Mobile Automation"
|
||||||
|
|
||||||
authors = [
|
authors = [
|
||||||
|
|||||||
@@ -57,15 +57,16 @@ def manifest_payload():
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def install(plugins, original_case=False):
|
def install(plugins, original_case=False, plugin_id="review"):
|
||||||
package = BytesIO()
|
package = BytesIO()
|
||||||
manifest = manifest_payload()
|
manifest = manifest_payload()
|
||||||
|
manifest["id"] = plugin_id
|
||||||
if original_case:
|
if original_case:
|
||||||
manifest["workspace"]["panels"][0]["kind"] = "original_case"
|
manifest["workspace"]["panels"][0]["kind"] = "original_case"
|
||||||
with zipfile.ZipFile(package, "w") as archive:
|
with zipfile.ZipFile(package, "w") as archive:
|
||||||
archive.writestr("plugin.json", json.dumps(manifest))
|
archive.writestr("plugin.json", json.dumps(manifest))
|
||||||
install_archive(plugins, package.getvalue())
|
install_archive(plugins, package.getvalue())
|
||||||
plugins.set_enabled("review", True)
|
plugins.set_enabled(plugin_id, True)
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
@@ -108,6 +109,71 @@ def save(client, values, revision=0, request_id=None):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_batch_review_badge_tracks_current_values_and_plugin_lifecycle(context, monkeypatch):
|
||||||
|
from apps.admin_console.routers import automation
|
||||||
|
|
||||||
|
store, plugins, client = context
|
||||||
|
monkeypatch.setattr(automation, "store", store)
|
||||||
|
result = {"status": "inconclusive", "source": "execution"}
|
||||||
|
monkeypatch.setattr(automation, "read_reviewed_result", lambda *args, **kwargs: result)
|
||||||
|
client.app.include_router(automation.router)
|
||||||
|
store.create_batch({"id": "review-batch", "status": "completed"}, [], {})
|
||||||
|
store.save_job(
|
||||||
|
{"session_id": "batch-task", "batch_id": "review-batch", "status": "completed", "case": {}}
|
||||||
|
)
|
||||||
|
store.save_job(
|
||||||
|
{"session_id": "new-run", "batch_id": "review-batch", "status": "completed", "case": {}}
|
||||||
|
)
|
||||||
|
|
||||||
|
def badge():
|
||||||
|
response = client.get("/api/automation/batches/review-batch")
|
||||||
|
assert response.status_code == 200
|
||||||
|
jobs = {job["session_id"]: job for job in response.json()["jobs"]}
|
||||||
|
assert "batch-task" in jobs, response.json()
|
||||||
|
assert jobs["batch-task"]["test_result"] == result
|
||||||
|
assert jobs["new-run"]["human_reviewed"] is False
|
||||||
|
return jobs["batch-task"]["human_reviewed"]
|
||||||
|
|
||||||
|
assert badge() is False
|
||||||
|
install(plugins)
|
||||||
|
assert (
|
||||||
|
client.put(
|
||||||
|
"/api/task-extensions/batch-task/review",
|
||||||
|
json={
|
||||||
|
"values": {"accuracy": "Pass"},
|
||||||
|
"expected_revision": 0,
|
||||||
|
"request_id": str(uuid4()),
|
||||||
|
},
|
||||||
|
).status_code
|
||||||
|
== 200
|
||||||
|
)
|
||||||
|
assert badge() is False # Other workspace plugins are not the human review plugin.
|
||||||
|
install(plugins, plugin_id="task-review")
|
||||||
|
assert badge() is False
|
||||||
|
for revision, (values, expected) in enumerate(
|
||||||
|
[
|
||||||
|
({}, False),
|
||||||
|
({"accuracy_note": " \n"}, False),
|
||||||
|
({"accuracy": "Pass"}, True),
|
||||||
|
({"accuracy": "Failed"}, True),
|
||||||
|
({"accuracy": "", "accuracy_note": ""}, False),
|
||||||
|
({"accuracy_note": "人工核对了截图"}, True),
|
||||||
|
]
|
||||||
|
):
|
||||||
|
response = client.put(
|
||||||
|
"/api/task-extensions/batch-task/task-review",
|
||||||
|
json={"values": values, "expected_revision": revision, "request_id": str(uuid4())},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert badge() is expected
|
||||||
|
plugins.set_enabled("task-review", False)
|
||||||
|
assert badge() is False
|
||||||
|
plugins.set_enabled("task-review", True)
|
||||||
|
assert badge() is True
|
||||||
|
plugins.uninstall("task-review")
|
||||||
|
assert badge() is False
|
||||||
|
|
||||||
|
|
||||||
def test_original_case_import_requires_enabled_plugin_and_keeps_execution_prompt(context):
|
def test_original_case_import_requires_enabled_plugin_and_keeps_execution_prompt(context):
|
||||||
from apps.admin_console.services import task_import
|
from apps.admin_console.services import task_import
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,59 @@
|
|||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from langchain_core.messages import AIMessageChunk
|
||||||
|
|
||||||
|
from artemis.llm.argument_fingerprint import ArgumentFingerprint, MAX_PREFIXES, PREFIX_BYTES
|
||||||
|
from artemis.llm.diagnostics import RequestAttempt
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("fragment_size", [1, 37, 4096, 100000])
|
||||||
|
def test_fingerprints_match_at_byte_offsets_despite_unicode_and_different_fragmentation(
|
||||||
|
fragment_size,
|
||||||
|
):
|
||||||
|
arguments = '{"target":"' + "私人内容🙂" * 8000 + '"}'
|
||||||
|
expected = arguments.encode("utf-8")
|
||||||
|
fingerprint = ArgumentFingerprint()
|
||||||
|
for offset in range(0, len(arguments), fragment_size):
|
||||||
|
fingerprint.observe(arguments[offset : offset + fragment_size])
|
||||||
|
result = fingerprint.snapshot()
|
||||||
|
assert result["argument_chars"] == len(arguments)
|
||||||
|
assert result["argument_bytes"] == len(expected)
|
||||||
|
assert result["argument_sha256"] == hashlib.sha256(expected).hexdigest()
|
||||||
|
assert result["argument_encoding_valid"] is True
|
||||||
|
assert result["argument_prefixes"] == [
|
||||||
|
{"bytes": size, "sha256": hashlib.sha256(expected[:size]).hexdigest()}
|
||||||
|
for size in range(PREFIX_BYTES, PREFIX_BYTES * MAX_PREFIXES + 1, PREFIX_BYTES)
|
||||||
|
]
|
||||||
|
assert "私人" not in json.dumps(result, ensure_ascii=False)
|
||||||
|
|
||||||
|
|
||||||
|
def test_invalid_utf8_is_marked_without_failing_or_forging_a_digest():
|
||||||
|
fingerprint = ArgumentFingerprint()
|
||||||
|
for value in ["safe", "\ud800", "later"]:
|
||||||
|
fingerprint.observe(value)
|
||||||
|
assert fingerprint.snapshot() == {
|
||||||
|
"argument_chars": 10,
|
||||||
|
"argument_bytes": None,
|
||||||
|
"argument_sha256": None,
|
||||||
|
"argument_deltas": 3,
|
||||||
|
"argument_prefixes": [],
|
||||||
|
"argument_encoding_valid": False,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_tool_tracking_is_bounded_but_total_count_keeps_all_received_arguments():
|
||||||
|
attempt = RequestAttempt(
|
||||||
|
None, request_id="bounded", attempt=1, timeout=300, max_retries=0, metadata={}
|
||||||
|
)
|
||||||
|
for index in range(65):
|
||||||
|
attempt.chunk(
|
||||||
|
AIMessageChunk(
|
||||||
|
content=[],
|
||||||
|
tool_call_chunks=[{"name": "click", "args": "{}", "index": index}],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert len(attempt.data["tools"]) == len(attempt.tool_fingerprints) == 64
|
||||||
|
assert attempt.data["tool_stats_truncated"] is True
|
||||||
|
assert attempt.data["tool_argument_chars"] == 130
|
||||||
@@ -2,6 +2,7 @@
|
|||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import json
|
import json
|
||||||
|
import hashlib
|
||||||
from types import SimpleNamespace
|
from types import SimpleNamespace
|
||||||
from unittest.mock import AsyncMock
|
from unittest.mock import AsyncMock
|
||||||
|
|
||||||
@@ -15,6 +16,7 @@ import pytest_asyncio
|
|||||||
from artemis.llm.reliability import FailureCategory, LLMExhaustedError
|
from artemis.llm.reliability import FailureCategory, LLMExhaustedError
|
||||||
from artemis.llm.response_guard import ResponseIntegrityError, ensure_response_complete
|
from artemis.llm.response_guard import ResponseIntegrityError, ensure_response_complete
|
||||||
from artemis.llm.router import ModelEndpoint, ModelFactory
|
from artemis.llm.router import ModelEndpoint, ModelFactory
|
||||||
|
from artemis.llm.diagnostics import RequestAttempt
|
||||||
from artemis.services import llm as llm_service
|
from artemis.services import llm as llm_service
|
||||||
from artemis.services.llm import RobustChatModelWrapper, acomplete
|
from artemis.services.llm import RobustChatModelWrapper, acomplete
|
||||||
|
|
||||||
@@ -195,6 +197,33 @@ async def test_completed_stream_returns_original_valid_arguments(transport_case)
|
|||||||
assert result.tool_calls[0]["args"] == {"passed": True}
|
assert result.tool_calls[0]["args"] == {"passed": True}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
@pytest.mark.parametrize("transport_case", ["fragmented"], indirect=True)
|
||||||
|
async def test_real_sdk_fingerprint_counts_received_arguments_once(transport_case):
|
||||||
|
case = transport_case
|
||||||
|
arguments = json.dumps({"passed": True, "evidence": "中文🙂" * 1000}, ensure_ascii=False)
|
||||||
|
case.outputs.append(("completed", arguments))
|
||||||
|
observation = RequestAttempt(
|
||||||
|
None, request_id="sdk-test", attempt=1, timeout=300, max_retries=0, metadata={}
|
||||||
|
)
|
||||||
|
merged = None
|
||||||
|
async for chunk in case.model.astream([]):
|
||||||
|
observation.chunk(chunk)
|
||||||
|
merged = chunk if merged is None else merged + chunk
|
||||||
|
observation.finish("completed", response=merged)
|
||||||
|
assert len(observation.data["tools"]) == 1
|
||||||
|
tool = observation.data["tools"][0]
|
||||||
|
assert tool["call_id"] == "call_test"
|
||||||
|
assert tool["argument_chars"] == len(arguments)
|
||||||
|
assert tool["argument_bytes"] == len(arguments.encode())
|
||||||
|
assert tool["argument_sha256"] == hashlib.sha256(arguments.encode()).hexdigest()
|
||||||
|
assert tool["argument_prefixes"][0] == {
|
||||||
|
"bytes": 4096,
|
||||||
|
"sha256": hashlib.sha256(arguments.encode()[:4096]).hexdigest(),
|
||||||
|
}
|
||||||
|
assert tool["argument_deltas"] == 1
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
@pytest.mark.parametrize("transport_case", ["fragmented"], indirect=True)
|
@pytest.mark.parametrize("transport_case", ["fragmented"], indirect=True)
|
||||||
@pytest.mark.parametrize("status", ["completed", None])
|
@pytest.mark.parametrize("status", ["completed", None])
|
||||||
|
|||||||
@@ -124,7 +124,9 @@ async def test_gateway_closes_bad_stream_discards_partial_and_bounds_retry(
|
|||||||
with pytest.raises(LLMExhaustedError) as caught:
|
with pytest.raises(LLMExhaustedError) as caught:
|
||||||
await wrapper.complete([])
|
await wrapper.complete([])
|
||||||
assert caught.value.failure.category is FailureCategory.INVALID_OUTPUT
|
assert caught.value.failure.category is FailureCategory.INVALID_OUTPUT
|
||||||
llm._wait_for_resume.assert_awaited_once()
|
llm._wait_for_resume.assert_not_awaited()
|
||||||
|
assert not llm.PAUSE_FILE.exists()
|
||||||
|
assert any(name == "llm_gave_up" and p["handover"] == "task_failed" for name, p in resets)
|
||||||
assert model.calls == 2
|
assert model.calls == 2
|
||||||
assert closed == [1, 2]
|
assert closed == [1, 2]
|
||||||
failure = next(
|
failure = next(
|
||||||
@@ -139,3 +141,47 @@ async def test_gateway_closes_bad_stream_discards_partial_and_bounds_retry(
|
|||||||
assert failure["usage"] is None
|
assert failure["usage"] is None
|
||||||
assert "private" not in json.dumps(records + resets, default=str)
|
assert "private" not in json.dumps(records + resets, default=str)
|
||||||
assert any(name == "llm_stream_reset" and p["action"] == "discard" for name, p in resets)
|
assert any(name == "llm_stream_reset" and p["action"] == "discard" for name, p in resets)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
@pytest.mark.parametrize("fallback_kind", ["same", "failing", "working"])
|
||||||
|
@pytest.mark.parametrize("already_paused", [False, True])
|
||||||
|
async def test_invalid_output_fallback_does_not_pause_or_clear_another_request(
|
||||||
|
fallback_kind, already_paused
|
||||||
|
):
|
||||||
|
calls = []
|
||||||
|
if already_paused:
|
||||||
|
llm.PAUSE_FILE.write_text("another request or user pause", encoding="utf-8")
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
calls.append("main")
|
||||||
|
raise ToolArgumentOverflow("click", SMALL_ACTION_ARGUMENT_LIMIT + 1)
|
||||||
|
|
||||||
|
async def fallback():
|
||||||
|
calls.append("fallback")
|
||||||
|
if fallback_kind == "working":
|
||||||
|
return "valid response"
|
||||||
|
raise ToolArgumentOverflow("click", SMALL_ACTION_ARGUMENT_LIMIT + 2)
|
||||||
|
|
||||||
|
async def run():
|
||||||
|
return await llm.with_fallback(
|
||||||
|
lambda: llm._run_with_recovery(main, endpoint_key="main", max_retries=5),
|
||||||
|
lambda: llm._run_with_recovery(
|
||||||
|
fallback,
|
||||||
|
endpoint_key="main" if fallback_kind == "same" else "fallback",
|
||||||
|
max_retries=5,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
if fallback_kind == "working":
|
||||||
|
assert await run() == "valid response"
|
||||||
|
else:
|
||||||
|
with pytest.raises(LLMExhaustedError) as caught:
|
||||||
|
await run()
|
||||||
|
assert caught.value.failure.category is FailureCategory.INVALID_OUTPUT
|
||||||
|
assert calls.count("main") == 2
|
||||||
|
assert calls.count("fallback") == {"same": 0, "failing": 2, "working": 1}[fallback_kind]
|
||||||
|
llm._wait_for_resume.assert_not_awaited()
|
||||||
|
assert llm.PAUSE_FILE.exists() is already_paused
|
||||||
|
if already_paused:
|
||||||
|
assert llm.PAUSE_FILE.read_text(encoding="utf-8") == "another request or user pause"
|
||||||
@@ -1,6 +1,7 @@
|
|||||||
import asyncio
|
import asyncio
|
||||||
from copy import deepcopy
|
from copy import deepcopy
|
||||||
import json
|
import json
|
||||||
|
import hashlib
|
||||||
from types import SimpleNamespace
|
from types import SimpleNamespace
|
||||||
from uuid import uuid4
|
from uuid import uuid4
|
||||||
|
|
||||||
@@ -36,6 +37,46 @@ class Engine:
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("headers", "expected"),
|
||||||
|
[
|
||||||
|
({"X-Oneapi-Request-Id": "gateway-request"}, "gateway-request"),
|
||||||
|
(
|
||||||
|
{"X-Oneapi-Request-Id": "gateway-request", "X-Request-Id": "provider-request"},
|
||||||
|
"gateway-request",
|
||||||
|
),
|
||||||
|
({"X-Request-Id": "provider-request"}, "provider-request"),
|
||||||
|
({"Request-Id": "legacy-request"}, "legacy-request"),
|
||||||
|
(
|
||||||
|
{"X-Oneapi-Request-Id": "invalid header value", "X-Request-Id": "standard-request"},
|
||||||
|
"standard-request",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_gateway_request_id_preserves_the_standard_header_without_assuming_its_source(
|
||||||
|
headers, expected
|
||||||
|
):
|
||||||
|
attempt = RequestAttempt(
|
||||||
|
Engine(), request_id="local-request", attempt=1, timeout=300, max_retries=1, metadata={}
|
||||||
|
)
|
||||||
|
attempt.headers(httpx.Response(200, headers=headers))
|
||||||
|
assert attempt.data["server_request_id"] == expected
|
||||||
|
assert attempt.data["response_x_request_id"] == headers.get("X-Request-Id")
|
||||||
|
|
||||||
|
|
||||||
|
def test_sdk_error_cannot_overwrite_the_observed_gateway_request_id():
|
||||||
|
class RequestError(Exception):
|
||||||
|
request_id = "sdk-request"
|
||||||
|
|
||||||
|
attempt = RequestAttempt(
|
||||||
|
Engine(), request_id="local-request", attempt=1, timeout=300, max_retries=1, metadata={}
|
||||||
|
)
|
||||||
|
attempt.headers(httpx.Response(400, headers={"X-Oneapi-Request-Id": "gateway-request"}))
|
||||||
|
attempt.finish("failed", error=RequestError("failed"))
|
||||||
|
assert attempt.data["server_request_id"] == "gateway-request"
|
||||||
|
assert attempt.data["error_request_id"] == "sdk-request"
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_request_hook_correlates_api_calls_without_logging_header_values():
|
async def test_request_hook_correlates_api_calls_without_logging_header_values():
|
||||||
engine = Engine()
|
engine = Engine()
|
||||||
@@ -55,13 +96,18 @@ async def test_request_hook_correlates_api_calls_without_logging_header_values()
|
|||||||
)
|
)
|
||||||
await observe_request_headers(request)
|
await observe_request_headers(request)
|
||||||
assert request.headers["x-client-request-id"] == "local-request"
|
assert request.headers["x-client-request-id"] == "local-request"
|
||||||
|
assert request.headers["x-client-attempt-id"] == str(attempt.trace_id)
|
||||||
|
assert attempt.data["client_attempt_id_matches"] is True
|
||||||
assert attempt.data["client_request_id_matches"] is True
|
assert attempt.data["client_request_id_matches"] is True
|
||||||
assert len(attempt.data["routing_header_fingerprints"]) == 2
|
assert len(attempt.data["routing_header_fingerprints"]) == 2
|
||||||
assert "private" not in json.dumps(engine.records, default=str)
|
assert "private" not in json.dumps(engine.records, default=str)
|
||||||
request.headers["X-Client-Request-Id"] = "caller-id"
|
request.headers["X-Client-Request-Id"] = "caller-id"
|
||||||
|
request.headers["X-Client-Attempt-Id"] = "caller-attempt"
|
||||||
await observe_request_headers(request)
|
await observe_request_headers(request)
|
||||||
assert request.headers["x-client-request-id"] == "caller-id"
|
assert request.headers["x-client-request-id"] == "caller-id"
|
||||||
assert attempt.data["client_request_id_matches"] is False
|
assert attempt.data["client_request_id_matches"] is False
|
||||||
|
assert request.headers["X-Client-Attempt-Id"] == "caller-attempt"
|
||||||
|
assert attempt.data["client_attempt_id_matches"] is False
|
||||||
finally:
|
finally:
|
||||||
ACTIVE_ATTEMPT.reset(token)
|
ACTIVE_ATTEMPT.reset(token)
|
||||||
|
|
||||||
@@ -75,6 +121,7 @@ async def test_headers_without_body_timeout_then_retry_keeps_evidence(monkeypatc
|
|||||||
)
|
)
|
||||||
llm._ENDPOINT_BREAKER._states.clear()
|
llm._ENDPOINT_BREAKER._states.clear()
|
||||||
calls = 0
|
calls = 0
|
||||||
|
wire_ids = []
|
||||||
|
|
||||||
class StalledBody(httpx.AsyncByteStream):
|
class StalledBody(httpx.AsyncByteStream):
|
||||||
async def __aiter__(self):
|
async def __aiter__(self):
|
||||||
@@ -84,6 +131,9 @@ async def test_headers_without_body_timeout_then_retry_keeps_evidence(monkeypatc
|
|||||||
async def transport(request):
|
async def transport(request):
|
||||||
nonlocal calls
|
nonlocal calls
|
||||||
calls += 1
|
calls += 1
|
||||||
|
wire_ids.append(
|
||||||
|
(request.headers["X-Client-Request-Id"], request.headers["X-Client-Attempt-Id"])
|
||||||
|
)
|
||||||
headers = {"x-request-id": f"server-{calls}", "set-cookie": "secret-cookie"}
|
headers = {"x-request-id": f"server-{calls}", "set-cookie": "secret-cookie"}
|
||||||
return httpx.Response(
|
return httpx.Response(
|
||||||
200, headers=headers, stream=StalledBody() if calls == 1 else httpx.ByteStream(b"ok")
|
200, headers=headers, stream=StalledBody() if calls == 1 else httpx.ByteStream(b"ok")
|
||||||
@@ -91,7 +141,7 @@ async def test_headers_without_body_timeout_then_retry_keeps_evidence(monkeypatc
|
|||||||
|
|
||||||
async with httpx.AsyncClient(
|
async with httpx.AsyncClient(
|
||||||
transport=httpx.MockTransport(transport),
|
transport=httpx.MockTransport(transport),
|
||||||
event_hooks={"response": [observe_response_headers]},
|
event_hooks={"request": [observe_request_headers], "response": [observe_response_headers]},
|
||||||
) as client:
|
) as client:
|
||||||
|
|
||||||
class Model:
|
class Model:
|
||||||
@@ -123,6 +173,10 @@ async def test_headers_without_body_timeout_then_retry_keeps_evidence(monkeypatc
|
|||||||
assert final["request_id"] == failed["request_id"]
|
assert final["request_id"] == failed["request_id"]
|
||||||
assert final["attempt_id"] != failed["attempt_id"] and final["attempt"] == 2
|
assert final["attempt_id"] != failed["attempt_id"] and final["attempt"] == 2
|
||||||
assert final["previous_attempt_id"] == failed["attempt_id"]
|
assert final["previous_attempt_id"] == failed["attempt_id"]
|
||||||
|
assert wire_ids == [
|
||||||
|
(failed["request_id"], failed["attempt_id"]),
|
||||||
|
(final["request_id"], final["attempt_id"]),
|
||||||
|
]
|
||||||
retry = next(r["payload"] for r in engine.records if r["name"] == "llm_retry")
|
retry = next(r["payload"] for r in engine.records if r["name"] == "llm_retry")
|
||||||
assert retry["request_id"] == failed["request_id"]
|
assert retry["request_id"] == failed["request_id"]
|
||||||
assert retry["attempt_id"] == failed["attempt_id"]
|
assert retry["attempt_id"] == failed["attempt_id"]
|
||||||
@@ -163,10 +217,26 @@ def test_tool_deltas_are_counted_once_without_retaining_arguments():
|
|||||||
attempt.finish("completed", response=combined)
|
attempt.finish("completed", response=combined)
|
||||||
assert attempt.data["output_chunks"] == 4
|
assert attempt.data["output_chunks"] == 4
|
||||||
assert attempt.data["tool_argument_chars"] == len('{"secret":"private"}{"seconds":1}')
|
assert attempt.data["tool_argument_chars"] == len('{"secret":"private"}{"seconds":1}')
|
||||||
assert attempt.data["tools"] == [
|
for tool, name, arguments, call_id, index, count in zip(
|
||||||
{"name": "click", "argument_chars": len('{"secret":"private"}')},
|
attempt.data["tools"],
|
||||||
{"name": "wait", "argument_chars": len('{"seconds":1}')},
|
["click", "wait"],
|
||||||
]
|
['{"secret":"private"}', '{"seconds":1}'],
|
||||||
|
["c1", "c2"],
|
||||||
|
[0, 1],
|
||||||
|
[2, 1],
|
||||||
|
strict=True,
|
||||||
|
):
|
||||||
|
assert tool == {
|
||||||
|
"name": name,
|
||||||
|
"call_id": call_id,
|
||||||
|
"chunk_index": index,
|
||||||
|
"argument_chars": len(arguments),
|
||||||
|
"argument_bytes": len(arguments.encode()),
|
||||||
|
"argument_sha256": hashlib.sha256(arguments.encode()).hexdigest(),
|
||||||
|
"argument_deltas": count,
|
||||||
|
"argument_prefixes": [],
|
||||||
|
"argument_encoding_valid": True,
|
||||||
|
}
|
||||||
assert attempt.data["response_status"] == "completed"
|
assert attempt.data["response_status"] == "completed"
|
||||||
assert "private" not in json.dumps(engine.records, default=str)
|
assert "private" not in json.dumps(engine.records, default=str)
|
||||||
assert "secret" not in json.dumps(engine.records, default=str)
|
assert "secret" not in json.dumps(engine.records, default=str)
|
||||||
@@ -205,7 +275,10 @@ async def test_timeout_during_tool_generation_keeps_last_output_and_tool_name(mo
|
|||||||
)
|
)
|
||||||
assert failed["failure_phase"] == "generating_tool_arguments"
|
assert failed["failure_phase"] == "generating_tool_arguments"
|
||||||
assert failed["last_output_at"] >= failed["first_output_at"]
|
assert failed["last_output_at"] >= failed["first_output_at"]
|
||||||
assert failed["tools"] == [{"name": "click", "argument_chars": 5}]
|
assert len(failed["tools"]) == 1
|
||||||
|
tool = failed["tools"][0]
|
||||||
|
assert tool["name"] == "click" and tool["argument_chars"] == tool["argument_bytes"] == 5
|
||||||
|
assert tool["argument_sha256"] == hashlib.sha256(b'{"x":').hexdigest()
|
||||||
assert failed["usage"] is None
|
assert failed["usage"] is None
|
||||||
assert failed["max_output_tokens"] == 12000
|
assert failed["max_output_tokens"] == 12000
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user