fix: keep Cua foreground actions on the target window

Correct native scroll scaling, reject stale or out-of-bounds pixel targets, and scope DPI awareness to geometry reads. Package adapter 0.36.0+artemis.3 and document the high-effort cross-device verification.

Validation: Go tests and vet, packaged MCP schema and handshake, Python checks, and user-run high-effort task completed with a passed verdict and explicit coverage limits.

README.md

@@ -1,6 +1,6 @@
 # Artemis Plugins
 
-ARTEMIS 2.0.0 的插件市场。当前电脑控制插件为 **Cua 0.36.0+artemis.1**，支持 Windows x64，通过窗口截图与可访问性树操作电脑应用和浏览器窗口，复用 ARTEMIS 的调度、资源互斥、步骤、截图附件和录像。
+ARTEMIS 2.0.0 的插件市场。当前电脑控制插件为 **Cua 0.36.0+artemis.4**，支持 Windows x64，通过窗口截图与可访问性树操作电脑应用和浏览器窗口，复用 ARTEMIS 的调度、资源互斥、步骤、截图附件和录像。
 
 在 ARTEMIS 的「插件」页面安装「电脑控制 · Cua」，授权并启用即可。安卓任务可根据提示词按需调用电脑控制，不需要单独配置 Codex MCP 或安装 Go、Node、Python。
 
@@ -16,6 +16,8 @@ Open Computer Use（OCU）已从市场索引下架，当前市场只提供 Cua
 
 已测试原生窗口表单、34 像素视觉目标、滚动、截图保存、ARTEMIS 窗口录像和子进程退出。对照测试独立验收 3/3 完成；ARTEMIS AI 裁定为 2 次通过、1 次未判定。浏览器复杂控制台、多 DPI、多显示器与安卓混合长程任务仍需进一步验证，详见 [验证记录](plugins/cua-driver/VERIFICATION.md)。
 
+适配修订另覆盖应用名发现、浏览器键盘与中文输入、截图缩放与坐标校验，以及前台激活后保持目标可见。Cua 需要前台输入时会占用用户桌面，不提供与用户操作互不干扰的执行环境；复杂业务是否通过仍以任务实际证据为准。
+
 ## 构建与验证
 
 构建机需要 Python 和 Go；最终用户只安装 ZIP。从固定上游发行下载 `cua-driver-rs-0.36.0-windows-x86_64-binary.zip`，构建脚本校验 SHA256 后打包，不执行全局安装器。
@@ -31,6 +33,6 @@ python tests/cua_discovery_smoke.py <安装后的artemis-cua-adapter.exe> --app
 
 `catalog.json` 是市场索引，包含清单、HTTPS 安装包地址、SHA256 和验证说明。`artifacts/` 保存本机安装包与测试资料，不进入 Git。版本文件不可原地覆盖，任务固定使用其启动时的版本。
 
-插件版本跟随所用 Cua Driver 上游版本：当前为 `0.36.0`，对应上游标签 `cua-driver-rs-v0.36.0`，当前适配修订为 `0.36.0+artemis.1`，本仓库发行标签为 `cua-driver-v0.36.0+artemis.1`。只修改 ARTEMIS 适配而未升级上游时使用 `+artemis.N` 后缀区分不可变安装包；包内 `provenance.json` 记录上游提交、原包校验值和适配修订，构建会检查版本对应关系。
+插件版本跟随所用 Cua Driver 上游版本：当前为 `0.36.0`，对应上游标签 `cua-driver-rs-v0.36.0`，当前适配修订为 `0.36.0+artemis.4`，本仓库发行标签为 `cua-driver-v0.36.0+artemis.4`。只修改 ARTEMIS 适配而未升级上游时使用 `+artemis.N` 后缀区分不可变安装包；包内 `provenance.json` 记录上游提交、原包校验值和适配修订，构建会检查版本对应关系。
 
 [发行说明](plugins/cua-driver/RELEASE.md) · [ARTEMIS 主项目](https://git.playones.com/chenmingxuan/Artemis-Eapil)

plugins/cua-driver/RELEASE.md

@@ -1,12 +1,16 @@
-# Cua 电脑控制 0.36.0+artemis.2
+# Cua 电脑控制 0.36.0+artemis.4
 
-作为 ARTEMIS 插件市场的默认电脑控制实现，替代已下架的 Open Computer Use。此修订修复模型同时填写控件令牌与坐标，导致浏览器按键连续被拒绝的问题；上游仍是固定的 Cua Driver 0.36.0 预发布版，市场标注开发预览。
+此修订修复前台操作完成后立即恢复其他应用、后续浏览器观察退回后台的问题。上游仍固定 Cua Driver 0.36.0 预发布版；仅增加 ARTEMIS 适配修订，没有接入独立环境、DOM 控制或新的后台通道。
 
-修复内容：`press_key` / `type_text` 只对目标窗口的当前焦点操作，不再暴露互斥的 element_token/x/y 参数；聚焦使用独立点击工具，控件赋值由必填 element_token 的 `set_value` 完成。`scroll` 仅按截图坐标定位滚动区域。旧格式在适配层返回明确未执行回执，不发送输入、不重新截图。补充浏览器前台输入与滚动后点击规则，不自动重试结果不确定的写操作。保留上一修订的应用名发现和无效 PID 修复。
+明确使用 foreground 的操作先调用上游自带 bring_to_front，核实目标激活后再发送一次输入。目标保持前台供后续渲染、观察和录像，不逐次切回原应用。激活失败或结果未知时不发送业务输入；写操作结果未知时不重发。独立 bring_to_front 工具可恢复被遮挡的窗口，不必刷新页面或重复业务动作。只读观察不抢焦点，capture_focus 记录采集前后的前台 HWND 和目标状态，供识别灰屏与焦点切换。PNG 保存成功仍不代表网页验收通过。
+
+保留前一修订的滚动缩放、capture_id、范围与窗口几何校验，以及键盘参数修复。原生后台操作继续遵循 Cua 支持范围；foreground 会占用用户前台，不承诺用户与模型可同时操作同一桌面。
+
+上一修订内容：`press_key` / `type_text` 只对目标窗口的当前焦点操作，不再暴露互斥的 element_token/x/y 参数；聚焦使用独立点击工具，控件赋值由必填 element_token 的 `set_value` 完成。`scroll` 仅按截图坐标定位滚动区域。旧格式在适配层返回明确未执行回执，不发送输入、不重新截图。补充浏览器前台输入与滚动后点击规则，不自动重试结果不确定的写操作。保留上一修订的应用名发现和无效 PID 修复。
 
 上游提交：`6d6c734c127025fb39ad5d585a10382808b39dc1`。Windows x64 原始 ZIP SHA256：`f8c47024b626f4e0e70f8428d2998df39e4879110c5722a29d5fd03e367353ad`。采用 MIT Driver，许可证随包提供；不包含 Cua Spaces、云环境或模型服务。
 
-提供 11 个窗口观察、控件/像素点击、输入、按键、拖动、滚动及截图工具。操作后返回真实窗口观察，保留动作回执，截图校验 PID/HWND 与物理边界，生成文件保存到当前任务 trace。共享 ARTEMIS 的 `desktop:local` 资源、任务固定版本、步骤和录像，不另建调度器。
+提供 12 个窗口激活、观察、控件/像素点击、输入、按键、拖动、滚动及截图工具。操作后返回真实窗口观察，保留动作回执，截图校验 PID/HWND 与物理边界，生成文件保存到当前任务 trace。共享 ARTEMIS 的 `desktop:local` 资源、任务固定版本、步骤和录像，不另建调度器。
 
 适配器与依赖随包分发，无需额外安装 Go、Python、Node 或另一份 Codex MCP。关闭、取消或宿主退出后通过 Windows Job Object 回收 Driver/UIA 子进程；仅在该子进程环境关闭遥测。
 

plugins/cua-driver/SKILL.md

@@ -6,8 +6,14 @@
 2. `get_window_state` 同时读取截图和 UIA 控件树。最新 `snapshot_id` 加 `:N` 即控件 `element_token`。优先用 `click` 点击最新控件；视觉目标用 `click_at`，坐标仅取本次截图上的窗口局部像素，不能用屏幕坐标，需携带本次 `capture_id`。两种点击参数互斥，不混用。
 3. 所有窗口操作明确传 `pid`、`window_id`。`set_value` 必须带最新 `element_token`，用于设置具有 set_value 动作的输入框；同名 Text 标签不能当成 Edit 输入框。`press_key` 和 `type_text` 只操作当前焦点，不能传 `element_token`、`x`、`y`（也不要补 0/null）。需要聚焦控件时先 `click` / `click_at`，再输入。`press_key` 传单键及 `modifiers` 数组。
 4. 每个动作后插件自动返回一次新截图和控件树，新 snapshot 替换旧 token。接着用新 token，无需重复获取相同状态。仅当画面确实还在变化或需要核验时再次观察。一次调用只执行一次动作，超时不自动重发。
-5. 默认后台操作；Brave/Chrome/Edge 等 Chromium 浏览器的点击（尤其滚动后）、快捷键、文字输入和滚动直接使用 `delivery_mode=foreground`。例如新标签页：`press_key(key="t",modifiers=["ctrl"],delivery_mode="foreground")`；地址导航：Ctrl+L → `type_text(text=网址,delivery_mode="foreground")` → Enter，各调用均带同一目标 pid/window_id。检查新页面再继续。`effect=unverifiable/suspected_noop` 仅代表投递/效果未确认，必须检查返回画面和读回值；`refused` 是未执行。其他应用后台无效时，可在任务授权范围内显式 foreground 后再次检查。不要仅凭工具成功声称测试通过。
+5. 默认后台操作；明确指定 foreground 时，适配器先使用 Cua 自带 bring_to_front 激活并核实目标，随后保持目标在前台，供页面渲染和连续观察；不会每次动作后切回原应用，也不能承诺不影响用户前台工作。Brave/Chrome/Edge 等 Chromium 浏览器的点击（尤其滚动后）、快捷键、文字输入和滚动直接使用 `delivery_mode=foreground`。例如新标签页：`press_key(key="t",modifiers=["ctrl"],delivery_mode="foreground")`；地址导航：Ctrl+L → `type_text(text=网址,delivery_mode="foreground")` → Enter，各调用均带同一目标 pid/window_id。检查新页面再继续。只读观察/保存截图不会抢焦点；若正文灰屏或控件树缺少网页正文，检查 capture_focus，调用同一 pid/window_id 的 bring_to_front，再观察页面；不要靠重复刷新或业务点击恢复。用户切换窗口后需要重新确认前台，不能拿灰屏判网站失败。`effect=unverifiable/suspected_noop` 仅代表投递/效果未确认，必须检查返回画面和读回值；`refused` 是未执行。其他应用后台无效时，可在任务授权范围内显式 foreground 后再次检查。不要仅凭工具成功声称测试通过。
 6. `scroll` 必须带最新截图中滚动区域内的 `x,y`，不能传 `element_token`。使用 up/down/left/right、amount、by=line/page；滚轮刻度不保证固定像素或整页距离，根据返回画面确认实际位移再继续。`get_window_state(since="latest")` 可在同一窗口读取差异，但新 snapshot_id 仍替换旧 token。
 7. `save_screenshot` 保存最新窗口 PNG 到当前任务 trace，filename 仅是文件名，不能传路径，不覆盖已有文件。实际成功返回 SHA256 和附件；不得把工具内存截图当成已保存文件。
 
 不提供 Cua 自带调度、录屏、回放、配置修改、安装器、任意脚本或自动升级接口。浏览器本轮使用窗口视觉/UIA 通路；未接入 DOM 或已有登录浏览器调试权限。此包基于上游 0.36.0 预发布版，通过已列出的本机验证，其他场景的支持范围以 VERIFICATION.md 为准。
+
+## 截图坐标与证据
+
+- `click_at`、`scroll`、`drag` 都携带最新返回截图的 `capture_id`，坐标取自同一张图的实际宽高。动作返回的新截图可能改变分辨率；旧编号、旧坐标不可沿用。窗口移动、缩放或坐标越界被拒绝后重新观察，不反复重试原参数。
+- 缩放转换由适配层处理，不自行乘 DPI 或加窗口偏移。优先用最新 `element_token` 点击可访问控件；UIA 的屏幕 `frame` 不能作为截图坐标。
+- 操作及保存截图前，从已有观察核对应用、窗口标题和页面身份（浏览器结合地址栏、选中标签及页面内容）。身份不符时定位正确页面再操作。无位移的滚动不能证明到底；先检查遮挡、焦点和滚动容器，再决定补一次边界验证或换路径。

plugins/cua-driver/VERIFICATION.md

@@ -60,3 +60,21 @@ Cua 第三轮虽然独立验收通过，ARTEMIS 最终核验却没有生成结
 `tests/cua_browser_smoke.py` 新增独立 Brave profile 和本地表单，全部操作经过打包后的 Cua MCP，未使用 CDP/DOM 控制。实测通过 Ctrl+T 新建标签、Ctrl+L 地址栏导航、UIA set_value、聚焦后的中文 type_text、表单恰好提交一次并读回、实际滚动位移、底部按钮及 PNG SHA256。测试窗口退出，不操作用户原有标签页。
 
 实测还发现 Chromium 后台点击可能沿用滚动前的控件位置；前台路径会解析可见位置，已验证滚动后点击成功，schema/Skill 明确要求浏览器使用 foreground。滚轮 amount/by 不是精确滚动像素或页数，测试按页面滚动事件决定是否继续。不能据小样本宣称所有浏览器、DPI、跨端业务都通过；原业务完整任务未重跑，移动端问题不在本修订范围。
+
+## 截图坐标修订 0.36.0+artemis.3
+
+对照实际任务中 2558×1538 原图、1280×770 后续截图，以及超界 (1700, 1000) 滚动仍被派发的回执，核对固定上游提交的 Windows ScrollTool：其前台路径直接 bitmap_to_screen，而点击/拖拽使用 capture 映射或 snapshot 缩放。修复仅对滚动补映射，不重复缩放点击和拖拽。
+
+Go 确定性回归覆盖上述尺寸、原始尺寸无二次缩放、不同裁边、负窗口原点、非整数点、缺失/过期编号、非数值/越界、拖拽两端、窗口移动/缩放及原始截图读取失败。Go test、vet 与构建通过。本修订没有重跑付费模型业务任务，也未声称真实多显示器、多 DPI 或跨端成功率已验证；待使用新插件进行业务复测。
+
+补充用户业务复测：2026-10-17 使用 `gpt-6-luna / high` 与 `0.36.0+artemis.3` 完成手机 Chrome、电脑 Brave 的混合页面探索，33 步、约 410 秒，执行状态为 completed，最终核验为 passed。记录确认移动端 Desktop site 未勾选、两端页底观察及追加滚动；报告明确保留 PC 中段局部未逐屏覆盖和文字证据冲突。此通过仅针对本次探索与报告要求，不表示页面所有区域无缺陷，也不外推多显示器、多 DPI 或整体成功率。
+
+## 前台取证修订 0.36.0+artemis.4
+
+2026-10-17 排查发现，前台点击后能短暂捕获网页正文，随后同窗口仅剩浏览器外壳和灰色正文，窗口录像也出现相同表现。固定上游在 foreground 输入后异步恢复原前台，ARTEMIS 的动作后截图则是独立调用；浏览器后台渲染和截图可能受到该时序影响。原业务没有重跑，保留 blocked，不能归因为网站缺陷。
+
+适配修复使用上游既有 bring_to_front，在显式 foreground 输入前完成持久激活并核实准确 HWND。拒绝、仅提升层级、激活后丢失前台及传输异常都不发送业务输入；不确定输入不重发。只读观察不改变前台，记录 capture_focus，模型可显式恢复同一目标后重新观察。
+
+Go 回归与 vet 通过。`tests/cua_browser_smoke.py --foreground-check` 在 ARTEMIS 的 desktop:local 资源锁内运行，使用临时 Brave profile、本地表单及另一个普通测试窗口代表原前台应用，不操作用户原有标签。38 次工具调用验证：从被遮挡窗口执行前台输入后，等待 5 秒（超过上游 3 秒恢复轮询）仍保持目标前台；人为切走后只读观察不抢焦点，bring_to_front 恢复并保存真实截图；导航后等待仍能读取正文；新标签、中文输入、表单恰好提交一次、实际滚动、页底点击和 PNG 校验均通过。
+
+本次未复测原业务全流程、Codex 窗口本身或多屏 DPI；以普通前台窗口验证同一前台恢复机制，不将局部回归称为原任务通过。此修订随 ARTEMIS 2.0.0 配套发布，市场继续保留预览标识；仅更新主程序不会自动更新已安装插件。

plugins/cua-driver/coordinates.go

@@ -0,0 +1,105 @@
+package main
+
+import (
+	"errors"
+	"fmt"
+	"math"
+)
+
+// A frame belongs to one adapter session and one concrete window. Encoded
+// pixels, DWM frame dimensions and Windows logical pixels are distinct spaces.
+type pointerFrame struct {
+	id            string
+	width, height int
+	window        obj
+}
+
+func targetKey(args obj) string {
+	return fmt.Sprintf("%d:%d", num(args["pid"]), num(args["window_id"]))
+}
+func prependFrameHint(r obj, f pointerFrame) {
+	// Keep coordinate metadata before long trees so bounded model context cannot
+	// silently drop the only capture ID while retaining the screenshot.
+	r["content"] = append([]any{textBlock(fmt.Sprintf("Screenshot capture_id=%s, size=%dx%d; pixel x=0..%d, y=0..%d. Use this new ID for click_at/scroll/drag.", f.id, f.width, f.height, f.width-1, f.height-1))}, items(r["content"])...)
+}
+func isPointerTool(name string) bool { return name == "click_at" || name == "drag" || name == "scroll" }
+func copyArgs(args obj) obj {
+	out := obj{}
+	for k, v := range args {
+		out[k] = v
+	}
+	return out
+}
+func capturedFrame(r, src obj) (pointerFrame, bool) {
+	w, h := imageSize(r)
+	id := str(object(r["structuredContent"])["capture_id"])
+	if src == nil || id == "" || w <= 0 || h <= 0 {
+		return pointerFrame{}, false
+	}
+	return pointerFrame{id, w, h, copyArgs(object(src["window"]))}, true
+}
+func coordinate(v any) (float64, bool) {
+	var n float64
+	switch x := v.(type) {
+	case float64:
+		n = x
+	case int:
+		n = float64(x)
+	case int64:
+		n = float64(x)
+	default:
+		return 0, false
+	}
+	return n, !math.IsNaN(n) && !math.IsInf(n, 0)
+}
+func (f pointerFrame) validate(name string, args obj) error {
+	if f.id == "" || str(args["capture_id"]) != f.id {
+		return errors.New("capture_id is missing or stale; use the latest screenshot for this window")
+	}
+	pairs := [][2]string{{"x", "y"}}
+	if name == "drag" {
+		pairs = [][2]string{{"from_x", "from_y"}, {"to_x", "to_y"}}
+	}
+	for _, pair := range pairs {
+		x, xok := coordinate(args[pair[0]])
+		y, yok := coordinate(args[pair[1]])
+		if !xok || !yok || x < 0 || y < 0 || x > float64(f.width-1) || y > float64(f.height-1) {
+			return fmt.Errorf("%s/%s must be inside screenshot %dx%d (0..%d, 0..%d); no screen or Android normalized coordinates", pair[0], pair[1], f.width, f.height, f.width-1, f.height-1)
+		}
+	}
+	return nil
+}
+func sameWindow(a, b obj) bool {
+	for _, k := range []string{"left", "top", "width", "height"} {
+		if num(a[k]) != num(b[k]) {
+			return false
+		}
+	}
+	return num(a["width"]) > 0 && num(a["height"]) > 0
+}
+
+// Only scroll needs pre-scaling in the pinned driver. click_at is capture-bound
+// upstream and drag uses the driver's snapshot ratio; never scale either twice.
+func nativeScrollArgs(f pointerFrame, args obj, readNative func() (pointerFrame, error)) (obj, error) {
+	if err := f.validate("scroll", args); err != nil {
+		return nil, err
+	}
+	native, err := readNative()
+	if err != nil {
+		return nil, err
+	}
+	if !sameWindow(f.window, native.window) || native.width < f.width || native.height < f.height {
+		return nil, errors.New("capture geometry changed; get_window_state again before scrolling")
+	}
+	// Reject a different crop/aspect ratio. One-pixel resize rounding is normal.
+	if math.Abs(float64(f.height)*float64(native.width)/float64(f.width)-float64(native.height)) > float64(native.width)/float64(f.width)+1 {
+		return nil, errors.New("capture aspect ratio changed; get_window_state again before scrolling")
+	}
+	x, _ := coordinate(args["x"])
+	y, _ := coordinate(args["y"])
+	out := copyArgs(args)
+	out["x"] = x * float64(native.width) / float64(f.width)
+	out["y"] = y * float64(native.height) / float64(f.height)
+	delete(out, "capture_id") // v0.36.0 scroll does not support capture binding.
+	return out, nil
+}

plugins/cua-driver/coordinates_test.go

@@ -0,0 +1,137 @@
+package main
+
+import (
+	"errors"
+	"math"
+	"testing"
+)
+
+func fixtureFrame(w, h int) pointerFrame {
+	return pointerFrame{"current", w, h, obj{"left": -2560, "top": 120, "width": 2560, "height": 1540}}
+}
+
+func TestScrollMapsActualNativeCaptureWithoutDPIOrWindowOffset(t *testing.T) {
+	// Reproduce the task's 2558x1538 -> 1280x770 image transition. Window
+	// position and OS logical-screen dimensions must never affect this ratio.
+	f := fixtureFrame(1280, 770)
+	args := obj{"pid": 1, "window_id": 2, "capture_id": "current", "x": 900, "y": 600, "direction": "down"}
+	got, err := nativeScrollArgs(f, args, func() (pointerFrame, error) { return fixtureFrame(2558, 1538), nil })
+	if err != nil {
+		t.Fatal(err)
+	}
+	if math.Abs(got["x"].(float64)-1798.59375) > 0.0001 || math.Abs(got["y"].(float64)-1198.441558) > 0.0001 {
+		t.Fatalf("incorrect mapping: %v", got)
+	}
+	if _, ok := got["capture_id"]; ok {
+		t.Fatal("upstream scroll does not support capture_id")
+	}
+	if args["x"] != 900 || args["capture_id"] != "current" {
+		t.Fatal("input evidence mutated")
+	}
+	if got["pid"] != 1 || got["window_id"] != 2 || got["direction"] != "down" {
+		t.Fatal("target changed")
+	}
+}
+
+func TestNativeScrollAndNonCroppedCapture(t *testing.T) {
+	for _, size := range [][2]int{{2558, 1538}, {2560, 1540}, {1280, 770}} {
+		f := fixtureFrame(size[0], size[1])
+		args := obj{"capture_id": "current", "x": 200.5, "y": 300.25}
+		got, err := nativeScrollArgs(f, args, func() (pointerFrame, error) { return f, nil })
+		if err != nil || got["x"] != 200.5 || got["y"] != 300.25 {
+			t.Fatalf("native coordinates changed: %v %v", got, err)
+		}
+	}
+	// WGC can have a different native extent than a one-pixel DWM crop.
+	f := fixtureFrame(1280, 770)
+	got, err := nativeScrollArgs(f, obj{"capture_id": "current", "x": 640, "y": 385}, func() (pointerFrame, error) { return fixtureFrame(2560, 1540), nil })
+	if err != nil || got["x"] != float64(1280) || got["y"] != float64(770) {
+		t.Fatalf("assumed DWM crop: %v %v", got, err)
+	}
+}
+
+func TestInvalidCoordinatesNeverReachNativeProbe(t *testing.T) {
+	f := fixtureFrame(1280, 770)
+	for _, bad := range []obj{
+		{"capture_id": "current", "x": 1700, "y": 1000}, // Actual failing task.
+		{"capture_id": "old", "x": 900, "y": 600},
+		{"x": 900, "y": 600},
+		{"capture_id": "current", "x": -1, "y": 0},
+		{"capture_id": "current", "x": 1280, "y": 0},
+		{"capture_id": "current", "x": 0, "y": 770},
+		{"capture_id": "current", "x": "900", "y": 600},
+		{"capture_id": "current", "x": math.NaN(), "y": 600},
+		{"capture_id": "current", "x": 900, "y": math.Inf(1)},
+	} {
+		_, err := nativeScrollArgs(f, bad, func() (pointerFrame, error) { t.Fatal("invalid coordinates reached probe"); return f, nil })
+		if err == nil {
+			t.Fatalf("accepted %v", bad)
+		}
+	}
+}
+
+func TestScrollRejectsMovedResizedOrDifferentCrop(t *testing.T) {
+	f := fixtureFrame(1280, 770)
+	for _, kind := range []string{"moved", "resized", "crop", "missing"} {
+		native := fixtureFrame(2558, 1538)
+		switch kind {
+		case "moved":
+			native.window["left"] = 0
+		case "resized":
+			native.window["width"] = 1920
+		case "crop":
+			native.height = 1000
+		case "missing":
+			native.width = 0
+		}
+		_, err := nativeScrollArgs(f, obj{"capture_id": "current", "x": 900, "y": 600}, func() (pointerFrame, error) { return native, nil })
+		if err == nil {
+			t.Fatal(kind)
+		}
+	}
+	_, err := nativeScrollArgs(f, obj{"capture_id": "current", "x": 900, "y": 600}, func() (pointerFrame, error) { return pointerFrame{}, errors.New("capture disconnected") })
+	if err == nil {
+		t.Fatal("missing capture was accepted")
+	}
+}
+
+func TestClickAndBothDragEndpointsUseLatestFrame(t *testing.T) {
+	f := fixtureFrame(1280, 770)
+	click := obj{"capture_id": "current", "x": 1279, "y": 769}
+	if err := f.validate("click_at", click); err != nil {
+		t.Fatal(err)
+	}
+	if click["x"] != 1279 || click["y"] != 769 {
+		t.Fatal("click scaled twice")
+	}
+	drag := obj{"capture_id": "current", "from_x": 0, "from_y": 0, "to_x": 1279, "to_y": 769}
+	if err := f.validate("drag", drag); err != nil {
+		t.Fatal(err)
+	}
+	for _, key := range []string{"from_x", "from_y", "to_x", "to_y"} {
+		bad := copyArgs(drag)
+		bad[key] = 3000
+		if f.validate("drag", bad) == nil {
+			t.Fatal("accepted out of bounds " + key)
+		}
+	}
+	drag["capture_id"] = "old"
+	if f.validate("drag", drag) == nil {
+		t.Fatal("accepted stale drag")
+	}
+}
+
+func TestAllPixelSchemasRequireCapture(t *testing.T) {
+	a := newAdapter(nil)
+	for _, name := range []string{"click_at", "scroll", "drag"} {
+		found := false
+		for _, key := range items(a.schemas[name]["required"]) {
+			if key == "capture_id" {
+				found = true
+			}
+		}
+		if !found {
+			t.Fatal(name + " lacks capture binding")
+		}
+	}
+}

plugins/cua-driver/foreground.go

@@ -0,0 +1,55 @@
+package main
+
+// Use Cua's existing persistent activation before explicit foreground input.
+// Cua then snapshots the target itself as its "previous foreground", so its
+// asynchronous restore cannot race ARTEMIS's separately requested observation.
+// Read-only calls and background delivery never activate or restore a window.
+func callWithForeground(call func(string, obj) (obj, error), name string, args obj, current func() int64) (obj, error) {
+	if name != "bring_to_front" && args["delivery_mode"] != "foreground" {
+		return call(name, args)
+	}
+	target := obj{"pid": args["pid"], "window_id": args["window_id"]}
+	focus, err := call("bring_to_front", target)
+	if err != nil {
+		// Only activation was attempted. Do not send (or replay) the business
+		// action when the activation result is unknown.
+		return failure("Foreground activation could not be confirmed; requested input was not sent: "+err.Error(), "not_started", "possible", "foreground_unavailable"), nil
+	}
+	activation := object(focus["structuredContent"])
+	actual := current()
+	activation["verified_foreground_window_id"] = actual
+	activation["policy"] = "keep_target_foreground"
+	if focus["isError"] == true || activation["landed_on_target"] != true || actual != num(args["window_id"]) {
+		r := failure("Target window did not retain foreground; requested input was not sent. Inspect the window before retrying.", "not_started", "possible", "foreground_unavailable")
+		object(r["structuredContent"])["foreground_activation"] = activation
+		return r, nil
+	}
+	if name == "bring_to_front" {
+		activation["artemis"] = receipt("completed", "confirmed", "")
+		focus["structuredContent"] = activation
+		return focus, nil
+	}
+	r, err := call(name, args)
+	if err == nil {
+		s := object(r["structuredContent"])
+		s["foreground_activation"] = activation
+		r["structuredContent"] = s
+	}
+	return r, err
+}
+
+func annotateCaptureFocus(r obj, target, before, after int64) {
+	s := object(r["structuredContent"])
+	s["capture_focus"] = obj{
+		"target_window_id":            target,
+		"foreground_window_id_before": before,
+		"foreground_window_id_after":  after,
+		"target_foreground_before":    before == target,
+		"target_foreground_after":     after == target,
+		"changed_during_capture":      before != after,
+	}
+	r["structuredContent"] = s
+	if before != target || after != target {
+		r["content"] = append(items(r["content"]), textBlock("Capture was not continuously foreground. If the page body is blank or missing from the tree, use bring_to_front for this same pid/window_id, then observe again; do not infer a website failure or repeat a business action."))
+	}
+}

plugins/cua-driver/foreground_test.go

@@ -0,0 +1,95 @@
+package main
+
+import (
+	"errors"
+	"reflect"
+	"testing"
+)
+
+func TestForegroundActivatesBeforeInputAndDoesNotRestore(t *testing.T) {
+	var calls []string
+	foreground := int64(10) // The user's foreground app, e.g. Codex.
+	call := func(name string, args obj) (obj, error) {
+		calls = append(calls, name)
+		if name == "bring_to_front" {
+			if len(args) != 2 || num(args["window_id"]) != 20 {
+				t.Fatal("activation changed the target or forwarded input arguments")
+			}
+			foreground = 20
+			return obj{"structuredContent": obj{"landed_on_target": true}}, nil
+		}
+		// Simulate upstream's activate/act/restore: it now saves the target,
+		// not the user's previous app, as its restore destination.
+		previous := foreground
+		foreground = 20
+		foreground = previous
+		return obj{"structuredContent": obj{"effect": "unverifiable"}}, nil
+	}
+	r, err := callWithForeground(call, "press_key", obj{"pid": 2, "window_id": 20, "key": "t", "delivery_mode": "foreground"}, func() int64 { return foreground })
+	if err != nil || r["isError"] == true || foreground != 20 || !reflect.DeepEqual(calls, []string{"bring_to_front", "press_key"}) {
+		t.Fatalf("foreground handoff failed: %v %v %v %v", calls, foreground, r, err)
+	}
+}
+
+func TestUnconfirmedActivationNeverSendsInput(t *testing.T) {
+	for _, mode := range []string{"refused", "raised_only", "lost_focus", "transport"} {
+		t.Run(mode, func(t *testing.T) {
+			calls := 0
+			r, err := callWithForeground(func(name string, _ obj) (obj, error) {
+				calls++
+				if name != "bring_to_front" {
+					t.Fatal("input sent without focus")
+				}
+				if mode == "transport" {
+					return nil, errors.New("disconnected")
+				}
+				return obj{"isError": mode == "refused", "structuredContent": obj{"raised": true, "landed_on_target": mode != "raised_only"}}, nil
+			}, "click", obj{"pid": 2, "window_id": 20, "delivery_mode": "foreground"}, func() int64 {
+				if mode == "lost_focus" {
+					return 10
+				}
+				return 20
+			})
+			rec := object(object(r["structuredContent"])["artemis"])
+			if err != nil || calls != 1 || r["isError"] != true || rec["execution"] != "not_started" || rec["retryable"] != false {
+				t.Fatalf("unsafe activation handling: %v %v", r, err)
+			}
+		})
+	}
+}
+
+func TestBackgroundAndObservationsNeverActivate(t *testing.T) {
+	for _, name := range []string{"get_window_state", "verify_state", "save_screenshot", "click"} {
+		var calls []string
+		_, err := callWithForeground(func(tool string, _ obj) (obj, error) {
+			calls = append(calls, tool)
+			return obj{}, nil
+		}, name, obj{"pid": 2, "window_id": 20}, func() int64 { t.Fatal("unexpected focus probe"); return 0 })
+		if err != nil || !reflect.DeepEqual(calls, []string{name}) {
+			t.Fatal(calls, err)
+		}
+	}
+}
+
+func TestUncertainInputIsNotReplayed(t *testing.T) {
+	calls := 0
+	_, err := callWithForeground(func(name string, _ obj) (obj, error) {
+		calls++
+		if name == "bring_to_front" {
+			return obj{"structuredContent": obj{"landed_on_target": true}}, nil
+		}
+		return nil, errors.New("input reply lost")
+	}, "type_text", obj{"pid": 2, "window_id": 20, "delivery_mode": "foreground"}, func() int64 { return 20 })
+	if calls != 2 || err == nil {
+		t.Fatal(calls, err)
+	}
+}
+
+func TestCaptureRecordsFocusWithoutHidingBackgroundEvidence(t *testing.T) {
+	r := obj{"content": []any{obj{"type": "image", "data": "original"}}}
+	annotateCaptureFocus(r, 20, 20, 10)
+	s := object(object(r["structuredContent"])["capture_focus"])
+	if s["changed_during_capture"] != true || s["target_foreground_after"] != false || len(items(r["content"])) != 2 || object(items(r["content"])[0])["data"] != "original" {
+		t.Fatal(r)
+	}
+}

plugins/cua-driver/main.go

@@ -25,7 +25,7 @@ import (
 //go:embed tools.json
 var toolsJSON []byte
 
-const adapterVersion = "0.36.0+artemis.2"
+const adapterVersion = "0.36.0+artemis.4"
 
 type obj = map[string]any
 type request struct {
@@ -175,10 +175,11 @@ type adapter struct {
 	peer     *peer
 	tools    []any
 	schemas  map[string]obj
+	captures map[string]pointerFrame
 }
 
 func newAdapter(p *peer) *adapter {
-	a := &adapter{peer: p, schemas: map[string]obj{}}
+	a := &adapter{peer: p, schemas: map[string]obj{}, captures: map[string]pointerFrame{}}
 	_ = json.Unmarshal(toolsJSON, &a.tools)
 	for _, v := range a.tools {
 		t := object(v)
@@ -287,12 +288,20 @@ func source(r obj, args obj, started float64) obj {
 	return obj{"kind": "computer", "target_id": fmt.Sprint(num(args["window_id"])), "label": s["window_title"], "captured_at": started, "width": w, "height": h, "window": rect, "screen": screen}
 }
 func (a *adapter) observe(args obj) (obj, obj, error) {
+	delete(a.captures, targetKey(args))
 	started := float64(time.Now().UnixMilli()) / 1000
+	before := foregroundWindow()
 	r, err := a.peer.tool("get_window_state", args)
 	if err != nil {
 		return nil, nil, err
 	}
-	return r, source(r, args, started), nil
+	annotateCaptureFocus(r, num(args["window_id"]), before, foregroundWindow())
+	src := source(r, args, started)
+	if frame, ok := capturedFrame(r, src); ok {
+		a.captures[targetKey(args)] = frame
+		prependFrameHint(r, frame)
+	}
+	return r, src, nil
 }
 func observationArgs(args obj) obj {
 	return obj{"pid": args["pid"], "window_id": args["window_id"], "max_image_dimension": 1280, "max_elements": 250, "tree_format": "markdown"}
@@ -325,11 +334,59 @@ func (a *adapter) handle(name string, args obj) obj {
 			r = windowListing(r, query)
 		}
 	} else {
+		if isPointerTool(name) {
+			frame, ok := a.captures[targetKey(args)]
+			if !ok {
+				return failure("No current screenshot; get_window_state before a pixel action", "not_started", "none", "invalid_coordinates")
+			}
+			if err := frame.validate(name, args); err != nil {
+				return failure(err.Error(), "not_started", "none", "invalid_coordinates")
+			}
+			if _, ok := verifyWindow(num(args["pid"]), num(args["window_id"]), frame.window); !ok {
+				delete(a.captures, targetKey(args))
+				return failure("Window moved or resized; get_window_state again", "not_started", "none", "stale_capture")
+			}
+			if name == "scroll" {
+				// v0.36.0 scroll omits screenshot scaling. Measure actual native
+				// capture pixels (not DWM/DPI estimates) without another model call.
+				// Do not expose this internal frame or reuse its tokens.
+				delete(a.captures, targetKey(args))
+				prepared, e := nativeScrollArgs(frame, args, func() (pointerFrame, error) {
+					nativeArgs := observationArgs(args)
+					nativeArgs["max_image_dimension"] = 0
+					nativeArgs["include_accessibility_tree"] = false
+					native, err := a.peer.tool("get_window_state", nativeArgs)
+					if err != nil {
+						return pointerFrame{}, err
+					}
+					src := source(native, args, float64(time.Now().UnixMilli())/1000)
+					f, ok := capturedFrame(native, src)
+					if !ok {
+						return pointerFrame{}, errors.New("native capture unavailable; get_window_state again")
+					}
+					return f, nil
+				})
+				if e != nil {
+					return failure(e.Error(), "not_started", "none", "invalid_coordinates")
+				}
+				args = prepared
+			} else if name == "drag" {
+				// Upstream drag already scales using its current screenshot.
+				args = copyArgs(args)
+				delete(args, "capture_id")
+			}
+		}
 		upstream := name
 		if name == "click_at" {
 			upstream = "click"
 		}
-		r, err = a.peer.tool(upstream, args)
+		if name == "verify_state" {
+			delete(a.captures, targetKey(args))
+		}
+		if writeTool(name) {
+			delete(a.captures, targetKey(args))
+		}
+		r, err = callWithForeground(a.peer.tool, upstream, args, foregroundWindow)
 	}
 	if err != nil {
 		a.peer.stop()
@@ -338,8 +395,11 @@ func (a *adapter) handle(name string, args obj) obj {
 		}
 		return failure(err.Error(), "not_started", "none", "transport_lost")
 	}
-	rec := actionReceipt(r, writeTool(name))
 	structured := object(r["structuredContent"])
+	rec := object(structured["artemis"])
+	if len(rec) == 0 {
+		rec = actionReceipt(r, writeTool(name))
+	}
 	if writeTool(name) && rec["execution"] != "not_started" {
 		// Observe once after delivery; NEVER repeat an uncertain write.
 		observation, origin, e := a.observe(observationArgs(args))
@@ -353,6 +413,10 @@ func (a *adapter) handle(name string, args obj) obj {
 	}
 	if name == "verify_state" { // Read-only verification may include a screenshot; it is not a task verdict.
 		src = source(r, args, float64(time.Now().UnixMilli())/1000)
+		if frame, ok := capturedFrame(r, src); ok {
+			a.captures[targetKey(args)] = frame
+			prependFrameHint(r, frame)
+		}
 	}
 	if src != nil {
 		rec["source"] = src

plugins/cua-driver/main_test.go

@@ -101,7 +101,7 @@ func TestInputContractsHaveOneTargetingMode(t *testing.T) {
 		"press_key": {"pid": 1, "window_id": 2, "key": "t", "modifiers": []any{"ctrl"}, "delivery_mode": "foreground"},
 		"type_text": {"pid": 1, "window_id": 2, "text": "中文"},
 		"set_value": {"pid": 1, "window_id": 2, "element_token": "s00000001:9", "value": "value"},
-		"scroll":    {"pid": 1, "window_id": 2, "direction": "down", "x": 100, "y": 100},
+		"scroll":    {"pid": 1, "window_id": 2, "direction": "down", "x": 100, "y": 100, "capture_id": "current"},
 	} {
 		if err := a.validate(name, args); err != nil {
 			t.Fatalf("valid %s: %v", name, err)

plugins/cua-driver/plugin.json

@@ -2,7 +2,7 @@
   "schema_version": 2,
   "id": "cua-driver",
   "name": "电脑控制 · Cua",
-  "version": "0.36.0+artemis.2",
+  "version": "0.36.0+artemis.4",
   "description": "通过 Cua Driver 的窗口截图与可访问性树操作 Windows 应用及浏览器窗口；复用 ARTEMIS 的任务调度、步骤证据与录像。",
   "artemis_version": ">=2.0.0,<3.0.0",
   "platforms": [
@@ -158,6 +158,18 @@
           "text",
           "image"
         ]
+      },
+      "bring_to_front": {
+        "effect": "write",
+        "observation": false,
+        "idempotent": true,
+        "target_argument": "window_id",
+        "cancellation": "unknown",
+        "evidence": [
+          "text",
+          "image",
+          "state"
+        ]
       }
     },
     "required_tools": [

plugins/cua-driver/tools.json

@@ -235,7 +235,7 @@
           "type": "integer"
         },
         "delivery_mode": {
-          "description": "background (default) avoids raising the window; unsupported input is refused. Use foreground for Chromium/Brave keyboard input and scrolling within task scope. Inspect the returned state before retrying an uncertain action.",
+          "description": "background does not activate the window. foreground uses Cua bring_to_front before input and leaves the target foreground for subsequent rendering and evidence; it does not switch back to the user's previous app after each action.",
           "enum": [
             "background",
             "foreground"
@@ -270,12 +270,12 @@
   },
   {
     "name": "drag",
-    "description": "Drag between screenshot-local coordinates. Returns actual delivery effect and fresh post-action observation.",
+    "description": "Drag between screenshot-local coordinates. Returns actual delivery effect and fresh post-action observation. Requires the latest capture_id. Stale, moved/resized or out-of-image targets are rejected before input.",
     "inputSchema": {
       "additionalProperties": false,
       "properties": {
         "delivery_mode": {
-          "description": "background (default) avoids raising the window; unsupported input is refused. Use foreground for Chromium/Brave keyboard input and scrolling within task scope. Inspect the returned state before retrying an uncertain action.",
+          "description": "background does not activate the window. foreground uses Cua bring_to_front before input and leaves the target foreground for subsequent rendering and evidence; it does not switch back to the user's previous app after each action.",
           "enum": [
             "background",
             "foreground"
@@ -311,6 +311,11 @@
         "window_id": {
           "description": "Exact HWND returned for this application by list_windows. Required and must belong to pid.",
           "type": "integer"
+        },
+        "capture_id": {
+          "type": "string",
+          "minLength": 1,
+          "description": "capture_id from the latest returned screenshot for this exact window. Each action returns a new screenshot; use its new ID and dimensions."
         }
       },
       "required": [
@@ -319,7 +324,8 @@
         "from_x",
         "from_y",
         "to_x",
-        "to_y"
+        "to_y",
+        "capture_id"
       ],
       "type": "object"
     },
@@ -335,7 +341,7 @@
       "additionalProperties": false,
       "properties": {
         "delivery_mode": {
-          "description": "background (default) avoids raising the window; unsupported input is refused. Use foreground for Chromium/Brave keyboard input and scrolling within task scope. Inspect the returned state before retrying an uncertain action.",
+          "description": "background does not activate the window. foreground uses Cua bring_to_front before input and leaves the target foreground for subsequent rendering and evidence; it does not switch back to the user's previous app after each action.",
           "enum": [
             "background",
             "foreground"
@@ -374,7 +380,7 @@
       "additionalProperties": false,
       "properties": {
         "delivery_mode": {
-          "description": "background (default) avoids raising the window; unsupported input is refused. Use foreground for Chromium/Brave keyboard input and scrolling within task scope. Inspect the returned state before retrying an uncertain action.",
+          "description": "background does not activate the window. foreground uses Cua bring_to_front before input and leaves the target foreground for subsequent rendering and evidence; it does not switch back to the user's previous app after each action.",
           "enum": [
             "background",
             "foreground"
@@ -420,7 +426,7 @@
       "additionalProperties": false,
       "properties": {
         "delivery_mode": {
-          "description": "background (default) avoids raising the window; unsupported input is refused. Use foreground for Chromium/Brave keyboard input and scrolling within task scope. Inspect the returned state before retrying an uncertain action.",
+          "description": "background does not activate the window. foreground uses Cua bring_to_front before input and leaves the target foreground for subsequent rendering and evidence; it does not switch back to the user's previous app after each action.",
           "enum": [
             "background",
             "foreground"
@@ -460,7 +466,7 @@
   },
   {
     "name": "scroll",
-    "description": "Scroll at x,y in the latest window screenshot. No element_token. Use a point inside the intended scroll surface, amount=1..50, by=line/page. Chromium/Brave requires foreground. Verify actual content movement in the returned observation.",
+    "description": "Scroll at x,y in the latest window screenshot. No element_token. Use a point inside the intended scroll surface, amount=1..50, by=line/page. Chromium/Brave requires foreground. Verify actual content movement in the returned observation. Requires the latest capture_id. Stale, moved/resized or out-of-image targets are rejected before input.",
     "inputSchema": {
       "additionalProperties": false,
       "properties": {
@@ -479,7 +485,7 @@
           "type": "string"
         },
         "delivery_mode": {
-          "description": "background (default) avoids raising the window; unsupported input is refused. Use foreground for Chromium/Brave keyboard input and scrolling within task scope. Inspect the returned state before retrying an uncertain action.",
+          "description": "background does not activate the window. foreground uses Cua bring_to_front before input and leaves the target foreground for subsequent rendering and evidence; it does not switch back to the user's previous app after each action.",
           "enum": [
             "background",
             "foreground"
@@ -511,6 +517,11 @@
         "y": {
           "description": "Y inside the scroll area in latest window-local screenshot pixels; never screen coordinates. Must be paired with the other coordinate.",
           "type": "number"
+        },
+        "capture_id": {
+          "type": "string",
+          "minLength": 1,
+          "description": "capture_id from the latest returned screenshot for this exact window. Each action returns a new screenshot; use its new ID and dimensions."
         }
       },
       "required": [
@@ -518,7 +529,8 @@
         "window_id",
         "direction",
         "x",
-        "y"
+        "y",
+        "capture_id"
       ],
       "type": "object"
     },
@@ -561,7 +573,7 @@
   },
   {
     "name": "click_at",
-    "description": "Click a visual target at x,y in the latest window screenshot, with its exact capture_id. Never pass an element token. Returns effect and fresh observation.",
+    "description": "Click a visual target at x,y in the latest window screenshot, with its exact capture_id. Never pass an element token. Returns effect and fresh observation. Use the latest returned image dimensions; no screen coordinates or Android normalization.",
     "inputSchema": {
       "additionalProperties": false,
       "properties": {
@@ -586,7 +598,7 @@
           "type": "integer"
         },
         "delivery_mode": {
-          "description": "background (default) avoids raising the window; unsupported input is refused. Use foreground for Chromium/Brave keyboard input and scrolling within task scope. Inspect the returned state before retrying an uncertain action.",
+          "description": "background does not activate the window. foreground uses Cua bring_to_front before input and leaves the target foreground for subsequent rendering and evidence; it does not switch back to the user's previous app after each action.",
           "enum": [
             "background",
             "foreground"
@@ -623,5 +635,34 @@
       "readOnlyHint": false,
       "idempotentHint": false
     }
+  },
+  {
+    "name": "bring_to_front",
+    "description": "Activate the exact authorized window and leave it foreground across observations and actions. Use when a background browser body is blank or its page tree is missing. This changes the user foreground; it does not navigate, refresh, or repeat any business action. Read the returned fresh screenshot before continuing.",
+    "inputSchema": {
+      "type": "object",
+      "required": [
+        "pid",
+        "window_id"
+      ],
+      "properties": {
+        "pid": {
+          "type": "integer",
+          "minimum": 1,
+          "maximum": 4294967295
+        },
+        "window_id": {
+          "type": "integer",
+          "minimum": 1
+        }
+      },
+      "additionalProperties": false
+    },
+    "annotations": {
+      "readOnlyHint": false,
+      "destructiveHint": false,
+      "idempotentHint": true,
+      "openWorldHint": false
+    }
   }
 ]

plugins/cua-driver/windows.go

@@ -3,6 +3,7 @@ package main
 import (
 	"fmt"
 	"os/exec"
+	"runtime"
 	"syscall"
 	"unsafe"
 )
@@ -10,9 +11,13 @@ import (
 var user32 = syscall.NewLazyDLL("user32.dll")
 var kernel32 = syscall.NewLazyDLL("kernel32.dll")
 
+func foregroundWindow() int64 {
+	hwnd, _, _ := user32.NewProc("GetForegroundWindow").Call()
+	return int64(hwnd)
+}
+
 func configureProcess(c *exec.Cmd) {
 	c.SysProcAttr = &syscall.SysProcAttr{HideWindow: true}
-	user32.NewProc("SetThreadDpiAwarenessContext").Call(^uintptr(3))
 }
 func windowOwned(pid, hwnd int64) bool {
 	var actual uint32
@@ -24,6 +29,17 @@ func windowOwned(pid, hwnd int64) bool {
 	return int64(actual) == pid
 }
 func verifyWindow(pid, hwnd int64, b obj) (obj, bool) {
+	// DPI context belongs to an OS thread, not a Go goroutine. Apply it around
+	// the actual geometry reads; setting it once while starting a child leaks
+	// context and later GetSystemMetrics can return virtualized dimensions.
+	runtime.LockOSThread()
+	defer runtime.UnlockOSThread()
+	setDPI := user32.NewProc("SetThreadDpiAwarenessContext")
+	previous, _, _ := setDPI.Call(^uintptr(3))
+	if previous == 0 {
+		return nil, false
+	}
+	defer setDPI.Call(previous)
 	if !windowOwned(pid, hwnd) {
 		return nil, false
 	}

scripts/build_cua_driver.py

@@ -99,7 +99,7 @@ def main():
                     "source_commit": COMMIT,
                     "upstream_version": UPSTREAM_VERSION,
                     "plugin_version": version,
-                    "adapter_revision": 3,
+                    "adapter_revision": 5,
                     "prerelease": True,
                     "url": URL,
                     "sha256": DIGEST,

tests/capture-fixture.ps1

@@ -1,4 +1,4 @@
-param([string]$Title, [string]$ReadyFile, [switch]$Cover, [int]$Width=500, [int]$Height=300, [switch]$PerMonitorDpi)
+param([string]$Title, [string]$ReadyFile, [switch]$Cover, [int]$Width=500, [int]$Height=300, [switch]$PerMonitorDpi, [switch]$NormalWindow)
 if ($PerMonitorDpi) {
     Add-Type -TypeDefinition 'using System; using System.Runtime.InteropServices; public static class FixtureDpi { [DllImport("user32.dll")] public static extern IntPtr SetThreadDpiAwarenessContext(IntPtr c); }'
     [void][FixtureDpi]::SetThreadDpiAwarenessContext([IntPtr](-4))
@@ -10,7 +10,7 @@ $form.Text = $Title
 $form.Size = New-Object System.Drawing.Size($Width, $Height)
 $form.StartPosition = 'Manual'
 $form.Location = New-Object System.Drawing.Point(100, 100)
-$form.TopMost = $true
+$form.TopMost = -not $NormalWindow
 $form.BackColor = if ($Cover) { [System.Drawing.Color]::Magenta } else { [System.Drawing.Color]::Lime }
 $label = New-Object System.Windows.Forms.Label
 $label.Text = if ($Cover) { 'OCCLUDER - NOT TEST EVIDENCE' } else { 'ARTEMIS window capture fixture' }

tests/cua_browser_smoke.py

@@ -22,6 +22,7 @@ import jsonschema
 import psutil
 from artemis.plugins.models import ServerConfig
 from artemis.plugins.session import PluginSession
+from artemis.runtime.device_lock import DeviceExecutionLock
 
 
 async def model_task(base, pid, url, output, events, traces, model):
@@ -54,7 +55,7 @@ async def model_task(base, pid, url, output, events, traces, model):
                     {
                         "id": "cua-driver",
                         "profile": "benchmark",
-                        "version": "0.36.0+artemis.2",
+                        "version": "0.36.0+artemis.3",
                     }
                 ],
                 "model": model,
@@ -133,6 +134,7 @@ async def run(
     base=None,
     traces=None,
     model="gpt-6.1-sol",
+    foreground_check=False,
 ):
     output.mkdir(parents=True, exist_ok=True)
     events = []
@@ -179,7 +181,7 @@ async def run(
     threading.Thread(target=server.serve_forever, daemon=True).start()
     url = f"http://127.0.0.1:{server.server_port}"
     session = None
-    process = None
+    process = cover_process = None
     try:
         with tempfile.TemporaryDirectory(prefix="artemis-cua-browser-") as profile:
             try:
@@ -276,6 +278,60 @@ async def run(
                     assert match, state["tree_markdown"]
                     return state["snapshot_id"] + ":" + match[1]
 
+                if foreground_check:
+                    ready = Path(profile) / "cover-ready"
+                    cover_process = await asyncio.create_subprocess_exec(
+                        "powershell.exe",
+                        "-NoProfile",
+                        "-File",
+                        str(Path(__file__).with_name("capture-fixture.ps1")),
+                        "-Title",
+                        "ARTEMIS foreground handoff fixture",
+                        "-ReadyFile",
+                        str(ready),
+                        "-Cover",
+                        "-NormalWindow",
+                        "-Width",
+                        "1100",
+                        "-Height",
+                        "900",
+                        stdout=subprocess.DEVNULL,
+                        stderr=subprocess.DEVNULL,
+                        creationflags=subprocess.CREATE_NO_WINDOW,
+                    )
+                    async with asyncio.timeout(20):
+                        while not ready.exists():
+                            await asyncio.sleep(0.1)
+                    cover_windows = await call(
+                        "list_windows", {"pid": cover_process.pid}
+                    )
+                    cover = {
+                        k: cover_windows["windows"][0][k] for k in ("pid", "window_id")
+                    }
+                    await call("bring_to_front", cover)
+                    hidden = await observe()
+                    assert not hidden["capture_focus"]["target_foreground_before"]
+                    assert not hidden["capture_focus"]["target_foreground_after"]
+                    # Explicit foreground input must not snap back to the cover.
+                    await key("l", ["ctrl"])
+                    await asyncio.sleep(5)  # Past Cua's 3-second restore poller.
+                    focused = await observe()
+                    assert focused["capture_focus"]["target_foreground_after"]
+                    await key("escape")
+                    # Manual focus loss is reported, never silently corrected by
+                    # read-only capture. Existing bring_to_front recovers it.
+                    await call("bring_to_front", cover)
+                    hidden = await observe()
+                    assert not hidden["capture_focus"]["target_foreground_after"]
+                    await call("bring_to_front", target)
+                    await asyncio.sleep(5)
+                    focused = await observe()
+                    assert focused["capture_focus"]["target_foreground_before"]
+                    assert focused["capture_focus"]["target_foreground_after"]
+                    await call(
+                        "save_screenshot",
+                        {**target, "filename": "foreground-recovered.png"},
+                    )
                 await wait_tree("ARTEMIS Cua Browser Start")
                 # Reproduce task f1928... invalid form before any keyboard dispatch.
                 bad = await call(
@@ -303,6 +359,11 @@ async def run(
                 )
                 await key("return")
                 state = await wait_tree("Test parameter")
+                if foreground_check:
+                    await asyncio.sleep(5)
+                    state = await observe()
+                    assert state["capture_focus"]["target_foreground_after"]
+                    assert "Test parameter" in state["tree_markdown"]
                 # Direct UIA replacement is a separate, unambiguous tool.
                 await call(
                     "set_value",
@@ -353,6 +414,7 @@ async def run(
                             "by": "page",
                             "x": source["width"] // 2,
                             "y": source["height"] // 2,
+                            "capture_id": state["capture_id"],
                             "delivery_mode": "foreground",
                         },
                     )
@@ -392,6 +454,7 @@ async def run(
                 )
                 report = {
                     "passed": True,
+                    "foreground_handoff_and_readonly_recovery": foreground_check,
                     "checks": [
                         "mixed parameters refused before dispatch",
                         "Ctrl+T new tab",
@@ -416,6 +479,9 @@ async def run(
                 )
                 if session:
                     await session.aclose()
+                if cover_process and cover_process.returncode is None:
+                    cover_process.terminate()
+                    await cover_process.wait()
                 if process:
                     try:
                         parent = psutil.Process(process.pid)
@@ -448,9 +514,21 @@ if __name__ == "__main__":
         "--traces", type=Path, help="That server's traces directory (model test only)"
     )
     parser.add_argument("--model", default="gpt-6.1-sol")
+    parser.add_argument("--foreground-check", action="store_true")
     args = parser.parse_args()
     if args.model_base and not args.traces:
         parser.error("--model-base requires --traces")
+    if args.foreground_check and args.model_base:
+        parser.error("--foreground-check is deterministic, not a model task")
+    lock = None
+    if not args.model_base:
+        lock = DeviceExecutionLock(
+            "cua-browser-regression",
+            description="Cua browser regression",
+            resource_ids=["desktop:local"],
+        )
+        lock.acquire(timeout=0)
+    try:
         asyncio.run(
             run(
                 args.executable.resolve(),
@@ -459,5 +537,9 @@ if __name__ == "__main__":
                 args.model_base,
                 args.traces,
                 args.model,
+                args.foreground_check,
             )
         )
+    finally:
+        if lock:
+            lock.release()

tests/cua_fixture_validation.py

@@ -79,12 +79,14 @@ async def main(executable):
                     "by": "page",
                     "x": 400,
                     "y": 470,
+                    "capture_id": pixel.structuredContent["observation"]["capture_id"],
                 },
             )
             if scrolled.isError:
                 assert (
                     scrolled.structuredContent["artemis"]["execution"] == "not_started"
                 ), scrolled
+                state = await session.call("get_window_state", target)
                 scrolled = await session.call(
                     "scroll",
                     {
@@ -94,6 +96,7 @@ async def main(executable):
                         "by": "page",
                         "x": 400,
                         "y": 470,
+                        "capture_id": state.structuredContent["capture_id"],
                         "delivery_mode": "foreground",
                     },
                 )