Sync third-party and MCP marketplace plugins
Constraint: Public skills are published only by explicit administrator action unless they are tracked third-party market sources. Confidence: high Scope-risk: narrow Directive: Keep private/internal skills out of the public marketplace and preserve normal incremental market Git history. Tested: Marketplace validation passed.
This commit is contained in:
KeyInfo Bot committed
1 parent
be7ce9bda2
commit
36c089fee9
298 files changed
+32935
-11681
No files matched your search
@@ -1,13 +1,13 @@
|
||||
{
|
||||
"name": "eapil-voice",
|
||||
"version": "0.1.0",
|
||||
"description": "当任务需要合成语音、根据指令设计语音、从参考音频克隆语音、使用 VoxCPM2 或 Qwen 提供商的服务、对音频进行增强或去噪、通过该服务转录音频、批量进行 ASR/警报音克隆、将生成的语音导出为 WAV/MP3 格式、检查模型状态，或生成用于 Eapil-Voice API 的 curl/Python 调用时，应使用此服务。",
|
||||
"version": "1.0.0",
|
||||
"description": "易瞳音频服务调用指南：语音设计、朗读、克隆、ASR、MossFormer2/RE-USE 降噪、Stable Audio 音乐与音效、Yue2 歌曲及批量导出。包含请求校验、错误处理和实测记录。",
|
||||
"author": {
|
||||
"name": "EAPIL",
|
||||
"url": "https://git.playones.com/arechen/EapilSkillMarket"
|
||||
},
|
||||
"homepage": "https://git.playones.com/arechen/EapilSkillMarket",
|
||||
"repository": "https://git.playones.com/arechen/EapilSkillMarket",
|
||||
"homepage": "https://git.playones.com/chenmingxuan/ManySkills/src/branch/main/plugins/eapil-voice",
|
||||
"repository": "https://git.playones.com/chenmingxuan/ManySkills",
|
||||
"license": "Proprietary",
|
||||
"keywords": [
|
||||
"eapil",
|
||||
@@ -16,9 +16,9 @@
|
||||
],
|
||||
"skills": "./skills/",
|
||||
"interface": {
|
||||
"displayName": "易瞳语音转录|设计|克隆|降噪",
|
||||
"shortDescription": "当任务需要合成语音、根据指令设计语音、从参考音频克隆语音、使用 VoxCPM2 或 Qwen 提供商的服务、对音频进行增强或去噪、通过该服务转录音频、批量进行 ASR/警报音克隆、将生成的语音导出为 WAV/MP3 格式、检查模型状态，或生成用于 Eapil-Voice API 的 curl/Python 调用时，应使用此服务。",
|
||||
"longDescription": "当任务需要合成语音、根据指令设计语音、从参考音频克隆语音、使用 VoxCPM2 或 Qwen 提供商的服务、对音频进行增强或去噪、通过该服务转录音频、批量进行 ASR/警报音克隆、将生成的语音导出为 WAV/MP3 格式、检查模型状态，或生成用于 Eapil-Voice API 的 curl/Python 调用时，应使用此服务。",
|
||||
"displayName": "易瞳音频：语音 / 转录 / 降噪 / 音乐 / 音效",
|
||||
"shortDescription": "语音、ASR、降噪、音乐、音效与歌曲的调用及验证指南。",
|
||||
"longDescription": "按需选择 Qwen/VoxCPM2 语音、转录、两类降噪、Stable Audio 和 Yue2 接口。配套单请求客户端与顺序批处理，校验 WAV/JSON 响应、使用准确参考转写，并明确超时不可盲目重发及部署限制。保留 eapil-voice 插件 ID。",
|
||||
"developerName": "EAPIL",
|
||||
"category": "Creativity",
|
||||
"capabilities": [
|
||||
@@ -26,7 +26,7 @@
|
||||
"Write"
|
||||
],
|
||||
"defaultPrompt": [
|
||||
"使用 易瞳语音转录|设计|克隆|降噪 帮我完成这个任务。"
|
||||
"使用 易瞳音频 帮我选择接口并完成音频任务。"
|
||||
],
|
||||
"websiteURL": "https://git.playones.com/arechen/EapilSkillMarket",
|
||||
"privacyPolicyURL": "https://git.playones.com/arechen/EapilSkillMarket",
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"sourceId": "eapil-voice",
|
||||
"repo": "https://git.playones.com/chenmingxuan/ManySkills.git",
|
||||
"ref": "v1.0.0",
|
||||
"commit": "e869eb169eeb5a7cd9e58c0e514fa2dbbf50987d",
|
||||
"adapter": "codex-plugin",
|
||||
"sourcePath": "plugins/eapil-voice",
|
||||
"syncedAt": "2026-10-06T07:26:41Z"
|
||||
}
|
||||
@@ -1,120 +1,47 @@
|
||||
---
|
||||
name: eapil-voice
|
||||
description: Guide agents to call the hosted Eapil-Voice/Qwen3 Voice Design FastAPI service by domain for audio generation workflows. Use when a task asks to synthesize speech, design a voice from an instruction, clone a voice from reference audio, use VoxCPM2 or Qwen providers, enhance or denoise audio, transcribe audio through this service, batch-ASR/clone alert sounds, export generated speech as WAV/MP3, check model health, or produce curl/Python calls for the Eapil-Voice API.
|
||||
description: Build, test or use EAPIL audio API calls for speech design and cloning, transcription, denoising, music, sound effects and lyric-conditioned songs; includes sequential batch speech export.
|
||||
---
|
||||
|
||||
# Eapil-Voice
|
||||
# EAPIL audio service
|
||||
|
||||
## Purpose
|
||||
Use the hosted service at `https://ades.playones.com`, or the deployment chosen by the user. Keep the historical skill/plugin ID for compatibility; this guide covers the whole audio service.
|
||||
|
||||
Use this skill to call the hosted Eapil-Voice API by domain. Prefer the existing API surface over inventing client contracts.
|
||||
| Need | Route | Read when relevant |
|
||||
|---|---|---|
|
||||
| Describe a voice and speak text | `/v1/voice/design` | [Speech](references/speech.md) |
|
||||
| Ordinary Qwen text reading | `/v1/voice/synthesize` | [Speech](references/speech.md) |
|
||||
| Clone from reference speech | `/v1/voice/clone` | [Speech](references/speech.md) |
|
||||
| Transcribe speech | `/v1/asr/transcribe` | [ASR and enhancement](references/recognition-enhancement.md) |
|
||||
| MossFormer2 enhancement / alias | `/v1/audio/enhance`, `/v1/audio/denoise` | [ASR and enhancement](references/recognition-enhancement.md) |
|
||||
| RE-USE denoising | `/v1/audio/reuse` | [ASR and enhancement](references/recognition-enhancement.md) |
|
||||
| Instrumental music or sound effects | `/v1/audio/music`, `/v1/audio/sfx` | [Stable Audio](references/music.md) |
|
||||
| Songs from lyrics or ABC arrangement | `/v1/audio/music/yue2` | [Yue2](references/yue2.md) |
|
||||
| Directory-scale ASR/clone/export | Sequential calls | [Batch audio](references/batch-audio.md) |
|
||||
|
||||
Default base URL is `https://ades.playones.com`. Override it only when the user gives another domain or `QWEN3_VOICE_API_BASE_URL`.
|
||||
Read [transport and errors](references/api.md) for client integration or troubleshooting. Consult `/openapi.json` and `/models` when deployment facts matter; a registered route or healthy service does not prove a model can infer. Dated evidence is in [verification](references/verification.md). Read only the mode needed.
|
||||
|
||||
Use `OPENAI_API_KEY` as the API key environment variable. If it is not set and the user did not provide a key directly, tell the user to get their key from `https://kinfo.playones.com`. Direct key use is allowed with `--token` or an `Authorization: Bearer <key>` header.
|
||||
## Call the actual contract
|
||||
|
||||
## Call Workflow
|
||||
Use `AUDIO_API_BASE` and `AUDIO_API_TOKEN`; old `QWEN3_VOICE_API_BASE_URL` and `OPENAI_API_KEY` remain fallback aliases. Explicit flags take precedence. An available legacy key is not proof of authorization on this deployment. Inference requires Bearer authentication; health/models are public. Obtain missing credentials from the provider without putting them in chat or tracked files.
|
||||
|
||||
1. Resolve the base URL first. Use `https://ades.playones.com` unless the user gives another domain.
|
||||
2. Check `GET /health` before expensive generation.
|
||||
3. Add `Authorization: Bearer <key>` from an explicit user-provided key or `OPENAI_API_KEY`.
|
||||
4. Choose the endpoint by desired output:
|
||||
- Voice from style instruction: `POST /v1/voice/design`.
|
||||
- Base TTS by speaker name: `POST /v1/voice/synthesize`.
|
||||
- Voice clone from reference audio: `POST /v1/voice/clone`.
|
||||
- Speech enhancement or denoise: `POST /v1/audio/enhance` or `POST /v1/audio/denoise`.
|
||||
- Transcription: `POST /v1/asr/transcribe` returns JSON, not audio.
|
||||
5. Save audio-output responses as `.wav`; these endpoints return `audio/wav`.
|
||||
6. Read `/openapi.json` from the active base URL when exact current form fields matter.
|
||||
7. Read `references/api.md` before writing detailed client code, exposing all services, troubleshooting errors, or selecting Qwen vs VoxCPM2 parameters.
|
||||
8. Read `references/batch-audio.md` before batch ASR/clone/export work, especially game filter alert sounds.
|
||||
Speech design/synthesis, music/SFX and Yue2 take JSON. Clone, ASR and enhancement use multipart with file field `file`; let the client create the boundary. Audio success is complete WAV bytes, not JSON/Base64/a URL or inference streaming. ASR returns JSON. Preserve explicit false/0; keep provider and endpoint controls separate.
|
||||
|
||||
## Provider Rules
|
||||
[call_voice_api.py](scripts/call_voice_api.py) makes one validated request. Existing command names remain; new commands are reuse, music, sfx, yue2. Dry run prints the payload without a key/network call, so share only synthetic inputs. Resolve the actual skill directory:
|
||||
|
||||
Use `provider=qwen` by default. Only `/v1/voice/design` and `/v1/voice/clone` accept `provider=voxcpm2`.
|
||||
~~~powershell
|
||||
python '<skill-dir>/scripts/call_voice_api.py' design --text '你好。' --instruct '温暖自然、清晰的女声。' --language Chinese --max-tokens 512 --out voice.wav
|
||||
python '<skill-dir>/scripts/call_voice_api.py' asr --file speech.wav --language zh
|
||||
~~~
|
||||
|
||||
For VoxCPM2:
|
||||
- Pass `provider=voxcpm2`.
|
||||
- Use `cfg_value`, `inference_timesteps`, `normalize`, `denoise`, and `retry_badcase` for generation controls.
|
||||
- Set `denoise=true` only when the server is configured with `VOXCPM2_LOAD_DENOISER=true`.
|
||||
- For clone with `ref_text`, expect Hi-Fi transcript-guided cloning.
|
||||
- For clone without `ref_text`, use `instruct` for controllable cloning.
|
||||
Prefix external commands with `rtk proxy` in this repository.
|
||||
|
||||
For Qwen clone, always provide both `text` and `ref_text`.
|
||||
## Verify and deliver
|
||||
|
||||
## Batch Audio Rules
|
||||
Check status/media type and complete WAV structure, then verify what matters: spoken content, accurate reference transcript, intelligibility/noise or musical content/duration. A valid WAV alone does not prove voice similarity or listening quality. Read actual rate/channels from the header; MP3 requires local transcoding.
|
||||
|
||||
- For many short alert sounds, ask only if the output mode is genuinely ambiguous: generated-only files, full replacement files, or overlay/mix.
|
||||
- Do not assume ASR segment timestamps are source-separation masks. ASR can locate/transcribe speech; it cannot remove an old voice from mixed music/SFX by itself.
|
||||
- If the user wants clean replacement assets for another tool, export generated speech only, named after the source files. Use `scripts/batch_clone_generated_audio.py`.
|
||||
- If the user wants to preserve music/SFX while replacing embedded speech, state that source separation or stems are required for clean removal. Do not mix the original voice under the generated voice unless the user explicitly asks for overlay.
|
||||
- Normalize generated text before clone: add expressive punctuation when useful; if the final character is not `。！？.!?~～`, append `。` for Chinese alert text.
|
||||
- For final MP3 delivery, convert the generated WAVs with `ffmpeg` and explicit gain such as `volume=4dB` only when requested.
|
||||
Do not invent transcripts from filenames, manufacture speech after empty ASR, silently rewrite requested text or treat ASR timestamps as source separation. Punctuation normalization, gain and overlays apply only when requested.
|
||||
|
||||
## Quick Examples
|
||||
Requests wait for the final result. There is no job ID, polling/cancel endpoint or idempotency key. Timeout/504 may leave inference running: do not automatically resubmit. Keep calls sequential initially; only an explicit busy rejection is eligible for bounded backoff.
|
||||
|
||||
Use JSON for voice design and synthesis:
|
||||
|
||||
```bash
|
||||
BASE_URL="${QWEN3_VOICE_API_BASE_URL:-https://ades.playones.com}"
|
||||
|
||||
curl -X POST "$BASE_URL/v1/voice/design" \
|
||||
-H "Authorization: Bearer $OPENAI_API_KEY" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"provider":"qwen","text":"Hello world","instruct":"Warm, calm, natural narration","language":"auto","temperature":0.9,"max_tokens":4096,"top_k":50,"top_p":1.0,"repetition_penalty":1.05}' \
|
||||
--output voice_design.wav
|
||||
```
|
||||
|
||||
Use multipart form data for clone, enhancement, denoise, and ASR:
|
||||
|
||||
```bash
|
||||
BASE_URL="${QWEN3_VOICE_API_BASE_URL:-https://ades.playones.com}"
|
||||
|
||||
curl -X POST "$BASE_URL/v1/voice/clone" \
|
||||
-H "Authorization: Bearer $OPENAI_API_KEY" \
|
||||
-F 'file=@reference.wav' \
|
||||
-F 'provider=qwen' \
|
||||
-F 'text=This is the cloned sentence.' \
|
||||
-F 'ref_text=Transcript of the reference audio.' \
|
||||
-F 'language=auto' \
|
||||
-F 'temperature=0.9' \
|
||||
-F 'max_tokens=4096' \
|
||||
-F 'top_k=50' \
|
||||
-F 'top_p=1.0' \
|
||||
-F 'repetition_penalty=1.05' \
|
||||
--output voice_clone.wav
|
||||
```
|
||||
|
||||
Use the bundled dependency-free client when convenient:
|
||||
|
||||
```bash
|
||||
python ~/.codex/skills/eapil-voice/scripts/call_voice_api.py design \
|
||||
--text "Hello world" \
|
||||
--instruct "Warm, calm, natural narration" \
|
||||
--out voice_design.wav
|
||||
```
|
||||
|
||||
Batch clone generated speech only, preserving source names:
|
||||
|
||||
```bash
|
||||
python ~/.codex/skills/eapil-voice/scripts/batch_clone_generated_audio.py \
|
||||
--base-url https://ades.playones.com \
|
||||
--token "$OPENAI_API_KEY" \
|
||||
--input-dir "/path/to/filter-sounds" \
|
||||
--reference-audio reference.wav \
|
||||
--output-dir "/path/to/generated-mp3" \
|
||||
--format mp3 \
|
||||
--gain-db 4
|
||||
```
|
||||
|
||||
On Windows, prefer direct command execution or a single quoted `Start-Process -ArgumentList` string for paths containing spaces or CJK characters.
|
||||
|
||||
## Source Anchors
|
||||
|
||||
When a local source checkout is available and the task depends on implementation details, use it only as source reference. Do not prefer local service startup over the hosted domain:
|
||||
|
||||
- `/Users/eapil/git/QWEN3-Voice-Design/app/api/routes/voice.py`
|
||||
- `/Users/eapil/git/QWEN3-Voice-Design/app/api/routes/audio.py`
|
||||
- `/Users/eapil/git/QWEN3-Voice-Design/app/api/routes/asr.py`
|
||||
- `/Users/eapil/git/QWEN3-Voice-Design/app/schemas/voice.py`
|
||||
- `/Users/eapil/git/QWEN3-Voice-Design/README.md`
|
||||
Return usable local audio, endpoint/provider and limitations. In Codex, preview delivered audio with an absolute path. Record actual coverage; keep credentials, probe outputs and private transcripts outside the repository.
|
||||
@@ -1,4 +1,4 @@
|
||||
interface:
|
||||
display_name: "Eapil-Voice"
|
||||
short_description: "Call hosted Eapil voice APIs"
|
||||
default_prompt: "Use $eapil-voice to transcribe, clone, or batch-export speech through the hosted Eapil-Voice API."
|
||||
display_name: "EAPIL Audio"
|
||||
short_description: "EAPIL speech, ASR, denoising, music and song APIs"
|
||||
default_prompt: "Use $eapil-voice to choose and call the EAPIL audio endpoint for speech, transcription, denoising, music or songs."
|
||||
@@ -1,359 +1,37 @@
|
||||
# Eapil-Voice API Reference
|
||||
# Transport and deployed contract
|
||||
|
||||
## Defaults
|
||||
Source: maintainer's API usage guide dated 2026-10-06 and live [OpenAPI](https://ades.playones.com/openapi.json). Schema limits, deployment caps and model recommendations differ; deployment settings can change.
|
||||
|
||||
- Base URL: `https://ades.playones.com`
|
||||
- Override env: `QWEN3_VOICE_API_BASE_URL`
|
||||
- API key env: `OPENAI_API_KEY`
|
||||
- Auth header: `Authorization: Bearer <key>`
|
||||
- If no key is available, get one from `https://kinfo.playones.com`.
|
||||
- Audio-output media type: `audio/wav`
|
||||
- Local source checkouts are source references only unless the user explicitly asks to run a local service.
|
||||
## Requests and credentials
|
||||
|
||||
## Endpoint Summary
|
||||
Base URL is the service root without /v1. Client precedence: explicit flags; AUDIO_API_BASE/AUDIO_API_TOKEN; legacy QWEN3_VOICE_API_BASE_URL/OPENAI_API_KEY; default hosted URL. --api-key-env NAME selects exactly that variable. --token remains for old scripts; environment credentials avoid process-argument exposure. GET health/models require no token.
|
||||
|
||||
| Task | Method | Path | Input | Output |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Voice design TTS | `POST` | `/v1/voice/design` | JSON | WAV |
|
||||
| Base TTS synthesis | `POST` | `/v1/voice/synthesize` | JSON | WAV |
|
||||
| Voice clone | `POST` | `/v1/voice/clone` | multipart form | WAV |
|
||||
| Speech enhancement | `POST` | `/v1/audio/enhance` | multipart form | WAV |
|
||||
| Denoise alias | `POST` | `/v1/audio/denoise` | multipart form | WAV |
|
||||
| ASR transcription | `POST` | `/v1/asr/transcribe` | multipart form | JSON |
|
||||
| Health | `GET` | `/health` | none | JSON |
|
||||
| Models | `GET` | `/models` | none | JSON |
|
||||
Inference uses Bearer auth. Serialize UTF-8 JSON, including multiline lyrics. File workflows prefer multipart with field file, automatically generated boundary and form-valued numbers/booleans. Raw binary with query fields is also supported by the API; the bundled client uses multipart to keep long text out of URLs.
|
||||
|
||||
## Parameter Limits
|
||||
Size limit: 26,214,400 bytes (25 MiB). Qwen clone, ASR, MossFormer2 and RE-USE enforce <=60 decoded seconds. VoxCPM2 shares the size limit but not that decoded-duration check; a clear 5–15 second reference is usually sufficient. A .wav extension is not conversion. WAV inputs can be preflight checked; other decoders depend on the server.
|
||||
|
||||
- `provider`: `qwen` or `voxcpm2`
|
||||
- `text`: 1 to 4096 characters
|
||||
- `instruct`: 1 to 1024 characters; optional for clone
|
||||
- `temperature`: 0 to 2
|
||||
- `max_tokens`: 1 to 8192
|
||||
- `top_k`: 0 to 1000
|
||||
- `top_p`: 0 to 1
|
||||
- `repetition_penalty`: 0.1 to 2
|
||||
- `cfg_value`: 0 to 10
|
||||
- `inference_timesteps`: 1 to 200
|
||||
- Uploaded audio defaults to a 25 MB and 60 second server-side limit, depending on deployment environment variables.
|
||||
## Results
|
||||
|
||||
## Voice Design
|
||||
Audio success is complete binary audio/wav. The helper checks media type and complete RIFF format/data chunks before saving. It handles PCM/IEEE float WAV; RF64/compressed WAV needs another validated decoder. ASR must have string text; an empty transcription can represent silence, but cannot drive a batch clone.
|
||||
|
||||
`POST /v1/voice/design`
|
||||
Read actual rate/channels. Music/SFX currently use 44.1 kHz stereo PCM16; Yue2 48 kHz stereo PCM16. Provider speech rates vary. RE-USE encodes the input rate; enhancement/RE-USE mix channels to mono and are not stereo mastering tools.
|
||||
|
||||
Use this for text-to-speech driven by a style instruction. Body is JSON.
|
||||
Server-side JavaScript reads response.arrayBuffer() for audio and response.json() for ASR, once per response body. Current deployment lacks general browser CORS; use a backend proxy or configured CORS. Keep service tokens on that backend.
|
||||
|
||||
Required:
|
||||
- `text`: text to speak
|
||||
- `instruct`: style, emotion, voice, or delivery instruction
|
||||
## Timeouts and concurrency
|
||||
|
||||
Common optional fields:
|
||||
- `provider`: `qwen` default, or `voxcpm2`
|
||||
- `language`: `auto` default, used by Qwen
|
||||
- `temperature`: `0.9` default, used by Qwen
|
||||
- `max_tokens`: `4096` default, used by Qwen
|
||||
- `top_k`: `50` default, used by Qwen
|
||||
- `top_p`: `1.0` default, used by Qwen
|
||||
- `repetition_penalty`: `1.05` default, used by Qwen
|
||||
- `cfg_value`: `2.0` default, used by VoxCPM2
|
||||
- `inference_timesteps`: `10` default, used by VoxCPM2
|
||||
- `normalize`: `false` default, used by VoxCPM2
|
||||
- `denoise`: `false` default, used by VoxCPM2
|
||||
- `retry_badcase`: `true` default, used by VoxCPM2
|
||||
Speech/ASR/enhancement timeout starts at 600 seconds; music/SFX/Yue2 at 1800. Client timeouts cannot extend a gateway timeout. HTTP chunked transport is not progressive generation.
|
||||
|
||||
Qwen example:
|
||||
There is no async job, task query, cancellation endpoint or idempotency key. Timeout/504 does not establish that inference stopped; investigate before repeating. Do not invent a status endpoint.
|
||||
|
||||
```bash
|
||||
curl -X POST https://ades.playones.com/v1/voice/design \
|
||||
-H 'Authorization: Bearer $OPENAI_API_KEY' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"provider": "qwen",
|
||||
"text": "Hello world",
|
||||
"instruct": "Warm, calm, natural narration",
|
||||
"language": "auto",
|
||||
"temperature": 0.9,
|
||||
"max_tokens": 4096,
|
||||
"top_k": 50,
|
||||
"top_p": 1.0,
|
||||
"repetition_penalty": 1.05
|
||||
}' \
|
||||
--output voice_design.wav
|
||||
```
|
||||
Qwen design can reject busy with 503 and Retry-After. Stable Audio music/SFX share a queue. Qwen synthesis/clone share serialized execution; ASR, VoxCPM2 and Yue2 are separately serialized; MossFormer2/RE-USE share execution. Loading/connection failures can also be 503: status alone is insufficient.
|
||||
|
||||
VoxCPM2 example:
|
||||
The single-call helper never retries. Batch --retries defaults to 1 total attempt, capped at 5. Additional attempts only follow explicit busy/running 503 detail without load/error/timeout indicators. It honors numeric Retry-After up to 60 seconds; longer backoff is left to the application, and fallback delays are bounded. Upload bodies are rebuilt in full. Ambiguous errors are failures, not retry signals.
|
||||
|
||||
```bash
|
||||
curl -X POST https://ades.playones.com/v1/voice/design \
|
||||
-H 'Authorization: Bearer $OPENAI_API_KEY' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"text": "Hello world",
|
||||
"instruct": "Warm, calm, natural narration",
|
||||
"provider": "voxcpm2",
|
||||
"cfg_value": 2.0,
|
||||
"inference_timesteps": 10,
|
||||
"normalize": false,
|
||||
"denoise": false,
|
||||
"retry_badcase": true
|
||||
}' \
|
||||
--output voxcpm2_voice_design.wav
|
||||
```
|
||||
## Errors and state
|
||||
|
||||
## Base TTS Synthesis
|
||||
Failure detail can be a string/array or gateway HTML. Check status/media before parsing. 400/422 usually require request correction; 401 credential correction; 413 smaller/shorter audio. 404 includes removed MiniMax. 500/501/502/503 need detail to distinguish input errors, unsupported backend, loading and busy. Yue2 upstream 4xx may appear as 502.
|
||||
|
||||
`POST /v1/voice/synthesize`
|
||||
Library callers can inspect APIError.status/detail/busy. CLI logs omit raw provider detail because it can echo private inputs; inspect locally and redact before sharing.
|
||||
|
||||
Use this for base Qwen TTS with an optional named speaker. Body is JSON. This endpoint does not accept `provider`.
|
||||
|
||||
Required:
|
||||
- `text`
|
||||
|
||||
Optional:
|
||||
- `speaker`: e.g. `Chelsie`; omit for model default
|
||||
- `language`: `auto` default
|
||||
- `temperature`: `0.9` default
|
||||
- `max_tokens`: `4096` default
|
||||
- `top_k`: `50` default
|
||||
- `top_p`: `1.0` default
|
||||
- `repetition_penalty`: `1.05` default
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
curl -X POST https://ades.playones.com/v1/voice/synthesize \
|
||||
-H 'Authorization: Bearer $OPENAI_API_KEY' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"text": "Hello world",
|
||||
"speaker": "Chelsie",
|
||||
"language": "auto",
|
||||
"temperature": 0.9,
|
||||
"max_tokens": 4096,
|
||||
"top_k": 50,
|
||||
"top_p": 1.0,
|
||||
"repetition_penalty": 1.05
|
||||
}' \
|
||||
--output synthesize.wav
|
||||
```
|
||||
|
||||
## Voice Clone
|
||||
|
||||
`POST /v1/voice/clone`
|
||||
|
||||
Use this for cloning from reference audio. Send multipart form upload with `file=@reference.wav`.
|
||||
|
||||
Audio input:
|
||||
- `file`: reference audio upload
|
||||
|
||||
Shared fields:
|
||||
- `text`: required output text
|
||||
- `provider`: `qwen` default, or `voxcpm2`
|
||||
- `language`: `auto` default, used by Qwen
|
||||
|
||||
Qwen required fields:
|
||||
- `text`
|
||||
- `ref_text`: transcript of the reference audio
|
||||
- `file`
|
||||
|
||||
Qwen optional fields:
|
||||
- `temperature`: default `0.9`
|
||||
- `max_tokens`: default `4096`
|
||||
- `top_k`: default `50`
|
||||
- `top_p`: default `1.0`
|
||||
- `repetition_penalty`: default `1.05`
|
||||
|
||||
VoxCPM2 required fields:
|
||||
- `text`
|
||||
- `file`
|
||||
- `provider=voxcpm2`
|
||||
|
||||
VoxCPM2 optional fields:
|
||||
- `ref_text`: when present, enables transcript-guided Hi-Fi clone
|
||||
- `instruct`: used only when `ref_text` is absent
|
||||
- `cfg_value`: default `2.0`
|
||||
- `inference_timesteps`: default `10`
|
||||
- `normalize`: default `false`
|
||||
- `denoise`: default `false`
|
||||
- `retry_badcase`: default `true`
|
||||
|
||||
Qwen clone example:
|
||||
|
||||
```bash
|
||||
curl -X POST https://ades.playones.com/v1/voice/clone \
|
||||
-H 'Authorization: Bearer $OPENAI_API_KEY' \
|
||||
-F 'file=@reference.wav' \
|
||||
-F 'provider=qwen' \
|
||||
-F 'text=This is the cloned sentence.' \
|
||||
-F 'ref_text=This is the transcript of the reference audio.' \
|
||||
-F 'language=auto' \
|
||||
-F 'temperature=0.9' \
|
||||
-F 'max_tokens=4096' \
|
||||
-F 'top_k=50' \
|
||||
-F 'top_p=1.0' \
|
||||
-F 'repetition_penalty=1.05' \
|
||||
--output voice_clone.wav
|
||||
```
|
||||
|
||||
VoxCPM2 controllable clone without transcript:
|
||||
|
||||
```bash
|
||||
curl -X POST https://ades.playones.com/v1/voice/clone \
|
||||
-H 'Authorization: Bearer $OPENAI_API_KEY' \
|
||||
-F 'file=@reference.wav' \
|
||||
-F 'text=This is the cloned sentence.' \
|
||||
-F 'instruct=Keep the same speaker identity with a calm delivery.' \
|
||||
-F 'provider=voxcpm2' \
|
||||
-F 'cfg_value=2.0' \
|
||||
-F 'inference_timesteps=10' \
|
||||
-F 'normalize=false' \
|
||||
-F 'denoise=false' \
|
||||
-F 'retry_badcase=true' \
|
||||
--output voxcpm2_controllable_clone.wav
|
||||
```
|
||||
|
||||
VoxCPM2 Hi-Fi clone with transcript:
|
||||
|
||||
```bash
|
||||
curl -X POST https://ades.playones.com/v1/voice/clone \
|
||||
-H 'Authorization: Bearer $OPENAI_API_KEY' \
|
||||
-F 'file=@reference.wav' \
|
||||
-F 'text=This is the cloned sentence.' \
|
||||
-F 'ref_text=This is the transcript of the reference audio.' \
|
||||
-F 'provider=voxcpm2' \
|
||||
-F 'cfg_value=2.0' \
|
||||
-F 'inference_timesteps=10' \
|
||||
-F 'normalize=false' \
|
||||
-F 'denoise=false' \
|
||||
-F 'retry_badcase=true' \
|
||||
--output voxcpm2_hifi_clone.wav
|
||||
```
|
||||
|
||||
## Speech Enhancement And Denoise
|
||||
|
||||
`POST /v1/audio/enhance`
|
||||
|
||||
`POST /v1/audio/denoise`
|
||||
|
||||
These two paths call the same service and return `enhanced.wav`. Send multipart form upload with `file=@noisy.wav`.
|
||||
|
||||
Optional:
|
||||
- `chunked`: form or query bool. Accepted truthy values include `1`, `true`, `yes`, `on`; falsy values include `0`, `false`, `no`, `off`.
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
curl -X POST https://ades.playones.com/v1/audio/enhance \
|
||||
-H 'Authorization: Bearer $OPENAI_API_KEY' \
|
||||
-F 'file=@noisy.wav' \
|
||||
-F 'chunked=false' \
|
||||
--output enhanced.wav
|
||||
```
|
||||
|
||||
Denoise alias:
|
||||
|
||||
```bash
|
||||
curl -X POST https://ades.playones.com/v1/audio/denoise \
|
||||
-H 'Authorization: Bearer $OPENAI_API_KEY' \
|
||||
-F 'file=@noisy.wav' \
|
||||
-F 'chunked=false' \
|
||||
--output denoise.wav
|
||||
```
|
||||
|
||||
## ASR Transcription
|
||||
|
||||
`POST /v1/asr/transcribe`
|
||||
|
||||
This is not audio generation. It returns JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"text": "...",
|
||||
"segments": [],
|
||||
"language": "English",
|
||||
"prompt_tokens": 0,
|
||||
"generation_tokens": 0,
|
||||
"total_tokens": 0,
|
||||
"prompt_tps": 0,
|
||||
"generation_tps": 0,
|
||||
"total_time": 0
|
||||
}
|
||||
```
|
||||
|
||||
Input is multipart form upload with `file=@speech.wav`.
|
||||
|
||||
Optional fields can be form or query params:
|
||||
- `language`: normalized aliases include `zh`, `zh-cn`, `en`, `ja`, `ko`; default is `English`
|
||||
- `temperature`: default `0.0`
|
||||
- `max_tokens`: default `8192`
|
||||
- `top_k`: default `0`
|
||||
- `top_p`: default `1.0`
|
||||
- `repetition_penalty`: default omitted
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
curl -X POST https://ades.playones.com/v1/asr/transcribe \
|
||||
-H 'Authorization: Bearer $OPENAI_API_KEY' \
|
||||
-F 'file=@speech.wav' \
|
||||
-F 'language=auto' \
|
||||
-F 'temperature=0.0' \
|
||||
-F 'max_tokens=8192' \
|
||||
-F 'top_k=0' \
|
||||
-F 'top_p=1.0'
|
||||
```
|
||||
|
||||
## Health And Models
|
||||
|
||||
```bash
|
||||
curl -H 'Authorization: Bearer $OPENAI_API_KEY' https://ades.playones.com/health
|
||||
curl -H 'Authorization: Bearer $OPENAI_API_KEY' https://ades.playones.com/models
|
||||
```
|
||||
|
||||
Model status keys:
|
||||
- `voice_design_tts`
|
||||
- `base_tts`
|
||||
- `asr`
|
||||
- `speech_enhancement`
|
||||
- `voxcpm2`
|
||||
|
||||
Models are lazy-loaded on first request, so `loaded: false` can be normal before use.
|
||||
|
||||
## Errors And Troubleshooting
|
||||
|
||||
- `401 unauthorized`: the request lacks a valid `Authorization: Bearer <key>`. Use `OPENAI_API_KEY`, pass `--token`, or get a key from `https://kinfo.playones.com`.
|
||||
- `400 provider must be one of: qwen, voxcpm2`: invalid or empty provider.
|
||||
- `400 audio file is required`: missing audio input for enhancement or ASR.
|
||||
- `400 reference audio file is required`: missing clone reference audio.
|
||||
- `400 text and ref_text are required`: Qwen clone requires both.
|
||||
- `400 text is required`: VoxCPM2 clone requires `text`.
|
||||
- `400 denoise=true requires VOXCPM2_LOAD_DENOISER=true`: retry with `denoise=false` or reconfigure server.
|
||||
- `501 VOXCPM2_BACKEND=vllm_omni is not implemented in this build`: local ARM Mac path supports `official`.
|
||||
- `503 failed to load ... model`: model package, model id, local weights, or optional `voxcpm` dependency is unavailable.
|
||||
- `504 Gateway Time-out`: the request passed HTTP validation but generation exceeded the gateway timeout. Retry after model warmup, reduce `max_tokens` for short test requests, or inspect server logs.
|
||||
- `500 Internal Server Error`: the request reached the application but failed internally. Re-run with `curl -i` and no `--output` to capture the response body; if the body is only `Internal Server Error`, inspect server logs.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Server-side:
|
||||
|
||||
```env
|
||||
OPENAI_API_KEY=your_key_here
|
||||
TTS_MODEL_ID=mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-bf16
|
||||
BASE_TTS_MODEL_ID=mlx-community/Qwen3-TTS-12Hz-0.6B-Base-8bit
|
||||
ASR_MODEL_ID=mlx-community/Qwen3-ASR-1.7B-8bit
|
||||
SE_MODEL_ID=starkdmi/MossFormer2_SE_48K_MLX
|
||||
SE_MODEL_PRECISION=int8
|
||||
VOXCPM2_MODEL_ID=openbmb/VoxCPM2
|
||||
VOXCPM2_BACKEND=official
|
||||
VOXCPM2_LOAD_DENOISER=false
|
||||
VOXCPM2_DEVICE=auto
|
||||
VOXCPM2_OPTIMIZE=true
|
||||
```
|
||||
|
||||
Client-side convenience:
|
||||
|
||||
```env
|
||||
QWEN3_VOICE_API_BASE_URL=https://ades.playones.com
|
||||
OPENAI_API_KEY=your_key_here
|
||||
```
|
||||
Health ok means API responsiveness. Models keys include voice_design_tts, base_tts, asr, speech_enhancement, reuse, stable_audio_3, yue2, voxcpm2. Loaded=false or Yue2 stopped can be normal lazy loading. Hosted API use does not require local model startup.
|
||||
@@ -1,60 +1,25 @@
|
||||
# Batch Audio Workflows
|
||||
# Sequential batch speech export
|
||||
|
||||
Use this reference for directory-scale ASR, clone, and export jobs such as game filter alert sounds.
|
||||
ASR each source, then clone speech under the source stem. Output is generated speech only, without overlay or embedded music/SFX preservation. Clean voice replacement inside mixed audio requires stems/source separation; ASR timestamps are not removal masks.
|
||||
|
||||
## Pick The Output Mode First
|
||||
~~~powershell
|
||||
python '<skill-dir>/scripts/batch_clone_generated_audio.py' --input-dir source-audio --reference-audio reference.wav --reference-text-file reference.txt --output-dir generated --provider qwen --asr-language zh --clone-language Chinese --format wav
|
||||
~~~
|
||||
|
||||
Clarify or infer the output mode before generating many files:
|
||||
## Transcripts and content
|
||||
|
||||
- **Generated-only assets**: create cloned speech files named after the source audio. This is the safest default when the user wants files they can manually replace elsewhere.
|
||||
- **Overlay/mix**: mix generated speech with the source audio only when the user explicitly asks for overlay.
|
||||
- **Speech replacement inside mixed audio**: do not promise clean removal from mixed music/SFX using ASR timestamps alone. Clean replacement requires source separation, stems, or a dedicated vocal-removal model.
|
||||
Supply reference-text/reference-text-file containing the recording's actual words. Otherwise transcribe the reference once with reference-language (default zh). Review ASR for important work; automatic recognition does not establish accuracy.
|
||||
|
||||
If the user says the old and new voices are both audible, stop overlay work and export generated-only assets instead unless they ask for source separation.
|
||||
Each source uses asr-language (default zh). Empty/invalid ASR fails that file, never substitutes its name. Recognized text is preserved apart from surrounding whitespace. Optional normalize-text collapses whitespace/adds terminal punctuation without invented interjections or rewriting semantics.
|
||||
|
||||
## Generated-Only Batch Export
|
||||
Both batch providers use transcript-guided clone. Use the single-call client for VoxCPM2 instruct-only clone; Qwen clone-language is not sent to VoxCPM2.
|
||||
|
||||
Prefer the bundled script:
|
||||
## Files and retries
|
||||
|
||||
```bash
|
||||
python ~/.codex/skills/eapil-voice/scripts/batch_clone_generated_audio.py \
|
||||
--base-url https://ades.playones.com \
|
||||
--token "$OPENAI_API_KEY" \
|
||||
--input-dir "/path/to/source-audio" \
|
||||
--reference-audio reference.wav \
|
||||
--output-dir "/path/to/generated-output" \
|
||||
--format mp3 \
|
||||
--gain-db 4 \
|
||||
--overwrite
|
||||
```
|
||||
Requests are sequential. Retries defaults to **1 total attempt**, only confirmed busy 503 permits extra attempts (max 5). Timeout, 500/502/504 and ambiguous load failures are not replayed. [Transport](api.md) explains classification. Multipart is rebuilt from the entire file.
|
||||
|
||||
The script:
|
||||
Preflight checks file presence/size, WAV limits where possible, output-name collisions, overlap with input/reference paths and required ffmpeg. Compressed decoding is server-dependent. Rename colliding stems before generation. Existing outputs skip unless overwrite is requested.
|
||||
|
||||
- ASRs each source file.
|
||||
- Normalizes alert text punctuation.
|
||||
- Uses the reference audio stem as `ref_text`.
|
||||
- Calls `/v1/voice/clone` with `provider=qwen` by default.
|
||||
- Saves `_raw_wav/` originals plus final WAV/MP3 named after the source files.
|
||||
- Writes `_manifests/` and `conversion_summary.json`.
|
||||
Raw WAVs go to _raw_wav; final outputs keep source stems. Per-file _manifests and conversion_summary.json track progress, saved after every file. Failed files do not become successes. Rerun skips completed files; review unknown outcomes before resubmitting failed files. This is local progress tracking, not server task resume. Keep private transcripts/manifests outside the repository.
|
||||
|
||||
If `OPENAI_API_KEY` is not set, ask the user to get their key from `https://kinfo.playones.com` or pass the key directly with `--token`.
|
||||
|
||||
## Text And Punctuation
|
||||
|
||||
For Chinese alert text, normalize whitespace and punctuation before clone. Add expressive punctuation when it matches the phrase. If the final character is not one of `。！？.!?~～`, append `。`.
|
||||
|
||||
## Gain And Format
|
||||
|
||||
Service clone responses are WAV. For MP3 delivery, use ffmpeg after generation:
|
||||
|
||||
```bash
|
||||
ffmpeg -y -i generated.wav -filter:a "volume=4dB" -codec:a libmp3lame -q:a 2 generated.mp3
|
||||
```
|
||||
|
||||
Use the user's requested gain literally. Do not normalize or limit unless requested.
|
||||
|
||||
## Windows Notes
|
||||
|
||||
Paths with spaces or CJK characters are common. Prefer direct shell invocation. If launching a background job with PowerShell `Start-Process`, pass a single fully quoted `-ArgumentList` string; arrays can be split incorrectly.
|
||||
|
||||
Use `subprocess.run(..., encoding="utf-8", errors="replace")` in Python when capturing ffmpeg output on Windows to avoid GBK decode crashes.
|
||||
WAV/no gain is copied; MP3 or requested gain uses ffmpeg. Export writes a temporary file and replaces the final output only after success; failure clears the temporary file and preserves any old completed output. Defaults are WAV and 0 dB. Set format mp3 or gain-db 4 only when wanted. No automatic normalization, limiter or overlay; subprocess capture uses UTF-8 for Windows CJK paths. For redirected Python output on Windows, use python -X utf8 so CJK diagnostics are encoded as UTF-8 too.
|
||||
@@ -0,0 +1,24 @@
|
||||
# Stable Audio music and sound effects
|
||||
|
||||
JSON music uses Medium + SAME-L; sfx uses Small-SFX + SAME-S. Text-to-audio only: no dedicated lyrics, reference audio, inpainting or continuation. Use Yue2 for explicit lyrics. MiniMax was removed.
|
||||
|
||||
| Field | Default | Schema / effective meaning |
|
||||
|---|---|---|
|
||||
| prompt | required | Nonblank 1–2000 characters; model uses <=256 tokenizer tokens |
|
||||
| negative_prompt | null | <=2000 characters; active only when cfg != 1 |
|
||||
| seconds | music 30, SFX 5 | >0 and <=380; current music cap 380 |
|
||||
| steps | 8 | Schema 1–64; current deployment cap **16** |
|
||||
| seed | null/random | 0–2147483647 |
|
||||
| cfg | 1 | 0–10; 1 is single-branch lower latency |
|
||||
| apg | 1 | 0–1; active only when cfg != 1 |
|
||||
|
||||
Effective limits are min(380, configured seconds) and min(64, configured steps). SFX shares the 380-second schema but its recommended window remains <=120 seconds. Schema acceptance is not a model recommendation.
|
||||
|
||||
Use one or two concrete sentences: source/genre, instruments, BPM, mood, development, space. Defaults or 4-step short previews are starting points; more steps/CFG do not guarantee quality. Fix seed to compare. Instrumental/no vocals is guidance, not a guarantee. Activate negative_prompt with cfg !=1 (e.g. 2.5), increasing compute. APG does nothing at cfg=1.
|
||||
|
||||
~~~powershell
|
||||
python '<skill-dir>/scripts/call_voice_api.py' music --prompt 'Instrumental piano and warm strings, 90 BPM, no vocals' --seconds 5 --steps 4 --seed 7 --out music.wav
|
||||
python '<skill-dir>/scripts/call_voice_api.py' sfx --prompt 'Three footsteps on wet gravel, realistic transients, no music' --seconds 5 --steps 8 --seed 7 --out sfx.wav
|
||||
~~~
|
||||
|
||||
Current output is 44.1 kHz stereo PCM16. Check duration, clipping, content and unwanted vocals by listening. Longer output increases wait/memory; music/SFX share a queue, so do not concurrently benchmark them merely to verify integration.
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
# ASR and speech enhancement
|
||||
|
||||
## ASR
|
||||
|
||||
Multipart /v1/asr/transcribe: file plus optional form fields. Default language is **English**, not automatic detection. Chinese recordings should explicitly use zh. Aliases: zh/zh-cn/zh-tw → Chinese, en → English, ja/jp → Japanese, ko/kr → Korean. Other values go to the model directly; auto is not an auto-detection guarantee.
|
||||
|
||||
Controls: temperature=0 (0–2), max_tokens=8192 (1–8192), top_k=0 (0–1000), top_p=1 (0–1), optional repetition_penalty (.1–2). Do not inherit TTS sampling settings without reason.
|
||||
|
||||
Text is the primary result; segments may be empty and does not guarantee word/sentence timestamps or speaker separation. Token/TPS and total_time are model statistics, not audio duration or total HTTP latency. Review uncertain transcription before cloning.
|
||||
|
||||
~~~powershell
|
||||
python '<skill-dir>/scripts/call_voice_api.py' asr --file speech.wav --language zh
|
||||
~~~
|
||||
|
||||
## Enhancement
|
||||
|
||||
MossFormer2 enhance/denoise are aliases. Optional chunked controls model chunking, not HTTP streaming; use form true/false. RE-USE requires only file and has no public strength/chunk/provider control.
|
||||
|
||||
~~~powershell
|
||||
python '<skill-dir>/scripts/call_voice_api.py' enhance --file noisy.wav --chunked true --out enhanced.wav
|
||||
python '<skill-dir>/scripts/call_voice_api.py' reuse --file noisy.wav --out reuse.wav
|
||||
~~~
|
||||
|
||||
Both return complete WAV and mix channels to mono; input is limited to 25 MiB/60 decoded seconds. Evaluate intelligibility, residual noise and speech damage with known speech/noise. Headers are not a quality score. These endpoints are not source separation, clean old-voice replacement inside mixed music, or stereo spatial preservation.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Speech design, reading and cloning
|
||||
|
||||
Design takes JSON text (spoken content) and instruct (voice/delivery), provider qwen by default or voxcpm2. Reading takes JSON text, Qwen only; optional speaker is meaningful only with an actual supported list. Omit it otherwise; do not invent speaker names or use instruct/provider here.
|
||||
|
||||
Clone takes multipart file (reference speech), text (new utterance) and ref_text (accurate reference transcript), with provider rules below. Text/ref_text limit 4096 characters, instruct 1024. Reject blank speech; split long content at complete sentences and verify continuity rather than assuming identical delivery.
|
||||
|
||||
## Qwen
|
||||
|
||||
TTS language names such as Chinese/English or auto go to the model; ASR alias normalization differs.
|
||||
|
||||
| Control | Default | Request range |
|
||||
|---|---|---|
|
||||
| temperature | .9 | 0–2 |
|
||||
| max_tokens | 4096 | 1–8192 |
|
||||
| top_k | 50 | 0–1000 |
|
||||
| top_p | 1 | 0–1 |
|
||||
| repetition_penalty | 1.05 | .1–2 |
|
||||
|
||||
Effective current caps: **design 512**, reading/clone **4096** tokens. Budgets are not characters/seconds; a higher accepted request cannot bypass the cap. Small budgets can truncate. Defaults are a starting point, not a quality guarantee.
|
||||
|
||||
Qwen clone requires nonblank accurate ref_text; instruct is ignored. Use the recording's actual text, a UTF-8 transcript file, or reviewed ASR, never a filename/stem.
|
||||
|
||||
~~~powershell
|
||||
python '<skill-dir>/scripts/call_voice_api.py' clone --provider qwen --file reference.wav --text '这是新的朗读内容。' --ref-text-file reference.txt --language Chinese --out clone.wav
|
||||
~~~
|
||||
|
||||
## VoxCPM2
|
||||
|
||||
Only design/clone support it. Controls: cfg_value=2 (0–10), inference_timesteps=10 (1–200), normalize=false, denoise=false, retry_badcase=true. More steps/CFG can increase time without better sound.
|
||||
|
||||
Current deployment has no built-in denoiser; denoise=true returns 400. Qwen language/sampling/token controls do not reach this provider. The helper rejects wrong-provider controls. Retry_badcase is a model option, separate from HTTP retries.
|
||||
|
||||
Clone with ref_text is transcript-guided Hi-Fi; instruct does not apply. Without ref_text, instruct guides delivery and audio supplies the reference. Design instruct is combined with text by this provider; verify the actual spoken utterance.
|
||||
|
||||
~~~powershell
|
||||
python '<skill-dir>/scripts/call_voice_api.py' clone --provider voxcpm2 --file reference.wav --text '今天的故事从这里开始。' --instruct '保持音色，轻松平静地朗读。' --denoise false --out clone.wav
|
||||
~~~
|
||||
@@ -0,0 +1,50 @@
|
||||
# Verification record
|
||||
|
||||
Date: **2026-10-06**. Target: `https://ades.playones.com`. Basis: maintainer's API usage document dated the same day, public health/models/OpenAPI inspection, actual synthetic requests and offline behavior tests. No service source or deployment configuration was modified.
|
||||
|
||||
## Live coverage
|
||||
|
||||
Existing environment credentials authenticated successfully via the legacy OPENAI_API_KEY alias. Inputs were a short Chinese utterance, its generated reference WAV, deterministic low-amplitude added noise, short music/SFX prompts and synthetic lyrics. Requests ran sequentially, without retries. All generated media and detailed diagnostics remain outside the repository.
|
||||
|
||||
| Operation | Probe and observed result |
|
||||
|---|---|
|
||||
| Qwen ordinary reading | Chinese text, 256-token budget; valid mono PCM16 24 kHz WAV, 2.64 s. This became the reference. |
|
||||
| Chinese ASR | Reference with language=zh; nonblank JSON text, known words matched. |
|
||||
| Qwen design | Short text/style, Chinese, 256-token budget; mono 24 kHz, 2.08 s. |
|
||||
| VoxCPM2 design | Short text/style, denoise=false; mono 48 kHz, 3.36 s. Generated utterance checked by ASR. |
|
||||
| Qwen clone | Accurate reference text and new utterance; mono 24 kHz, 1.60 s. Utterance checked by ASR. |
|
||||
| VoxCPM2 Hi-Fi clone | Same reference with ref_text; mono 48 kHz, 2.08 s. Utterance checked by ASR. |
|
||||
| VoxCPM2 controlled clone | No ref_text, delivery instruct, denoise=false; mono 48 kHz, 1.92 s. Utterance checked by ASR. |
|
||||
| MossFormer2 enhance | Noisy synthetic reference, chunked=true; mono 48 kHz, 2.64 s. |
|
||||
| RE-USE | Same noisy reference; mono 24 kHz, 2.64 s. |
|
||||
| Stable Audio music | 5 s, 4 steps, seed 7, cfg=1; stereo 44.1 kHz PCM16, exactly 5 s. |
|
||||
| Stable Audio SFX | Same timing/settings, footsteps prompt; stereo 44.1 kHz PCM16, exactly 5 s. |
|
||||
| Yue2 | Synthetic tagged Chinese lyrics, direct, 4 steps, seed 7, min 8/max 12 s; stereo 48 kHz PCM16, 11.999 s. |
|
||||
| Real single-file batch | Explicit accurate reference text; source ASR followed by Qwen clone succeeded, CJK output names/raw WAV/manifest created. Rerun skipped the completed output. |
|
||||
| Local MP3 | Generated batch WAV converted with explicit +4 dB, produced a usable nonempty MP3. Export was rechecked locally after the atomic-write fix. |
|
||||
|
||||
There were 18 small inference calls including four ASR content rechecks and the two batch calls. After punctuation removal, those four transcripts matched the intended utterances. All initial generated WAVs had nonzero PCM samples and no samples at PCM16 clipping extremes. These checks establish playable-format/signal/content evidence; they do **not** establish perceptual quality, speaker similarity, genre correctness, lyric completeness or denoising improvement.
|
||||
|
||||
Public GETs confirmed service responsiveness and current route/schema availability. Models not loaded or Yue2 stopped before inference were treated as lazy loading, not a failed model test. Stable Audio's OpenAPI step cap was 64 while the supplied deployment guide states effective cap 16; schema presence alone does not override runtime constraints.
|
||||
|
||||
## Local and independent validation
|
||||
|
||||
Run from the repository root:
|
||||
|
||||
```powershell
|
||||
rtk proxy python -X utf8 -m unittest discover -s plugins/eapil-voice/skills/eapil-voice/tests -v
|
||||
```
|
||||
|
||||
**22 regression tests passed**, with network mocked. Coverage includes public GET without credentials, environment precedence, Unicode/multiline/zero/false wire values, real multipart boundary/file content, provider controls, Yue2 mode/ABC constraints, WAV media/header/truncation, decoded duration, ASR shape/empty text, string/array/HTML errors, explicit busy-only retries/Retry-After, whole-file reupload, name collisions, overlap, per-file continuation, existing-output skip and failed conversion cleanup/preservation/manual rerun.
|
||||
|
||||
An independent offline use pass constructed VoxCPM2 clone, 5-second music and UTF-8 lyrics/ABC melody requests through the actual CLI. Mock HTTP drove actual multipart and batch processing; local ffmpeg exported two MP3s, WAVs and manifests. Empty ASR did not clone, a timeout was not replayed, and mixed successes/failures produced the correct summary/exit status. It discovered the partial-MP3 skip problem; atomic export and targeted failure/rerun tests fixed it. This pass did not call live models.
|
||||
|
||||
Skill frontmatter/UI metadata, local documentation links, root AGENTS.md and all five plugin manifests/assets/version were checked. IDs remain eapil-voice and version remains 1.0.0. Changed scripts also passed Ruff checks.
|
||||
|
||||
## Limits of this run
|
||||
|
||||
- No blind listening test, human voice-similarity evaluation, objective denoising score or long-audio benchmark was performed.
|
||||
- Yue2 direct was live tested; melody/full/external ABC were checked offline only. No claim of live coverage for those modes.
|
||||
- Denoise alias was checked through the client/offline routing; live enhancement used the primary enhance endpoint.
|
||||
- Deployment caps, maximum-duration behavior, denied denoise=true, invalid-token behavior, real busy/503 handling, raw-binary uploads and browser CORS were not load-tested. Their guidance comes from the supplied contract/public schema; retry/error behavior was simulated offline.
|
||||
- No broad concurrency tests, timeout resubmission, API/service reconfiguration, commit, push, tag or market publication occurred.
|
||||
@@ -0,0 +1,26 @@
|
||||
# Yue2 songs and ABC arrangements
|
||||
|
||||
JSON /v1/audio/music/yue2, current Yue2 3B Q8. Required nonblank style (1–4000 characters) and lyrics (1–30000). Unknown fields are rejected: Stable Audio prompt/seconds/cfg and legacy duration_seconds do not belong here.
|
||||
|
||||
| Field | Default | Range / meaning |
|
||||
|---|---|---|
|
||||
| mode | full | direct, melody, full |
|
||||
| abc | null | Nonblank if supplied, <=100000 characters; melody/full only |
|
||||
| steps | 8 | 1–64 acoustic denoising steps |
|
||||
| seed | 831001 | 0–9223372036854775807 |
|
||||
| cfg_scale | 1 | 0–20 |
|
||||
| min_duration_seconds | 8 | Integer 8–360 |
|
||||
| max_duration_seconds | 120 | Integer 8–360, >=minimum |
|
||||
|
||||
Direct skips ABC planning. Melody plans melody or uses supplied ABC; full plans melody/harmony or uses supplied ABC. No WAV/MP3 reference is accepted; score arrangement is not recording-to-recording singing conversion.
|
||||
|
||||
Duration bounds map to 25 semantic tokens/second; they guide stopping, not exact length or natural endings. Large minimum can force repetition; short maximum can cut lyrics. Start at minimum 8, maximum 60–120 for real songs. Short probes do not establish full-song quality; fixed-length products may need requested trimming/fades.
|
||||
|
||||
Use [Verse]/[Chorus]/[Bridge] and UTF-8 lyric/ABC files. JS Number cannot represent all allowed 64-bit seeds; keep interoperable seeds <=2147483647 or handle exact integers.
|
||||
|
||||
~~~powershell
|
||||
python '<skill-dir>/scripts/call_voice_api.py' yue2 --style 'Mandopop ballad, female vocal, piano and strings' --lyrics-file lyrics.txt --mode full --steps 8 --seed 7 --min-duration-seconds 8 --max-duration-seconds 60 --out song.wav
|
||||
python '<skill-dir>/scripts/call_voice_api.py' yue2 --style 'Acoustic folk, intimate vocal' --lyrics-file lyrics.txt --mode melody --abc-file melody.abc --max-duration-seconds 30 --out arrangement.wav
|
||||
~~~
|
||||
|
||||
Lyrics must not be blank/absent. [Instrumental] is experimental, not a guarantee; prefer Stable Audio for pure instrumentals. Current output is 48 kHz stereo PCM16. Lazy startup is handled by the service. Verify lyrics/music by listening; one direct probe does not establish melody/full/ABC.
|
||||
+284
-207
@@ -1,249 +1,326 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Batch ASR + Eapil-Voice clone and export generated speech by source name."""
|
||||
"""Sequential ASR/clone export. Generated speech only; no timeout retries."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import mimetypes
|
||||
import os
|
||||
import math
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from types import SimpleNamespace
|
||||
import call_voice_api as api
|
||||
|
||||
import requests
|
||||
|
||||
|
||||
DEFAULT_BASE_URL = "https://ades.playones.com"
|
||||
KEY_HELP = "missing API key: set OPENAI_API_KEY, pass --token directly, or get a key from https://kinfo.playones.com"
|
||||
AUDIO_EXTENSIONS = {".mp3", ".wav", ".m4a", ".flac", ".ogg", ".aac", ".wma"}
|
||||
END_PUNCTUATION = "。！？.!?~～"
|
||||
SAFE_NAME_RE = re.compile(r'[<>:"/\\|?*\x00-\x1f]')
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--base-url", default=os.getenv("QWEN3_VOICE_API_BASE_URL", DEFAULT_BASE_URL))
|
||||
parser.add_argument("--token", default=os.getenv("OPENAI_API_KEY"), required=False)
|
||||
parser.add_argument("--input-dir", required=True, help="Directory of source alert audio files")
|
||||
parser.add_argument("--reference-audio", required=True, help="Voice clone reference audio")
|
||||
parser.add_argument("--output-dir", required=True, help="Directory for generated files")
|
||||
parser.add_argument("--format", choices=["wav", "mp3"], default="wav")
|
||||
parser.add_argument("--gain-db", type=float, default=0.0, help="Output gain applied during export")
|
||||
parser.add_argument("--provider", choices=["qwen", "voxcpm2"], default="qwen")
|
||||
parser.add_argument("--asr-language", default="zh")
|
||||
parser.add_argument("--clone-language", default="auto")
|
||||
parser.add_argument("--timeout", type=int, default=900)
|
||||
parser.add_argument("--retries", type=int, default=3)
|
||||
parser.add_argument("--overwrite", action="store_true")
|
||||
return parser.parse_args()
|
||||
def parse_args(argv=None):
|
||||
p = argparse.ArgumentParser(description=__doc__)
|
||||
p.add_argument("--base-url", default=api.env_base_url())
|
||||
p.add_argument("--token", help="legacy direct token; prefer environment")
|
||||
p.add_argument("--api-key-env")
|
||||
p.add_argument("--input-dir", required=True)
|
||||
p.add_argument("--reference-audio", required=True)
|
||||
ref = p.add_mutually_exclusive_group()
|
||||
ref.add_argument("--reference-text")
|
||||
ref.add_argument("--reference-text-file")
|
||||
p.add_argument("--reference-language", default="zh")
|
||||
p.add_argument("--output-dir", required=True)
|
||||
p.add_argument("--format", choices=["wav", "mp3"], default="wav")
|
||||
p.add_argument("--gain-db", type=float, default=0)
|
||||
p.add_argument("--provider", choices=["qwen", "voxcpm2"], default="qwen")
|
||||
p.add_argument("--asr-language", default="zh")
|
||||
p.add_argument("--clone-language", default="auto")
|
||||
p.add_argument("--timeout", type=float, default=600)
|
||||
p.add_argument(
|
||||
"--retries",
|
||||
type=int,
|
||||
default=1,
|
||||
help="maximum attempts for explicit busy 503 only; default 1",
|
||||
)
|
||||
p.add_argument(
|
||||
"--normalize-text",
|
||||
action="store_true",
|
||||
help="opt in to whitespace normalization and terminal punctuation",
|
||||
)
|
||||
p.add_argument("--overwrite", action="store_true")
|
||||
return p.parse_args(argv)
|
||||
|
||||
|
||||
def log(message: str) -> None:
|
||||
print(message, flush=True)
|
||||
|
||||
|
||||
def bearer_token(token: str | None) -> str:
|
||||
if not token:
|
||||
raise SystemExit(KEY_HELP)
|
||||
token = token.strip()
|
||||
if token.lower().startswith("bearer "):
|
||||
token = token[7:].strip()
|
||||
if not token:
|
||||
raise SystemExit(KEY_HELP)
|
||||
return token
|
||||
|
||||
|
||||
def request_headers(token: str | None) -> dict[str, str]:
|
||||
return {"Authorization": f"Bearer {bearer_token(token)}"}
|
||||
|
||||
|
||||
def audio_files(input_dir: Path) -> list[Path]:
|
||||
def audio_files(input_dir):
|
||||
return sorted(
|
||||
[path for path in input_dir.iterdir() if path.is_file() and path.suffix.lower() in AUDIO_EXTENSIONS],
|
||||
key=lambda path: path.name.casefold(),
|
||||
(
|
||||
p
|
||||
for p in input_dir.iterdir()
|
||||
if p.is_file() and p.suffix.lower() in AUDIO_EXTENSIONS
|
||||
),
|
||||
key=lambda p: p.name.casefold(),
|
||||
)
|
||||
|
||||
|
||||
def safe_stem(text: str) -> str:
|
||||
cleaned = SAFE_NAME_RE.sub("_", text).strip().strip(".")
|
||||
return cleaned or "untitled"
|
||||
def safe_stem(text):
|
||||
name = SAFE_NAME_RE.sub("_", text).strip().strip(".")
|
||||
if not name:
|
||||
name = "untitled"
|
||||
if re.fullmatch(r"(CON|PRN|AUX|NUL|COM[1-9]|LPT[1-9])", name, re.I):
|
||||
name = "_" + name
|
||||
return name
|
||||
|
||||
|
||||
def normalize_text(raw: str, fallback: str) -> str:
|
||||
text = " ".join((raw or "").replace("\u3000", " ").split()).strip() or fallback.strip()
|
||||
text = text.translate(str.maketrans({",": "，", ":": "：", ";": "；", "(": "（", ")": "）"}))
|
||||
if text.startswith("哇") and len(text) > 1 and text[1] not in "，,。！？!?":
|
||||
text = "哇，" + text[1:]
|
||||
if text.startswith("拜托") and "，" not in text[:4] and len(text) > 2:
|
||||
text = "拜托，" + text[2:]
|
||||
if text and text[-1] not in END_PUNCTUATION:
|
||||
def normalize_text(raw):
|
||||
if not isinstance(raw, str) or not raw.strip():
|
||||
raise ValueError(
|
||||
"ASR returned no usable transcript; file names are not transcripts"
|
||||
)
|
||||
text = " ".join(raw.split())
|
||||
if text[-1] not in "。！？.!?~～":
|
||||
text += "。"
|
||||
return text
|
||||
|
||||
|
||||
def request_with_retries(
|
||||
method: str,
|
||||
url: str,
|
||||
*,
|
||||
retries: int,
|
||||
timeout: int,
|
||||
**kwargs: Any,
|
||||
) -> requests.Response:
|
||||
last_error: Exception | None = None
|
||||
for attempt in range(1, retries + 1):
|
||||
def call_with_busy_retry(operation, attempts):
|
||||
for index in range(attempts):
|
||||
try:
|
||||
response = requests.request(method, url, timeout=timeout, **kwargs)
|
||||
if response.status_code < 500:
|
||||
return response
|
||||
last_error = RuntimeError(f"HTTP {response.status_code}: {response.text[:500]}")
|
||||
except Exception as exc: # noqa: BLE001
|
||||
last_error = exc
|
||||
wait = min(30, 2**attempt)
|
||||
log(f"retry attempt={attempt}/{retries} wait={wait}s error={last_error}")
|
||||
time.sleep(wait)
|
||||
if last_error:
|
||||
raise last_error
|
||||
raise RuntimeError("request failed")
|
||||
return operation()
|
||||
except api.APIError as exc:
|
||||
if not exc.busy or index + 1 >= attempts:
|
||||
raise
|
||||
delay = (
|
||||
exc.retry_after
|
||||
if exc.retry_after is not None
|
||||
else min(30, 2 ** (index + 1))
|
||||
)
|
||||
if delay > 60:
|
||||
raise # Leave unusually long backoff to the calling application.
|
||||
time.sleep(delay)
|
||||
raise ValueError("attempt count must be at least 1")
|
||||
|
||||
|
||||
def transcribe(args: argparse.Namespace, source: Path, headers: dict[str, str]) -> dict[str, Any]:
|
||||
mime = mimetypes.guess_type(source.name)[0] or "application/octet-stream"
|
||||
with source.open("rb") as handle:
|
||||
response = request_with_retries(
|
||||
"POST",
|
||||
f"{args.base_url.rstrip('/')}/v1/asr/transcribe",
|
||||
retries=args.retries,
|
||||
timeout=args.timeout,
|
||||
headers=headers,
|
||||
files={"file": (source.name, handle, mime)},
|
||||
data={"language": args.asr_language, "temperature": "0"},
|
||||
)
|
||||
if response.status_code != 200:
|
||||
raise RuntimeError(f"ASR HTTP {response.status_code}: {response.text[:1000]}")
|
||||
return response.json()
|
||||
def transcribe(args, source):
|
||||
# Each optional busy retry rebuilds multipart from the complete file.
|
||||
data = call_with_busy_retry(
|
||||
lambda: api.post_multipart(
|
||||
args,
|
||||
api.PATHS["asr"],
|
||||
{"language": args.asr_language, "temperature": 0},
|
||||
str(source),
|
||||
),
|
||||
args.retries,
|
||||
)
|
||||
result = json.loads(data)
|
||||
if not isinstance(result.get("text"), str) or not result["text"].strip():
|
||||
raise ValueError("ASR returned an empty transcript")
|
||||
return result
|
||||
|
||||
|
||||
def clone(args: argparse.Namespace, reference_audio: Path, ref_text: str, text: str, headers: dict[str, str]) -> bytes:
|
||||
mime = mimetypes.guess_type(reference_audio.name)[0] or "audio/wav"
|
||||
with reference_audio.open("rb") as handle:
|
||||
response = request_with_retries(
|
||||
"POST",
|
||||
f"{args.base_url.rstrip('/')}/v1/voice/clone",
|
||||
retries=args.retries,
|
||||
timeout=args.timeout,
|
||||
headers=headers,
|
||||
files={"file": (reference_audio.name, handle, mime)},
|
||||
data={
|
||||
"text": text,
|
||||
"ref_text": ref_text,
|
||||
"language": args.clone_language,
|
||||
"provider": args.provider,
|
||||
},
|
||||
)
|
||||
if response.status_code != 200:
|
||||
raise RuntimeError(f"CLONE HTTP {response.status_code}: {response.text[:1000]}")
|
||||
return response.content
|
||||
|
||||
|
||||
def export_audio(raw_wav: Path, output: Path, gain_db: float, output_format: str) -> None:
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
if output_format == "wav":
|
||||
command = [
|
||||
"ffmpeg",
|
||||
"-hide_banner",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-y",
|
||||
"-i",
|
||||
str(raw_wav),
|
||||
"-filter:a",
|
||||
f"volume={gain_db:.6f}dB",
|
||||
str(output),
|
||||
]
|
||||
def reference_transcript(args):
|
||||
if args.reference_text_file:
|
||||
text = Path(args.reference_text_file).read_text(encoding="utf-8-sig")
|
||||
elif args.reference_text is not None:
|
||||
text = args.reference_text
|
||||
else:
|
||||
command = [
|
||||
"ffmpeg",
|
||||
"-hide_banner",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-y",
|
||||
"-i",
|
||||
str(raw_wav),
|
||||
"-filter:a",
|
||||
f"volume={gain_db:.6f}dB",
|
||||
"-codec:a",
|
||||
"libmp3lame",
|
||||
"-q:a",
|
||||
"2",
|
||||
str(output),
|
||||
ref_args = SimpleNamespace(**vars(args))
|
||||
ref_args.asr_language = args.reference_language
|
||||
text = transcribe(ref_args, Path(args.reference_audio))["text"]
|
||||
if not isinstance(text, str) or not text.strip() or len(text) > 4096:
|
||||
raise ValueError(
|
||||
"reference transcript must be nonblank and at most 4096 characters"
|
||||
)
|
||||
return text.strip()
|
||||
|
||||
|
||||
def clone(args, ref_text, text):
|
||||
fields = {"text": text, "ref_text": ref_text, "provider": args.provider}
|
||||
if args.provider == "qwen":
|
||||
fields["language"] = args.clone_language
|
||||
api.validate_payload("clone", fields)
|
||||
return call_with_busy_retry(
|
||||
lambda: api.post_multipart(
|
||||
args, api.PATHS["clone"], fields, args.reference_audio
|
||||
),
|
||||
args.retries,
|
||||
)
|
||||
|
||||
|
||||
def export_audio(raw_wav, output, gain_db, output_format):
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
with tempfile.NamedTemporaryFile(
|
||||
dir=output.parent, prefix=".audio-export-", suffix=output.suffix, delete=False
|
||||
) as handle:
|
||||
temporary = Path(handle.name)
|
||||
try:
|
||||
if output_format == "wav" and gain_db == 0:
|
||||
api.wav_info(raw_wav.read_bytes())
|
||||
temporary.write_bytes(raw_wav.read_bytes())
|
||||
else:
|
||||
command = [
|
||||
"ffmpeg",
|
||||
"-hide_banner",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-y",
|
||||
"-i",
|
||||
str(raw_wav),
|
||||
]
|
||||
if gain_db != 0:
|
||||
command += ["-filter:a", f"volume={gain_db:.6f}dB"]
|
||||
if output_format == "mp3":
|
||||
command += ["-codec:a", "libmp3lame", "-q:a", "2"]
|
||||
command.append(str(temporary))
|
||||
if shutil.which("rtk"):
|
||||
command = ["rtk", "proxy", *command]
|
||||
result = subprocess.run(
|
||||
command,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
)
|
||||
if result.returncode or not temporary.stat().st_size:
|
||||
raise RuntimeError("ffmpeg export failed; inspect conversion locally")
|
||||
if output_format == "wav":
|
||||
api.wav_info(temporary.read_bytes())
|
||||
temporary.replace(output)
|
||||
finally:
|
||||
temporary.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def prepare(args):
|
||||
if (
|
||||
not math.isfinite(args.gain_db)
|
||||
or not math.isfinite(args.timeout)
|
||||
or args.timeout <= 0
|
||||
or args.retries < 1
|
||||
or args.retries > 5
|
||||
):
|
||||
raise ValueError("invalid gain/timeout; retries must be 1–5")
|
||||
api.build_url(args.base_url, api.PATHS["clone"])
|
||||
source = Path(args.input_dir)
|
||||
if not source.is_dir():
|
||||
raise ValueError("input directory does not exist")
|
||||
api.preflight_audio(
|
||||
args.reference_audio,
|
||||
args.provider == "qwen"
|
||||
or not (args.reference_text or args.reference_text_file),
|
||||
)
|
||||
if (args.format == "mp3" or args.gain_db != 0) and not shutil.which("ffmpeg"):
|
||||
raise ValueError("ffmpeg is required for MP3 or gain conversion")
|
||||
files = audio_files(source)
|
||||
names = [safe_stem(p.stem).casefold() for p in files]
|
||||
if len(names) != len(set(names)):
|
||||
raise ValueError(
|
||||
"source names collide after output normalization; rename before generating"
|
||||
)
|
||||
output = Path(args.output_dir)
|
||||
inputs = {p.resolve() for p in files} | {Path(args.reference_audio).resolve()}
|
||||
for p in files:
|
||||
for target in (
|
||||
output / f"{safe_stem(p.stem)}.{args.format}",
|
||||
output / "_raw_wav" / f"{safe_stem(p.stem)}.wav",
|
||||
):
|
||||
if target.resolve() in inputs:
|
||||
raise ValueError("output must not overwrite input/reference audio")
|
||||
api.preflight_audio(str(p))
|
||||
return files
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
args = parse_args(argv)
|
||||
try:
|
||||
files = prepare(args)
|
||||
import os
|
||||
|
||||
args.token = args.token or (
|
||||
os.getenv(args.api_key_env) if args.api_key_env else api.env_token()
|
||||
)
|
||||
api.bearer_token(args.token)
|
||||
output_dir = Path(args.output_dir)
|
||||
raw_dir, manifest_dir = output_dir / "_raw_wav", output_dir / "_manifests"
|
||||
for path in (output_dir, raw_dir, manifest_dir):
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
pending = [
|
||||
p
|
||||
for p in files
|
||||
if args.overwrite
|
||||
or not (output_dir / f"{safe_stem(p.stem)}.{args.format}").exists()
|
||||
]
|
||||
result = subprocess.run(command, capture_output=True, text=True, encoding="utf-8", errors="replace")
|
||||
if result.returncode != 0:
|
||||
raise RuntimeError(result.stderr[-2000:] or result.stdout[-2000:] or "ffmpeg failed")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = parse_args()
|
||||
if not args.token:
|
||||
log(KEY_HELP)
|
||||
return 2
|
||||
|
||||
input_dir = Path(args.input_dir)
|
||||
reference_audio = Path(args.reference_audio)
|
||||
output_dir = Path(args.output_dir)
|
||||
raw_dir = output_dir / "_raw_wav"
|
||||
manifest_dir = output_dir / "_manifests"
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
raw_dir.mkdir(parents=True, exist_ok=True)
|
||||
manifest_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
headers = request_headers(args.token)
|
||||
files = audio_files(input_dir)
|
||||
ref_text = reference_audio.stem
|
||||
summary: list[dict[str, Any]] = []
|
||||
|
||||
log(f"base_url={args.base_url.rstrip('/')}")
|
||||
log(f"input_files={len(files)} output_dir={output_dir} format={args.format} gain_db={args.gain_db}")
|
||||
|
||||
for index, source in enumerate(files, 1):
|
||||
output = output_dir / f"{safe_stem(source.stem)}.{args.format}"
|
||||
raw_output = raw_dir / f"{safe_stem(source.stem)}.wav"
|
||||
try:
|
||||
ref_text = reference_transcript(args) if pending else ""
|
||||
summary = []
|
||||
for source in files:
|
||||
output = output_dir / f"{safe_stem(source.stem)}.{args.format}"
|
||||
raw = raw_dir / f"{safe_stem(source.stem)}.wav"
|
||||
if output.exists() and not args.overwrite:
|
||||
record = {"source": str(source), "output": str(output), "status": "skipped", "reason": "exists"}
|
||||
else:
|
||||
asr = transcribe(args, source, headers)
|
||||
text = normalize_text(str(asr.get("text") or ""), source.stem)
|
||||
raw_output.write_bytes(clone(args, reference_audio, ref_text, text, headers))
|
||||
export_audio(raw_output, output, args.gain_db, args.format)
|
||||
record = {
|
||||
"source": str(source),
|
||||
"output": str(output),
|
||||
"raw_wav": str(raw_output),
|
||||
"status": "ok",
|
||||
"text": text,
|
||||
"asr": asr,
|
||||
"status": "skipped",
|
||||
"reason": "exists",
|
||||
}
|
||||
(manifest_dir / f"{safe_stem(source.stem)}.json").write_text(
|
||||
json.dumps(record, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
log(f"{index}/{len(files)} {record['status'].upper()} {source.name}")
|
||||
except Exception as exc: # noqa: BLE001
|
||||
record = {"source": str(source), "status": "error", "error": str(exc)}
|
||||
log(f"{index}/{len(files)} ERROR {source.name}: {exc}")
|
||||
summary.append(record)
|
||||
|
||||
summary_path = output_dir / "conversion_summary.json"
|
||||
summary_path.write_text(json.dumps(summary, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
errors = sum(1 for record in summary if record.get("status") == "error")
|
||||
log(f"done files={len(files)} errors={errors} summary={summary_path}")
|
||||
return 1 if errors else 0
|
||||
else:
|
||||
try:
|
||||
asr = transcribe(args, source)
|
||||
text = (
|
||||
normalize_text(asr["text"])
|
||||
if args.normalize_text
|
||||
else asr["text"].strip()
|
||||
)
|
||||
audio = clone(args, ref_text, text)
|
||||
api.write_output(str(raw), audio)
|
||||
export_audio(raw, output, args.gain_db, args.format)
|
||||
record = {
|
||||
"source": str(source),
|
||||
"output": str(output),
|
||||
"raw_wav": str(raw),
|
||||
"status": "ok",
|
||||
"text": text,
|
||||
"reference_text": ref_text,
|
||||
"asr": asr,
|
||||
"audio": api.wav_info(audio),
|
||||
}
|
||||
(manifest_dir / f"{safe_stem(source.stem)}.json").write_text(
|
||||
json.dumps(record, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
except (api.APIError, OSError, ValueError, RuntimeError) as exc:
|
||||
record = {
|
||||
"source": str(source),
|
||||
"status": "error",
|
||||
"error_type": type(exc).__name__,
|
||||
}
|
||||
if isinstance(exc, api.APIError):
|
||||
record["http_status"] = exc.status
|
||||
summary.append(record)
|
||||
# Persist progress after each file; existing successful outputs are skipped on rerun.
|
||||
(output_dir / "conversion_summary.json").write_text(
|
||||
json.dumps(summary, ensure_ascii=False, indent=2), encoding="utf-8"
|
||||
)
|
||||
print(
|
||||
json.dumps(
|
||||
{"file": source.name, "status": record["status"]},
|
||||
ensure_ascii=False,
|
||||
),
|
||||
flush=True,
|
||||
)
|
||||
if not files:
|
||||
(output_dir / "conversion_summary.json").write_text("[]", encoding="utf-8")
|
||||
return int(any(r["status"] == "error" for r in summary))
|
||||
except (api.APIError, OSError, ValueError, RuntimeError) as exc:
|
||||
report = {"ok": False, "error_type": type(exc).__name__}
|
||||
if isinstance(exc, api.APIError):
|
||||
report["http_status"] = exc.status
|
||||
else:
|
||||
message = str(exc)
|
||||
if args.token:
|
||||
message = message.replace(args.token, "[redacted]")
|
||||
report["message"] = message[:240]
|
||||
print(json.dumps(report), file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
raise SystemExit(main())
|
||||
@@ -1,42 +1,107 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Dependency-free client for the hosted Eapil-Voice API."""
|
||||
"""Dependency-free EAPIL audio client. One request, no inference retries."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import mimetypes
|
||||
import os
|
||||
import struct
|
||||
import sys
|
||||
import uuid
|
||||
from http.client import HTTPException
|
||||
from pathlib import Path
|
||||
from urllib.error import HTTPError, URLError
|
||||
from urllib.parse import urljoin
|
||||
from urllib.parse import urlsplit
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
|
||||
DEFAULT_BASE_URL = "https://ades.playones.com"
|
||||
KEY_HELP = "missing API key: set OPENAI_API_KEY, pass --token directly, or get a key from https://kinfo.playones.com"
|
||||
MAX_UPLOAD_BYTES = 26_214_400
|
||||
PATHS = {
|
||||
"health": "/health",
|
||||
"models": "/models",
|
||||
"design": "/v1/voice/design",
|
||||
"synthesize": "/v1/voice/synthesize",
|
||||
"clone": "/v1/voice/clone",
|
||||
"asr": "/v1/asr/transcribe",
|
||||
"enhance": "/v1/audio/enhance",
|
||||
"denoise": "/v1/audio/denoise",
|
||||
"reuse": "/v1/audio/reuse",
|
||||
"music": "/v1/audio/music",
|
||||
"sfx": "/v1/audio/sfx",
|
||||
"yue2": "/v1/audio/music/yue2",
|
||||
}
|
||||
QWEN = ("language", "temperature", "max_tokens", "top_k", "top_p", "repetition_penalty")
|
||||
VOX = ("cfg_value", "inference_timesteps", "normalize", "denoise", "retry_badcase")
|
||||
|
||||
|
||||
def env_base_url() -> str:
|
||||
return os.getenv("QWEN3_VOICE_API_BASE_URL", DEFAULT_BASE_URL).rstrip("/")
|
||||
class APIError(RuntimeError):
|
||||
def __init__(self, status, detail, retry_after=None):
|
||||
self.status, self.detail = status, detail
|
||||
try:
|
||||
delay = float(retry_after)
|
||||
self.retry_after = delay if math.isfinite(delay) and delay >= 0 else None
|
||||
except (TypeError, ValueError):
|
||||
self.retry_after = None
|
||||
text = detail.lower() if isinstance(detail, str) else ""
|
||||
self.busy = (
|
||||
status == 503
|
||||
and ("busy" in text or "already running" in text)
|
||||
and not any(
|
||||
word in text
|
||||
for word in ("failed", "error", "load", "connect", "timeout")
|
||||
)
|
||||
)
|
||||
super().__init__(f"HTTP {status}; provider detail omitted, no automatic retry")
|
||||
|
||||
|
||||
def env_token() -> str | None:
|
||||
return os.getenv("OPENAI_API_KEY")
|
||||
def env_base_url():
|
||||
return (
|
||||
os.getenv("AUDIO_API_BASE")
|
||||
or os.getenv("QWEN3_VOICE_API_BASE_URL")
|
||||
or DEFAULT_BASE_URL
|
||||
)
|
||||
|
||||
|
||||
def add_common_args(parser: argparse.ArgumentParser) -> None:
|
||||
parser.add_argument("--base-url", default=env_base_url(), help="API base URL")
|
||||
parser.add_argument("--token", default=env_token(), help="Bearer API key; defaults to OPENAI_API_KEY")
|
||||
def env_token():
|
||||
return os.getenv("AUDIO_API_TOKEN") or os.getenv("OPENAI_API_KEY")
|
||||
|
||||
|
||||
def clean_payload(values: dict[str, object | None]) -> dict[str, object]:
|
||||
return {key: value for key, value in values.items() if value is not None}
|
||||
def bearer_token(token):
|
||||
value = (token or "").strip()
|
||||
if value.lower().startswith("bearer "):
|
||||
value = value[7:].strip()
|
||||
if not value or "\r" in value or "\n" in value:
|
||||
raise ValueError(
|
||||
"set AUDIO_API_TOKEN (legacy OPENAI_API_KEY supported), or --api-key-env"
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def bool_value(raw: str | bool | None) -> bool | None:
|
||||
def build_url(base_url, path):
|
||||
parts = urlsplit(base_url)
|
||||
if (
|
||||
parts.scheme not in {"http", "https"}
|
||||
or not parts.netloc
|
||||
or parts.username
|
||||
or parts.password
|
||||
or parts.query
|
||||
or parts.fragment
|
||||
):
|
||||
raise ValueError(
|
||||
"base URL must be an HTTP(S) service root without credentials/query/fragment"
|
||||
)
|
||||
if parts.path.rstrip("/").endswith("/v1"):
|
||||
raise ValueError("audio base URL is the service root, without /v1")
|
||||
return base_url.rstrip("/") + path
|
||||
|
||||
|
||||
def clean_payload(values):
|
||||
return {k: v for k, v in values.items() if v is not None}
|
||||
|
||||
|
||||
def bool_value(raw):
|
||||
if raw is None or isinstance(raw, bool):
|
||||
return raw
|
||||
value = raw.strip().lower()
|
||||
@@ -44,326 +109,458 @@ def bool_value(raw: str | bool | None) -> bool | None:
|
||||
return True
|
||||
if value in {"0", "false", "no", "off"}:
|
||||
return False
|
||||
raise argparse.ArgumentTypeError(f"invalid boolean value: {raw}")
|
||||
raise argparse.ArgumentTypeError("boolean must be true or false")
|
||||
|
||||
|
||||
def bearer_token(token: str | None) -> str:
|
||||
if not token:
|
||||
raise SystemExit(KEY_HELP)
|
||||
token = token.strip()
|
||||
if token.lower().startswith("bearer "):
|
||||
token = token[7:].strip()
|
||||
if not token:
|
||||
raise SystemExit(KEY_HELP)
|
||||
return token
|
||||
def upload_bytes(file_path):
|
||||
path = Path(file_path)
|
||||
if not path.is_file() or not 0 < path.stat().st_size <= MAX_UPLOAD_BYTES:
|
||||
raise ValueError("upload must be a nonempty file of at most 25 MiB")
|
||||
return path.read_bytes()
|
||||
|
||||
|
||||
def request_headers(token: str | None, extra: dict[str, str] | None = None) -> dict[str, str]:
|
||||
headers = dict(extra or {})
|
||||
headers["Authorization"] = f"Bearer {bearer_token(token)}"
|
||||
return headers
|
||||
def preflight_audio(file_path, limit_seconds=True):
|
||||
data = upload_bytes(file_path)
|
||||
if Path(file_path).suffix.lower() == ".wav":
|
||||
info = wav_info(data)
|
||||
if limit_seconds and info["frames"] / info["sample_rate"] > 60:
|
||||
raise ValueError("decoded WAV exceeds the current 60-second input limit")
|
||||
|
||||
|
||||
def build_url(base_url: str, path: str) -> str:
|
||||
return urljoin(base_url.rstrip("/") + "/", path.lstrip("/"))
|
||||
def multipart_body(*, fields, file_path, file_field="file"):
|
||||
audio = upload_bytes(file_path)
|
||||
boundary = "----eapil-audio-" + uuid.uuid4().hex
|
||||
parts = []
|
||||
for name, value in clean_payload(fields).items():
|
||||
if any(c in name for c in '\r\n"'):
|
||||
raise ValueError("invalid multipart field name")
|
||||
value = str(value).lower() if isinstance(value, bool) else str(value)
|
||||
parts.append(
|
||||
f'--{boundary}\r\nContent-Disposition: form-data; name="{name}"\r\n\r\n{value}\r\n'.encode(
|
||||
"utf-8"
|
||||
)
|
||||
)
|
||||
filename = (
|
||||
Path(file_path).name.replace('"', "_").replace("\r", "_").replace("\n", "_")
|
||||
)
|
||||
mime = mimetypes.guess_type(filename)[0] or "application/octet-stream"
|
||||
parts.append(
|
||||
f'--{boundary}\r\nContent-Disposition: form-data; name="{file_field}"; filename="{filename}"\r\nContent-Type: {mime}\r\n\r\n'.encode(
|
||||
"utf-8"
|
||||
)
|
||||
)
|
||||
parts.extend((audio, f"\r\n--{boundary}--\r\n".encode("ascii")))
|
||||
return b"".join(parts), f"multipart/form-data; boundary={boundary}"
|
||||
|
||||
|
||||
def wav_info(data):
|
||||
"""Check complete little-endian PCM/float WAV, including extensible headers."""
|
||||
if len(data) < 12 or data[:4] != b"RIFF" or data[8:12] != b"WAVE":
|
||||
raise ValueError("response is not RIFF/WAVE")
|
||||
end = struct.unpack_from("<I", data, 4)[0] + 8
|
||||
if end != len(data):
|
||||
raise ValueError("WAV size mismatch or truncated response")
|
||||
fmt, audio, pos = None, None, 12
|
||||
while pos + 8 <= end:
|
||||
kind, size = data[pos : pos + 4], struct.unpack_from("<I", data, pos + 4)[0]
|
||||
start, stop = pos + 8, pos + 8 + size
|
||||
if stop > end:
|
||||
raise ValueError("truncated WAV chunk")
|
||||
if kind == b"fmt ":
|
||||
if size < 16:
|
||||
raise ValueError("invalid WAV format")
|
||||
fmt = struct.unpack_from("<HHIIHH", data, start)
|
||||
if fmt[0] == 0xFFFE:
|
||||
if size < 40 or struct.unpack_from("<H", data, start + 16)[0] < 22:
|
||||
raise ValueError("invalid extensible WAV")
|
||||
fmt = (struct.unpack_from("<H", data, start + 24)[0], *fmt[1:])
|
||||
elif kind == b"data":
|
||||
audio = size
|
||||
pos = stop + size % 2
|
||||
if pos != end or fmt is None or not audio:
|
||||
raise ValueError("WAV lacks complete format/audio chunks")
|
||||
code, channels, rate, byte_rate, align, bits = fmt
|
||||
if (
|
||||
code not in {1, 3}
|
||||
or not channels
|
||||
or not rate
|
||||
or bits not in {8, 16, 24, 32, 64}
|
||||
or align != channels * bits // 8
|
||||
or byte_rate != rate * align
|
||||
or audio % align
|
||||
):
|
||||
raise ValueError("invalid or unsupported WAV sample format")
|
||||
return {
|
||||
"sample_rate": rate,
|
||||
"channels": channels,
|
||||
"bits": bits,
|
||||
"duration_s": round(audio / byte_rate, 3),
|
||||
"frames": audio // align,
|
||||
"bytes": len(data),
|
||||
}
|
||||
|
||||
|
||||
def http_request(
|
||||
*,
|
||||
method: str,
|
||||
base_url: str,
|
||||
path: str,
|
||||
token: str | None,
|
||||
body: bytes | None = None,
|
||||
headers: dict[str, str] | None = None,
|
||||
) -> bytes:
|
||||
request = Request(
|
||||
build_url(base_url, path),
|
||||
data=body,
|
||||
method=method,
|
||||
headers=request_headers(token, headers),
|
||||
)
|
||||
method,
|
||||
base_url,
|
||||
path,
|
||||
token=None,
|
||||
body=None,
|
||||
headers=None,
|
||||
timeout=600,
|
||||
expected="json",
|
||||
):
|
||||
if not math.isfinite(timeout) or timeout <= 0:
|
||||
raise ValueError("timeout must be positive and finite")
|
||||
hdr = dict(headers or {})
|
||||
if path.startswith("/v1/"):
|
||||
hdr["Authorization"] = "Bearer " + bearer_token(token)
|
||||
hdr["Accept"] = "audio/wav" if expected == "wav" else "application/json"
|
||||
req = Request(build_url(base_url, path), data=body, method=method, headers=hdr)
|
||||
try:
|
||||
with urlopen(request) as response:
|
||||
return response.read()
|
||||
with urlopen(req, timeout=timeout) as response:
|
||||
media = (
|
||||
response.headers.get("Content-Type", "")
|
||||
.split(";", 1)[0]
|
||||
.lower()
|
||||
.strip()
|
||||
)
|
||||
data = response.read()
|
||||
except HTTPError as exc:
|
||||
detail = exc.read().decode("utf-8", errors="replace")
|
||||
raise SystemExit(f"HTTP {exc.code} {exc.reason}: {detail}") from exc
|
||||
except URLError as exc:
|
||||
raise SystemExit(f"request failed: {exc.reason}") from exc
|
||||
raw = exc.read().decode("utf-8", "replace")
|
||||
try:
|
||||
parsed = json.loads(raw)
|
||||
detail = (
|
||||
parsed.get("detail", parsed) if isinstance(parsed, dict) else parsed
|
||||
)
|
||||
except ValueError:
|
||||
detail = "non-JSON error response"
|
||||
raise APIError(
|
||||
exc.code, detail, exc.headers.get("Retry-After") if exc.headers else None
|
||||
) from exc
|
||||
except HTTPException as exc:
|
||||
raise RuntimeError(
|
||||
"incomplete HTTP response; outcome unknown, no automatic retry"
|
||||
) from exc
|
||||
if expected == "wav":
|
||||
if media not in {"audio/wav", "audio/x-wav", "audio/wave"}:
|
||||
raise ValueError("successful response is not WAV media")
|
||||
wav_info(data)
|
||||
else:
|
||||
if media != "application/json":
|
||||
raise ValueError("successful response is not JSON media")
|
||||
value = json.loads(data)
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError("expected a JSON object")
|
||||
if path == PATHS["asr"] and not isinstance(value.get("text"), str):
|
||||
raise ValueError("ASR response must contain string text")
|
||||
return data
|
||||
|
||||
|
||||
def post_json(args: argparse.Namespace, path: str, payload: dict[str, object]) -> bytes:
|
||||
body = json.dumps(payload, ensure_ascii=False).encode("utf-8")
|
||||
def post_json(args, path, payload):
|
||||
return http_request(
|
||||
method="POST",
|
||||
base_url=args.base_url,
|
||||
path=path,
|
||||
token=args.token,
|
||||
body=body,
|
||||
body=json.dumps(payload, ensure_ascii=False, allow_nan=False).encode("utf-8"),
|
||||
headers={"Content-Type": "application/json"},
|
||||
timeout=args.timeout,
|
||||
expected="wav",
|
||||
)
|
||||
|
||||
|
||||
def multipart_body(
|
||||
*,
|
||||
fields: dict[str, object | None],
|
||||
file_field: str | None = None,
|
||||
file_path: str | None = None,
|
||||
) -> tuple[bytes, str]:
|
||||
boundary = f"----eapil-voice-{uuid.uuid4().hex}"
|
||||
chunks: list[bytes] = []
|
||||
|
||||
def add_line(value: str) -> None:
|
||||
chunks.append(value.encode("utf-8"))
|
||||
|
||||
def field_value(value: object) -> str:
|
||||
if isinstance(value, bool):
|
||||
return "true" if value else "false"
|
||||
return str(value)
|
||||
|
||||
for name, value in clean_payload(fields).items():
|
||||
add_line(f"--{boundary}\r\n")
|
||||
add_line(f'Content-Disposition: form-data; name="{name}"\r\n\r\n')
|
||||
add_line(f"{field_value(value)}\r\n")
|
||||
|
||||
if file_field and file_path:
|
||||
path = Path(file_path)
|
||||
content_type = mimetypes.guess_type(path.name)[0] or "application/octet-stream"
|
||||
add_line(f"--{boundary}\r\n")
|
||||
add_line(
|
||||
f'Content-Disposition: form-data; name="{file_field}"; filename="{path.name}"\r\n'
|
||||
)
|
||||
add_line(f"Content-Type: {content_type}\r\n\r\n")
|
||||
chunks.append(path.read_bytes())
|
||||
add_line("\r\n")
|
||||
|
||||
add_line(f"--{boundary}--\r\n")
|
||||
return b"".join(chunks), f"multipart/form-data; boundary={boundary}"
|
||||
|
||||
|
||||
def post_multipart(
|
||||
args: argparse.Namespace,
|
||||
path: str,
|
||||
fields: dict[str, object | None],
|
||||
file_path: str | None,
|
||||
) -> bytes:
|
||||
body, content_type = multipart_body(fields=fields, file_field="file", file_path=file_path)
|
||||
def post_multipart(args, path, fields, file_path):
|
||||
body, media = multipart_body(fields=fields, file_path=file_path)
|
||||
return http_request(
|
||||
method="POST",
|
||||
base_url=args.base_url,
|
||||
path=path,
|
||||
token=args.token,
|
||||
body=body,
|
||||
headers={"Content-Type": content_type},
|
||||
headers={"Content-Type": media},
|
||||
timeout=args.timeout,
|
||||
expected="json" if path == PATHS["asr"] else "wav",
|
||||
)
|
||||
|
||||
|
||||
def write_output(path: str, data: bytes) -> None:
|
||||
Path(path).write_bytes(data)
|
||||
print(path)
|
||||
def write_output(path, data):
|
||||
wav_info(data)
|
||||
target = Path(path)
|
||||
if target.suffix.lower() != ".wav":
|
||||
raise ValueError("API output is WAV; transcode separately for MP3")
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_bytes(data)
|
||||
|
||||
|
||||
def cmd_health(args: argparse.Namespace) -> None:
|
||||
data = http_request(
|
||||
method="GET",
|
||||
base_url=args.base_url,
|
||||
path="/health",
|
||||
token=args.token,
|
||||
)
|
||||
print_json(data)
|
||||
def validate_payload(command, payload):
|
||||
required = {
|
||||
"design": ("text", "instruct"),
|
||||
"synthesize": ("text",),
|
||||
"clone": ("text",),
|
||||
"music": ("prompt",),
|
||||
"sfx": ("prompt",),
|
||||
"yue2": ("style", "lyrics"),
|
||||
}.get(command, ())
|
||||
limits = {
|
||||
"text": 4096,
|
||||
"ref_text": 4096,
|
||||
"instruct": 1024,
|
||||
"prompt": 2000,
|
||||
"negative_prompt": 2000,
|
||||
"style": 4000,
|
||||
"lyrics": 30000,
|
||||
"abc": 100000,
|
||||
}
|
||||
for key in required:
|
||||
if not isinstance(payload.get(key), str) or not payload[key].strip():
|
||||
raise ValueError(f"{key} must be nonblank text")
|
||||
for key, limit in limits.items():
|
||||
if key in payload and (
|
||||
not isinstance(payload[key], str)
|
||||
or len(payload[key]) > limit
|
||||
or key in {"ref_text", "abc"}
|
||||
and not payload[key].strip()
|
||||
):
|
||||
raise ValueError(f"invalid {key} or exceeds {limit} characters")
|
||||
ranges = {
|
||||
"temperature": (0, 2),
|
||||
"max_tokens": (1, 8192),
|
||||
"top_k": (0, 1000),
|
||||
"top_p": (0, 1),
|
||||
"repetition_penalty": (0.1, 2),
|
||||
"cfg_value": (0, 10),
|
||||
"inference_timesteps": (1, 200),
|
||||
"cfg": (0, 10),
|
||||
"apg": (0, 1),
|
||||
"steps": (1, 64),
|
||||
"cfg_scale": (0, 20),
|
||||
"min_duration_seconds": (8, 360),
|
||||
"max_duration_seconds": (8, 360),
|
||||
"seed": (0, 9223372036854775807 if command == "yue2" else 2147483647),
|
||||
}
|
||||
for key, (low, high) in ranges.items():
|
||||
if key in payload and (
|
||||
isinstance(payload[key], bool)
|
||||
or not isinstance(payload[key], (int, float))
|
||||
or not math.isfinite(payload[key])
|
||||
or not low <= payload[key] <= high
|
||||
):
|
||||
raise ValueError(f"{key} must be between {low} and {high}")
|
||||
if "seconds" in payload and not (
|
||||
math.isfinite(payload["seconds"]) and 0 < payload["seconds"] <= 380
|
||||
):
|
||||
raise ValueError("seconds must be greater than 0 and at most 380")
|
||||
if command in {"design", "clone"}:
|
||||
provider = payload.get("provider", "qwen")
|
||||
if provider not in {"qwen", "voxcpm2"}:
|
||||
raise ValueError("provider must be qwen or voxcpm2")
|
||||
ignored = VOX if provider == "qwen" else QWEN
|
||||
if any(k in payload for k in ignored):
|
||||
raise ValueError("control fields must match the selected provider")
|
||||
if command == "clone":
|
||||
if provider == "qwen" and not payload.get("ref_text", "").strip():
|
||||
raise ValueError(
|
||||
"Qwen clone requires the accurate reference transcript"
|
||||
)
|
||||
if "instruct" in payload and (provider == "qwen" or "ref_text" in payload):
|
||||
raise ValueError(
|
||||
"clone instruct only applies to VoxCPM2 without ref_text"
|
||||
)
|
||||
if command == "yue2":
|
||||
if payload.get("mode", "full") == "direct" and "abc" in payload:
|
||||
raise ValueError("ABC is allowed only in melody/full mode")
|
||||
if payload.get("max_duration_seconds", 120) < payload.get(
|
||||
"min_duration_seconds", 8
|
||||
):
|
||||
raise ValueError("maximum duration must not be less than minimum")
|
||||
|
||||
|
||||
def cmd_models(args: argparse.Namespace) -> None:
|
||||
data = http_request(
|
||||
method="GET",
|
||||
base_url=args.base_url,
|
||||
path="/models",
|
||||
token=args.token,
|
||||
)
|
||||
print_json(data)
|
||||
|
||||
|
||||
def print_json(data: bytes) -> None:
|
||||
try:
|
||||
parsed = json.loads(data.decode("utf-8"))
|
||||
except json.JSONDecodeError:
|
||||
sys.stdout.buffer.write(data)
|
||||
return
|
||||
print(json.dumps(parsed, ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
def cmd_design(args: argparse.Namespace) -> None:
|
||||
payload = clean_payload(
|
||||
{
|
||||
"text": args.text,
|
||||
"instruct": args.instruct,
|
||||
"provider": args.provider,
|
||||
"language": args.language,
|
||||
"temperature": args.temperature,
|
||||
"max_tokens": args.max_tokens,
|
||||
"top_k": args.top_k,
|
||||
"top_p": args.top_p,
|
||||
"repetition_penalty": args.repetition_penalty,
|
||||
"cfg_value": args.cfg_value,
|
||||
"inference_timesteps": args.inference_timesteps,
|
||||
"normalize": args.normalize,
|
||||
"denoise": args.denoise,
|
||||
"retry_badcase": args.retry_badcase,
|
||||
}
|
||||
)
|
||||
write_output(args.out, post_json(args, "/v1/voice/design", payload))
|
||||
|
||||
|
||||
def cmd_synthesize(args: argparse.Namespace) -> None:
|
||||
payload = clean_payload(
|
||||
{
|
||||
"text": args.text,
|
||||
"speaker": args.speaker,
|
||||
"language": args.language,
|
||||
"temperature": args.temperature,
|
||||
"max_tokens": args.max_tokens,
|
||||
"top_k": args.top_k,
|
||||
"top_p": args.top_p,
|
||||
"repetition_penalty": args.repetition_penalty,
|
||||
}
|
||||
)
|
||||
write_output(args.out, post_json(args, "/v1/voice/synthesize", payload))
|
||||
|
||||
|
||||
def cmd_clone(args: argparse.Namespace) -> None:
|
||||
fields = clean_payload(
|
||||
{
|
||||
"text": args.text,
|
||||
"ref_text": args.ref_text,
|
||||
"language": args.language,
|
||||
"temperature": args.temperature,
|
||||
"max_tokens": args.max_tokens,
|
||||
"top_k": args.top_k,
|
||||
"top_p": args.top_p,
|
||||
"repetition_penalty": args.repetition_penalty,
|
||||
"provider": args.provider,
|
||||
"instruct": args.instruct,
|
||||
"cfg_value": args.cfg_value,
|
||||
"inference_timesteps": args.inference_timesteps,
|
||||
"normalize": args.normalize,
|
||||
"denoise": args.denoise,
|
||||
"retry_badcase": args.retry_badcase,
|
||||
}
|
||||
)
|
||||
write_output(args.out, post_multipart(args, "/v1/voice/clone", fields, args.file))
|
||||
|
||||
|
||||
def cmd_enhance(args: argparse.Namespace) -> None:
|
||||
fields = {"chunked": args.chunked}
|
||||
write_output(args.out, post_multipart(args, "/v1/audio/enhance", fields, args.file))
|
||||
|
||||
|
||||
def cmd_denoise(args: argparse.Namespace) -> None:
|
||||
fields = {"chunked": args.chunked}
|
||||
write_output(args.out, post_multipart(args, "/v1/audio/denoise", fields, args.file))
|
||||
|
||||
|
||||
def cmd_asr(args: argparse.Namespace) -> None:
|
||||
fields = clean_payload(
|
||||
{
|
||||
"language": args.language,
|
||||
"temperature": args.temperature,
|
||||
"max_tokens": args.max_tokens,
|
||||
"top_k": args.top_k,
|
||||
"top_p": args.top_p,
|
||||
"repetition_penalty": args.repetition_penalty,
|
||||
}
|
||||
)
|
||||
print_json(post_multipart(args, "/v1/asr/transcribe", fields, args.file))
|
||||
|
||||
|
||||
def add_generation_args(parser: argparse.ArgumentParser) -> None:
|
||||
parser.add_argument("--language", default=None)
|
||||
parser.add_argument("--temperature", type=float, default=None)
|
||||
parser.add_argument("--max-tokens", type=int, default=None)
|
||||
parser.add_argument("--top-k", type=int, default=None)
|
||||
parser.add_argument("--top-p", type=float, default=None)
|
||||
parser.add_argument("--repetition-penalty", type=float, default=None)
|
||||
|
||||
|
||||
def add_voxcpm2_args(parser: argparse.ArgumentParser) -> None:
|
||||
parser.add_argument("--cfg-value", type=float, default=None)
|
||||
parser.add_argument("--inference-timesteps", type=int, default=None)
|
||||
parser.add_argument("--normalize", type=bool_value, default=None)
|
||||
parser.add_argument("--denoise", type=bool_value, default=None)
|
||||
parser.add_argument("--retry-badcase", type=bool_value, default=None)
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
def build_parser():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
subparsers = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
health = subparsers.add_parser("health", help="Check service health")
|
||||
add_common_args(health)
|
||||
health.set_defaults(func=cmd_health)
|
||||
|
||||
models = subparsers.add_parser("models", help="Show configured model status")
|
||||
add_common_args(models)
|
||||
models.set_defaults(func=cmd_models)
|
||||
|
||||
design = subparsers.add_parser("design", help="Generate voice from text and instruction")
|
||||
add_common_args(design)
|
||||
design.add_argument("--text", required=True)
|
||||
design.add_argument("--instruct", required=True)
|
||||
design.add_argument("--provider", choices=["qwen", "voxcpm2"], default=None)
|
||||
design.add_argument("--out", default="voice_design.wav")
|
||||
add_generation_args(design)
|
||||
add_voxcpm2_args(design)
|
||||
design.set_defaults(func=cmd_design)
|
||||
|
||||
synthesize = subparsers.add_parser("synthesize", help="Base TTS synthesis")
|
||||
add_common_args(synthesize)
|
||||
synthesize.add_argument("--text", required=True)
|
||||
synthesize.add_argument("--speaker", default=None)
|
||||
synthesize.add_argument("--out", default="synthesize.wav")
|
||||
add_generation_args(synthesize)
|
||||
synthesize.set_defaults(func=cmd_synthesize)
|
||||
|
||||
clone = subparsers.add_parser("clone", help="Clone a voice from reference audio")
|
||||
add_common_args(clone)
|
||||
clone.add_argument("--file", required=True, help="Reference audio path")
|
||||
clone.add_argument("--text", required=True)
|
||||
clone.add_argument("--ref-text", default=None)
|
||||
clone.add_argument("--provider", choices=["qwen", "voxcpm2"], default=None)
|
||||
clone.add_argument("--instruct", default=None)
|
||||
clone.add_argument("--out", default="voice_clone.wav")
|
||||
add_generation_args(clone)
|
||||
add_voxcpm2_args(clone)
|
||||
clone.set_defaults(func=cmd_clone)
|
||||
|
||||
enhance = subparsers.add_parser("enhance", help="Enhance speech audio")
|
||||
add_common_args(enhance)
|
||||
enhance.add_argument("--file", required=True)
|
||||
enhance.add_argument("--chunked", type=bool_value, default=None)
|
||||
enhance.add_argument("--out", default="enhanced.wav")
|
||||
enhance.set_defaults(func=cmd_enhance)
|
||||
|
||||
denoise = subparsers.add_parser("denoise", help="Denoise speech audio")
|
||||
add_common_args(denoise)
|
||||
denoise.add_argument("--file", required=True)
|
||||
denoise.add_argument("--chunked", type=bool_value, default=None)
|
||||
denoise.add_argument("--out", default="enhanced.wav")
|
||||
denoise.set_defaults(func=cmd_denoise)
|
||||
|
||||
asr = subparsers.add_parser("asr", help="Transcribe audio to JSON")
|
||||
add_common_args(asr)
|
||||
asr.add_argument("--file", required=True)
|
||||
add_generation_args(asr)
|
||||
asr.set_defaults(func=cmd_asr)
|
||||
|
||||
subs = parser.add_subparsers(dest="command", required=True)
|
||||
for command in PATHS:
|
||||
sub = subs.add_parser(command)
|
||||
sub.add_argument("--base-url", default=env_base_url())
|
||||
sub.add_argument("--token", help="legacy direct token; prefer environment")
|
||||
sub.add_argument("--api-key-env", help="explicit token environment variable")
|
||||
sub.add_argument(
|
||||
"--timeout",
|
||||
type=float,
|
||||
default=1800 if command in {"music", "sfx", "yue2"} else 600,
|
||||
)
|
||||
sub.add_argument("--dry-run", action="store_true")
|
||||
if command in {"health", "models"}:
|
||||
continue
|
||||
if command != "asr":
|
||||
sub.add_argument(
|
||||
"--out",
|
||||
default={
|
||||
"design": "voice_design.wav",
|
||||
"clone": "voice_clone.wav",
|
||||
"enhance": "enhanced.wav",
|
||||
"denoise": "enhanced.wav",
|
||||
}.get(command, f"{command}.wav"),
|
||||
)
|
||||
if command in {"clone", "asr", "enhance", "denoise", "reuse"}:
|
||||
sub.add_argument("--file", required=True)
|
||||
if command in {"design", "synthesize", "clone"}:
|
||||
sub.add_argument("--text", required=True)
|
||||
if command in {"design", "clone"}:
|
||||
sub.add_argument("--instruct", required=command == "design")
|
||||
sub.add_argument("--provider", choices=["qwen", "voxcpm2"], default="qwen")
|
||||
for key in VOX:
|
||||
kind = (
|
||||
bool_value
|
||||
if key in {"normalize", "denoise", "retry_badcase"}
|
||||
else int
|
||||
if key == "inference_timesteps"
|
||||
else float
|
||||
)
|
||||
sub.add_argument("--" + key.replace("_", "-"), type=kind)
|
||||
if command == "clone":
|
||||
sub.add_argument("--ref-text")
|
||||
sub.add_argument("--ref-text-file")
|
||||
if command == "synthesize":
|
||||
sub.add_argument("--speaker")
|
||||
if command in {"design", "synthesize", "clone", "asr"}:
|
||||
for key in QWEN:
|
||||
kind = (
|
||||
str
|
||||
if key == "language"
|
||||
else int
|
||||
if key in {"max_tokens", "top_k"}
|
||||
else float
|
||||
)
|
||||
sub.add_argument("--" + key.replace("_", "-"), type=kind)
|
||||
if command in {"enhance", "denoise"}:
|
||||
sub.add_argument("--chunked", type=bool_value)
|
||||
if command in {"music", "sfx"}:
|
||||
sub.add_argument("--prompt", required=True)
|
||||
sub.add_argument("--negative-prompt")
|
||||
sub.add_argument(
|
||||
"--seconds", type=float, default=30 if command == "music" else 5
|
||||
)
|
||||
sub.add_argument("--steps", type=int, default=8)
|
||||
sub.add_argument("--seed", type=int)
|
||||
sub.add_argument("--cfg", type=float, default=1.0)
|
||||
sub.add_argument("--apg", type=float, default=1.0)
|
||||
if command == "yue2":
|
||||
sub.add_argument("--style", required=True)
|
||||
lyrics = sub.add_mutually_exclusive_group(required=True)
|
||||
lyrics.add_argument("--lyrics")
|
||||
lyrics.add_argument("--lyrics-file")
|
||||
sub.add_argument(
|
||||
"--mode", choices=["direct", "melody", "full"], default="full"
|
||||
)
|
||||
sub.add_argument("--abc-file")
|
||||
sub.add_argument("--steps", type=int, default=8)
|
||||
sub.add_argument("--seed", type=int, default=831001)
|
||||
sub.add_argument("--cfg-scale", type=float, default=1.0)
|
||||
sub.add_argument("--min-duration-seconds", type=int, default=8)
|
||||
sub.add_argument("--max-duration-seconds", type=int, default=120)
|
||||
return parser
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = build_parser().parse_args()
|
||||
args.func(args)
|
||||
def payload_from_args(args):
|
||||
excluded = {
|
||||
"command",
|
||||
"base_url",
|
||||
"token",
|
||||
"api_key_env",
|
||||
"timeout",
|
||||
"dry_run",
|
||||
"out",
|
||||
"file",
|
||||
"ref_text_file",
|
||||
"lyrics_file",
|
||||
"abc_file",
|
||||
}
|
||||
payload = clean_payload({k: v for k, v in vars(args).items() if k not in excluded})
|
||||
for key in ("ref_text", "lyrics", "abc"):
|
||||
path = getattr(args, key + "_file", None)
|
||||
if path:
|
||||
if key in payload:
|
||||
raise ValueError(f"supply {key} text or file, not both")
|
||||
payload[key] = Path(path).read_text(encoding="utf-8-sig")
|
||||
validate_payload(args.command, payload)
|
||||
return payload
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
args = build_parser().parse_args(argv)
|
||||
try:
|
||||
url = build_url(args.base_url, PATHS[args.command])
|
||||
if not math.isfinite(args.timeout) or args.timeout <= 0:
|
||||
raise ValueError("timeout must be positive and finite")
|
||||
payload = payload_from_args(args)
|
||||
file = getattr(args, "file", None)
|
||||
if file:
|
||||
preflight_audio(
|
||||
file, not (args.command == "clone" and args.provider == "voxcpm2")
|
||||
)
|
||||
if hasattr(args, "out") and Path(args.out).suffix.lower() != ".wav":
|
||||
raise ValueError("use a .wav output path")
|
||||
if args.dry_run:
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"url": url,
|
||||
"payload": payload,
|
||||
"file": file,
|
||||
"timeout": args.timeout,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
indent=2,
|
||||
)
|
||||
)
|
||||
return 0
|
||||
args.token = args.token or (
|
||||
os.getenv(args.api_key_env) if args.api_key_env else env_token()
|
||||
)
|
||||
if args.command in {"health", "models"}:
|
||||
data = http_request(
|
||||
method="GET",
|
||||
base_url=args.base_url,
|
||||
path=PATHS[args.command],
|
||||
timeout=args.timeout,
|
||||
)
|
||||
elif file:
|
||||
data = post_multipart(args, PATHS[args.command], payload, file)
|
||||
else:
|
||||
data = post_json(args, PATHS[args.command], payload)
|
||||
if args.command in {"asr", "health", "models"}:
|
||||
print(json.dumps(json.loads(data), ensure_ascii=False, indent=2))
|
||||
else:
|
||||
write_output(args.out, data)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"ok": True,
|
||||
"output": str(Path(args.out).resolve()),
|
||||
**wav_info(data),
|
||||
},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
)
|
||||
return 0
|
||||
except (APIError, URLError, OSError, ValueError, RuntimeError) as exc:
|
||||
report = {"ok": False, "error_type": type(exc).__name__}
|
||||
if isinstance(exc, APIError):
|
||||
report.update(http_status=exc.status, confirmed_busy=exc.busy)
|
||||
elif isinstance(exc, (URLError, TimeoutError)):
|
||||
report["message"] = (
|
||||
"connection/timeout; outcome unknown, no automatic retry"
|
||||
)
|
||||
else:
|
||||
message = str(exc)
|
||||
if args.token:
|
||||
message = message.replace(args.token, "[redacted]")
|
||||
report["message"] = message[:240]
|
||||
print(json.dumps(report, ensure_ascii=False), file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,486 @@
|
||||
from __future__ import annotations
|
||||
import contextlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import wave
|
||||
from email.parser import BytesParser
|
||||
from email.policy import default
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
from urllib.error import HTTPError
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
|
||||
import call_voice_api as api
|
||||
import batch_clone_generated_audio as batch
|
||||
|
||||
|
||||
def wav(seconds=0.02):
|
||||
out = io.BytesIO()
|
||||
with wave.open(out, "wb") as w:
|
||||
w.setnchannels(1)
|
||||
w.setsampwidth(2)
|
||||
w.setframerate(24000)
|
||||
w.writeframes(b"\x01\x00" * int(24000 * seconds))
|
||||
return out.getvalue()
|
||||
|
||||
|
||||
def response(data, media):
|
||||
r = io.BytesIO(data)
|
||||
r.headers = {"Content-Type": media}
|
||||
return r
|
||||
|
||||
|
||||
class ClientTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
p = mock.patch.object(
|
||||
api, "urlopen", side_effect=AssertionError("unexpected network")
|
||||
)
|
||||
self.net = p.start()
|
||||
self.addCleanup(p.stop)
|
||||
|
||||
def test_public_status_needs_no_auth(self):
|
||||
self.net.side_effect = None
|
||||
self.net.return_value = response(b'{"status":"ok"}', "application/json")
|
||||
api.http_request(method="GET", base_url="https://example.test", path="/health")
|
||||
req = self.net.call_args.args[0]
|
||||
self.assertIsNone(req.get_header("Authorization"))
|
||||
self.assertEqual(self.net.call_args.kwargs["timeout"], 600)
|
||||
|
||||
def test_environment_priority_and_explicit_variable(self):
|
||||
with mock.patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"AUDIO_API_TOKEN": "new",
|
||||
"OPENAI_API_KEY": "old",
|
||||
"AUDIO_API_BASE": "https://new.test",
|
||||
"QWEN3_VOICE_API_BASE_URL": "https://old.test",
|
||||
},
|
||||
clear=True,
|
||||
):
|
||||
self.assertEqual(api.env_token(), "new")
|
||||
self.assertEqual(api.env_base_url(), "https://new.test")
|
||||
for base in (
|
||||
"file:///tmp",
|
||||
"https://token@example.test",
|
||||
"https://example.test/v1",
|
||||
"https://example.test?q=secret",
|
||||
):
|
||||
with self.subTest(base=base), self.assertRaises(ValueError):
|
||||
api.build_url(base, "/health")
|
||||
|
||||
def test_json_wire_preserves_multiline_unicode_zero_and_false(self):
|
||||
args = api.build_parser().parse_args(
|
||||
[
|
||||
"yue2",
|
||||
"--style",
|
||||
"中文",
|
||||
"--lyrics",
|
||||
"[Verse]\n海风",
|
||||
"--seed",
|
||||
"0",
|
||||
"--cfg-scale",
|
||||
"0",
|
||||
]
|
||||
)
|
||||
args.token = "test-key"
|
||||
payload = api.payload_from_args(args)
|
||||
self.net.side_effect = None
|
||||
self.net.return_value = response(wav(), "audio/wav")
|
||||
api.post_json(args, api.PATHS["yue2"], payload)
|
||||
req = self.net.call_args.args[0]
|
||||
self.assertEqual(json.loads(req.data), payload)
|
||||
self.assertEqual(payload["seed"], 0)
|
||||
self.assertEqual(payload["lyrics"], "[Verse]\n海风")
|
||||
self.assertNotIn("seconds", payload)
|
||||
vox = api.build_parser().parse_args(
|
||||
[
|
||||
"design",
|
||||
"--provider",
|
||||
"voxcpm2",
|
||||
"--text",
|
||||
"你好",
|
||||
"--instruct",
|
||||
"平静",
|
||||
"--denoise",
|
||||
"false",
|
||||
"--cfg-value",
|
||||
"0",
|
||||
]
|
||||
)
|
||||
self.assertIs(api.payload_from_args(vox)["denoise"], False)
|
||||
|
||||
def test_multipart_has_real_boundary_complete_file_and_false_zero(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
file = Path(d) / "参考.wav"
|
||||
file.write_bytes(wav())
|
||||
body, media = api.multipart_body(
|
||||
fields={"chunked": False, "temperature": 0, "ref_text": "你好"},
|
||||
file_path=str(file),
|
||||
)
|
||||
msg = BytesParser(policy=default).parsebytes(
|
||||
("Content-Type: " + media + "\r\nMIME-Version: 1.0\r\n\r\n").encode()
|
||||
+ body
|
||||
)
|
||||
parts = list(msg.iter_parts())
|
||||
values = {
|
||||
p.get_param("name", header="content-disposition"): p.get_payload(
|
||||
decode=True
|
||||
)
|
||||
for p in parts
|
||||
}
|
||||
self.assertEqual(values["file"], file.read_bytes())
|
||||
self.assertEqual(values["chunked"], b"false")
|
||||
self.assertEqual(values["temperature"], b"0")
|
||||
self.assertEqual(values["ref_text"].decode(), "你好")
|
||||
|
||||
def test_success_media_and_truncated_wav_fail_before_writing(self):
|
||||
cases = [
|
||||
(b'{"error":"bad"}', "application/json"),
|
||||
(b"<html>error</html>", "text/html"),
|
||||
(b"garbage", "audio/wav"),
|
||||
(wav()[:-2], "audio/wav"),
|
||||
]
|
||||
for data, media in cases:
|
||||
with self.subTest(media=media), self.assertRaises(ValueError):
|
||||
self.net.side_effect = None
|
||||
self.net.return_value = response(data, media)
|
||||
api.http_request(
|
||||
method="POST",
|
||||
base_url="https://example.test",
|
||||
path=api.PATHS["music"],
|
||||
token="test",
|
||||
expected="wav",
|
||||
)
|
||||
|
||||
def test_wav_header_durations_and_preflight(self):
|
||||
self.assertEqual(api.wav_info(wav())["sample_rate"], 24000)
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
file = Path(d) / "long.wav"
|
||||
file.write_bytes(wav(60.1))
|
||||
with self.assertRaises(ValueError):
|
||||
api.preflight_audio(str(file))
|
||||
api.preflight_audio(str(file), False)
|
||||
file.write_bytes(wav(60 + 1 / 24000))
|
||||
with self.assertRaises(ValueError):
|
||||
api.preflight_audio(str(file))
|
||||
file.write_bytes(b"not a WAV")
|
||||
with self.assertRaises(ValueError):
|
||||
api.preflight_audio(str(file))
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
out = Path(d) / "out.wav"
|
||||
with self.assertRaises(ValueError):
|
||||
api.write_output(str(out), wav()[:-1])
|
||||
self.assertFalse(out.exists())
|
||||
|
||||
def test_asr_invalid_json_or_wrong_text_type_fails(self):
|
||||
for data in (b"not json", b'{"text":7}', b"[]"):
|
||||
self.net.side_effect = None
|
||||
self.net.return_value = response(data, "application/json")
|
||||
with self.assertRaises(ValueError):
|
||||
api.http_request(
|
||||
method="POST",
|
||||
base_url="https://example.test",
|
||||
path=api.PATHS["asr"],
|
||||
token="test",
|
||||
)
|
||||
|
||||
def test_error_detail_string_array_html_and_no_retry(self):
|
||||
for raw in (
|
||||
b'{"detail":"busy"}',
|
||||
b'{"detail":[{"msg":"invalid"}]}',
|
||||
b"<html>bad</html>",
|
||||
):
|
||||
self.net.reset_mock()
|
||||
self.net.side_effect = HTTPError(
|
||||
"https://example.test", 503, "error", {}, io.BytesIO(raw)
|
||||
)
|
||||
with self.assertRaises(api.APIError) as caught:
|
||||
api.http_request(
|
||||
method="POST",
|
||||
base_url="https://example.test",
|
||||
path=api.PATHS["music"],
|
||||
token="test",
|
||||
)
|
||||
self.assertEqual(caught.exception.status, 503)
|
||||
self.net.assert_called_once()
|
||||
self.assertFalse(caught.exception.busy)
|
||||
|
||||
def test_provider_constraints_and_yue2_combinations(self):
|
||||
for command, payload in [
|
||||
("clone", {"provider": "qwen", "text": "new"}),
|
||||
(
|
||||
"design",
|
||||
{
|
||||
"provider": "voxcpm2",
|
||||
"text": "new",
|
||||
"instruct": "style",
|
||||
"temperature": 0,
|
||||
},
|
||||
),
|
||||
(
|
||||
"clone",
|
||||
{
|
||||
"provider": "voxcpm2",
|
||||
"text": "new",
|
||||
"ref_text": "reference",
|
||||
"instruct": "ignored",
|
||||
},
|
||||
),
|
||||
(
|
||||
"yue2",
|
||||
{"style": "folk", "lyrics": "song", "mode": "direct", "abc": "X:1"},
|
||||
),
|
||||
(
|
||||
"yue2",
|
||||
{
|
||||
"style": "folk",
|
||||
"lyrics": "song",
|
||||
"min_duration_seconds": 30,
|
||||
"max_duration_seconds": 8,
|
||||
},
|
||||
),
|
||||
("music", {"prompt": " "}),
|
||||
("synthesize", {"text": "new", "temperature": float("nan")}),
|
||||
]:
|
||||
with self.subTest(command=command), self.assertRaises(ValueError):
|
||||
api.validate_payload(command, payload)
|
||||
|
||||
def test_new_routes_and_lyric_files_dry_run_require_no_key(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
file = Path(d) / "speech.wav"
|
||||
file.write_bytes(wav())
|
||||
lyrics = Path(d) / "歌词.txt"
|
||||
lyrics.write_text("[Verse]\n你好", encoding="utf-8-sig")
|
||||
commands = [
|
||||
["reuse", "--file", str(file)],
|
||||
["music", "--prompt", "piano"],
|
||||
["sfx", "--prompt", "rain"],
|
||||
["yue2", "--style", "folk", "--lyrics-file", str(lyrics)],
|
||||
]
|
||||
for argv in commands:
|
||||
output = io.StringIO()
|
||||
with (
|
||||
mock.patch.dict(os.environ, {}, clear=True),
|
||||
contextlib.redirect_stdout(output),
|
||||
):
|
||||
self.assertEqual(api.main(argv + ["--dry-run"]), 0)
|
||||
value = json.loads(output.getvalue())
|
||||
self.assertTrue(value["url"].endswith(api.PATHS[argv[0]]))
|
||||
self.assertNotIn("token", value)
|
||||
self.net.assert_not_called()
|
||||
|
||||
|
||||
class BatchTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
self.root = Path(self.tmp.name)
|
||||
(self.root / "inputs").mkdir()
|
||||
self.ref = self.root / "misleading-name.wav"
|
||||
self.ref.write_bytes(wav())
|
||||
self.src = self.root / "inputs" / "source.wav"
|
||||
self.src.write_bytes(wav())
|
||||
self.argv = [
|
||||
"--input-dir",
|
||||
str(self.root / "inputs"),
|
||||
"--reference-audio",
|
||||
str(self.ref),
|
||||
"--output-dir",
|
||||
str(self.root / "out"),
|
||||
"--token",
|
||||
"test",
|
||||
]
|
||||
p = mock.patch.object(
|
||||
api, "urlopen", side_effect=AssertionError("unexpected network")
|
||||
)
|
||||
self.net = p.start()
|
||||
self.addCleanup(p.stop)
|
||||
|
||||
def test_reference_uses_explicit_or_one_asr_never_stem(self):
|
||||
args = batch.parse_args(self.argv)
|
||||
with mock.patch.object(
|
||||
batch, "transcribe", return_value={"text": "actual words"}
|
||||
) as transcribe:
|
||||
self.assertEqual(batch.reference_transcript(args), "actual words")
|
||||
transcribe.assert_called_once()
|
||||
args.reference_text = "explicit actual words"
|
||||
with mock.patch.object(batch, "transcribe") as transcribe:
|
||||
self.assertEqual(batch.reference_transcript(args), "explicit actual words")
|
||||
transcribe.assert_not_called()
|
||||
|
||||
def test_empty_asr_never_clones_and_next_file_continues(self):
|
||||
(self.root / "inputs" / "second.wav").write_bytes(wav())
|
||||
with (
|
||||
mock.patch.object(
|
||||
batch, "transcribe", side_effect=[ValueError("empty"), {"text": "你好"}]
|
||||
),
|
||||
mock.patch.object(batch, "clone", return_value=wav()) as clone,
|
||||
contextlib.redirect_stdout(io.StringIO()),
|
||||
):
|
||||
code = batch.main(self.argv + ["--reference-text", "known reference"])
|
||||
self.assertEqual(code, 1)
|
||||
clone.assert_called_once()
|
||||
summary = json.loads(
|
||||
(self.root / "out" / "conversion_summary.json").read_text(encoding="utf-8")
|
||||
)
|
||||
self.assertEqual([r["status"] for r in summary], ["error", "ok"])
|
||||
self.assertEqual(summary[1]["text"], "你好")
|
||||
|
||||
def test_successful_outputs_skip_without_reference_asr_or_clone(self):
|
||||
(self.root / "out").mkdir()
|
||||
(self.root / "out" / "source.wav").write_bytes(wav())
|
||||
with (
|
||||
mock.patch.object(batch, "transcribe") as transcribe,
|
||||
mock.patch.object(batch, "clone") as clone,
|
||||
contextlib.redirect_stdout(io.StringIO()),
|
||||
):
|
||||
self.assertEqual(batch.main(self.argv), 0)
|
||||
transcribe.assert_not_called()
|
||||
clone.assert_not_called()
|
||||
|
||||
def test_empty_recognition_does_not_fall_back_to_name(self):
|
||||
args = batch.parse_args(self.argv)
|
||||
with (
|
||||
mock.patch.object(api, "post_multipart", return_value=b'{"text":""}'),
|
||||
self.assertRaises(ValueError),
|
||||
):
|
||||
batch.transcribe(args, self.src)
|
||||
with self.assertRaises(ValueError):
|
||||
batch.normalize_text("")
|
||||
self.assertEqual(batch.normalize_text("你好"), "你好。")
|
||||
|
||||
def test_only_confirmed_busy_retries_and_last_failure_never_sleeps(self):
|
||||
op = mock.Mock(side_effect=[api.APIError(503, "model busy"), b"ok"])
|
||||
with mock.patch.object(batch.time, "sleep") as sleep:
|
||||
self.assertEqual(batch.call_with_busy_retry(op, 2), b"ok")
|
||||
sleep.assert_called_once()
|
||||
for err in [
|
||||
api.APIError(500, "busy"),
|
||||
api.APIError(502, "busy"),
|
||||
api.APIError(504, "timeout"),
|
||||
api.APIError(503, "failed to load model"),
|
||||
TimeoutError("unknown outcome"),
|
||||
]:
|
||||
op = mock.Mock(side_effect=err)
|
||||
with (
|
||||
mock.patch.object(batch.time, "sleep") as sleep,
|
||||
self.assertRaises(type(err)),
|
||||
):
|
||||
batch.call_with_busy_retry(op, 3)
|
||||
op.assert_called_once()
|
||||
sleep.assert_not_called()
|
||||
|
||||
def test_busy_retry_reuploads_full_file(self):
|
||||
args = batch.parse_args(self.argv)
|
||||
seen = []
|
||||
|
||||
def send(**kw):
|
||||
seen.append(kw["body"])
|
||||
if len(seen) == 1:
|
||||
raise api.APIError(503, "busy")
|
||||
return b'{"text":"known"}'
|
||||
|
||||
with (
|
||||
mock.patch.object(api, "http_request", side_effect=send),
|
||||
mock.patch.object(batch.time, "sleep"),
|
||||
):
|
||||
args.retries = 2
|
||||
batch.transcribe(args, self.src)
|
||||
self.assertEqual(len(seen), 2)
|
||||
self.assertTrue(all(self.src.read_bytes() in body for body in seen))
|
||||
|
||||
def test_busy_retry_honors_server_delay_and_stops_on_long_delay(self):
|
||||
op = mock.Mock(side_effect=[api.APIError(503, "busy", "7"), b"ok"])
|
||||
with mock.patch.object(batch.time, "sleep") as sleep:
|
||||
batch.call_with_busy_retry(op, 2)
|
||||
sleep.assert_called_once_with(7)
|
||||
op = mock.Mock(side_effect=api.APIError(503, "busy", "120"))
|
||||
with (
|
||||
mock.patch.object(batch.time, "sleep") as sleep,
|
||||
self.assertRaises(api.APIError),
|
||||
):
|
||||
batch.call_with_busy_retry(op, 2)
|
||||
op.assert_called_once()
|
||||
sleep.assert_not_called()
|
||||
|
||||
def test_collision_overlap_and_missing_file_fail_before_inference(self):
|
||||
args = batch.parse_args(self.argv)
|
||||
(self.root / "inputs" / "source.mp3").write_bytes(b"compressed")
|
||||
with self.assertRaises(ValueError):
|
||||
batch.prepare(args)
|
||||
(self.root / "inputs" / "source.mp3").unlink()
|
||||
args.output_dir = str(self.root / "inputs")
|
||||
with self.assertRaises(ValueError):
|
||||
batch.prepare(args)
|
||||
args.output_dir = str(self.root / "out")
|
||||
args.reference_audio = str(self.root / "missing.wav")
|
||||
with self.assertRaises(ValueError):
|
||||
batch.prepare(args)
|
||||
self.net.assert_not_called()
|
||||
|
||||
def test_vox_batch_does_not_send_qwen_language(self):
|
||||
args = batch.parse_args(self.argv + ["--provider", "voxcpm2"])
|
||||
with mock.patch.object(api, "post_multipart", return_value=wav()) as send:
|
||||
batch.clone(args, "reference words", "new text")
|
||||
self.assertNotIn("language", send.call_args.args[2])
|
||||
self.assertEqual(send.call_args.args[2]["ref_text"], "reference words")
|
||||
|
||||
def test_wav_no_gain_export_preserves_bytes_without_ffmpeg(self):
|
||||
raw = self.root / "raw.wav"
|
||||
out = self.root / "out.wav"
|
||||
raw.write_bytes(wav())
|
||||
with mock.patch.object(batch.subprocess, "run") as run:
|
||||
batch.export_audio(raw, out, 0, "wav")
|
||||
run.assert_not_called()
|
||||
self.assertEqual(out.read_bytes(), raw.read_bytes())
|
||||
|
||||
def test_failed_conversion_leaves_no_partial_final_and_preserves_old_output(self):
|
||||
raw = self.root / "raw.wav"
|
||||
raw.write_bytes(wav())
|
||||
output = self.root / "out.mp3"
|
||||
|
||||
def failed(command, **kwargs):
|
||||
Path(command[-1]).write_bytes(b"partial invalid MP3")
|
||||
return mock.Mock(returncode=1)
|
||||
|
||||
with (
|
||||
mock.patch.object(batch.subprocess, "run", side_effect=failed),
|
||||
self.assertRaises(RuntimeError),
|
||||
):
|
||||
batch.export_audio(raw, output, 0, "mp3")
|
||||
self.assertFalse(output.exists())
|
||||
self.assertEqual(list(self.root.glob(".audio-export-*")), [])
|
||||
output.write_bytes(b"existing completed output")
|
||||
with (
|
||||
mock.patch.object(batch.subprocess, "run", side_effect=failed),
|
||||
self.assertRaises(RuntimeError),
|
||||
):
|
||||
batch.export_audio(raw, output, 0, "mp3")
|
||||
self.assertEqual(output.read_bytes(), b"existing completed output")
|
||||
|
||||
def test_failed_conversion_is_not_skipped_on_manual_rerun(self):
|
||||
def failed(command, **kwargs):
|
||||
Path(command[-1]).write_bytes(b"partial MP3")
|
||||
return mock.Mock(returncode=1)
|
||||
|
||||
argv = self.argv + ["--reference-text", "known", "--format", "mp3"]
|
||||
with (
|
||||
mock.patch.object(
|
||||
batch, "transcribe", return_value={"text": "known"}
|
||||
) as transcribe,
|
||||
mock.patch.object(batch, "clone", return_value=wav()),
|
||||
mock.patch.object(batch.subprocess, "run", side_effect=failed),
|
||||
contextlib.redirect_stdout(io.StringIO()),
|
||||
):
|
||||
self.assertEqual(batch.main(argv), 1)
|
||||
self.assertFalse((self.root / "out" / "source.mp3").exists())
|
||||
self.assertEqual(batch.main(argv), 1)
|
||||
self.assertEqual(transcribe.call_count, 2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user