Sync third-party and MCP marketplace plugins

Constraint: Public skills are published only by explicit administrator action unless they are tracked third-party market sources.
Confidence: high
Scope-risk: narrow
Directive: Keep private/internal skills out of the public marketplace and preserve normal incremental market Git history.
Tested: Marketplace validation passed.
This commit is contained in:
KeyInfo Bot committed 2026-10-12 00:02:42 +08:00
1 parent cbde777e05
commit 9e74428671
15 files changed
+571 -42

No files matched your search

+8 -8
View File
@@ -105,8 +105,8 @@
"repo": "https://github.com/nextlevelbuilder/ui-ux-pro-max-skill.git",
"ref": "main",
"adapter": "claude-skill",
"commit": "50d8a7de0900119855614541f15a1a616691eb33",
"syncedAt": "2026-10-09T10:26:29Z"
"commit": "67803eac81607188194aea9842a723b89326d90d",
"syncedAt": "2026-10-11T16:00:00Z"
},
{
"id": "caveman",
@@ -132,8 +132,8 @@
"repo": "https://github.com/shadcn-ui/ui.git",
"ref": "main",
"adapter": "claude-skill",
"commit": "c2a67849be260701852b0977ba15eb5f3a0ce2a4",
"syncedAt": "2026-10-10T16:00:00Z"
"commit": "02cad61166c4c20edbc8a3f71bd720eed4c1f49b",
"syncedAt": "2026-10-11T16:00:00Z"
},
{
"id": "frontend-slides",
@@ -168,8 +168,8 @@
"repo": "https://github.com/hugohe3/ppt-master.git",
"ref": "main",
"adapter": "claude-skill",
"commit": "b4efe3ddf237a97a42880164759162f8e2412f2f",
"syncedAt": "2026-10-08T15:59:59Z"
"commit": "ca04346de145ed6462ddf34682848e71df5e20a2",
"syncedAt": "2026-10-11T16:00:00Z"
},
{
"id": "grill-me",
@@ -258,8 +258,8 @@
"repo": "https://github.com/vercel/next.js.git",
"ref": "canary",
"adapter": "skill-collection",
"commit": "d8bfc3c994670c367e6e31853b78f42e8b7fba85",
"syncedAt": "2026-10-10T16:00:00Z"
"commit": "d8c1c1b0002db8ccb63b58095f338e8bf423fa99",
"syncedAt": "2026-10-11T16:00:00Z"
},
{
"id": "shuorenhua",
@@ -3,5 +3,5 @@
"name": "playwright浏览器自动化操作",
"version": "20260605",
"keySource": "none",
"syncedAt": "2026-10-10T16:02:37Z"
"syncedAt": "2026-10-11T16:02:41Z"
}
@@ -2,8 +2,8 @@
"sourceId": "next-skills",
"repo": "https://github.com/vercel/next.js.git",
"ref": "canary",
"commit": "d8bfc3c994670c367e6e31853b78f42e8b7fba85",
"commit": "d8c1c1b0002db8ccb63b58095f338e8bf423fa99",
"adapter": "skill-collection",
"sourcePath": "skills",
"syncedAt": "2026-10-10T16:00:00Z"
"syncedAt": "2026-10-11T16:00:00Z"
}
@@ -2,8 +2,8 @@
"sourceId": "ppt-master",
"repo": "https://github.com/hugohe3/ppt-master.git",
"ref": "main",
"commit": "b4efe3ddf237a97a42880164759162f8e2412f2f",
"commit": "ca04346de145ed6462ddf34682848e71df5e20a2",
"adapter": "claude-skill",
"sourcePath": "skills/ppt-master",
"syncedAt": "2026-10-08T15:59:59Z"
"syncedAt": "2026-10-11T16:00:00Z"
}
@@ -24,6 +24,14 @@ unified `source_to_md.py` dispatcher writes a missing profile before success.
| Image manifest | `<stem>_files/image_manifest.json` when image metadata is available |
| Conversion profile | `<stem>.conversion_profile.json` beside the Markdown output when written |
For multi-file or directory inputs, the unified entry and direct PDF, document,
Excel, and PowerPoint backends allocate distinct Markdown paths within the run.
Without `-o`, each output stays in its source's directory; with `-o`, outputs
share that directory. Collisions with inputs or other planned outputs gain
`_2`, `_3`, etc., reported as `[INFO] Renamed output ...` on stderr. Assets and
profiles follow the chosen Markdown stem. Separate runs retain the existing
overwrite behavior for unrelated outputs.
When present, the conversion profile is metadata only: converter, source path,
Markdown structure counts, asset directory, image manifest path, and image
count. Downstream PPT workflows still use Markdown and the image manifest as
@@ -76,8 +84,13 @@ Useful options:
refuses an existing file except its own Markdown/text passthrough.
For multi-source project intake, use `project_manager.py import-sources` with
all source paths / URLs. For local files, the default is to keep generated
Markdown and profile outputs beside the original source.
all source paths / URLs. Same-stem convertible sources receive distinct
archive names such as `budget_2.xlsm`, so their Markdown, profiles, assets, and
PPTX intake artifacts stay separate. Pre-existing Markdown and explicitly
supplied same-stem Markdown still suppress automatic conversion; PPTX intake
remains separate for each archived deck. See
[`project.md`](project.md). For standalone local conversion, the default is to
keep generated Markdown and profiles beside the original source.
`source_to_md.py` and the backend converters support single files, explicit
multi-file inputs, and non-recursive directory inputs.
@@ -124,7 +137,8 @@ Native path (no external binary required):
- `.docx` — via `mammoth`; text-only tables (with footnotes) and chart data become pipe Markdown, and OMML / Office Math equations (Word-native or MathType "Convert to Office Math") are rewritten to inline LaTeX. Classic MathType OLE objects carry no OMML and are kept only as their preview image.
- `.html` / `.htm` — via `markdownify` + `beautifulsoup4`
- `.epub` — via `ebooklib` + `markdownify`
- `.ipynb` — via `nbconvert`
- `.ipynb` — via `nbconvert`; code-cell output images and Markdown attachments
use `<output-stem>_files/` with relative Markdown references
Pandoc fallback (only if you need these):
- `.doc`, `.odt`, `.rtf`, `.tex`/`.latex`, `.rst`, `.org`, `.typ`
@@ -155,9 +169,10 @@ pip install mammoth markdownify ebooklib nbconvert beautifulsoup4
# Windows: https://pandoc.org/installing.html
```
All paths produce `<input>.md`. Extracted assets use a sibling `<input>_files/`
By default, each input produces `<input-stem>.md`, subject to batch collision
renaming. Extracted assets use a sibling `<output-stem>_files/`
directory with relative references. Without assets, that directory need not remain.
The direct backend then attempts `<input>.conversion_profile.json` under the
The direct backend then attempts `<output-stem>.conversion_profile.json` under the
shared best-effort sidecar contract.
## `source_to_md/excel_to_md.py`
@@ -250,7 +265,7 @@ Outputs (per source deck, prefixed by file stem):
- `<stem>.slide_library.json` — text slots, geometry, native tables, native chart display caches, and SmartArt nodes/connections
- `source_profile.json` — the single multi-deck index: a compact Strategist-facing digest per deck (over identity, tables, charts, SmartArt, and page types) under `decks[]`, with prefixed artifact pointers
`project_manager.py import-sources` runs this automatically for PPTX/PPTM/PPSX/PPSM/POTX/POTM inputs and stores the bundle directly under `analysis/`. Multi-deck per project: importing several PPTX files gives each its own `<stem>.*` artifacts and a `decks[]` entry in the shared `source_profile.json` index (re-importing the same stem replaces its entry). The beautify profile stays single-deck and reads one chosen deck's `<stem>.*` artifacts.
`project_manager.py import-sources` runs this automatically for PPTX/PPTM/PPSX/PPSM/POTX/POTM inputs and stores the bundle directly under `analysis/`. Multi-deck per project: each deck uses its archived stem, including any collision suffix, for `<stem>.*` artifacts and its `decks[]` entry in the shared `source_profile.json` index. Re-running intake for an existing archived stem replaces its entry. The beautify profile stays single-deck and reads one chosen deck's `<stem>.*` artifacts.
Usage boundary:
- Standard generation uses these fields as facts and recommendation candidates; it does not inherit source slide coordinates or page order by default.
@@ -58,6 +58,11 @@ Notes:
pass the source file/directory once when generated Markdown lives beside the
original source. If Step 1 used `-o` to write Markdown elsewhere, pass both
the original source path/directory and the Markdown output path/directory.
- Distinct same-stem convertible sources in one `import-sources` call
receive archive suffixes (`budget.xlsm` → `budget_2.xlsm`). Markdown, conversion
profiles, `_files/` assets, and PPTX analysis follow the archived stem.
Pre-existing Markdown and explicitly supplied same-stem Markdown still
suppress automatic conversion; PPTX intake still keeps each deck separate.
- A projects-local supplied source directory left strictly empty after import
(or empty from the start) is removed; every directory outside `projects/`
remains untouched. `--copy` never removes directories.
@@ -101,8 +106,9 @@ Notes:
- PPTX-family inputs are enriched automatically under `analysis/` with
per-deck `<stem>.identity.json` / `<stem>.slide_library.json` plus the shared
multi-deck index `source_profile.json` (`decks[]`).
Multi-deck per project: several PPTX imports each get their own `<stem>.*`
artifacts and a `decks[]` entry; re-importing the same stem replaces its entry.
Multi-deck per project: `<stem>` is the archived stem, including any collision
suffix. Each deck gets its own artifacts and `decks[]` entry; re-running
intake for an existing archived stem replaces its entry.
### On-demand page execution view
@@ -820,6 +820,8 @@ class ProjectManager:
raise ValueError("At least one source path or URL is required")
sources_dir = self._source_dir(project_dir)
existing_markdown = {path.resolve() for path in sources_dir.glob("*.md")}
reserved_markdown: set[Path] = set()
summary: dict[str, list[str]] = {
"archived": [],
"url_records": [],
@@ -998,12 +1000,48 @@ class ProjectManager:
summary["notes"].append(note)
continue
archive_target = sources_dir / source_path.name
if source_path.resolve() != archive_target.resolve():
archive_target = self._ensure_unique_path(archive_target)
canonical_markdown_path = archive_target.with_suffix(".md")
has_explicit_markdown = archive_target.stem in explicit_markdown_stems
if suffix in PDF_SUFFIXES | PRESENTATION_SUFFIXES | EXCEL_SUFFIXES | DOC_SUFFIXES:
markdown_target = canonical_markdown_path
keep_markdown = (
has_explicit_markdown or markdown_target.resolve() in existing_markdown
)
if not keep_markdown or markdown_target.resolve() in reserved_markdown:
# Reserve the source stem before conversion, including failed
# conversions that may already have written intake or assets.
stem = archive_target.stem
counter = 2
while (
markdown_target.resolve() in reserved_markdown
or markdown_target.exists()
or archive_target.stem in explicit_markdown_stems
or (archive_target.exists() and archive_target.resolve() != source_path.resolve())
or markdown_target.with_suffix(".conversion_profile.json").exists()
or (sources_dir / f"{archive_target.stem}_files").exists()
or (project_dir / "analysis" / f"{archive_target.stem}.identity.json").exists()
or (project_dir / "analysis" / f"{archive_target.stem}.slide_library.json").exists()
):
archive_target = archive_target.with_name(f"{stem}_{counter}{source_path.suffix}")
markdown_target = archive_target.with_suffix(".md")
counter += 1
# PPTX intake still writes analysis when Markdown is reused.
reserved_markdown.add(markdown_target.resolve())
if not keep_markdown:
canonical_markdown_path = markdown_target
archived_path = self._copy_or_move_file(
source_path,
sources_dir / source_path.name,
archive_target,
move=effective_move,
)
summary["archived"].append(str(archived_path))
if archived_path.name != source_path.name:
summary["notes"].append(
f"{item}: archived as {archived_path.name} to avoid a filename or source-stem collision"
)
if suffix in BITMAP_IMAGE_SUFFIXES:
images_dir = project_dir / "images"
@@ -1017,8 +1055,7 @@ class ProjectManager:
"to avoid a filename collision"
)
elif suffix in PDF_SUFFIXES:
canonical_markdown_path = sources_dir / f"{archived_path.stem}.md"
if archived_path.stem in explicit_markdown_stems:
if has_explicit_markdown:
summary["notes"].append(
f"{item}: skipped PDF auto-conversion because a same-stem Markdown source was provided"
)
@@ -1040,7 +1077,6 @@ class ProjectManager:
except Exception as exc: # pragma: no cover - summary path
summary["skipped"].append(f"{item}: PDF conversion failed ({exc})")
elif suffix in PRESENTATION_SUFFIXES:
canonical_markdown_path = sources_dir / f"{archived_path.stem}.md"
try:
intake_dir = self._import_pptx_intake(archived_path, project_dir)
intake_str = str(intake_dir)
@@ -1048,7 +1084,7 @@ class ProjectManager:
summary["analysis"].append(intake_str)
except Exception as exc: # pragma: no cover - summary path
summary["notes"].append(f"{item}: PPTX intake analysis failed ({exc})")
if archived_path.stem in explicit_markdown_stems:
if has_explicit_markdown:
summary["notes"].append(
f"{item}: skipped presentation auto-conversion because a same-stem Markdown source was provided"
)
@@ -1070,8 +1106,7 @@ class ProjectManager:
except Exception as exc: # pragma: no cover - summary path
summary["skipped"].append(f"{item}: presentation conversion failed ({exc})")
elif suffix in EXCEL_SUFFIXES:
canonical_markdown_path = sources_dir / f"{archived_path.stem}.md"
if archived_path.stem in explicit_markdown_stems:
if has_explicit_markdown:
summary["notes"].append(
f"{item}: skipped Excel auto-conversion because a same-stem Markdown source was provided"
)
@@ -1102,8 +1137,7 @@ class ProjectManager:
f"{item}: archived as a plain-text table source; no Markdown conversion needed"
)
elif suffix in DOC_SUFFIXES:
canonical_markdown_path = sources_dir / f"{archived_path.stem}.md"
if archived_path.stem in explicit_markdown_stems:
if has_explicit_markdown:
summary["notes"].append(
f"{item}: skipped document auto-conversion because a same-stem Markdown source was provided"
)
@@ -84,8 +84,16 @@ def _output_for(
batch_mode: bool,
used_outputs: set[Path],
) -> Path:
if output_arg and batch_mode:
return unique_output_path(Path(output_arg), source.stem, used_outputs)
if batch_mode:
output_dir = Path(output_arg) if output_arg else source.parent
default_output = output_dir / f"{source.stem}.md"
output = unique_output_path(output_dir, source.stem, used_outputs)
if output != default_output:
_print_status(
f"[INFO] Renamed output for {source}: {default_output} -> {output} "
"(input/output collision)"
)
return output
if output_arg:
return Path(output_arg)
return source.with_suffix(".md")
@@ -115,7 +123,7 @@ def run_path_batch(
success_count = 0
failed: list[str] = []
skipped: list[str] = list(expansion_errors)
used_outputs: set[Path] = set()
used_outputs = {source.resolve() for source in sources}
for source in sources:
output = _output_for(source, output_arg, batch_mode, used_outputs)
@@ -1498,18 +1498,23 @@ def _convert_ipynb(input_file: Path, out_file: Path) -> str:
print("[ERROR] nbconvert not installed. Run: pip install nbconvert")
return ""
# Pre-process cell-level markdown attachments: nbconvert leaves
# `attachment:<name>` references intact but doesn't write the files.
# Extract them into our outputs dict so FilesWriter picks them up.
# Extract attachments once with notebook-wide names; nbconvert's own
# attachment extractor can reuse names from different Markdown cells.
nb = nbformat.read(str(input_file), as_version=4)
extra_outputs: dict[str, bytes] = {}
rel_media_dir = f"{out_file.stem}_files"
output_stems = {
Path(output.metadata["filename"]).stem.casefold()
for cell in nb.cells
for output in cell.get("outputs", [])
if output.get("metadata", {}).get("filename")
}
attach_counter = 0
for cell in nb.cells:
if cell.cell_type != "markdown":
continue
attachments = getattr(cell, "attachments", None) or {}
attachments = cell.pop("attachments", None) or {}
if not attachments:
continue
for att_name, mime_data in attachments.items():
@@ -1519,6 +1524,9 @@ def _convert_ipynb(input_file: Path, out_file: Path) -> str:
if ext == ".jpe":
ext = ".jpg"
filename = f"attachment_{attach_counter:03d}{ext}"
while Path(filename).stem.casefold() in output_stems:
attach_counter += 1
filename = f"attachment_{attach_counter:03d}{ext}"
out_path = f"{rel_media_dir}/{filename}"
try:
extra_outputs[out_path] = base64.b64decode(b64)
@@ -1530,7 +1538,9 @@ def _convert_ipynb(input_file: Path, out_file: Path) -> str:
cell.source = src
exporter = MarkdownExporter()
body, resources = exporter.from_notebook_node(nb)
body, resources = exporter.from_notebook_node(
nb, resources={"output_files_dir": rel_media_dir},
)
# Merge attachment outputs with whatever nbconvert collected
resources.setdefault("outputs", {}).update(extra_outputs)
@@ -13,6 +13,7 @@ Dependencies:
from __future__ import annotations
import contextlib
import base64
import io
import json
import socket
@@ -412,6 +413,226 @@ class ArchivedProjectImportTests(unittest.TestCase):
)
class SameStemProjectImportTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name)
self.manager = project_cli.ProjectManager(base_dir=self.root / "projects")
def _project(self, name="target") -> Path:
with contextlib.redirect_stdout(io.StringIO()):
return Path(self.manager.init_project(name, quick_generate=True))
def _workbooks(self, root: Path) -> list[Path]:
from openpyxl import Workbook
root.mkdir(parents=True)
sources = []
for suffix, total in ((".xlsx", 20000), (".xlsm", 22500)):
workbook = Workbook()
workbook.active.append(["Category", "Amount"])
workbook.active.append(["Total", total])
source = root / f"budget{suffix}"
workbook.save(source)
workbook.close()
sources.append(source)
return sources
def _import(self, project: Path, sources: list[Path]) -> dict:
with contextlib.redirect_stdout(io.StringIO()):
return self.manager.import_sources(str(project), list(map(str, sources)), copy=True)
def _assert_conversions(self, project: Path, summary: dict, labels: list[str]) -> None:
self.assertEqual(summary["skipped"], [])
self.assertEqual(len(summary["markdown"]), len(labels))
self.assertEqual(len(set(summary["markdown"])), len(labels))
for raw_path, markdown_path, label in zip(summary["archived"], summary["markdown"], labels):
source = Path(raw_path)
markdown = Path(markdown_path)
self.assertEqual(source.stem, markdown.stem)
content = markdown.read_text(encoding="utf-8").replace(r"\-", "-")
self.assertIn(label, content)
for other_label in labels:
if other_label != label:
self.assertNotIn(other_label, content)
profile = json.loads(profile_path_for(markdown).read_text(encoding="utf-8"))
self.assertEqual(profile["source"]["name"], source.name)
self.assertEqual(profile["outputs"]["markdown"], markdown.name)
def test_same_stem_workbooks_each_convert_with_matching_archived_names(self) -> None:
for reverse in (False, True):
with self.subTest(reverse=reverse):
sources = self._workbooks(self.root / f"inputs_{reverse}")
labels = ["20000", "22500"]
if reverse:
sources.reverse()
labels.reverse()
originals = {source: source.read_bytes() for source in sources}
project = self._project(f"target_{reverse}")
summary = self._import(project, sources)
self._assert_conversions(project, summary, labels)
self.assertEqual([Path(path).name for path in summary["markdown"]],
["budget.md", "budget_2.md"])
for source, archived in zip(sources, summary["archived"]):
self.assertEqual(Path(archived).read_bytes(), originals[source])
self.assertEqual(source.read_bytes(), originals[source])
def test_document_and_presentation_assets_and_intake_keep_each_source(self) -> None:
from docx import Document
from PIL import Image
from pptx import Presentation
from pptx.util import Inches
for kind in ("document", "presentation"):
with self.subTest(kind=kind):
inputs = self.root / kind
inputs.mkdir()
sources = []
expected_images = []
labels = ["FIRST-SOURCE-20000", "SECOND-SOURCE-22500"]
for index, color in enumerate(("red", "blue")):
buffer = io.BytesIO()
Image.new("RGB", (16, 16), color).save(buffer, format="PNG")
raw = buffer.getvalue()
expected_images.append(raw)
if kind == "document":
source = inputs / ("report.docx" if index == 0 else "report.html")
if index == 0:
document = Document()
document.add_paragraph(labels[index])
document.add_picture(io.BytesIO(raw))
document.save(source)
else:
encoded = base64.b64encode(raw).decode("ascii")
source.write_text(
f'<p>{labels[index]}</p><img src="data:image/png;base64,{encoded}">',
encoding="utf-8",
)
else:
source = inputs / ("deck.pptx" if index == 0 else "deck.potx")
presentation = Presentation()
slide = presentation.slides.add_slide(presentation.slide_layouts[6])
slide.shapes.add_textbox(Inches(1), Inches(1), Inches(5), Inches(1)).text = labels[index]
slide.shapes.add_picture(io.BytesIO(raw), Inches(1), Inches(2))
presentation.save(source)
sources.append(source)
project = self._project(kind)
summary = self._import(project, sources)
self._assert_conversions(project, summary, labels)
stems = {Path(path).stem for path in summary["archived"]}
manifest = json.loads((project / "images/image_manifest.json").read_text(encoding="utf-8"))
self.assertEqual({row["source_namespace"] for row in manifest}, stems)
self.assertEqual({(project / "images" / row["filename"]).read_bytes() for row in manifest},
set(expected_images))
for source, raw in zip(summary["archived"], expected_images):
assets = project / "sources" / f"{Path(source).stem}_files"
self.assertEqual(len(list(assets.glob("*.png"))), 1)
self.assertEqual(next(assets.glob("*.png")).read_bytes(), raw)
if kind == "presentation":
index = json.loads((project / "analysis/source_profile.json").read_text(encoding="utf-8"))
self.assertEqual(index["deck_count"], 2)
self.assertEqual({deck["stem"] for deck in index["decks"]}, stems)
self.assertEqual({deck["source_pptx"] for deck in index["decks"]}, set(summary["archived"]))
for stem, label in zip(("deck", "deck_2"), labels):
self.assertTrue((project / "analysis" / f"{stem}.identity.json").is_file())
library = project / "analysis" / f"{stem}.slide_library.json"
self.assertIn(label, library.read_text(encoding="utf-8"))
def test_reused_markdown_does_not_merge_same_stem_presentation_intake(self) -> None:
from pptx import Presentation
from pptx.util import Inches
for explicit in (False, True):
with self.subTest(explicit=explicit):
inputs = self.root / f"companion_{explicit}"
inputs.mkdir()
sources = []
labels = ["FIRST-SOURCE-20000", "SECOND-SOURCE-22500"]
for suffix, label in zip((".pptx", ".potx"), labels):
presentation = Presentation()
slide = presentation.slides.add_slide(presentation.slide_layouts[6])
slide.shapes.add_textbox(Inches(1), Inches(1), Inches(5), Inches(1)).text = label
source = inputs / f"deck{suffix}"
presentation.save(source)
sources.append(source)
project = self._project(f"companion_{explicit}")
(project / "sources").mkdir()
markdown = (inputs if explicit else project / "sources") / "deck.md"
markdown.write_text("# Authoritative Markdown\n", encoding="utf-8")
summary = self._import(project, [*sources, markdown] if explicit else sources)
self.assertEqual(summary["skipped"], [])
self.assertEqual([Path(path).name for path in summary["archived"][:2]],
["deck.pptx", "deck_2.potx"])
self.assertEqual((project / "sources/deck.md").read_text(encoding="utf-8"),
"# Authoritative Markdown\n")
self.assertEqual(list((project / "sources").glob("*.conversion_profile.json")), [])
self.assertEqual(sum("skipped presentation auto-conversion" in note
for note in summary["notes"]), 2)
index = json.loads((project / "analysis/source_profile.json").read_text(encoding="utf-8"))
self.assertEqual(index["deck_count"], 2)
self.assertEqual({deck["stem"] for deck in index["decks"]}, {"deck", "deck_2"})
self.assertEqual({deck["source_pptx"] for deck in index["decks"]},
set(summary["archived"][:2]))
for stem, label, other_label in zip(("deck", "deck_2"), labels, reversed(labels)):
self.assertTrue((project / "analysis" / f"{stem}.identity.json").is_file())
library = (project / "analysis" / f"{stem}.slide_library.json").read_text(encoding="utf-8")
self.assertIn(label, library)
self.assertNotIn(other_label, library)
def test_explicit_markdown_still_suppresses_both_conversions(self) -> None:
for markdown_first in (False, True):
with self.subTest(markdown_first=markdown_first):
sources = self._workbooks(self.root / f"explicit_{markdown_first}")
markdown = sources[0].with_suffix(".md")
markdown.write_text("# Authoritative Markdown\n", encoding="utf-8")
inputs = [markdown, *sources] if markdown_first else [*sources, markdown]
project = self._project(f"explicit_{markdown_first}")
summary = self._import(project, inputs)
self.assertEqual(summary["skipped"], [])
self.assertEqual((project / "sources/budget.md").read_text(encoding="utf-8"),
"# Authoritative Markdown\n")
self.assertEqual(list((project / "sources").glob("*.conversion_profile.json")), [])
self.assertEqual(sum("same-stem Markdown source was provided" in note
for note in summary["notes"]), 2)
def test_markdown_existing_before_import_is_reused(self) -> None:
sources = self._workbooks(self.root / "existing")
project = self._project()
(project / "sources").mkdir()
markdown = project / "sources/budget.md"
markdown.write_text("# Previous conversion\n", encoding="utf-8")
summary = self._import(project, sources)
self.assertEqual(summary["markdown"], [str(markdown), str(markdown)])
self.assertEqual(markdown.read_text(encoding="utf-8"), "# Previous conversion\n")
self.assertEqual(sum("budget.md already exists" in note for note in summary["notes"]), 2)
self.assertEqual(list((project / "sources").glob("*.conversion_profile.json")), [])
def test_generated_suffix_avoids_existing_markdown_and_source_names(self) -> None:
sources = self._workbooks(self.root / "suffixes")
project = self._project()
(project / "sources").mkdir()
existing = project / "sources/budget_2.md"
existing.write_text("# Unrelated existing source\n", encoding="utf-8")
occupied = project / "sources/budget_3.xlsm"
occupied.write_bytes(b"archived original")
summary = self._import(project, sources)
self._assert_conversions(project, summary, ["20000", "22500"])
self.assertEqual([Path(path).name for path in summary["markdown"]],
["budget.md", "budget_4.md"])
self.assertEqual(existing.read_text(encoding="utf-8"), "# Unrelated existing source\n")
self.assertEqual(occupied.read_bytes(), b"archived original")
def test_failed_conversion_still_reserves_its_source_stem(self) -> None:
sources = self._workbooks(self.root / "failure")
sources[0].write_bytes(b"invalid workbook")
project = self._project()
summary = self._import(project, sources)
self.assertEqual(len(summary["skipped"]), 1)
self.assertEqual([Path(path).name for path in summary["markdown"]], ["budget_2.md"])
self.assertIn("22500", Path(summary["markdown"][0]).read_text(encoding="utf-8"))
class _CurlHeaders(dict):
"""Represent repeated curl headers without importing optional curl_cffi."""
@@ -271,6 +271,73 @@ class SourceCollisionTests(unittest.TestCase):
for line in result.stdout.splitlines() if line.startswith("{")
]
def test_direct_pdf_batch_preserves_inputs_and_reports_renames(self) -> None:
try:
import fitz
except ImportError:
self.skipTest("PyMuPDF unavailable")
for reverse in (False, True):
for separate_output in (False, True):
with self.subTest(reverse=reverse, output=separate_output):
root = self.root / f"pdf_{reverse}_{separate_output}"
root.mkdir()
sources = [root / "same.pdf", root / "same.md", root / "same_2.md"]
originals = {}
for index, source in enumerate(sources):
with fitz.open() as document:
page = document.new_page()
page.insert_text((72, 150), f"PDF-SOURCE-{index}")
originals[source] = document.tobytes()
source.write_bytes(originals[source])
if reverse:
sources.reverse()
arguments = [str(path) for path in sources]
if separate_output:
arguments += ["-o", str(root)]
result = subprocess.run(
[sys.executable, str(WEB_BACKEND_DIR / "pdf_to_md.py"),
*arguments, "--images", "none"],
capture_output=True, text=True, encoding="utf-8", check=False,
)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
for source, content in originals.items():
self.assertEqual(source.read_bytes(), content)
profiles = sorted(root.glob("*.conversion_profile.json"))
self.assertEqual(len(profiles), 3)
for path in profiles:
profile = json.loads(path.read_text(encoding="utf-8"))
source = root / profile["source"]["name"]
output = root / profile["outputs"]["markdown"]
self.assertNotIn(output, sources)
self.assertIn(
f"[INFO] Renamed output for {source}: {source.with_suffix('.md')} "
f"-> {output} (input/output collision)", result.stderr,
)
def test_direct_batch_keeps_outputs_in_each_source_directory(self) -> None:
from openpyxl import Workbook
sources = []
for name in ("first", "second"):
directory = self.root / name
directory.mkdir()
source = directory / "same.xlsx"
workbook = Workbook()
workbook.active.append([name])
workbook.save(source)
workbook.close()
sources.append(source)
for _ in range(2):
result = subprocess.run(
[sys.executable, str(WEB_BACKEND_DIR / "excel_to_md.py"), *map(str, sources)],
capture_output=True, text=True, encoding="utf-8", check=False,
)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertNotIn("Renamed output", result.stderr)
for source in sources:
self.assertEqual(list(source.parent.glob("*.md")), [source.with_suffix(".md")])
self.assertIn(source.parent.name, source.with_suffix(".md").read_text(encoding="utf-8"))
def test_same_stem_batch_preserves_markdown_in_both_orders(self) -> None:
for reverse in (False, True):
for output_directory in (False, True):
@@ -5,8 +5,11 @@ Usage: python3 -m unittest tests.test_source_to_md_content_loss_docx
Dependencies: python-docx, mammoth
"""
import base64
import io
import json
import re
import subprocess
import sys
import tempfile
import unittest
@@ -26,6 +29,128 @@ sys.path.insert(0, str(SCRIPTS_DIR / "source_to_md"))
import doc_to_md # noqa: E402
class NotebookContentLossTests(unittest.TestCase):
def test_attachments_do_not_overwrite_named_outputs_or_each_other(self) -> None:
try:
import nbconvert # noqa: F401
import nbformat
except ImportError:
self.skipTest("optional nbconvert/nbformat unavailable")
from PIL import Image
for filename in ("attachment_001", "attachment_001.png"):
with self.subTest(filename=filename), tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
expected = []
encoded = []
for color in ("red", "blue", "green"):
buffer = io.BytesIO()
Image.new("RGB", (8, 8), color).save(buffer, format="PNG")
expected.append(buffer.getvalue())
encoded.append(base64.b64encode(buffer.getvalue()).decode("ascii"))
notebook = nbformat.v4.new_notebook(cells=[
nbformat.v4.new_code_cell("# Named output", outputs=[
nbformat.v4.new_output(
"display_data", data={"image/png": encoded[0]},
metadata={"filename": filename},
),
]),
*[
nbformat.v4.new_markdown_cell(
"![attachment](attachment:picture.png)",
attachments={"picture.png": {"image/png": data}},
)
for data in encoded[1:]
],
])
source = root / "book.ipynb"
nbformat.write(notebook, source)
with redirect_stdout(io.StringIO()):
markdown = doc_to_md.convert_to_markdown(str(source))
refs = re.findall(r"!\[[^\]]*\]\(([^)]+)\)", markdown)
self.assertEqual(len(refs), 3)
self.assertEqual(len(set(refs)), 3)
for ref, raw in zip(refs, expected):
self.assertEqual(Path(ref).parent, Path("book_files"))
self.assertEqual((root / ref).read_bytes(), raw)
manifest = json.loads((root / "book_files/image_manifest.json").read_text(encoding="utf-8"))
self.assertEqual({row["filename"] for row in manifest}, {Path(ref).name for ref in refs})
self.assertTrue(all(len(row["occurrences"]) == 1 for row in manifest))
def test_batch_images_and_attachments_remain_independent_after_import(self) -> None:
try:
import nbconvert # noqa: F401
import nbformat
except ImportError:
self.skipTest("optional nbconvert/nbformat unavailable")
from PIL import Image
from project_management.cli import ProjectManager
for same_stem in (False, True):
with self.subTest(same_stem=same_stem), tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
sources = []
expected = []
for index, color in enumerate(("red", "blue")):
source_dir = root / str(index)
source_dir.mkdir()
source = source_dir / ("book.ipynb" if same_stem else f"book{index}.ipynb")
buffer = io.BytesIO()
Image.new("RGB", (8, 8), color).save(buffer, format="PNG")
raw = buffer.getvalue()
encoded = base64.b64encode(raw).decode("ascii")
output = nbformat.v4.new_output(
"display_data", data={"image/png": encoded},
metadata={"filename": "figure"} if same_stem else {},
)
notebook = nbformat.v4.new_notebook(cells=[
nbformat.v4.new_code_cell(f"# Notebook {index}", outputs=[output]),
nbformat.v4.new_markdown_cell(
"![attachment](attachment:picture.png)",
attachments={"picture.png": {"image/png": encoded}},
),
])
nbformat.write(notebook, source)
sources.append(source)
expected.append(raw)
output_dir = root / "converted"
result = subprocess.run(
[sys.executable, str(SCRIPTS_DIR / "source_to_md/doc_to_md.py"),
*map(str, sources), "-o", str(output_dir)],
capture_output=True, text=True, encoding="utf-8", check=False,
)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
markdown_paths = sorted(output_dir.glob("*.md"))
self.assertEqual(len(markdown_paths), 2)
self.assertEqual(list(output_dir.glob("*.png")), [])
for markdown, raw in zip(markdown_paths, expected):
refs = re.findall(r"!\[[^\]]*\]\(([^)]+)\)", markdown.read_text(encoding="utf-8"))
self.assertEqual(len(refs), 2)
for ref in refs:
self.assertEqual(Path(ref).parent, Path(f"{markdown.stem}_files"))
self.assertEqual((output_dir / ref).read_bytes(), raw)
profile = json.loads(markdown.with_suffix(".conversion_profile.json").read_text())
self.assertEqual(profile["outputs"]["image_count"], 2)
manifest = json.loads((output_dir / profile["outputs"]["image_manifest"]).read_text())
self.assertEqual(len(manifest), 2)
manager = ProjectManager(base_dir=root / "projects")
with redirect_stdout(io.StringIO()):
project = Path(manager.init_project("notebooks", quick_generate=True))
summary = manager.import_sources(str(project), list(map(str, markdown_paths)), copy=True)
self.assertEqual(summary["skipped"], [])
self.assertEqual(len(summary["assets"]), 2)
for path, raw in zip(summary["markdown"], expected):
markdown = Path(path)
refs = re.findall(r"!\[[^\]]*\]\(([^)]+)\)", markdown.read_text(encoding="utf-8"))
for ref in refs:
self.assertEqual((markdown.parent / ref).read_bytes(), raw)
manifest = json.loads((project / "images/image_manifest.json").read_text())
self.assertEqual({row["source_namespace"] for row in manifest},
{path.stem for path in markdown_paths})
self.assertEqual({(project / "images" / row["filename"]).read_bytes() for row in manifest},
set(expected))
class DocxContentLossTests(unittest.TestCase):
def _convert(self, document, patch_archive=None):
with tempfile.TemporaryDirectory() as tmp:
@@ -24,6 +24,49 @@ import excel_to_md # noqa: E402
class ExcelContentLossTests(unittest.TestCase):
def test_same_stem_batch_preserves_each_workbook_and_profile(self) -> None:
for directory_input in (False, True):
for separate_output in (False, True):
with self.subTest(directory=directory_input, output=separate_output):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
sources = []
totals = {"budget.xlsx": 20000, "budget.xlsm": 22500}
for name, total in totals.items():
workbook = Workbook()
workbook.active.append(["Category", "Amount"])
workbook.active.append(["Total", total])
source = root / name
workbook.save(source)
workbook.close()
sources.append(source)
output = root / "output" if separate_output else root
arguments = [root] if directory_input else sources
if separate_output:
arguments += ["-o", output]
result = subprocess.run(
[sys.executable, str(SCRIPTS_DIR / "source_to_md/excel_to_md.py"),
*map(str, arguments)],
capture_output=True, text=True, encoding="utf-8", check=False,
)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertIn("Success: 2/2, Failed: 0", result.stderr)
self.assertIn("[INFO] Renamed output for", result.stderr)
self.assertEqual(sorted(p.name for p in output.glob("*.md")),
["budget.md", "budget_2.md"])
profiles = sorted(output.glob("*.conversion_profile.json"))
self.assertEqual(len(profiles), 2)
seen = set()
for path in profiles:
profile = json.loads(path.read_text(encoding="utf-8"))
source_name = profile["source"]["name"]
seen.add(source_name)
markdown = output / profile["outputs"]["markdown"]
self.assertIn(f"| Total | {totals[source_name]} |",
markdown.read_text(encoding="utf-8"))
self.assertEqual(path, markdown.with_suffix(".conversion_profile.json"))
self.assertEqual(seen, set(totals))
def _convert(self, workbook, **options):
with tempfile.TemporaryDirectory() as tmp:
source = Path(tmp) / "book.xlsx"
@@ -2,8 +2,8 @@
"sourceId": "shadcn",
"repo": "https://github.com/shadcn-ui/ui.git",
"ref": "main",
"commit": "c2a67849be260701852b0977ba15eb5f3a0ce2a4",
"commit": "02cad61166c4c20edbc8a3f71bd720eed4c1f49b",
"adapter": "claude-skill",
"sourcePath": "skills/shadcn",
"syncedAt": "2026-10-10T16:00:00Z"
"syncedAt": "2026-10-11T16:00:00Z"
}
@@ -2,8 +2,8 @@
"sourceId": "ui-ux-pro-max",
"repo": "https://github.com/nextlevelbuilder/ui-ux-pro-max-skill.git",
"ref": "main",
"commit": "50d8a7de0900119855614541f15a1a616691eb33",
"commit": "67803eac81607188194aea9842a723b89326d90d",
"adapter": "claude-skill",
"sourcePath": ".claude/skills/ui-ux-pro-max",
"syncedAt": "2026-10-09T10:26:29Z"
"syncedAt": "2026-10-11T16:00:00Z"
}