diff --git a/.agents/skills/ofb/SKILL.md b/.agents/skills/ofb/SKILL.md index e5eb874..c77aae9 100644 --- a/.agents/skills/ofb/SKILL.md +++ b/.agents/skills/ofb/SKILL.md @@ -32,11 +32,13 @@ frontend/ → catalog-app, Mermaid map view, site chrome ```bash bun install -bun run build:js # after frontend/ changes -bun run generate:site-metadata # regenerate static/sitemap.xml + static/feed.xml from content + git log -bun run index:search # once per dev session (copies public/pagefind → static/pagefind) -zola serve # http://127.0.0.1:1111 -bun run build # production: JS + Zola + pagefind → public/ +bun run dev # instant local serve (zola only) +bun run build:chapter math/measures-dispersion math/loci +bun run build:chapter math/* +bun run build:chapter # auto-detect changed chapters (interactive confirm) +bun run build:site # full rebuild (sync external data + all chapters + site) +bun run build # alias of build:site +bun test ``` ## Non-negotiables @@ -65,7 +67,7 @@ bun run build # production: JS + Zola + pagefind → public/ | Sitemap/feed coverage | `scripts/generate-site-metadata.ts`, `static/sitemap.xml`, `static/feed.xml`, `package.json` build script | | Homepage subject card | `themes/.../templates/index.html` (not yet data-driven) | | Browse / nav URL | `zola.toml` `[extra] browse_url` + `base.html` site-config + `main.tsx` fallback | -| Search UI / index | `base.html` (Pagefind assets + modal), `content/search.md`, `_search.scss`; `bun run index:search` for dev | +| Search UI / index | `base.html` (Pagefind assets + modal), `content/search.md`, `_search.scss`; chapter/site builds refresh Pagefind | ## Sitemap / feed auto-generation @@ -73,7 +75,7 @@ bun run build # production: JS + Zola + pagefind → public/ - `static/sitemap.xml` (all user-facing built pages + live chapters) - `static/feed.xml` (RSS 2.0 for pages + live chapters) - URL timestamps come from `git log -1` on content-source paths. If a page/chapter has no commit yet, generator falls back to filesystem mtime. -- Build pipeline runs this automatically before `zola build` via `package.json`. +- Build pipeline runs this automatically before `zola build`. - Include only resolvable pages; planned chapters without live content are excluded. ## Search (Pagefind) @@ -87,7 +89,7 @@ bun run build # production: JS + Zola + pagefind → public/ | Theming | `themes/.../sass/_search.scss` (`--pf-*` → site tokens) | - **Component UI** only (`pagefind-component-ui.js`), not legacy `pagefind-ui.js`. -- Index runs **after** `zola build`. Production: `bun run build`. Dev: `bun run index:search` copies bundle to `static/pagefind/` (gitignored) for `zola serve`. +- Index runs **after** `zola build` in build commands (`build:chapter`/`build:site`). - v1 indexes **live chapter pages only** (`data-pagefind-body` on `templates/chapter.html`); home, about, catalog, credits, and other site pages are excluded. Detailed checklists: [curriculum-data.md](curriculum-data.md). Human-facing guide: [CONTRIBUTING.md](../../../CONTRIBUTING.md). diff --git a/.agents/skills/zola/SKILL.md b/.agents/skills/zola/SKILL.md index 479d7ce..d1bd716 100644 --- a/.agents/skills/zola/SKILL.md +++ b/.agents/skills/zola/SKILL.md @@ -28,23 +28,22 @@ See [reference.md](reference.md) for Tera pitfalls and build checks. ## Commands ```bash -bun run build:js # sync chapters + Vite build (JS/CSS bundle outputs) -bun run build # smart local build (incremental; uses build cache) -bun run build:full # full deterministic rebuild (use in CI / clean validation) -bun run dev # smart local dev startup (incremental chapter sync + watch + zola serve) -bun run dev:full # force full chapter sync before starting dev server -zola build # required before indexing -bun run index:search # pagefind on public/ → copies to static/pagefind/ for zola serve -zola serve # dev at http://127.0.0.1:1111 +bun run dev # instant serve (zola only) +bun run build:chapter math/loci math/measures-dispersion +bun run build:chapter math/* # wildcard chapter builds +bun run build:chapter # auto-detect changed chapters (interactive confirm) +bun run build:site # full site rebuild (includes sponsor sync when token exists) +bun run build # alias of build:site +bun test ``` After Sass or template changes, run `zola build` (or `zola serve`). Do **not** copy `public/` into `themes/openfreebooks/static/`. ### Build mode behavior -- **First clone / cold start:** `bun run build` behaves like a full pipeline because caches are absent. -- **Continuous local development:** `bun run build` reuses caches and skips unchanged steps where possible. -- **CI:** prefer `bun run build:full` for deterministic, correctness-first builds. +- **Chapter workflow:** prefer `build:chapter` for normal content work; it rebuilds chapter assets and only runs Vite when frontend/site-shell code changes. +- **Site workflow:** use `build:site`/`build` for full rebuilds and release verification. +- **CI:** use `bun run build:site` and `bun test`. **Search (Pagefind):** read [zola-pagefind.md](zola-pagefind.md) before changing search UI, indexing, or `data-pagefind-*` attributes. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c219962..90afe29 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,11 +19,11 @@ Thank you for helping grow free, open textbooks. | Add a **catalog subject banner** | [§ Catalog subject banner](#catalog-subject-banner) | | Add a **curriculum** filter (e.g. GCSE) | [§ New curriculum label](#add-a-curriculum-label) | -**Browse the catalog locally:** `bun run build:js && zola serve` → [http://127.0.0.1:1111/catalog/](http://127.0.0.1:1111/catalog/) +**Browse the catalog locally:** `bun run dev` → [http://127.0.0.1:1111/catalog/](http://127.0.0.1:1111/catalog/) -**Search locally:** After `bun run build:js`, run `bun run index:search` once (or again after changing chapter HTML), then `zola serve`. Try [http://127.0.0.1:1111/search/](http://127.0.0.1:1111/search/) or press ⌘K / Ctrl+K. +**Search locally:** Run `bun run build:chapter ...` or `bun run build:site` after content changes to refresh the index, then use `bun run dev`. Try [http://127.0.0.1:1111/search/](http://127.0.0.1:1111/search/) or press ⌘K / Ctrl+K. -**Build modes:** `bun run build` is smart/incremental for local iteration; use `bun run build:full` for a deterministic full rebuild (CI/release checks). +**Build modes:** use `bun run build:chapter` for targeted chapter builds and `bun run build:site` for full rebuilds. --- @@ -37,7 +37,7 @@ content/{subject}/{slug}/ → everything for the live page (one folder) - **Catalog** (`/catalog/`) — list, map (`?view=tree`), and for Mathematics a Compare doc (`?view=compare`, HTML partial `themes/openfreebooks/templates/partials/math-curricula-compare.html`). Curriculum names (DSE, IB, …) appear only in the catalog — not inside chapter prose ([spec.md](spec.md)). - Every chapter needs a **`description`** in curriculum JSON (map clamps to three lines). -- **`bun run sync:chapters`** (also run automatically by `bun run build:js`) merges `core.html` + `supplement.html` into `_index.md` for Zola. Edit the HTML files, not the merged body in `_index.md`. +- Chapter sync (merging `core.html` + `supplement.html` into `_index.md`) runs automatically in build commands. Edit HTML sources, not merged `_index.md`. ### Map graph rules (summary) @@ -130,15 +130,15 @@ strand = "Number & Algebra" ### 4. Build and check ```bash -bun run build:js # sync:chapters + Vite bundle -zola serve +bun run build:chapter math/your-chapter-slug +bun run dev ``` - `/catalog/?subject=math` — chapter appears and links correctly. - `/math/your-chapter-slug/` — page renders. - `bun run build` — full production build. -New subjects with shared CSS/JS: add `content/{subject}/subject.scss` and/or `subject.ts`; `sync:chapters` wires them automatically. No per-chapter template branches. +New subjects with shared CSS/JS: add `content/{subject}/subject.scss` and/or `subject.ts`; chapter sync during build wires them automatically. No per-chapter template branches. --- @@ -163,7 +163,7 @@ Add edges in `data/{subject}-curriculum.json`: 1. `data/catalog.json` — add the subject to `subjects`. 2. Create `data/{id}-curriculum.json` (same schema as `math-curriculum.json`; `strands` + optional `graph.edges`). 3. Add `content/{subject}/` when you have live chapters. -4. Run `bun run build:js` (this syncs chapter content and auto-registers subject CSS/JS modules). +4. Run `bun run build:chapter {subject}/*` (this syncs chapters, rebuilds relevant widgets, and updates global outputs). ### Subject scaffold (copyable) @@ -210,7 +210,7 @@ cwebp -resize 1400 0 -q 82 your-banner.jpg -o static/catalog/banners/math.webp { "id": "math", "name": "Mathematics", "banner": "/catalog/banners/math.webp" } ``` -4. Check locally: `bun run build:js && zola serve` → `/catalog/?subject=math` +4. Check locally: `bun run build:chapter math/* && bun run dev` → `/catalog/?subject=math` Omit `banner` for subjects that should stay unchanged (e.g. roadmap-only placeholders). @@ -228,14 +228,13 @@ Omit `banner` for subjects that should stay unchanged (e.g. roadmap-only placeho ```bash bun install -bun run sync:chapters # after content/ chapter changes -bun run build:js # sync + Vite -bun run dev # incremental sync + watch + zola serve -bun run dev:full # force full sync before dev -zola serve +bun run dev +bun run build:chapter math/measures-dispersion math/loci +bun run build:chapter math/* +bun run build:chapter # auto-detect changed chapters, list, confirm +bun run build:site bun test bun run build -bun run build:full ``` - Do **not** commit `themes/openfreebooks/static/` (Vite output). diff --git a/README.md b/README.md index 6ac3df5..ee9de77 100644 --- a/README.md +++ b/README.md @@ -55,29 +55,37 @@ Install JS dependencies: bun install ``` -For day-to-day local development, run JS watch + Zola serve together: +Serve locally: ```bash bun run dev ``` -Open [http://127.0.0.1:1111](http://127.0.0.1:1111). Re-run `bun run index:search` when you change chapter HTML or add pages so search stays up to date. +Open [http://127.0.0.1:1111](http://127.0.0.1:1111). -Use `bun run dev:full` when you want to force a full chapter sync before starting dev servers. +Search UI is available in local serve; run a build command to refresh the search index (`static/pagefind/` is gitignored and copied from build output). -Search UI is available immediately; results require the index step above (`static/pagefind/` is gitignored and copied from the build output). - -## Production build +## Build commands ```bash -bun run build +bun run build:chapter math/measures-dispersion math/loci +bun run build:chapter math/* +bun run build:chapter # auto-detect changed chapters, list them, ask for confirmation +bun run build:site # full site build (includes sponsor sync when GITHUB_TOKEN is set) +bun run build # alias of build:site ``` -`bun run build` is smart/incremental for local development: -- first run (or missing cache) behaves like a full build, -- repeat runs skip unchanged steps. +`build:chapter` updates chapter sync, chapter widget JS (targeted), metadata, site output, and search index. -Use `bun run build:full` for a deterministic full rebuild (recommended for CI/release validation). +In non-interactive environments, `build:chapter` requires explicit chapter arguments. + +## Testing + +```bash +bun test +``` + +`bun test` includes curriculum validation. ## Deploy to Cloudflare diff --git a/bun.lock b/bun.lock index 6da812f..58c49ff 100644 --- a/bun.lock +++ b/bun.lock @@ -13,6 +13,7 @@ "three": "^0.184.0", }, "devDependencies": { + "@types/bun": "^1.3.14", "pagefind": "^1.5.0", "sass": "^1.89.2", "typescript": "^5.8.3", @@ -234,6 +235,8 @@ "@types/babel__traverse": ["@types/babel__traverse@7.28.0", "", { "dependencies": { "@babel/types": "^7.28.2" } }, "sha512-8PvcXf70gTDZBgt9ptxJ8elBeBjcLOAcOtoO/mPJjtji1+CdGbHgm77om1GrsPxsiE+uXIpNSK64UYaIwQXd4Q=="], + "@types/bun": ["@types/bun@1.3.14", "", { "dependencies": { "bun-types": "1.3.14" } }, "sha512-h1hFqFVcvAvD9j9K7ZW7vd82aSA+rTdznZa+5bwvCwqSB1jmmfLcbIWhOLx1/+boy/xmjgCs/OMUL8hRJSmnPw=="], + "@types/d3": ["@types/d3@7.4.3", "", { "dependencies": { "@types/d3-array": "*", "@types/d3-axis": "*", "@types/d3-brush": "*", "@types/d3-chord": "*", "@types/d3-color": "*", "@types/d3-contour": "*", "@types/d3-delaunay": "*", "@types/d3-dispatch": "*", "@types/d3-drag": "*", "@types/d3-dsv": "*", "@types/d3-ease": "*", "@types/d3-fetch": "*", "@types/d3-force": "*", "@types/d3-format": "*", "@types/d3-geo": "*", "@types/d3-hierarchy": "*", "@types/d3-interpolate": "*", "@types/d3-path": "*", "@types/d3-polygon": "*", "@types/d3-quadtree": "*", "@types/d3-random": "*", "@types/d3-scale": "*", "@types/d3-scale-chromatic": "*", "@types/d3-selection": "*", "@types/d3-shape": "*", "@types/d3-time": "*", "@types/d3-time-format": "*", "@types/d3-timer": "*", "@types/d3-transition": "*", "@types/d3-zoom": "*" } }, "sha512-lZXZ9ckh5R8uiFVt8ogUNf+pIrK4EsWrx2Np75WvF/eTpJ0FMHNhjXk8CKEx/+gpHbNQyJWehbFaTvqmHWB3ww=="], "@types/d3-array": ["@types/d3-array@3.2.2", "", {}, "sha512-hOLWVbm7uRza0BYXpIIW5pxfrKe0W+D5lrFiAEYR+pb6w3N2SwSMaJbXdUfSEv+dT4MfHBLtn5js0LAWaO6otw=="], @@ -300,6 +303,8 @@ "@types/geojson": ["@types/geojson@7946.0.16", "", {}, "sha512-6C8nqWur3j98U6+lXDfTUWIfgvZU+EumvpHKcYjujKH7woYyLj2sUmff0tRhrqM7BohUw7Pz3ZB1jj2gW9Fvmg=="], + "@types/node": ["@types/node@25.9.1", "", { "dependencies": { "undici-types": ">=7.24.0 <7.24.7" } }, "sha512-xfrlY7UD5rMJk3ZVJP8BNzS28J36YJg+xp+LPXV1TdWxr8uMH5A860QNxYDGQe/ylDSgjxE52Q9VnO7p75tJxg=="], + "@types/trusted-types": ["@types/trusted-types@2.0.7", "", {}, "sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw=="], "@upsetjs/venn.js": ["@upsetjs/venn.js@2.0.0", "", { "optionalDependencies": { "d3-selection": "^3.0.0", "d3-transition": "^3.0.1" } }, "sha512-WbBhLrooyePuQ1VZxrJjtLvTc4NVfpOyKx0sKqioq9bX1C1m7Jgykkn8gLrtwumBioXIqam8DLxp88Adbue6Hw=="], @@ -316,6 +321,8 @@ "browserslist": ["browserslist@4.28.2", "", { "dependencies": { "baseline-browser-mapping": "^2.10.12", "caniuse-lite": "^1.0.30001782", "electron-to-chromium": "^1.5.328", "node-releases": "^2.0.36", "update-browserslist-db": "^1.2.3" }, "bin": { "browserslist": "cli.js" } }, "sha512-48xSriZYYg+8qXna9kwqjIVzuQxi+KYWp2+5nCYnYKPTr0LvD89Jqk2Or5ogxz0NUMfIjhh2lIUX/LyX9B4oIg=="], + "bun-types": ["bun-types@1.3.14", "", { "dependencies": { "@types/node": "*" } }, "sha512-4N0ig0fEomHt5R0KCFWjovxow98rIoRwKolrYdCcknNwMekCXRnWEUvgu5soYV8QXtVsrUD8B95MBOZGPvr6KQ=="], + "caniuse-lite": ["caniuse-lite@1.0.30001793", "", {}, "sha512-iwSsYWaCOoh26cV8NwNRViHlrfUvYsHDfRVcbtmw0Kg6PJIZZXwMkj1442FYLBGkeUf1juAsU3DTfxW579mrPA=="], "chokidar": ["chokidar@4.0.3", "", { "dependencies": { "readdirp": "^4.0.1" } }, "sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA=="], @@ -564,6 +571,8 @@ "uhyphen": ["uhyphen@0.2.0", "", {}, "sha512-qz3o9CHXmJJPGBdqzab7qAYuW8kQGKNEuoHFYrBwV6hWIMcpAmxDLXojcHfFr9US1Pe6zUswEIJIbLI610fuqA=="], + "undici-types": ["undici-types@7.24.6", "", {}, "sha512-WRNW+sJgj5OBN4/0JpHFqtqzhpbnV0GuB+OozA9gCL7a993SmU+1JBZCzLNxYsbMfIeDL+lTsphD5jN5N+n0zg=="], + "update-browserslist-db": ["update-browserslist-db@1.2.3", "", { "dependencies": { "escalade": "^3.2.0", "picocolors": "^1.1.1" }, "peerDependencies": { "browserslist": ">= 4.21.0" }, "bin": { "update-browserslist-db": "cli.js" } }, "sha512-Js0m9cx+qOgDxo0eMiFGEueWztz+d4+M3rGlmKPT+T4IS/jP4ylw3Nwpu6cpTTP8R1MAC1kF4VbdLt3ARf209w=="], "uuid": ["uuid@14.0.0", "", { "bin": { "uuid": "dist-node/bin/uuid" } }, "sha512-Qo+uWgilfSmAhXCMav1uYFynlQO7fMFiMVZsQqZRMIXp0O7rR7qjkj+cPvBHLgBqi960QCoo/PH2/6ZtVqKvrg=="], diff --git a/data/github-sponsors.json b/data/github-sponsors.json index bdbe449..441e3cf 100644 --- a/data/github-sponsors.json +++ b/data/github-sponsors.json @@ -2,7 +2,7 @@ "title": "GitHub Sponsors", "intro": "Supporters who help keep Open Free Books free and open for learners everywhere.", "profile_url": "https://github.com/sponsors/HananoshikaYomaru", - "fetched_at": "2026-05-21T18:06:17.197Z", + "fetched_at": "2026-05-22T18:41:21.577Z", "sponsors": [ { "login": "cybersader", diff --git a/frontend/src/main.tsx b/frontend/src/main.tsx index 6798395..c057ad4 100644 --- a/frontend/src/main.tsx +++ b/frontend/src/main.tsx @@ -82,20 +82,20 @@ function mountChapterWidgets() { if (mounts.length === 0) return; - void import("./generated/chapter-widgets").then(({ chapterWidgetLoaders }) => { + void import("./generated/chapter-widgets").then(({ chapterWidgetUrls }) => { const loadPromises: Promise[] = []; for (const el of mounts) { const widget = el.dataset.widget; if (!widget) continue; const loaderKey = `${chapterKey}/${widget}`; - const loader = chapterWidgetLoaders[loaderKey]; - if (!loader) { - console.warn(`No widget loader registered for ${loaderKey}`); + const moduleUrl = chapterWidgetUrls[loaderKey]; + if (!moduleUrl) { + console.warn(`No widget module URL registered for ${loaderKey}`); continue; } loadPromises.push( - loader().then((mod) => { + import(/* @vite-ignore */ moduleUrl).then((mod) => { render(() => mod.default({}), el); }) ); diff --git a/package.json b/package.json index 5ea30af..057f61a 100644 --- a/package.json +++ b/package.json @@ -4,21 +4,11 @@ "private": true, "type": "module", "scripts": { - "sync:chapters": "bun scripts/sync-chapters.ts", - "sync:sponsors": "bun scripts/fetch-github-sponsors.ts", - "generate:site-metadata": "bun scripts/generate-site-metadata.ts", - "build:js": "bun run sync:chapters && vite build", - "build:site": "bun run generate:site-metadata && zola build && bun run index:search", - "build:full": "bun run check:static-html && bun run validate:curriculum && rm -f themes/openfreebooks/static/main.css public/main.css && bun run sync:chapters -- --full && vite build && bun run build:site", - "build": "bun scripts/build-smart.ts", - "index:search": "pagefind --site public --glob \"**/index.html\" && rm -rf static/pagefind && cp -R public/pagefind static/pagefind", - "dev:js": "vite build --watch", - "dev:full": "bash scripts/dev.sh --full", - "dev": "bash scripts/dev.sh", - "check:static-html": "bash scripts/check-static-html.sh", - "scan:size": "bun scripts/scan-output-size.ts", - "test": "bun test", - "validate:curriculum": "bun scripts/validate-curriculum.ts" + "build:chapter": "bun scripts/build-chapter.ts", + "build:site": "bun scripts/build-site.ts", + "build": "bun run build:site", + "dev": "zola serve", + "test": "bun test" }, "dependencies": { "defuddle": "^0.18.1", @@ -33,6 +23,7 @@ "pagefind": "^1.5.0", "typescript": "^5.8.3", "vite": "^6.3.5", - "vite-plugin-solid": "^2.11.6" + "vite-plugin-solid": "^2.11.6", + "@types/bun": "^1.3.14" } } diff --git a/scripts/build-chapter-widgets.ts b/scripts/build-chapter-widgets.ts new file mode 100644 index 0000000..faf71e3 --- /dev/null +++ b/scripts/build-chapter-widgets.ts @@ -0,0 +1,137 @@ +import { existsSync } from "node:fs"; +import { mkdir, readdir, readFile, rm, writeFile } from "node:fs/promises"; +import { basename, dirname, join } from "node:path"; +import { build } from "vite"; +import solid from "vite-plugin-solid"; +import { fingerprintPaths } from "./build-change-detector"; + +const ROOT = join(import.meta.dir, ".."); +const CONTENT = join(ROOT, "content"); +const OUTPUT_ROOT = join(ROOT, "themes/openfreebooks/static/js/chapter-widgets"); +const CACHE_PATH = join(ROOT, "data/_generated/chapter-widgets-cache.json"); + +type Chapter = { subject: string; slug: string; dir: string; key: string }; +type CacheFile = { version: 1; chapters: Record }; + +async function discoverChapters(): Promise { + const output: Chapter[] = []; + const subjects = await readdir(CONTENT, { withFileTypes: true }); + for (const subjectEntry of subjects) { + if (!subjectEntry.isDirectory() || subjectEntry.name.startsWith(".")) continue; + const subject = subjectEntry.name; + const subjectDir = join(CONTENT, subject); + const chapterEntries = await readdir(subjectDir, { withFileTypes: true }); + for (const chapterEntry of chapterEntries) { + if (!chapterEntry.isDirectory() || chapterEntry.name.startsWith(".")) continue; + const slug = chapterEntry.name; + const dir = join(subjectDir, slug); + const indexPath = join(dir, "_index.md"); + if (!existsSync(indexPath)) continue; + const indexRaw = await readFile(indexPath, "utf8"); + if (!/template\s*=\s*["']chapter\.html["']/.test(indexRaw)) continue; + output.push({ subject, slug, dir, key: `${subject}/${slug}` }); + } + } + return output; +} + +async function listWidgets(chapter: Chapter): Promise { + const widgetsDir = join(chapter.dir, "widgets"); + if (!existsSync(widgetsDir)) return []; + const entries = await readdir(widgetsDir, { withFileTypes: true }); + return entries + .filter((entry) => entry.isFile() && entry.name.endsWith(".tsx") && !entry.name.startsWith("_")) + .map((entry) => basename(entry.name, ".tsx")) + .sort(); +} + +async function chapterFingerprint(chapter: Chapter): Promise { + return fingerprintPaths(ROOT, [join(chapter.dir, "widgets")]); +} + +async function readCache(): Promise { + if (!existsSync(CACHE_PATH)) return { version: 1, chapters: {} }; + try { + const parsed = JSON.parse(await readFile(CACHE_PATH, "utf8")) as CacheFile; + if (parsed.version !== 1) return { version: 1, chapters: {} }; + return parsed; + } catch { + return { version: 1, chapters: {} }; + } +} + +async function buildWidget(chapter: Chapter, widget: string) { + const entry = join(chapter.dir, "widgets", `${widget}.tsx`); + const outDir = join(OUTPUT_ROOT, chapter.subject, chapter.slug); + await mkdir(outDir, { recursive: true }); + await build({ + configFile: false, + plugins: [solid()], + publicDir: false, + resolve: { + alias: { + "@ofb/katex": join(ROOT, "frontend/src/lib/katex.ts"), + }, + }, + build: { + outDir, + emptyOutDir: false, + lib: { + entry, + formats: ["es"], + fileName: () => `${widget}.js`, + }, + rollupOptions: { + treeshake: false, + output: { + chunkFileNames: "assets/[name]-[hash].js", + assetFileNames: "assets/[name]-[hash][extname]", + }, + }, + }, + }); +} + +function parseTargets(args: string[]): { full: boolean; chapterKeys: string[] } { + const full = args.includes("--full"); + const chapterKeys = args.filter((arg) => !arg.startsWith("--")); + return { full, chapterKeys }; +} + +async function main() { + const { full, chapterKeys } = parseTargets(process.argv.slice(2)); + const chapters = await discoverChapters(); + const chapterByKey = new Map(chapters.map((chapter) => [chapter.key, chapter])); + const targets = + chapterKeys.length === 0 + ? chapters + : chapterKeys + .map((key) => chapterByKey.get(key)) + .filter((chapter): chapter is Chapter => Boolean(chapter)); + + const prev = await readCache(); + const next: CacheFile = { version: 1, chapters: { ...prev.chapters } }; + + for (const chapter of targets) { + const fingerprint = await chapterFingerprint(chapter); + const unchanged = !full && prev.chapters[chapter.key] === fingerprint; + if (unchanged) { + console.log(`Unchanged widgets ${chapter.key} (skipped)`); + continue; + } + + const widgets = await listWidgets(chapter); + const chapterOutDir = join(OUTPUT_ROOT, chapter.subject, chapter.slug); + await rm(chapterOutDir, { recursive: true, force: true }); + for (const widget of widgets) { + await buildWidget(chapter, widget); + } + next.chapters[chapter.key] = fingerprint; + console.log(`Built widgets ${chapter.key} (${widgets.length})`); + } + + await mkdir(dirname(CACHE_PATH), { recursive: true }); + await writeFile(CACHE_PATH, `${JSON.stringify(next, null, 2)}\n`); +} + +main(); diff --git a/scripts/build-chapter.ts b/scripts/build-chapter.ts new file mode 100644 index 0000000..fe2a368 --- /dev/null +++ b/scripts/build-chapter.ts @@ -0,0 +1,186 @@ +import { spawn } from "node:child_process"; +import { existsSync } from "node:fs"; +import { mkdir, readdir, readFile, writeFile } from "node:fs/promises"; +import { dirname, join } from "node:path"; +import { createInterface } from "node:readline/promises"; +import { stdin as input, stdout as output } from "node:process"; +import { $ } from "bun"; +import { fingerprintPaths } from "./build-change-detector"; + +const ROOT = join(import.meta.dir, ".."); +const CONTENT = join(ROOT, "content"); +const BUNDLE_PATH = join(ROOT, "themes/openfreebooks/static/js/bundle.js"); +const BUILD_CACHE_PATH = join(ROOT, "data/_generated/build-chapter-cache.json"); + +type Chapter = { key: string; subject: string; slug: string }; +type BuildCache = { version: 1; shellFingerprint: string }; + +async function runStep(command: string) { + const result = await $`bash -lc ${command}`; + if (result.exitCode !== 0) process.exit(result.exitCode || 1); +} + +async function discoverChapters(): Promise { + const output: Chapter[] = []; + const subjects = await readdir(CONTENT, { withFileTypes: true }); + for (const subjectEntry of subjects) { + if (!subjectEntry.isDirectory() || subjectEntry.name.startsWith(".")) continue; + const subject = subjectEntry.name; + const subjectDir = join(CONTENT, subject); + const chapterEntries = await readdir(subjectDir, { withFileTypes: true }); + for (const chapterEntry of chapterEntries) { + if (!chapterEntry.isDirectory() || chapterEntry.name.startsWith(".")) continue; + const slug = chapterEntry.name; + const indexPath = join(subjectDir, slug, "_index.md"); + if (!existsSync(indexPath)) continue; + const raw = await readFile(indexPath, "utf8"); + if (!/template\s*=\s*["']chapter\.html["']/.test(raw)) continue; + output.push({ key: `${subject}/${slug}`, subject, slug }); + } + } + return output.sort((a, b) => a.key.localeCompare(b.key)); +} + +function patternToRegex(pattern: string): RegExp { + const escaped = pattern.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*"); + return new RegExp(`^${escaped}$`); +} + +function resolvePatterns(chapters: Chapter[], patterns: string[]): Chapter[] { + const results = new Map(); + for (const pattern of patterns) { + const regex = patternToRegex(pattern); + const matches = chapters.filter((chapter) => regex.test(chapter.key)); + for (const chapter of matches) { + results.set(chapter.key, chapter); + } + } + return [...results.values()].sort((a, b) => a.key.localeCompare(b.key)); +} + +async function changedChapterKeys(chapterKeys: Set): Promise { + const trackedDiff = await $`git --no-pager diff --name-only HEAD -- content`.text(); + const untracked = await $`git --no-pager ls-files --others --exclude-standard -- content`.text(); + const files = new Set( + [...trackedDiff.split("\n"), ...untracked.split("\n")] + .map((line) => line.trim()) + .filter((line) => line.length > 0) + ); + + const keys = new Set(); + for (const file of files) { + const match = file.match(/^content\/([^/]+)\/([^/]+)\//); + if (!match) continue; + const key = `${match[1]}/${match[2]}`; + if (chapterKeys.has(key)) keys.add(key); + } + return [...keys].sort(); +} + +async function confirmDetectedChanges(chapters: Chapter[], frontendChanged: boolean): Promise { + if (chapters.length > 0) { + console.log("[build:chapter] Auto-detected changed chapters:"); + for (const chapter of chapters) { + console.log(` - ${chapter.key}`); + } + } + if (frontendChanged) { + console.log("[build:chapter] Auto-detected frontend/site-shell changes."); + } + const rl = createInterface({ input, output }); + const answer = (await rl.question("Proceed? [y/N] ")).trim().toLowerCase(); + rl.close(); + return answer === "y" || answer === "yes"; +} + +async function readBuildCache(): Promise { + if (!existsSync(BUILD_CACHE_PATH)) return null; + try { + const parsed = JSON.parse(await readFile(BUILD_CACHE_PATH, "utf8")) as BuildCache; + if (parsed.version !== 1) return null; + return parsed; + } catch { + return null; + } +} + +async function writeBuildCache(cache: BuildCache) { + await mkdir(dirname(BUILD_CACHE_PATH), { recursive: true }); + await writeFile(BUILD_CACHE_PATH, `${JSON.stringify(cache, null, 2)}\n`); +} + +async function shellFingerprint(): Promise { + return fingerprintPaths(ROOT, [join(ROOT, "frontend"), join(ROOT, "vite.config.ts")]); +} + +function runPagefindInBackground() { + const command = + 'pagefind --site public --glob "**/index.html" && rm -rf static/pagefind && cp -R public/pagefind static/pagefind'; + const child = spawn("bash", ["-lc", command], { + cwd: ROOT, + detached: true, + stdio: "ignore", + }); + child.unref(); + console.log("[build:chapter] Pagefind indexing started in background."); +} + +async function main() { + const patterns = process.argv.slice(2).filter((arg) => !arg.startsWith("--")); + const chapters = await discoverChapters(); + const chapterByKey = new Map(chapters.map((chapter) => [chapter.key, chapter])); + const previousBuild = await readBuildCache(); + const currentShellFingerprint = await shellFingerprint(); + const frontendChanged = + !existsSync(BUNDLE_PATH) || previousBuild?.shellFingerprint !== currentShellFingerprint; + + let targets: Chapter[] = []; + + if (patterns.length > 0) { + targets = resolvePatterns(chapters, patterns); + if (targets.length === 0) { + console.error(`[build:chapter] No chapters matched patterns: ${patterns.join(", ")}`); + process.exit(1); + } + } else { + if (!input.isTTY || !output.isTTY) { + console.error("[build:chapter] Non-interactive mode requires explicit chapter arguments."); + process.exit(1); + } + const changedKeys = await changedChapterKeys(new Set(chapters.map((chapter) => chapter.key))); + targets = changedKeys.map((key) => chapterByKey.get(key)).filter((c): c is Chapter => Boolean(c)); + + if (targets.length === 0 && !frontendChanged) { + console.log("[build:chapter] No changed chapters or frontend changes detected."); + return; + } + + const confirmed = await confirmDetectedChanges(targets, frontendChanged); + if (!confirmed) { + console.log("[build:chapter] Cancelled."); + return; + } + } + + if (targets.length > 0) { + await runStep("bun scripts/sync-chapters.ts"); + await runStep(`bun scripts/build-chapter-widgets.ts ${targets.map((t) => t.key).join(" ")}`); + } + + const needsShellBuild = !existsSync(BUNDLE_PATH) || frontendChanged; + if (needsShellBuild) { + await runStep("rm -f themes/openfreebooks/static/main.css public/main.css"); + await runStep("vite build"); + } + + await runStep("bun scripts/generate-site-metadata.ts"); + await runStep("rm -rf public"); + await runStep("zola build"); + runPagefindInBackground(); + + await writeBuildCache({ version: 1, shellFingerprint: currentShellFingerprint }); + const summaryTargets = targets.length > 0 ? targets.map((target) => target.key).join(", ") : "(frontend only)"; + console.log(`[build:chapter] Core build done: ${summaryTargets}`); +} + +void main(); diff --git a/scripts/build-site.ts b/scripts/build-site.ts new file mode 100644 index 0000000..7714b37 --- /dev/null +++ b/scripts/build-site.ts @@ -0,0 +1,25 @@ +import { $ } from "bun"; + +async function runStep(command: string) { + const result = await $`bash -lc ${command}`; + if (result.exitCode !== 0) process.exit(result.exitCode || 1); +} + +async function main() { + await runStep("bash scripts/check-static-html.sh"); + await runStep("rm -f themes/openfreebooks/static/main.css public/main.css"); + if (process.env.GITHUB_TOKEN?.trim()) { + await runStep("bun scripts/fetch-github-sponsors.ts"); + } else { + console.log("[build:site] GITHUB_TOKEN not set; skipping sponsor sync."); + } + await runStep("bun scripts/sync-chapters.ts --full"); + await runStep("bun scripts/build-chapter-widgets.ts --full"); + await runStep("vite build"); + await runStep("bun scripts/generate-site-metadata.ts"); + await runStep("rm -rf public"); + await runStep("zola build"); + await runStep('pagefind --site public --glob "**/index.html" && rm -rf static/pagefind && cp -R public/pagefind static/pagefind'); +} + +void main(); diff --git a/scripts/dev.sh b/scripts/dev.sh deleted file mode 100644 index cb2e091..0000000 --- a/scripts/dev.sh +++ /dev/null @@ -1,21 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -cleanup() { - if [[ -n "${JS_PID:-}" ]] && kill -0 "${JS_PID}" 2>/dev/null; then - kill "${JS_PID}" - fi -} - -trap cleanup EXIT INT TERM - -if [[ "${1:-}" == "--full" ]]; then - bun run sync:chapters -- --full -else - bun run sync:chapters -fi - -bun run dev:js & -JS_PID=$! - -zola serve diff --git a/scripts/sync-chapters.ts b/scripts/sync-chapters.ts index d0c27be..c509a56 100644 --- a/scripts/sync-chapters.ts +++ b/scripts/sync-chapters.ts @@ -123,6 +123,13 @@ async function copyDir(srcDir: string, destDir: string) { } } +async function writeIfChanged(path: string, content: string) { + const current = existsSync(path) ? await readFile(path, "utf8") : null; + if (current === content) return; + await mkdir(dirname(path), { recursive: true }); + await writeFile(path, content); +} + async function compileChapterScss( subject: string, slug: string, @@ -361,18 +368,14 @@ function buildWidgetRegistry( const lines: string[] = [ "// Auto-generated by scripts/sync-chapters.ts — do not edit.", "", - "import type { Component } from \"solid-js\";", - "", - "export type ChapterWidgetModule = { default: Component };", - "", - "export const chapterWidgetLoaders: Record Promise> = {", + "export const chapterWidgetUrls: Record = {", ]; - for (const { key, widgets, dir } of entries) { - const relFromGenerated = relative(dirname(GENERATED_WIDGETS), dir).replace(/\\/g, "/"); + for (const { key, widgets } of entries) { + const [subject, slug] = key.split("/"); for (const widget of widgets) { - const importPath = `${relFromGenerated}/widgets/${widget}`; - lines.push(` "${key}/${widget}": () => import("${importPath}"),`); + const importPath = `/js/chapter-widgets/${subject}/${slug}/${widget}.js`; + lines.push(` "${key}/${widget}": "${importPath}",`); } } @@ -411,14 +414,11 @@ async function main() { widgetEntries.push({ key, widgets, dir: chapter.dir }); } - await mkdir(dirname(GENERATED_META), { recursive: true }); - await writeFile(GENERATED_META, `${JSON.stringify(metaFile, null, 2)}\n`); - - await mkdir(dirname(GENERATED_WIDGETS), { recursive: true }); - await writeFile(GENERATED_WIDGETS, buildWidgetRegistry(widgetEntries)); + await writeIfChanged(GENERATED_META, `${JSON.stringify(metaFile, null, 2)}\n`); + await writeIfChanged(GENERATED_WIDGETS, buildWidgetRegistry(widgetEntries)); const chapterNav = await buildChapterNav(chapters); - await writeFile(GENERATED_NAV, `${JSON.stringify(chapterNav, null, 2)}\n`); + await writeIfChanged(GENERATED_NAV, `${JSON.stringify(chapterNav, null, 2)}\n`); const subjectEntries: Array<{ id: string; dir: string; hasJs: boolean }> = []; const subjectsMeta: SubjectsMetaFile = { subjects: {} }; @@ -446,9 +446,8 @@ async function main() { subjectsMeta.subjects[subject.id] = meta; subjectEntries.push({ id: subject.id, dir: subject.dir, hasJs: meta.hasJs }); } - await writeFile(GENERATED_SUBJECTS_META, `${JSON.stringify(subjectsMeta, null, 2)}\n`); - await mkdir(dirname(GENERATED_SUBJECT_MODULES), { recursive: true }); - await writeFile(GENERATED_SUBJECT_MODULES, buildSubjectModuleRegistry(subjectEntries)); + await writeIfChanged(GENERATED_SUBJECTS_META, `${JSON.stringify(subjectsMeta, null, 2)}\n`); + await writeIfChanged(GENERATED_SUBJECT_MODULES, buildSubjectModuleRegistry(subjectEntries)); await mkdir(dirname(SYNC_CACHE_PATH), { recursive: true }); await writeFile(SYNC_CACHE_PATH, `${JSON.stringify(nextCache, null, 2)}\n`); diff --git a/scripts/validate-curriculum.test.ts b/scripts/validate-curriculum.test.ts new file mode 100644 index 0000000..4c4a22d --- /dev/null +++ b/scripts/validate-curriculum.test.ts @@ -0,0 +1,15 @@ +import { describe, expect, it } from "bun:test"; +import { validateCurriculum } from "./validate-curriculum"; + +describe("validate-curriculum", () => { + it("has no curriculum validation errors", async () => { + const result = await validateCurriculum(); + if (result.errors.length > 0) { + const details = result.errors + .map((error) => `[${error.file}] ${error.message}`) + .join("\n"); + throw new Error(`Curriculum validation failed:\n${details}`); + } + expect(result.errors).toHaveLength(0); + }); +}); diff --git a/scripts/validate-curriculum.ts b/scripts/validate-curriculum.ts index 89436de..4d4f795 100644 --- a/scripts/validate-curriculum.ts +++ b/scripts/validate-curriculum.ts @@ -97,14 +97,24 @@ async function readJsonFile(path: string): Promise { } } -async function main() { +export async function validateCurriculum(): Promise<{ + curriculumFileCount: number; + errors: ValidationError[]; +}> { const errors: ValidationError[] = []; const catalogPath = join(DATA_DIR, "catalog.json"); const catalog = await readJsonFile(catalogPath); if (!catalog || !Array.isArray(catalog.curriculums)) { - console.error("Cannot validate curriculums: data/catalog.json must contain a curriculums array."); - process.exit(1); + return { + curriculumFileCount: 0, + errors: [ + { + file: "catalog.json", + message: "Cannot validate curriculums: data/catalog.json must contain a curriculums array.", + }, + ], + }; } const globalCurriculums = new Set( @@ -263,15 +273,21 @@ async function main() { } } - if (errors.length > 0) { - console.error(`\n✖ Curriculum validation failed with ${errors.length} error(s):`); - for (const error of errors) { + return { curriculumFileCount: curriculumFiles.length, errors }; +} + +async function main() { + const result = await validateCurriculum(); + if (result.errors.length > 0) { + console.error(`\n✖ Curriculum validation failed with ${result.errors.length} error(s):`); + for (const error of result.errors) { console.error(`- [${error.file}] ${error.message}`); } process.exit(1); } - - console.log(`✓ Curriculum validation passed for ${curriculumFiles.length} curriculum file(s).`); + console.log(`✓ Curriculum validation passed for ${result.curriculumFileCount} curriculum file(s).`); } -main(); +if (import.meta.main) { + void main(); +} diff --git a/spec.md b/spec.md index 24984bf..2de0844 100644 --- a/spec.md +++ b/spec.md @@ -35,4 +35,14 @@ each chapter needs to have the following other than the core content: 5. questions bank - some exams questions from easy, intermediate, difficult, hardcore 6. reference (if any) -the order can be adjustable base on importance. \ No newline at end of file +the order can be adjustable base on importance. + +## Non-functional requirements + +### Dev experience and performance + +1. `bun run dev` must start fast and directly run `zola serve` only (no implicit sync/build/watch pipeline). +2. Chapter build workflow must prioritize fast feedback for content edits: + - avoid full frontend bundle rebuild unless frontend/site-shell code actually changed, + - keep chapter core rebuild path as lightweight as possible. +3. `build:chapter` is the default iterative workflow for chapter updates; it should minimize unnecessary I/O and avoid rebuilding unrelated assets. \ No newline at end of file diff --git a/static/feed.xml b/static/feed.xml index 3a6db22..3818856 100644 --- a/static/feed.xml +++ b/static/feed.xml @@ -4,7 +4,14 @@ Open Free Books https://openfreebooks.org/ Recent updates to Open Free Books pages and live chapters. - Fri, 22 May 2026 06:55:33 GMT + Fri, 22 May 2026 07:32:28 GMT + + Written calculation + https://openfreebooks.org/math/written-calculation/ + https://openfreebooks.org/math/written-calculation/ + Fri, 22 May 2026 07:32:28 GMT + Add, subtract, multiply, and divide whole numbers using vertical written form: align columns, regroup, and read the result. + Pythagoras' theorem https://openfreebooks.org/math/pythagoras-theorem/ @@ -250,13 +257,6 @@ Thu, 21 May 2026 06:54:57 GMT Solve, represent, and apply linear inequalities in one unknown using number lines and interval notation. - - Written calculation - https://openfreebooks.org/math/written-calculation/ - https://openfreebooks.org/math/written-calculation/ - Thu, 21 May 2026 05:05:38 GMT - Add, subtract, multiply, and divide whole numbers using vertical written form: align columns, regroup, and read the result. - FAQ https://openfreebooks.org/faq/ diff --git a/static/sitemap.xml b/static/sitemap.xml index 77e9c50..26907f0 100644 --- a/static/sitemap.xml +++ b/static/sitemap.xml @@ -174,7 +174,7 @@ https://openfreebooks.org/math/written-calculation/ - 2026-05-21T05:05:38.000Z + 2026-05-22T07:32:28.000Z https://openfreebooks.org/math/zero-and-one/ diff --git a/themes/openfreebooks/sass/_catalog.scss b/themes/openfreebooks/sass/_catalog.scss index 47d40db..464cc25 100644 --- a/themes/openfreebooks/sass/_catalog.scss +++ b/themes/openfreebooks/sass/_catalog.scss @@ -741,6 +741,7 @@ display: -webkit-box; -webkit-line-clamp: 3; -webkit-box-orient: vertical; + line-clamp: 3; overflow: hidden; }