Update build scripts and enhance development workflow: Revise package.json to streamline build commands, replacing deprecated scripts with new chapter-specific build commands. Update CONTRIBUTING.md to clarify local development instructions and improve chapter sync processes. Enhance README.md with updated serving and build commands for better user guidance. Add new types for Bun in devDependencies and update the feed and sitemap to reflect recent content changes.

This commit is contained in:
Hananoshika Yomaru 2026-05-22 11:42:00 -07:00
parent 61963350e7
commit 5ac8e330ca
19 changed files with 501 additions and 125 deletions

View file

@ -32,11 +32,13 @@ frontend/ → catalog-app, Mermaid map view, site chrome
```bash
bun install
bun run build:js # after frontend/ changes
bun run generate:site-metadata # regenerate static/sitemap.xml + static/feed.xml from content + git log
bun run index:search # once per dev session (copies public/pagefind → static/pagefind)
zola serve # http://127.0.0.1:1111
bun run build # production: JS + Zola + pagefind → public/
bun run dev # instant local serve (zola only)
bun run build:chapter math/measures-dispersion math/loci
bun run build:chapter math/*
bun run build:chapter # auto-detect changed chapters (interactive confirm)
bun run build:site # full rebuild (sync external data + all chapters + site)
bun run build # alias of build:site
bun test
```
## Non-negotiables
@ -65,7 +67,7 @@ bun run build # production: JS + Zola + pagefind → public/
| Sitemap/feed coverage | `scripts/generate-site-metadata.ts`, `static/sitemap.xml`, `static/feed.xml`, `package.json` build script |
| Homepage subject card | `themes/.../templates/index.html` (not yet data-driven) |
| Browse / nav URL | `zola.toml` `[extra] browse_url` + `base.html` site-config + `main.tsx` fallback |
| Search UI / index | `base.html` (Pagefind assets + modal), `content/search.md`, `_search.scss`; `bun run index:search` for dev |
| Search UI / index | `base.html` (Pagefind assets + modal), `content/search.md`, `_search.scss`; chapter/site builds refresh Pagefind |
## Sitemap / feed auto-generation
@ -73,7 +75,7 @@ bun run build # production: JS + Zola + pagefind → public/
- `static/sitemap.xml` (all user-facing built pages + live chapters)
- `static/feed.xml` (RSS 2.0 for pages + live chapters)
- URL timestamps come from `git log -1` on content-source paths. If a page/chapter has no commit yet, generator falls back to filesystem mtime.
- Build pipeline runs this automatically before `zola build` via `package.json`.
- Build pipeline runs this automatically before `zola build`.
- Include only resolvable pages; planned chapters without live content are excluded.
## Search (Pagefind)
@ -87,7 +89,7 @@ bun run build # production: JS + Zola + pagefind → public/
| Theming | `themes/.../sass/_search.scss` (`--pf-*` → site tokens) |
- **Component UI** only (`pagefind-component-ui.js`), not legacy `pagefind-ui.js`.
- Index runs **after** `zola build`. Production: `bun run build`. Dev: `bun run index:search` copies bundle to `static/pagefind/` (gitignored) for `zola serve`.
- Index runs **after** `zola build` in build commands (`build:chapter`/`build:site`).
- v1 indexes **live chapter pages only** (`data-pagefind-body` on `templates/chapter.html`); home, about, catalog, credits, and other site pages are excluded.
Detailed checklists: [curriculum-data.md](curriculum-data.md). Human-facing guide: [CONTRIBUTING.md](../../../CONTRIBUTING.md).

View file

@ -28,23 +28,22 @@ See [reference.md](reference.md) for Tera pitfalls and build checks.
## Commands
```bash
bun run build:js # sync chapters + Vite build (JS/CSS bundle outputs)
bun run build # smart local build (incremental; uses build cache)
bun run build:full # full deterministic rebuild (use in CI / clean validation)
bun run dev # smart local dev startup (incremental chapter sync + watch + zola serve)
bun run dev:full # force full chapter sync before starting dev server
zola build # required before indexing
bun run index:search # pagefind on public/ → copies to static/pagefind/ for zola serve
zola serve # dev at http://127.0.0.1:1111
bun run dev # instant serve (zola only)
bun run build:chapter math/loci math/measures-dispersion
bun run build:chapter math/* # wildcard chapter builds
bun run build:chapter # auto-detect changed chapters (interactive confirm)
bun run build:site # full site rebuild (includes sponsor sync when token exists)
bun run build # alias of build:site
bun test
```
After Sass or template changes, run `zola build` (or `zola serve`). Do **not** copy `public/` into `themes/openfreebooks/static/`.
### Build mode behavior
- **First clone / cold start:** `bun run build` behaves like a full pipeline because caches are absent.
- **Continuous local development:** `bun run build` reuses caches and skips unchanged steps where possible.
- **CI:** prefer `bun run build:full` for deterministic, correctness-first builds.
- **Chapter workflow:** prefer `build:chapter` for normal content work; it rebuilds chapter assets and only runs Vite when frontend/site-shell code changes.
- **Site workflow:** use `build:site`/`build` for full rebuilds and release verification.
- **CI:** use `bun run build:site` and `bun test`.
**Search (Pagefind):** read [zola-pagefind.md](zola-pagefind.md) before changing search UI, indexing, or `data-pagefind-*` attributes.

View file

@ -19,11 +19,11 @@ Thank you for helping grow free, open textbooks.
| Add a **catalog subject banner** | [§ Catalog subject banner](#catalog-subject-banner) |
| Add a **curriculum** filter (e.g. GCSE) | [§ New curriculum label](#add-a-curriculum-label) |
**Browse the catalog locally:** `bun run build:js && zola serve` → [http://127.0.0.1:1111/catalog/](http://127.0.0.1:1111/catalog/)
**Browse the catalog locally:** `bun run dev` → [http://127.0.0.1:1111/catalog/](http://127.0.0.1:1111/catalog/)
**Search locally:** After `bun run build:js`, run `bun run index:search` once (or again after changing chapter HTML), then `zola serve`. Try [http://127.0.0.1:1111/search/](http://127.0.0.1:1111/search/) or press ⌘K / Ctrl+K.
**Search locally:** Run `bun run build:chapter ...` or `bun run build:site` after content changes to refresh the index, then use `bun run dev`. Try [http://127.0.0.1:1111/search/](http://127.0.0.1:1111/search/) or press ⌘K / Ctrl+K.
**Build modes:** `bun run build` is smart/incremental for local iteration; use `bun run build:full` for a deterministic full rebuild (CI/release checks).
**Build modes:** use `bun run build:chapter` for targeted chapter builds and `bun run build:site` for full rebuilds.
---
@ -37,7 +37,7 @@ content/{subject}/{slug}/ → everything for the live page (one folder)
- **Catalog** (`/catalog/`) — list, map (`?view=tree`), and for Mathematics a Compare doc (`?view=compare`, HTML partial `themes/openfreebooks/templates/partials/math-curricula-compare.html`). Curriculum names (DSE, IB, …) appear only in the catalog — not inside chapter prose ([spec.md](spec.md)).
- Every chapter needs a **`description`** in curriculum JSON (map clamps to three lines).
- **`bun run sync:chapters`** (also run automatically by `bun run build:js`) merges `core.html` + `supplement.html` into `_index.md` for Zola. Edit the HTML files, not the merged body in `_index.md`.
- Chapter sync (merging `core.html` + `supplement.html` into `_index.md`) runs automatically in build commands. Edit HTML sources, not merged `_index.md`.
### Map graph rules (summary)
@ -130,15 +130,15 @@ strand = "Number & Algebra"
### 4. Build and check
```bash
bun run build:js # sync:chapters + Vite bundle
zola serve
bun run build:chapter math/your-chapter-slug
bun run dev
```
- `/catalog/?subject=math` — chapter appears and links correctly.
- `/math/your-chapter-slug/` — page renders.
- `bun run build` — full production build.
New subjects with shared CSS/JS: add `content/{subject}/subject.scss` and/or `subject.ts`; `sync:chapters` wires them automatically. No per-chapter template branches.
New subjects with shared CSS/JS: add `content/{subject}/subject.scss` and/or `subject.ts`; chapter sync during build wires them automatically. No per-chapter template branches.
---
@ -163,7 +163,7 @@ Add edges in `data/{subject}-curriculum.json`:
1. `data/catalog.json` — add the subject to `subjects`.
2. Create `data/{id}-curriculum.json` (same schema as `math-curriculum.json`; `strands` + optional `graph.edges`).
3. Add `content/{subject}/` when you have live chapters.
4. Run `bun run build:js` (this syncs chapter content and auto-registers subject CSS/JS modules).
4. Run `bun run build:chapter {subject}/*` (this syncs chapters, rebuilds relevant widgets, and updates global outputs).
### Subject scaffold (copyable)
@ -210,7 +210,7 @@ cwebp -resize 1400 0 -q 82 your-banner.jpg -o static/catalog/banners/math.webp
{ "id": "math", "name": "Mathematics", "banner": "/catalog/banners/math.webp" }
```
4. Check locally: `bun run build:js && zola serve` → `/catalog/?subject=math`
4. Check locally: `bun run build:chapter math/* && bun run dev` → `/catalog/?subject=math`
Omit `banner` for subjects that should stay unchanged (e.g. roadmap-only placeholders).
@ -228,14 +228,13 @@ Omit `banner` for subjects that should stay unchanged (e.g. roadmap-only placeho
```bash
bun install
bun run sync:chapters # after content/ chapter changes
bun run build:js # sync + Vite
bun run dev # incremental sync + watch + zola serve
bun run dev:full # force full sync before dev
zola serve
bun run dev
bun run build:chapter math/measures-dispersion math/loci
bun run build:chapter math/*
bun run build:chapter # auto-detect changed chapters, list, confirm
bun run build:site
bun test
bun run build
bun run build:full
```
- Do **not** commit `themes/openfreebooks/static/` (Vite output).

View file

@ -55,29 +55,37 @@ Install JS dependencies:
bun install
```
For day-to-day local development, run JS watch + Zola serve together:
Serve locally:
```bash
bun run dev
```
Open [http://127.0.0.1:1111](http://127.0.0.1:1111). Re-run `bun run index:search` when you change chapter HTML or add pages so search stays up to date.
Open [http://127.0.0.1:1111](http://127.0.0.1:1111).
Use `bun run dev:full` when you want to force a full chapter sync before starting dev servers.
Search UI is available in local serve; run a build command to refresh the search index (`static/pagefind/` is gitignored and copied from build output).
Search UI is available immediately; results require the index step above (`static/pagefind/` is gitignored and copied from the build output).
## Production build
## Build commands
```bash
bun run build
bun run build:chapter math/measures-dispersion math/loci
bun run build:chapter math/*
bun run build:chapter # auto-detect changed chapters, list them, ask for confirmation
bun run build:site # full site build (includes sponsor sync when GITHUB_TOKEN is set)
bun run build # alias of build:site
```
`bun run build` is smart/incremental for local development:
- first run (or missing cache) behaves like a full build,
- repeat runs skip unchanged steps.
`build:chapter` updates chapter sync, chapter widget JS (targeted), metadata, site output, and search index.
Use `bun run build:full` for a deterministic full rebuild (recommended for CI/release validation).
In non-interactive environments, `build:chapter` requires explicit chapter arguments.
## Testing
```bash
bun test
```
`bun test` includes curriculum validation.
## Deploy to Cloudflare

View file

@ -13,6 +13,7 @@
"three": "^0.184.0",
},
"devDependencies": {
"@types/bun": "^1.3.14",
"pagefind": "^1.5.0",
"sass": "^1.89.2",
"typescript": "^5.8.3",
@ -234,6 +235,8 @@
"@types/babel__traverse": ["@types/babel__traverse@7.28.0", "", { "dependencies": { "@babel/types": "^7.28.2" } }, "sha512-8PvcXf70gTDZBgt9ptxJ8elBeBjcLOAcOtoO/mPJjtji1+CdGbHgm77om1GrsPxsiE+uXIpNSK64UYaIwQXd4Q=="],
"@types/bun": ["@types/bun@1.3.14", "", { "dependencies": { "bun-types": "1.3.14" } }, "sha512-h1hFqFVcvAvD9j9K7ZW7vd82aSA+rTdznZa+5bwvCwqSB1jmmfLcbIWhOLx1/+boy/xmjgCs/OMUL8hRJSmnPw=="],
"@types/d3": ["@types/d3@7.4.3", "", { "dependencies": { "@types/d3-array": "*", "@types/d3-axis": "*", "@types/d3-brush": "*", "@types/d3-chord": "*", "@types/d3-color": "*", "@types/d3-contour": "*", "@types/d3-delaunay": "*", "@types/d3-dispatch": "*", "@types/d3-drag": "*", "@types/d3-dsv": "*", "@types/d3-ease": "*", "@types/d3-fetch": "*", "@types/d3-force": "*", "@types/d3-format": "*", "@types/d3-geo": "*", "@types/d3-hierarchy": "*", "@types/d3-interpolate": "*", "@types/d3-path": "*", "@types/d3-polygon": "*", "@types/d3-quadtree": "*", "@types/d3-random": "*", "@types/d3-scale": "*", "@types/d3-scale-chromatic": "*", "@types/d3-selection": "*", "@types/d3-shape": "*", "@types/d3-time": "*", "@types/d3-time-format": "*", "@types/d3-timer": "*", "@types/d3-transition": "*", "@types/d3-zoom": "*" } }, "sha512-lZXZ9ckh5R8uiFVt8ogUNf+pIrK4EsWrx2Np75WvF/eTpJ0FMHNhjXk8CKEx/+gpHbNQyJWehbFaTvqmHWB3ww=="],
"@types/d3-array": ["@types/d3-array@3.2.2", "", {}, "sha512-hOLWVbm7uRza0BYXpIIW5pxfrKe0W+D5lrFiAEYR+pb6w3N2SwSMaJbXdUfSEv+dT4MfHBLtn5js0LAWaO6otw=="],
@ -300,6 +303,8 @@
"@types/geojson": ["@types/geojson@7946.0.16", "", {}, "sha512-6C8nqWur3j98U6+lXDfTUWIfgvZU+EumvpHKcYjujKH7woYyLj2sUmff0tRhrqM7BohUw7Pz3ZB1jj2gW9Fvmg=="],
"@types/node": ["@types/node@25.9.1", "", { "dependencies": { "undici-types": ">=7.24.0 <7.24.7" } }, "sha512-xfrlY7UD5rMJk3ZVJP8BNzS28J36YJg+xp+LPXV1TdWxr8uMH5A860QNxYDGQe/ylDSgjxE52Q9VnO7p75tJxg=="],
"@types/trusted-types": ["@types/trusted-types@2.0.7", "", {}, "sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw=="],
"@upsetjs/venn.js": ["@upsetjs/venn.js@2.0.0", "", { "optionalDependencies": { "d3-selection": "^3.0.0", "d3-transition": "^3.0.1" } }, "sha512-WbBhLrooyePuQ1VZxrJjtLvTc4NVfpOyKx0sKqioq9bX1C1m7Jgykkn8gLrtwumBioXIqam8DLxp88Adbue6Hw=="],
@ -316,6 +321,8 @@
"browserslist": ["browserslist@4.28.2", "", { "dependencies": { "baseline-browser-mapping": "^2.10.12", "caniuse-lite": "^1.0.30001782", "electron-to-chromium": "^1.5.328", "node-releases": "^2.0.36", "update-browserslist-db": "^1.2.3" }, "bin": { "browserslist": "cli.js" } }, "sha512-48xSriZYYg+8qXna9kwqjIVzuQxi+KYWp2+5nCYnYKPTr0LvD89Jqk2Or5ogxz0NUMfIjhh2lIUX/LyX9B4oIg=="],
"bun-types": ["bun-types@1.3.14", "", { "dependencies": { "@types/node": "*" } }, "sha512-4N0ig0fEomHt5R0KCFWjovxow98rIoRwKolrYdCcknNwMekCXRnWEUvgu5soYV8QXtVsrUD8B95MBOZGPvr6KQ=="],
"caniuse-lite": ["caniuse-lite@1.0.30001793", "", {}, "sha512-iwSsYWaCOoh26cV8NwNRViHlrfUvYsHDfRVcbtmw0Kg6PJIZZXwMkj1442FYLBGkeUf1juAsU3DTfxW579mrPA=="],
"chokidar": ["chokidar@4.0.3", "", { "dependencies": { "readdirp": "^4.0.1" } }, "sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA=="],
@ -564,6 +571,8 @@
"uhyphen": ["uhyphen@0.2.0", "", {}, "sha512-qz3o9CHXmJJPGBdqzab7qAYuW8kQGKNEuoHFYrBwV6hWIMcpAmxDLXojcHfFr9US1Pe6zUswEIJIbLI610fuqA=="],
"undici-types": ["undici-types@7.24.6", "", {}, "sha512-WRNW+sJgj5OBN4/0JpHFqtqzhpbnV0GuB+OozA9gCL7a993SmU+1JBZCzLNxYsbMfIeDL+lTsphD5jN5N+n0zg=="],
"update-browserslist-db": ["update-browserslist-db@1.2.3", "", { "dependencies": { "escalade": "^3.2.0", "picocolors": "^1.1.1" }, "peerDependencies": { "browserslist": ">= 4.21.0" }, "bin": { "update-browserslist-db": "cli.js" } }, "sha512-Js0m9cx+qOgDxo0eMiFGEueWztz+d4+M3rGlmKPT+T4IS/jP4ylw3Nwpu6cpTTP8R1MAC1kF4VbdLt3ARf209w=="],
"uuid": ["uuid@14.0.0", "", { "bin": { "uuid": "dist-node/bin/uuid" } }, "sha512-Qo+uWgilfSmAhXCMav1uYFynlQO7fMFiMVZsQqZRMIXp0O7rR7qjkj+cPvBHLgBqi960QCoo/PH2/6ZtVqKvrg=="],

View file

@ -2,7 +2,7 @@
"title": "GitHub Sponsors",
"intro": "Supporters who help keep Open Free Books free and open for learners everywhere.",
"profile_url": "https://github.com/sponsors/HananoshikaYomaru",
"fetched_at": "2026-05-21T18:06:17.197Z",
"fetched_at": "2026-05-22T18:41:21.577Z",
"sponsors": [
{
"login": "cybersader",

View file

@ -82,20 +82,20 @@ function mountChapterWidgets() {
if (mounts.length === 0) return;
void import("./generated/chapter-widgets").then(({ chapterWidgetLoaders }) => {
void import("./generated/chapter-widgets").then(({ chapterWidgetUrls }) => {
const loadPromises: Promise<void>[] = [];
for (const el of mounts) {
const widget = el.dataset.widget;
if (!widget) continue;
const loaderKey = `${chapterKey}/${widget}`;
const loader = chapterWidgetLoaders[loaderKey];
if (!loader) {
console.warn(`No widget loader registered for ${loaderKey}`);
const moduleUrl = chapterWidgetUrls[loaderKey];
if (!moduleUrl) {
console.warn(`No widget module URL registered for ${loaderKey}`);
continue;
}
loadPromises.push(
loader().then((mod) => {
import(/* @vite-ignore */ moduleUrl).then((mod) => {
render(() => mod.default({}), el);
})
);

View file

@ -4,21 +4,11 @@
"private": true,
"type": "module",
"scripts": {
"sync:chapters": "bun scripts/sync-chapters.ts",
"sync:sponsors": "bun scripts/fetch-github-sponsors.ts",
"generate:site-metadata": "bun scripts/generate-site-metadata.ts",
"build:js": "bun run sync:chapters && vite build",
"build:site": "bun run generate:site-metadata && zola build && bun run index:search",
"build:full": "bun run check:static-html && bun run validate:curriculum && rm -f themes/openfreebooks/static/main.css public/main.css && bun run sync:chapters -- --full && vite build && bun run build:site",
"build": "bun scripts/build-smart.ts",
"index:search": "pagefind --site public --glob \"**/index.html\" && rm -rf static/pagefind && cp -R public/pagefind static/pagefind",
"dev:js": "vite build --watch",
"dev:full": "bash scripts/dev.sh --full",
"dev": "bash scripts/dev.sh",
"check:static-html": "bash scripts/check-static-html.sh",
"scan:size": "bun scripts/scan-output-size.ts",
"test": "bun test",
"validate:curriculum": "bun scripts/validate-curriculum.ts"
"build:chapter": "bun scripts/build-chapter.ts",
"build:site": "bun scripts/build-site.ts",
"build": "bun run build:site",
"dev": "zola serve",
"test": "bun test"
},
"dependencies": {
"defuddle": "^0.18.1",
@ -33,6 +23,7 @@
"pagefind": "^1.5.0",
"typescript": "^5.8.3",
"vite": "^6.3.5",
"vite-plugin-solid": "^2.11.6"
"vite-plugin-solid": "^2.11.6",
"@types/bun": "^1.3.14"
}
}

View file

@ -0,0 +1,137 @@
import { existsSync } from "node:fs";
import { mkdir, readdir, readFile, rm, writeFile } from "node:fs/promises";
import { basename, dirname, join } from "node:path";
import { build } from "vite";
import solid from "vite-plugin-solid";
import { fingerprintPaths } from "./build-change-detector";
const ROOT = join(import.meta.dir, "..");
const CONTENT = join(ROOT, "content");
const OUTPUT_ROOT = join(ROOT, "themes/openfreebooks/static/js/chapter-widgets");
const CACHE_PATH = join(ROOT, "data/_generated/chapter-widgets-cache.json");
type Chapter = { subject: string; slug: string; dir: string; key: string };
type CacheFile = { version: 1; chapters: Record<string, string> };
async function discoverChapters(): Promise<Chapter[]> {
const output: Chapter[] = [];
const subjects = await readdir(CONTENT, { withFileTypes: true });
for (const subjectEntry of subjects) {
if (!subjectEntry.isDirectory() || subjectEntry.name.startsWith(".")) continue;
const subject = subjectEntry.name;
const subjectDir = join(CONTENT, subject);
const chapterEntries = await readdir(subjectDir, { withFileTypes: true });
for (const chapterEntry of chapterEntries) {
if (!chapterEntry.isDirectory() || chapterEntry.name.startsWith(".")) continue;
const slug = chapterEntry.name;
const dir = join(subjectDir, slug);
const indexPath = join(dir, "_index.md");
if (!existsSync(indexPath)) continue;
const indexRaw = await readFile(indexPath, "utf8");
if (!/template\s*=\s*["']chapter\.html["']/.test(indexRaw)) continue;
output.push({ subject, slug, dir, key: `${subject}/${slug}` });
}
}
return output;
}
async function listWidgets(chapter: Chapter): Promise<string[]> {
const widgetsDir = join(chapter.dir, "widgets");
if (!existsSync(widgetsDir)) return [];
const entries = await readdir(widgetsDir, { withFileTypes: true });
return entries
.filter((entry) => entry.isFile() && entry.name.endsWith(".tsx") && !entry.name.startsWith("_"))
.map((entry) => basename(entry.name, ".tsx"))
.sort();
}
async function chapterFingerprint(chapter: Chapter): Promise<string> {
return fingerprintPaths(ROOT, [join(chapter.dir, "widgets")]);
}
async function readCache(): Promise<CacheFile> {
if (!existsSync(CACHE_PATH)) return { version: 1, chapters: {} };
try {
const parsed = JSON.parse(await readFile(CACHE_PATH, "utf8")) as CacheFile;
if (parsed.version !== 1) return { version: 1, chapters: {} };
return parsed;
} catch {
return { version: 1, chapters: {} };
}
}
async function buildWidget(chapter: Chapter, widget: string) {
const entry = join(chapter.dir, "widgets", `${widget}.tsx`);
const outDir = join(OUTPUT_ROOT, chapter.subject, chapter.slug);
await mkdir(outDir, { recursive: true });
await build({
configFile: false,
plugins: [solid()],
publicDir: false,
resolve: {
alias: {
"@ofb/katex": join(ROOT, "frontend/src/lib/katex.ts"),
},
},
build: {
outDir,
emptyOutDir: false,
lib: {
entry,
formats: ["es"],
fileName: () => `${widget}.js`,
},
rollupOptions: {
treeshake: false,
output: {
chunkFileNames: "assets/[name]-[hash].js",
assetFileNames: "assets/[name]-[hash][extname]",
},
},
},
});
}
function parseTargets(args: string[]): { full: boolean; chapterKeys: string[] } {
const full = args.includes("--full");
const chapterKeys = args.filter((arg) => !arg.startsWith("--"));
return { full, chapterKeys };
}
async function main() {
const { full, chapterKeys } = parseTargets(process.argv.slice(2));
const chapters = await discoverChapters();
const chapterByKey = new Map(chapters.map((chapter) => [chapter.key, chapter]));
const targets =
chapterKeys.length === 0
? chapters
: chapterKeys
.map((key) => chapterByKey.get(key))
.filter((chapter): chapter is Chapter => Boolean(chapter));
const prev = await readCache();
const next: CacheFile = { version: 1, chapters: { ...prev.chapters } };
for (const chapter of targets) {
const fingerprint = await chapterFingerprint(chapter);
const unchanged = !full && prev.chapters[chapter.key] === fingerprint;
if (unchanged) {
console.log(`Unchanged widgets ${chapter.key} (skipped)`);
continue;
}
const widgets = await listWidgets(chapter);
const chapterOutDir = join(OUTPUT_ROOT, chapter.subject, chapter.slug);
await rm(chapterOutDir, { recursive: true, force: true });
for (const widget of widgets) {
await buildWidget(chapter, widget);
}
next.chapters[chapter.key] = fingerprint;
console.log(`Built widgets ${chapter.key} (${widgets.length})`);
}
await mkdir(dirname(CACHE_PATH), { recursive: true });
await writeFile(CACHE_PATH, `${JSON.stringify(next, null, 2)}\n`);
}
main();

186
scripts/build-chapter.ts Normal file
View file

@ -0,0 +1,186 @@
import { spawn } from "node:child_process";
import { existsSync } from "node:fs";
import { mkdir, readdir, readFile, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { createInterface } from "node:readline/promises";
import { stdin as input, stdout as output } from "node:process";
import { $ } from "bun";
import { fingerprintPaths } from "./build-change-detector";
const ROOT = join(import.meta.dir, "..");
const CONTENT = join(ROOT, "content");
const BUNDLE_PATH = join(ROOT, "themes/openfreebooks/static/js/bundle.js");
const BUILD_CACHE_PATH = join(ROOT, "data/_generated/build-chapter-cache.json");
type Chapter = { key: string; subject: string; slug: string };
type BuildCache = { version: 1; shellFingerprint: string };
async function runStep(command: string) {
const result = await $`bash -lc ${command}`;
if (result.exitCode !== 0) process.exit(result.exitCode || 1);
}
async function discoverChapters(): Promise<Chapter[]> {
const output: Chapter[] = [];
const subjects = await readdir(CONTENT, { withFileTypes: true });
for (const subjectEntry of subjects) {
if (!subjectEntry.isDirectory() || subjectEntry.name.startsWith(".")) continue;
const subject = subjectEntry.name;
const subjectDir = join(CONTENT, subject);
const chapterEntries = await readdir(subjectDir, { withFileTypes: true });
for (const chapterEntry of chapterEntries) {
if (!chapterEntry.isDirectory() || chapterEntry.name.startsWith(".")) continue;
const slug = chapterEntry.name;
const indexPath = join(subjectDir, slug, "_index.md");
if (!existsSync(indexPath)) continue;
const raw = await readFile(indexPath, "utf8");
if (!/template\s*=\s*["']chapter\.html["']/.test(raw)) continue;
output.push({ key: `${subject}/${slug}`, subject, slug });
}
}
return output.sort((a, b) => a.key.localeCompare(b.key));
}
function patternToRegex(pattern: string): RegExp {
const escaped = pattern.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*");
return new RegExp(`^${escaped}$`);
}
function resolvePatterns(chapters: Chapter[], patterns: string[]): Chapter[] {
const results = new Map<string, Chapter>();
for (const pattern of patterns) {
const regex = patternToRegex(pattern);
const matches = chapters.filter((chapter) => regex.test(chapter.key));
for (const chapter of matches) {
results.set(chapter.key, chapter);
}
}
return [...results.values()].sort((a, b) => a.key.localeCompare(b.key));
}
async function changedChapterKeys(chapterKeys: Set<string>): Promise<string[]> {
const trackedDiff = await $`git --no-pager diff --name-only HEAD -- content`.text();
const untracked = await $`git --no-pager ls-files --others --exclude-standard -- content`.text();
const files = new Set(
[...trackedDiff.split("\n"), ...untracked.split("\n")]
.map((line) => line.trim())
.filter((line) => line.length > 0)
);
const keys = new Set<string>();
for (const file of files) {
const match = file.match(/^content\/([^/]+)\/([^/]+)\//);
if (!match) continue;
const key = `${match[1]}/${match[2]}`;
if (chapterKeys.has(key)) keys.add(key);
}
return [...keys].sort();
}
async function confirmDetectedChanges(chapters: Chapter[], frontendChanged: boolean): Promise<boolean> {
if (chapters.length > 0) {
console.log("[build:chapter] Auto-detected changed chapters:");
for (const chapter of chapters) {
console.log(` - ${chapter.key}`);
}
}
if (frontendChanged) {
console.log("[build:chapter] Auto-detected frontend/site-shell changes.");
}
const rl = createInterface({ input, output });
const answer = (await rl.question("Proceed? [y/N] ")).trim().toLowerCase();
rl.close();
return answer === "y" || answer === "yes";
}
async function readBuildCache(): Promise<BuildCache | null> {
if (!existsSync(BUILD_CACHE_PATH)) return null;
try {
const parsed = JSON.parse(await readFile(BUILD_CACHE_PATH, "utf8")) as BuildCache;
if (parsed.version !== 1) return null;
return parsed;
} catch {
return null;
}
}
async function writeBuildCache(cache: BuildCache) {
await mkdir(dirname(BUILD_CACHE_PATH), { recursive: true });
await writeFile(BUILD_CACHE_PATH, `${JSON.stringify(cache, null, 2)}\n`);
}
async function shellFingerprint(): Promise<string> {
return fingerprintPaths(ROOT, [join(ROOT, "frontend"), join(ROOT, "vite.config.ts")]);
}
function runPagefindInBackground() {
const command =
'pagefind --site public --glob "**/index.html" && rm -rf static/pagefind && cp -R public/pagefind static/pagefind';
const child = spawn("bash", ["-lc", command], {
cwd: ROOT,
detached: true,
stdio: "ignore",
});
child.unref();
console.log("[build:chapter] Pagefind indexing started in background.");
}
async function main() {
const patterns = process.argv.slice(2).filter((arg) => !arg.startsWith("--"));
const chapters = await discoverChapters();
const chapterByKey = new Map(chapters.map((chapter) => [chapter.key, chapter]));
const previousBuild = await readBuildCache();
const currentShellFingerprint = await shellFingerprint();
const frontendChanged =
!existsSync(BUNDLE_PATH) || previousBuild?.shellFingerprint !== currentShellFingerprint;
let targets: Chapter[] = [];
if (patterns.length > 0) {
targets = resolvePatterns(chapters, patterns);
if (targets.length === 0) {
console.error(`[build:chapter] No chapters matched patterns: ${patterns.join(", ")}`);
process.exit(1);
}
} else {
if (!input.isTTY || !output.isTTY) {
console.error("[build:chapter] Non-interactive mode requires explicit chapter arguments.");
process.exit(1);
}
const changedKeys = await changedChapterKeys(new Set(chapters.map((chapter) => chapter.key)));
targets = changedKeys.map((key) => chapterByKey.get(key)).filter((c): c is Chapter => Boolean(c));
if (targets.length === 0 && !frontendChanged) {
console.log("[build:chapter] No changed chapters or frontend changes detected.");
return;
}
const confirmed = await confirmDetectedChanges(targets, frontendChanged);
if (!confirmed) {
console.log("[build:chapter] Cancelled.");
return;
}
}
if (targets.length > 0) {
await runStep("bun scripts/sync-chapters.ts");
await runStep(`bun scripts/build-chapter-widgets.ts ${targets.map((t) => t.key).join(" ")}`);
}
const needsShellBuild = !existsSync(BUNDLE_PATH) || frontendChanged;
if (needsShellBuild) {
await runStep("rm -f themes/openfreebooks/static/main.css public/main.css");
await runStep("vite build");
}
await runStep("bun scripts/generate-site-metadata.ts");
await runStep("rm -rf public");
await runStep("zola build");
runPagefindInBackground();
await writeBuildCache({ version: 1, shellFingerprint: currentShellFingerprint });
const summaryTargets = targets.length > 0 ? targets.map((target) => target.key).join(", ") : "(frontend only)";
console.log(`[build:chapter] Core build done: ${summaryTargets}`);
}
void main();

25
scripts/build-site.ts Normal file
View file

@ -0,0 +1,25 @@
import { $ } from "bun";
async function runStep(command: string) {
const result = await $`bash -lc ${command}`;
if (result.exitCode !== 0) process.exit(result.exitCode || 1);
}
async function main() {
await runStep("bash scripts/check-static-html.sh");
await runStep("rm -f themes/openfreebooks/static/main.css public/main.css");
if (process.env.GITHUB_TOKEN?.trim()) {
await runStep("bun scripts/fetch-github-sponsors.ts");
} else {
console.log("[build:site] GITHUB_TOKEN not set; skipping sponsor sync.");
}
await runStep("bun scripts/sync-chapters.ts --full");
await runStep("bun scripts/build-chapter-widgets.ts --full");
await runStep("vite build");
await runStep("bun scripts/generate-site-metadata.ts");
await runStep("rm -rf public");
await runStep("zola build");
await runStep('pagefind --site public --glob "**/index.html" && rm -rf static/pagefind && cp -R public/pagefind static/pagefind');
}
void main();

View file

@ -1,21 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
cleanup() {
if [[ -n "${JS_PID:-}" ]] && kill -0 "${JS_PID}" 2>/dev/null; then
kill "${JS_PID}"
fi
}
trap cleanup EXIT INT TERM
if [[ "${1:-}" == "--full" ]]; then
bun run sync:chapters -- --full
else
bun run sync:chapters
fi
bun run dev:js &
JS_PID=$!
zola serve

View file

@ -123,6 +123,13 @@ async function copyDir(srcDir: string, destDir: string) {
}
}
async function writeIfChanged(path: string, content: string) {
const current = existsSync(path) ? await readFile(path, "utf8") : null;
if (current === content) return;
await mkdir(dirname(path), { recursive: true });
await writeFile(path, content);
}
async function compileChapterScss(
subject: string,
slug: string,
@ -361,18 +368,14 @@ function buildWidgetRegistry(
const lines: string[] = [
"// Auto-generated by scripts/sync-chapters.ts — do not edit.",
"",
"import type { Component } from \"solid-js\";",
"",
"export type ChapterWidgetModule = { default: Component };",
"",
"export const chapterWidgetLoaders: Record<string, () => Promise<ChapterWidgetModule>> = {",
"export const chapterWidgetUrls: Record<string, string> = {",
];
for (const { key, widgets, dir } of entries) {
const relFromGenerated = relative(dirname(GENERATED_WIDGETS), dir).replace(/\\/g, "/");
for (const { key, widgets } of entries) {
const [subject, slug] = key.split("/");
for (const widget of widgets) {
const importPath = `${relFromGenerated}/widgets/${widget}`;
lines.push(` "${key}/${widget}": () => import("${importPath}"),`);
const importPath = `/js/chapter-widgets/${subject}/${slug}/${widget}.js`;
lines.push(` "${key}/${widget}": "${importPath}",`);
}
}
@ -411,14 +414,11 @@ async function main() {
widgetEntries.push({ key, widgets, dir: chapter.dir });
}
await mkdir(dirname(GENERATED_META), { recursive: true });
await writeFile(GENERATED_META, `${JSON.stringify(metaFile, null, 2)}\n`);
await mkdir(dirname(GENERATED_WIDGETS), { recursive: true });
await writeFile(GENERATED_WIDGETS, buildWidgetRegistry(widgetEntries));
await writeIfChanged(GENERATED_META, `${JSON.stringify(metaFile, null, 2)}\n`);
await writeIfChanged(GENERATED_WIDGETS, buildWidgetRegistry(widgetEntries));
const chapterNav = await buildChapterNav(chapters);
await writeFile(GENERATED_NAV, `${JSON.stringify(chapterNav, null, 2)}\n`);
await writeIfChanged(GENERATED_NAV, `${JSON.stringify(chapterNav, null, 2)}\n`);
const subjectEntries: Array<{ id: string; dir: string; hasJs: boolean }> = [];
const subjectsMeta: SubjectsMetaFile = { subjects: {} };
@ -446,9 +446,8 @@ async function main() {
subjectsMeta.subjects[subject.id] = meta;
subjectEntries.push({ id: subject.id, dir: subject.dir, hasJs: meta.hasJs });
}
await writeFile(GENERATED_SUBJECTS_META, `${JSON.stringify(subjectsMeta, null, 2)}\n`);
await mkdir(dirname(GENERATED_SUBJECT_MODULES), { recursive: true });
await writeFile(GENERATED_SUBJECT_MODULES, buildSubjectModuleRegistry(subjectEntries));
await writeIfChanged(GENERATED_SUBJECTS_META, `${JSON.stringify(subjectsMeta, null, 2)}\n`);
await writeIfChanged(GENERATED_SUBJECT_MODULES, buildSubjectModuleRegistry(subjectEntries));
await mkdir(dirname(SYNC_CACHE_PATH), { recursive: true });
await writeFile(SYNC_CACHE_PATH, `${JSON.stringify(nextCache, null, 2)}\n`);

View file

@ -0,0 +1,15 @@
import { describe, expect, it } from "bun:test";
import { validateCurriculum } from "./validate-curriculum";
describe("validate-curriculum", () => {
it("has no curriculum validation errors", async () => {
const result = await validateCurriculum();
if (result.errors.length > 0) {
const details = result.errors
.map((error) => `[${error.file}] ${error.message}`)
.join("\n");
throw new Error(`Curriculum validation failed:\n${details}`);
}
expect(result.errors).toHaveLength(0);
});
});

View file

@ -97,14 +97,24 @@ async function readJsonFile<T>(path: string): Promise<T | null> {
}
}
async function main() {
export async function validateCurriculum(): Promise<{
curriculumFileCount: number;
errors: ValidationError[];
}> {
const errors: ValidationError[] = [];
const catalogPath = join(DATA_DIR, "catalog.json");
const catalog = await readJsonFile<CatalogFile>(catalogPath);
if (!catalog || !Array.isArray(catalog.curriculums)) {
console.error("Cannot validate curriculums: data/catalog.json must contain a curriculums array.");
process.exit(1);
return {
curriculumFileCount: 0,
errors: [
{
file: "catalog.json",
message: "Cannot validate curriculums: data/catalog.json must contain a curriculums array.",
},
],
};
}
const globalCurriculums = new Set(
@ -263,15 +273,21 @@ async function main() {
}
}
if (errors.length > 0) {
console.error(`\n✖ Curriculum validation failed with ${errors.length} error(s):`);
for (const error of errors) {
return { curriculumFileCount: curriculumFiles.length, errors };
}
async function main() {
const result = await validateCurriculum();
if (result.errors.length > 0) {
console.error(`\n✖ Curriculum validation failed with ${result.errors.length} error(s):`);
for (const error of result.errors) {
console.error(`- [${error.file}] ${error.message}`);
}
process.exit(1);
}
console.log(`✓ Curriculum validation passed for ${curriculumFiles.length} curriculum file(s).`);
console.log(`✓ Curriculum validation passed for ${result.curriculumFileCount} curriculum file(s).`);
}
main();
if (import.meta.main) {
void main();
}

12
spec.md
View file

@ -35,4 +35,14 @@ each chapter needs to have the following other than the core content:
5. questions bank - some exams questions from easy, intermediate, difficult, hardcore
6. reference (if any)
the order can be adjustable base on importance.
the order can be adjustable base on importance.
## Non-functional requirements
### Dev experience and performance
1. `bun run dev` must start fast and directly run `zola serve` only (no implicit sync/build/watch pipeline).
2. Chapter build workflow must prioritize fast feedback for content edits:
- avoid full frontend bundle rebuild unless frontend/site-shell code actually changed,
- keep chapter core rebuild path as lightweight as possible.
3. `build:chapter` is the default iterative workflow for chapter updates; it should minimize unnecessary I/O and avoid rebuilding unrelated assets.

View file

@ -4,7 +4,14 @@
<title>Open Free Books</title>
<link>https://openfreebooks.org/</link>
<description>Recent updates to Open Free Books pages and live chapters.</description>
<lastBuildDate>Fri, 22 May 2026 06:55:33 GMT</lastBuildDate>
<lastBuildDate>Fri, 22 May 2026 07:32:28 GMT</lastBuildDate>
<item>
<title>Written calculation</title>
<link>https://openfreebooks.org/math/written-calculation/</link>
<guid isPermaLink="true">https://openfreebooks.org/math/written-calculation/</guid>
<pubDate>Fri, 22 May 2026 07:32:28 GMT</pubDate>
<description>Add, subtract, multiply, and divide whole numbers using vertical written form: align columns, regroup, and read the result.</description>
</item>
<item>
<title>Pythagoras&apos; theorem</title>
<link>https://openfreebooks.org/math/pythagoras-theorem/</link>
@ -250,13 +257,6 @@
<pubDate>Thu, 21 May 2026 06:54:57 GMT</pubDate>
<description>Solve, represent, and apply linear inequalities in one unknown using number lines and interval notation.</description>
</item>
<item>
<title>Written calculation</title>
<link>https://openfreebooks.org/math/written-calculation/</link>
<guid isPermaLink="true">https://openfreebooks.org/math/written-calculation/</guid>
<pubDate>Thu, 21 May 2026 05:05:38 GMT</pubDate>
<description>Add, subtract, multiply, and divide whole numbers using vertical written form: align columns, regroup, and read the result.</description>
</item>
<item>
<title>FAQ</title>
<link>https://openfreebooks.org/faq/</link>

View file

@ -174,7 +174,7 @@
</url>
<url>
<loc>https://openfreebooks.org/math/written-calculation/</loc>
<lastmod>2026-05-21T05:05:38.000Z</lastmod>
<lastmod>2026-05-22T07:32:28.000Z</lastmod>
</url>
<url>
<loc>https://openfreebooks.org/math/zero-and-one/</loc>

View file

@ -741,6 +741,7 @@
display: -webkit-box;
-webkit-line-clamp: 3;
-webkit-box-orient: vertical;
line-clamp: 3;
overflow: hidden;
}