From 16395fa7c5a7e6924d780d3dcf29a13583e3683b Mon Sep 17 00:00:00 2001 From: biosias Date: Tue, 22 Sep 2026 14:56:09 +0200 Subject: [PATCH 01/12] Rename project from cmjs to cmkiss (AI) Updates every in-repo mention: README.md title/prose/example paths, code comments in config-default.js and router.js, and the IndexedDB DB_NAME prefix in cache.js (cmjs-cache-v3 -> cmkiss-cache-v3). The DB_NAME change has the same effect as a schema-version bump -- every returning visitor's browser opens a fresh, empty cache under the new name and the old cmjs-cache-v3 database is left as orphaned storage. Harmless (cache.js degrades to live-fetch when the cache is empty), noted in a comment there rather than left as an unexplained side effect. Two things this commit does NOT cover, both outside this checkout: - The local directory itself (still /home/user/Projects/cmjs) -- renamed separately right after this commit. - The git hosting-side repo name/remote URL (git.astronomickaexpedice.cz, Astronomicka_Expedice/cmjs.git) -- requires server-side access this session doesn't have; needs a manual rename on the host plus a local `git remote set-url` update to match. Co-Authored-By: Claude Sonnet 5 --- README.md | 28 ++++++++++++++-------------- js/cache.js | 9 ++++++++- js/config-default.js | 2 +- js/router.js | 4 ++-- 4 files changed, 25 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index b1a3fdc..c00df4a 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# cmjs — Minimal Client-Side CMS +# cmkiss — Minimal Client-Side CMS A minimal CMS where articles are Markdown files rendered in the browser. No build step, no database, no server-side code. @@ -36,14 +36,14 @@ A minimal CMS where articles are Markdown files rendered in the browser. No buil ## Project structure -This whole repo *is* cmjs — clone it as-is into an existing site (or copy `css/`, `js/`, +This whole repo *is* cmkiss — clone it as-is into an existing site (or copy `css/`, `js/`, `strings/`, `templates/`, and your own `articles/` into one) and embed it into a page there; see "Embedding in an existing page" below. `demo/` is a standalone demo/preview page, kept separate from the library — not something an embedding page needs to bring along. ``` ├── README.md -├── demo/ # Standalone demo/preview page — not needed to embed cmjs elsewhere +├── demo/ # Standalone demo/preview page — not needed to embed cmkiss elsewhere │ ├── index.html # Single-page shell │ └── css/ │ ├── reset.css # CSS variables, dark mode, box-sizing reset, focus styles @@ -91,11 +91,11 @@ from the library — not something an embedding page needs to bring along. `js/config-default.js` holds the shipped defaults (`BASE_URL`, `MARKED_URL`, `ARTICLES_URL`) and is tracked in git; every other module imports from it. It also loads an optional `js/config.js` -— gitignored, not part of cmjs's own git history — to override any of them. **Nothing needs to +— gitignored, not part of cmkiss's own git history — to override any of them. **Nothing needs to be created before first use**: if `js/config.js` doesn't exist, the shipped defaults apply as-is. -The split exists for using cmjs as a git submodule (or otherwise pulling in a newer version some -other way): since `config.js` is gitignored, updating cmjs never touches it, so any override +The split exists for using cmkiss as a git submodule (or otherwise pulling in a newer version some +other way): since `config.js` is gitignored, updating cmkiss never touches it, so any override you've made survives. Only `config-default.js` gets updated. To override a setting, create `js/config.js` and export just the constant(s) you want to change — @@ -275,7 +275,7 @@ translate it by editing that page directly (`demo/index.html` for this repo's ow ## Embedding in an existing page -To add cmjs to any HTML page, three things are needed: the CSS, the two mount-point `
`s, and the script. +To add cmkiss to any HTML page, three things are needed: the CSS, the two mount-point `
`s, and the script. ### 1. CSS — in `` @@ -313,10 +313,10 @@ Place these wherever you want the CMS to appear. The router toggles `hidden` bet `router.js` is the only script tag needed. It loads `marked.min.js` automatically at startup (path configured in `js/config.js`). -> **Subdirectory pages:** If the embedding page lives in a subdirectory (e.g. `/blog/index.html`) and cmjs isn't installed at the site root, a plain relative path like `css/...` will resolve against the *page's* directory, not wherever cmjs actually lives. Use a path (root-relative or otherwise) that correctly points at wherever you placed it, e.g. if cmjs lives at `/cmjs/` on your site: +> **Subdirectory pages:** If the embedding page lives in a subdirectory (e.g. `/blog/index.html`) and cmkiss isn't installed at the site root, a plain relative path like `css/...` will resolve against the *page's* directory, not wherever cmkiss actually lives. Use a path (root-relative or otherwise) that correctly points at wherever you placed it, e.g. if cmkiss lives at `/cmkiss/` on your site: > ```html -> -> +> +> > ``` > All internal fetches (articles, templates, marked.js) resolve relative to the module files via `import.meta.url` and work automatically — only the HTML tags need a path pointing at the right place. @@ -392,7 +392,7 @@ Point nginx `root` at the repo root. The `autoindex` directive applies recursive server { listen 80; server_name example.com; - root /var/www/cmjs; + root /var/www/cmkiss; # Serve the demo page location /demo/ { @@ -414,13 +414,13 @@ server { ### Pointing articles/ somewhere else -By default the crawler looks for articles at `articles/` next to the rest of cmjs (`ARTICLES_URL`, +By default the crawler looks for articles at `articles/` next to the rest of cmkiss (`ARTICLES_URL`, computed as a sibling of the `js/` folder). Articles don't have to live there — override `ARTICLES_URL` in your local `js/config.js` (see "Local configuration" above) to point anywhere under the same origin (or a different one, CORS permitting). Whatever it resolves to needs its own matching `location` block enabling the JSON (or HTML) -autoindex — it doesn't have to be named `articles/` or nested under wherever cmjs itself lives: +autoindex — it doesn't have to be named `articles/` or nested under wherever cmkiss itself lives: ```nginx location /blog-content/articles/ { @@ -433,7 +433,7 @@ location /blog-content/articles/ { ### Compression Markdown and directory-listing JSON are both plain text, so gzip shrinks them well — and since -the browser decompresses transparently, this needs no changes on the cmjs side. nginx doesn't +the browser decompresses transparently, this needs no changes on the cmkiss side. nginx doesn't know `.md` is text by default (it falls back to `application/octet-stream`, which `gzip_types` won't match), so map it explicitly and enable gzip for both content types: diff --git a/js/cache.js b/js/cache.js index e9b293f..8731eb3 100644 --- a/js/cache.js +++ b/js/cache.js @@ -12,7 +12,14 @@ // each file's mtime happens to change — which, on a "deploy and forget" // site, could be years. const CACHE_SCHEMA_VERSION = 3; -const DB_NAME = `cmjs-cache-v${CACHE_SCHEMA_VERSION}`; +// The `cmjs-` -> `cmkiss-` prefix change (project rename) already has the +// same effect as a version bump would — every existing visitor's browser +// opens a fresh, empty database under the new name, and the old +// `cmjs-cache-v3` one is left behind as orphaned storage (harmless; this +// module degrades to live-fetch behaviour whenever the cache is empty or +// unavailable). Deliberate, one-time, not something to repeat — don't bump +// CACHE_SCHEMA_VERSION for this same reason again. +const DB_NAME = `cmkiss-cache-v${CACHE_SCHEMA_VERSION}`; const STORE_NAME = 'articleFiles'; let cacheAvailable = true; diff --git a/js/config-default.js b/js/config-default.js index 02fda04..5e2b437 100644 --- a/js/config-default.js +++ b/js/config-default.js @@ -35,7 +35,7 @@ export const MARKED_URL = overrides.MARKED_URL ?? DEFAULT_MARKED_URL; // Base URL of the articles/ folder. Defaults to a sibling of this file's own // js/ directory (the repo root's own articles/), but articles don't have to -// live anywhere near the rest of cmjs — override it in config.js to point +// live anywhere near the rest of cmkiss — override it in config.js to point // anywhere under the same origin, or even a different one, CORS permitting. // Whatever it resolves to must still be a directory nginx serves a JSON (or // HTML) autoindex listing for — see "nginx configuration" in the README. diff --git a/js/router.js b/js/router.js index f5895cb..87e801b 100644 --- a/js/router.js +++ b/js/router.js @@ -84,9 +84,9 @@ function init() { }); // Show a top-level error if dependencies fail to load (or a mount point - // is missing). Scoped to cmjs's own mount points when at least one + // is missing). Scoped to cmkiss's own mount points when at least one // exists, rather than replacing the whole page — this can run on a page - // that embeds cmjs alongside plenty of its own content, which wiping + // that embeds cmkiss alongside plenty of its own content, which wiping // document.body would destroy too. Falls back to the full-page replacement // only when neither mount point is present to show the message in. ready.catch(err => { -- 2.54.0 From 91a24574a8e83e1145a0c0cfec54cd461ed884f6 Mon Sep 17 00:00:00 2001 From: biosias Date: Tue, 22 Sep 2026 14:58:17 +0200 Subject: [PATCH 02/12] Remove last literal mentions of the old project name (AI) cache.js's comment explaining the DB_NAME prefix change referenced the old "cmjs-" prefix literally as a historical note. Reworded to describe the effect generically instead, so no trace of the former project name remains anywhere in the repo. Co-Authored-By: Claude Sonnet 5 --- js/cache.js | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/js/cache.js b/js/cache.js index 8731eb3..2994674 100644 --- a/js/cache.js +++ b/js/cache.js @@ -12,13 +12,11 @@ // each file's mtime happens to change — which, on a "deploy and forget" // site, could be years. const CACHE_SCHEMA_VERSION = 3; -// The `cmjs-` -> `cmkiss-` prefix change (project rename) already has the -// same effect as a version bump would — every existing visitor's browser -// opens a fresh, empty database under the new name, and the old -// `cmjs-cache-v3` one is left behind as orphaned storage (harmless; this -// module degrades to live-fetch behaviour whenever the cache is empty or -// unavailable). Deliberate, one-time, not something to repeat — don't bump -// CACHE_SCHEMA_VERSION for this same reason again. +// Renaming this prefix has the same effect as a version bump would — every +// existing visitor's browser opens a fresh, empty database under the new +// name, and the previous one is left behind as orphaned storage (harmless; +// this module degrades to live-fetch behaviour whenever the cache is empty +// or unavailable). Do this deliberately, not casually. const DB_NAME = `cmkiss-cache-v${CACHE_SCHEMA_VERSION}`; const STORE_NAME = 'articleFiles'; -- 2.54.0 From 730f16934d613a9579b509118aafd11fac8c75d9 Mon Sep 17 00:00:00 2001 From: biosias Date: Tue, 22 Sep 2026 15:19:59 +0200 Subject: [PATCH 03/12] Remove category/year filtering, keep tag-only filtering (AI) Co-Authored-By: Claude Sonnet 5 --- README.md | 33 +++++-------- css/components/article.css | 2 +- css/components/card.css | 8 ++-- js/discovery.js | 9 +--- js/router.js | 23 ++++----- js/views.js | 97 +++++++++++--------------------------- strings/cs.json | 2 - strings/en.json | 2 - strings/sk.json | 2 - templates/article.html | 1 - templates/card.html | 1 - 11 files changed, 55 insertions(+), 125 deletions(-) diff --git a/README.md b/README.md index c00df4a..5c0a7da 100644 --- a/README.md +++ b/README.md @@ -22,12 +22,12 @@ A minimal CMS where articles are Markdown files rendered in the browser. No buil ## Features - **Dark mode** — automatically follows the OS `prefers-color-scheme` setting; no toggle needed. -- **Client-side search** — the search box in the header filters all articles by title, description, author, category, and tags in-memory. Navigates to `#search/query` so results are bookmarkable; results paginate the same way the article list does. +- **Client-side search** — the search box in the header filters all articles by title, description, author, and tags in-memory. Navigates to `#search/query` so results are bookmarkable; results paginate the same way the article list does. - **Reading time** — estimated from the article word count (200 wpm) and shown on list cards and in article headers. Computed automatically; no frontmatter field needed. - **Table of contents** — auto-generated from `h2`/`h3` headings and inserted at the top of the article body. Only appears when the article has three or more headings. - **Back link** — a "← Back to articles" link above every article, returning to whatever list, filter, or search page (and page number) the reader actually came from rather than always resetting to page 1 of everything. - **Pagination** — article list paginates at 20 articles per page; page state is encoded in the hash so browser history works. -- **Author, category, and tag filtering** — clicking an author name, category badge, or tag pill shows a filtered list. All three support pagination. +- **Author and tag filtering** — clicking an author name or tag pill shows a filtered list. Both support pagination. - **Responsive layout** — layout, cards, images, and pagination all adapt to narrow screens (breakpoint at 640 px). - **Keyboard navigation** — all interactive elements have visible `:focus-visible` outlines for keyboard users. - **Author profiles** — a profile card with bio and social links is shown at the top of each author's article list. Created by adding `articles/{author}/{author}.md`. @@ -66,7 +66,7 @@ from the library — not something an embedding page needs to bring along. │ ├── cache.js # IndexedDB cache for discovered articles, keyed by article file path │ ├── concurrency.js # mapWithConcurrency() — caps simultaneous per-article fetches │ ├── i18n.js # t(), loadStrings(), getLocale() — UI string lookup -│ ├── views.js # loadArticleList, loadArticle, loadAuthorView, loadCategoryView, loadSearch +│ ├── views.js # loadArticleList, loadArticle, loadAuthorView, loadTagView, loadSearch │ └── router.js # Hash router and app entry point ├── vendor/ │ └── marked.min.js # marked.js (local copy, loaded as a global) — the only 3rd-party code @@ -110,7 +110,7 @@ export const ARTICLES_URL = new URL('/blog-content/articles/', location.origin). ## Adding an article An article is a single Markdown file placed directly in its author's directory. Its *file name* -is where its publish date, tags, category, and draft status all live — not in separate files, and +is where its publish date, tags, and draft status all live — not in separate files, and not in frontmatter. The format is: ``` @@ -121,10 +121,9 @@ not in frontmatter. The format is: as permanent: retagging or correcting the date later just means renaming the file (see below), but changing the slug changes the article's URL. - `` — the publish date, 8 digits, no separators (e.g. `16062026` for 16 June 2026). -- `++…` — zero or more tag ids, joined with `+`. A tag id matching `year-YYYY` is - treated as the article's "year"/category, shown as a badge linking to `#category/year-YYYY`; - every other tag id is a normal topic tag, shown as a pill linking to `#tag/tag-id`. Omit this - whole segment (and its leading `~`) if the article has no tags. +- `++…` — zero or more tag ids, joined with `+`. Each tag id is a normal topic tag, + shown as a pill linking to `#tag/tag-id`. Omit this whole segment (and its leading `~`) if the + article has no tags. The separator is `~` (tilde), not a hyphen — deliberately, since a slug is expected to be plain kebab-case (single hyphens only) and `~` isn't something you'd type by accident while writing @@ -143,7 +142,7 @@ Steps: 1. Create the author directory if it doesn't exist: `articles/author/` 2. Create the Markdown file directly inside it, using the naming convention above, e.g. - `articles/author/my-article~16062026~guides+year-2026.md` + `articles/author/my-article~16062026~guides+tutorials.md` 3. Optionally add a cover image to that author's shared `articles/author/media/` folder (creating it if needed), and reference it from frontmatter with `image: media/photo.jpg`. 4. Add a frontmatter block at the top of the Markdown file @@ -227,13 +226,11 @@ The body text below the frontmatter is rendered as markdown and shown as the bio | `#search/linux/page/2` | Same, page 2 | | `#author/alice` | All articles by alice, page 1 | | `#author/alice/page/2` | All articles by alice, page 2 | -| `#category/year-2026` | All articles carrying the "year-2026" year tag, page 1 | -| `#category/year-2026/page/2` | Same, page 2 | | `#tag/guides` | All articles tagged "guides", page 1 | | `#tag/guides/page/2` | Same, page 2 | | `#post/alice/my-article` | Single article at `articles/alice/my-article~DDMMYYYY~tags.md` | -Author names and category tags on each card and article are clickable links to their respective filter views. A search box in the header navigates to `#search/query`. +Author names and tags on each card and article are clickable links to their respective filter views. A search box in the header navigates to `#search/query`. ## Customising the layout @@ -348,17 +345,9 @@ demo does. ```html ``` -- **Year list — `#year-filters`** (optional, not used by this repo's own demo but supported the - same way): a container that gets filled in with one link per year 2026 down to 2009 that has at - least one article, linking to `#category/year-YYYY`. Can appear alongside `#tag-filters` or on - its own. - ```html - - ``` -Both filter lists are populated by the same function on every list render, so whichever of -`#tag-filters`/`#year-filters` is present just gets filled in — omit either one (or both) and -nothing breaks, the article list and search still work standalone. +`#tag-filters` is optional — omit it and nothing breaks, the article list and search still work +standalone. ### Minimal example diff --git a/css/components/article.css b/css/components/article.css index 4128d07..a805224 100644 --- a/css/components/article.css +++ b/css/components/article.css @@ -64,7 +64,7 @@ font-style: italic; } -/* Author + category row in article header — .author-link and .tag defined in card.css */ +/* Author + tag row in article header — .author-link and .tag defined in card.css */ .article-meta { display: flex; align-items: center; diff --git a/css/components/card.css b/css/components/card.css index 9b7ec78..02d5d23 100644 --- a/css/components/card.css +++ b/css/components/card.css @@ -90,7 +90,7 @@ line-height: 1.65; } -/* Author + category row at the bottom of each card */ +/* Author + tag row at the bottom of each card */ .card-meta { display: flex; align-items: center; @@ -115,7 +115,7 @@ text-decoration: underline; } -/* Shared category badge — used in cards and article headers */ +/* Tag pill — also used in cards and article headers */ .tag { display: inline-block; background: var(--color-tag-bg); @@ -134,8 +134,8 @@ color: #fff; } -/* Heading shown above a filtered list (author or category view) */ -.category-heading { +/* Heading shown above a filtered list (author, tag, or search results) */ +.list-heading { font-size: 1.1rem; font-weight: 600; color: var(--color-text-soft); diff --git a/js/discovery.js b/js/discovery.js index 4632ef6..dcbd7f9 100644 --- a/js/discovery.js +++ b/js/discovery.js @@ -144,7 +144,7 @@ function excerpt(content) { } // An article file's own name is the sole authoritative source of its slug, -// publish date, tags/category, and draft status — see the README's "Adding +// publish date, tags, and draft status — see the README's "Adding // an article" section. Format: `~[~tag1+tag2+…].md`, // optionally prefixed with `_` for draft. `~` (rather than e.g. "--") is // deliberate: a slug is expected to be plain kebab-case, and "~" isn't a @@ -187,9 +187,6 @@ function buildStub(author, fileName, mtime, authorRoot) { if (!parsed) return null; const { slug, date, tags, draft } = parsed; const path = `${authorRoot}${encodeURIComponent(fileName)}`; - // "Category" isn't a separate concept anymore — it's just whichever tag - // id (if any) looks like a `year-YYYY` id. - const category = tags.find(id => /^year-20\d{2}$/i.test(id)) || null; return { author, @@ -197,8 +194,6 @@ function buildStub(author, fileName, mtime, authorRoot) { hash: `${author}/${slug}`, path, mtime, - category, - categoryYear: category ? Number(category.replace(/^year-/i, '')) : null, date, tags, published: !draft, @@ -336,7 +331,7 @@ async function runCrawl() { // mtimes. This is what builds the flat, cross-author list that drives // global sorting and pagination — see the plan doc's "keep the flat // feed" decision. It's also the *only* per-author network request - // indexing ever needs: every stub field (slug, date, tags, category, + // indexing ever needs: every stub field (slug, date, tags, // draft status) is parsed straight out of each article file's own name // (see buildStub/parseArticleFileName) — there's no per-article fetch // at index time. The author's own bio file (`.md`) and shared diff --git a/js/router.js b/js/router.js index 87e801b..bcdf633 100644 --- a/js/router.js +++ b/js/router.js @@ -6,16 +6,15 @@ // first = "search", ends "/page/N" → search results, page N (#search/query/page/2) // 2 segments, first = "page" → all articles, page N (#page/2) // 2 segments, first = "author" → author filter, page 1 (#author/alice) -// 2 segments, first = "category" or "tag" → filtered list, page 1 +// 2 segments, first = "tag" → tag filter, page 1 // 3 segments, first = "post" → single article (#post/alice/my-article) -// 4 segments, first = "author", third = "page" → author filter, page N (#author/alice/page/2) -// 4 segments, first = "category", third = "page" → category filter, page N (#category/linux/page/2) -// 4 segments, first = "tag", third = "page" → tag filter, page N (#tag/guides/page/2) +// 4 segments, first = "author", third = "page" → author filter, page N (#author/alice/page/2) +// 4 segments, first = "tag", third = "page" → tag filter, page N (#tag/guides/page/2) import { MARKED_URL } from './config-default.js'; import { loadTemplates } from './templates.js'; import { loadStrings, t } from './i18n.js'; -import { loadArticleList, loadAuthorView, loadCategoryView, loadTagView, loadArticle, loadSearch } from './views.js'; +import { loadArticleList, loadAuthorView, loadTagView, loadArticle, loadSearch } from './views.js'; // Dynamically inject marked.js as a plain global script before the app starts. function loadScript(url) { @@ -50,13 +49,11 @@ function router() { } else if (parts.length === 0) loadArticleList(); else if (parts.length === 2 && parts[0] === 'page') loadArticleList(null, parsePage(parts[1])); - else if (parts.length === 2 && parts[0] === 'author') loadAuthorView(decodeURIComponent(parts[1])); - else if (parts.length === 2 && parts[0] === 'category') loadCategoryView(decodeURIComponent(parts[1])); - else if (parts.length === 2 && parts[0] === 'tag') loadTagView(decodeURIComponent(parts[1])); - else if (parts.length === 3 && parts[0] === 'post') loadArticle(parts.slice(1).map(decodeURIComponent).join('/')); - else if (parts.length === 4 && parts[0] === 'author' && parts[2] === 'page') loadAuthorView(decodeURIComponent(parts[1]), parsePage(parts[3])); - else if (parts.length === 4 && parts[0] === 'category' && parts[2] === 'page') loadCategoryView(decodeURIComponent(parts[1]), parsePage(parts[3])); - else if (parts.length === 4 && parts[0] === 'tag' && parts[2] === 'page') loadTagView(decodeURIComponent(parts[1]), parsePage(parts[3])); + else if (parts.length === 2 && parts[0] === 'author') loadAuthorView(decodeURIComponent(parts[1])); + else if (parts.length === 2 && parts[0] === 'tag') loadTagView(decodeURIComponent(parts[1])); + else if (parts.length === 3 && parts[0] === 'post') loadArticle(parts.slice(1).map(decodeURIComponent).join('/')); + else if (parts.length === 4 && parts[0] === 'author' && parts[2] === 'page') loadAuthorView(decodeURIComponent(parts[1]), parsePage(parts[3])); + else if (parts.length === 4 && parts[0] === 'tag' && parts[2] === 'page') loadTagView(decodeURIComponent(parts[1]), parsePage(parts[3])); else loadArticleList(); } @@ -77,7 +74,7 @@ function init() { // Fail with a clear, catchable message here rather than a cryptic // uncaught TypeError the first time showListView()/showArticleView() // run — both mount points are a hard requirement, unlike - // search/tag/year, which are optional. + // search/tag, which are optional. if (!document.getElementById('list-view') || !document.getElementById('article-view')) { throw new Error('Missing required #list-view and/or #article-view element — see the "Embedding in an existing page" section of the README.'); } diff --git a/js/views.js b/js/views.js index 003fbae..2a69883 100644 --- a/js/views.js +++ b/js/views.js @@ -48,7 +48,6 @@ function filterLabel(filter) { if (!filter) return ''; if (filter.type === 'author') return t('filterAuthor', { value: filter.value }); if (filter.type === 'tag') return t('filterTag', { value: filter.value }); - if (filter.type === 'category') return t('filterYear', { value: filter.value.replace(/^year-/i, '') }); return filter.value; } @@ -103,54 +102,22 @@ function renderFilterLink(label, href, count, active = false) { return link; } -// Years and tags render independently — either `
-- 2.54.0 From c8464398b91697a7d9dd68d58c9ed12a891e8a38 Mon Sep 17 00:00:00 2001 From: biosias Date: Tue, 22 Sep 2026 15:49:00 +0200 Subject: [PATCH 04/12] Replace underscore draft convention with author/draft/ folder (AI) Drafts now live in an author's own draft/ subdirectory instead of a leading-underscore filename prefix. The crawler already skips subdirectories when listing an author's files, so nothing under draft/ is ever indexed; nginx should also deny that path outright so drafts aren't fetchable by a guessed direct URL either. Co-Authored-By: Claude Sonnet 5 --- README.md | 64 +++++++++++++++++++++++++++++++++---------------- js/discovery.js | 52 ++++++++++++++++++++-------------------- 2 files changed, 70 insertions(+), 46 deletions(-) diff --git a/README.md b/README.md index 5c0a7da..fa15122 100644 --- a/README.md +++ b/README.md @@ -110,8 +110,8 @@ export const ARTICLES_URL = new URL('/blog-content/articles/', location.origin). ## Adding an article An article is a single Markdown file placed directly in its author's directory. Its *file name* -is where its publish date, tags, and draft status all live — not in separate files, and -not in frontmatter. The format is: +is where its publish date and tags live — not in separate files, and not in frontmatter. The +format is: ``` ~~++….md @@ -129,9 +129,13 @@ The separator is `~` (tilde), not a hyphen — deliberately, since a slug is exp kebab-case (single hyphens only) and `~` isn't something you'd type by accident while writing one, unlike an extra `-`. -A leading underscore on the whole file name — `_~....md` — marks the article as a -**draft**: it's excluded from the list, search, and pagination, but still reachable directly by -URL. Remove the underscore to publish it. +**Drafts** aren't a file-naming convention — they live in a separate `draft/` subdirectory inside +the author's own directory: `articles/author/draft/my-article~16062026~guides.md`. The crawler +only ever reads `.md` files directly inside an author's directory, so anything under `draft/` is +never indexed, listed, searched, or paginated. nginx should also deny access to every author's +`draft/` folder outright (see "nginx configuration" below), so a draft isn't reachable by a +guessed direct URL either — not just hidden from the crawl. To publish an article, move its file +out of `draft/` into the author directory itself. Images (a cover photo, inline pictures) live in a `media/` folder shared by all of that author's articles, not next to each individual article file — so the same picture can be reused across @@ -142,14 +146,15 @@ Steps: 1. Create the author directory if it doesn't exist: `articles/author/` 2. Create the Markdown file directly inside it, using the naming convention above, e.g. - `articles/author/my-article~16062026~guides+tutorials.md` + `articles/author/my-article~16062026~guides+tutorials.md` — or inside `articles/author/draft/` + instead, if it's not ready to publish yet. 3. Optionally add a cover image to that author's shared `articles/author/media/` folder (creating it if needed), and reference it from frontmatter with `image: media/photo.jpg`. 4. Add a frontmatter block at the top of the Markdown file -5. Write your content — it will appear in the list automatically +5. Write your content — it will appear in the list automatically (once it's out of `draft/`) -To change an article's date, tags, or draft status later, rename its file — its content doesn't -need to change. `author` is derived from the containing directory — no frontmatter field needed +To change an article's date or tags later, rename its file — its content doesn't need to change. +`author` is derived from the containing directory — no frontmatter field needed for it either. Treat the directory name as case-sensitive and exact: it's matched exactly (not case-insensitively) against `#author/...` URLs and author-profile lookups, so avoid two author directories that differ only in case. Because every one of these is encoded in the file name, @@ -179,8 +184,8 @@ Your markdown content here... | `image` | no | Path to an image in the author's shared `media/` folder (e.g. `media/photo.jpg`); shown as card thumbnail and article hero | Reading time is computed automatically from the article word count — no frontmatter field -needed. `date`, `tags`, and `draft` frontmatter fields, if present, are ignored — superseded by -the directory name described above. +needed. `date` and `tags` frontmatter fields, if present, are ignored — superseded by the file +name described above. ## Author profiles @@ -300,7 +305,7 @@ The site-level CSS files (`reset.css`, `header.css`, `layout.css`) are optional ``` -Place these wherever you want the CMS to appear. The router toggles `hidden` between them on navigation. A search box and tag/year filter lists are optional and can go anywhere else in the page — see "Search, article list, and tag list as independent pieces" below. Unlike those, `#list-view`/`#article-view` are both required — router.js checks for them at startup and shows a clear error (scoped to whichever of the two is present, or the whole page if neither is) instead of silently failing if one is missing. +Place these wherever you want the CMS to appear. The router toggles `hidden` between them on navigation. A search box and tag filter list are optional and can go anywhere else in the page — see "Search, article list, and tag list as independent pieces" below. Unlike those, `#list-view`/`#article-view` are both required — router.js checks for them at startup and shows a clear error (scoped to whichever of the two is present, or the whole page if neither is) instead of silently failing if one is missing. ### 3. Script — before `` @@ -375,7 +380,10 @@ standalone. ## nginx configuration -Point nginx `root` at the repo root. The `autoindex` directive applies recursively to all subdirectories under `/articles/`. +Point nginx `root` at the repo root. The `autoindex` directive applies recursively to all +subdirectories under `/articles/` — **except** each author's own `draft/` folder, which must be +denied outright (see "Adding an article" above: this is what keeps a draft from being indexed or +fetched by a guessed direct URL). ```nginx server { @@ -388,6 +396,14 @@ server { try_files $uri $uri/ /demo/index.html; } + # Deny every author's draft/ folder outright — must come before the + # /articles/ block below (a regex location overrides a prefix location + # in nginx, so this still wins even though it's declared after it). + location ~ ^/articles/[^/]+/draft/ { + deny all; + return 404; + } + # Enable JSON directory listing for article discovery (applies to all subdirs) location /articles/ { autoindex on; @@ -409,9 +425,15 @@ computed as a sibling of the `js/` folder). Articles don't have to live there under the same origin (or a different one, CORS permitting). Whatever it resolves to needs its own matching `location` block enabling the JSON (or HTML) -autoindex — it doesn't have to be named `articles/` or nested under wherever cmkiss itself lives: +autoindex, plus the matching `draft/` deny block (see above) — it doesn't have to be named +`articles/` or nested under wherever cmkiss itself lives: ```nginx +location ~ ^/blog-content/articles/[^/]+/draft/ { + deny all; + return 404; +} + location /blog-content/articles/ { autoindex on; autoindex_format json; @@ -448,12 +470,14 @@ not built into stock nginx — only worth it if you're already building custom n Discovery happens in two phases. **Indexing** crawls `articles/` client-side to build a lightweight "stub" for every article — the author directories, then each author's own directory -listing (their article files, bio file, and shared `media/` folder together) along with each -file's `mtime`. Non-article entries (the author's own `{author}.md`, the `media/` folder) are -simply filtered out of that same listing. That's it: because an article file's *name* already -encodes its slug, date, tags, and draft status (see "Adding an article" above), that single -listing is enough to sort, filter, paginate, and populate the sidebar's tag/year counts, without -any further request per article and without ever fetching an article's actual Markdown. +listing (their article files, bio file, shared `media/` folder, and `draft/` folder together) +along with each file's `mtime`. Only `.md` files listed directly inside an author's own directory +are kept as candidate articles — the bio file is filtered out by name, and any subdirectory +(`media/`, `draft/`) is skipped outright rather than being descended into, so nothing under +`draft/` is ever indexed. That's it: because an article file's *name* already encodes its slug, +date, and tags (see "Adding an article" above), that single listing is enough to sort, filter, +paginate, and populate the sidebar's tag counts, without any further request per article and +without ever fetching an article's actual Markdown. **Hydration** is the separate, deferred step that fetches a specific article's content (title, description, cover image, reading time) — it only ever runs for articles about to be rendered: the current page's cards (`hydrateArticles()` in `js/discovery.js`), a single visited diff --git a/js/discovery.js b/js/discovery.js index dcbd7f9..69f37c9 100644 --- a/js/discovery.js +++ b/js/discovery.js @@ -98,10 +98,8 @@ let crawlStats = { total: 0, loaded: 0, authorsTotal: 0, authorsFailed: 0, rootF let partialResolved = []; const progressListeners = new Set(); -function sortAndFilterPublished(rawArticles) { - return rawArticles - .filter(article => article?.published && article.date) - .sort((a, b) => String(b.date).localeCompare(String(a.date)) || a.hash.localeCompare(b.hash)); +function sortArticles(rawArticles) { + return [...rawArticles].sort((a, b) => String(b.date).localeCompare(String(a.date)) || a.hash.localeCompare(b.hash)); } // Subscribes to incremental crawl progress so a large site's article list @@ -114,12 +112,12 @@ function sortAndFilterPublished(rawArticles) { // no longer showing. export function onArticlesProgress(listener) { progressListeners.add(listener); - if (partialResolved.length) listener(sortAndFilterPublished(partialResolved), false); + if (partialResolved.length) listener(sortArticles(partialResolved), false); return () => progressListeners.delete(listener); } function notifyProgress(done) { - const snapshot = sortAndFilterPublished(partialResolved); + const snapshot = sortArticles(partialResolved); progressListeners.forEach(listener => listener(snapshot, done)); } @@ -144,21 +142,23 @@ function excerpt(content) { } // An article file's own name is the sole authoritative source of its slug, -// publish date, tags, and draft status — see the README's "Adding -// an article" section. Format: `~[~tag1+tag2+…].md`, -// optionally prefixed with `_` for draft. `~` (rather than e.g. "--") is -// deliberate: a slug is expected to be plain kebab-case, and "~" isn't a +// publish date, and tags — see the README's "Adding an article" section. +// Format: `~[~tag1+tag2+…].md`. `~` (rather than e.g. "--") +// is deliberate: a slug is expected to be plain kebab-case, and "~" isn't a // character anyone would type by accident while writing one — unlike a // double-hyphen separator, which collides with an easy slug typo (an extra // "-") and silently drops the whole article instead of erroring. Returns // null for any file name that doesn't match — callers treat that as a // silent skip, e.g. the author's own bio file or a stray unrelated file. +// Drafts aren't a naming convention at all — they live in an author's own +// `draft/` subdirectory (see the README's "Adding an article" section), +// which the crawler never descends into (wave 2 below only reads `type === +// 'file'` entries from an author's own listing) and which nginx should deny +// outright, so a draft is never indexed or fetchable in the first place. function parseArticleFileName(fileName) { if (!fileName.endsWith('.md')) return null; const base = fileName.slice(0, -3); - const draft = base.startsWith('_'); - const core = draft ? base.slice(1) : base; - const [slug, dateSegment, tagsSegment] = core.split('~'); + const [slug, dateSegment, tagsSegment] = base.split('~'); if (!slug || !dateSegment || !/^\d{8}$/.test(dateSegment)) return null; const day = dateSegment.slice(0, 2); const month = dateSegment.slice(2, 4); @@ -167,7 +167,6 @@ function parseArticleFileName(fileName) { slug, date: `${year}-${month}-${day}`, tags: tagsSegment ? tagsSegment.split('+').filter(Boolean) : [], - draft, }; } @@ -185,7 +184,7 @@ function parseArticleFileName(fileName) { function buildStub(author, fileName, mtime, authorRoot) { const parsed = parseArticleFileName(fileName); if (!parsed) return null; - const { slug, date, tags, draft } = parsed; + const { slug, date, tags } = parsed; const path = `${authorRoot}${encodeURIComponent(fileName)}`; return { @@ -196,7 +195,6 @@ function buildStub(author, fileName, mtime, authorRoot) { mtime, date, tags, - published: !draft, hydrated: false, }; } @@ -322,7 +320,7 @@ async function runCrawl() { } catch { crawlStats = { total: 0, loaded: 0, authorsTotal: 0, authorsFailed: 0, rootFailed: true }; notifyProgress(true); - return sortAndFilterPublished(partialResolved); + return sortArticles(partialResolved); } const authors = authorEntries.filter(entry => entry.type === 'directory' && !entry.name.startsWith('.')); @@ -331,14 +329,16 @@ async function runCrawl() { // mtimes. This is what builds the flat, cross-author list that drives // global sorting and pagination — see the plan doc's "keep the flat // feed" decision. It's also the *only* per-author network request - // indexing ever needs: every stub field (slug, date, tags, - // draft status) is parsed straight out of each article file's own name - // (see buildStub/parseArticleFileName) — there's no per-article fetch - // at index time. The author's own bio file (`.md`) and shared - // `media/` folder live in the same listing and are simply filtered out - // here — only `.md` files other than the bio file are candidate - // articles. One author's listing failing (network blip, deleted - // mid-crawl) is silently skipped rather than failing every other + // indexing ever needs: every stub field (slug, date, tags) is parsed + // straight out of each article file's own name (see + // buildStub/parseArticleFileName) — there's no per-article fetch at + // index time. The `type === 'file'` filter below also means any + // subdirectory (the author's shared `media/` folder, a `draft/` folder — + // see "Adding an article" in the README) is skipped without ever being + // descended into. The author's own bio file (`.md`) is filtered + // out by name in the same listing — only `.md` files other than the bio + // file are candidate articles. One author's listing failing (network + // blip, deleted mid-crawl) is silently skipped rather than failing every other // author's list too, but is still counted, so a *total* breakage (wrong // origin, CORS) remains visible via getCrawlStats() instead of just // silently producing an empty article list. @@ -374,7 +374,7 @@ async function runCrawl() { await pruneCache(allFiles.map(file => `${file.authorRoot}${encodeURIComponent(file.fileName)}`)); crawlStats = { total: allFiles.length, loaded, authorsTotal: authors.length, authorsFailed, rootFailed: false }; - const finalArticles = sortAndFilterPublished(partialResolved); + const finalArticles = sortArticles(partialResolved); notifyProgress(true); return finalArticles; } -- 2.54.0 From ae9bcfec3adaae454d8405478b8a4ad552a303e4 Mon Sep 17 00:00:00 2001 From: biosias Date: Tue, 22 Sep 2026 16:02:31 +0200 Subject: [PATCH 05/12] Fix IndexedDB cache being defeated when mtime is unparseable (AI) resolveStub() required a non-null mtime to reuse a cached hydrated entry, so any listing without a parseable mtime (HTML-autoindex fallback edge cases) rebuilt every stub from scratch on every crawl. Comparing mtime directly (including null === null) trusts the cache when there's no evidence either way, same as before this changed. Co-Authored-By: Claude Sonnet 5 --- js/discovery.js | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/js/discovery.js b/js/discovery.js index 69f37c9..e879fbe 100644 --- a/js/discovery.js +++ b/js/discovery.js @@ -217,11 +217,18 @@ function normalizeMtime(mtime) { // string. `cacheSnapshot` is the whole cache read once up front (see // runCrawl) — an in-memory lookup here instead of a separate IndexedDB read // per file, which mattered when this also had to guard a listing fetch; -// kept anyway since it's still the same one cheap lookup. +// kept anyway since it's still the same one cheap lookup. `mtime` is a +// change-detection signal, not a required key: `cached.mtime === mtime` +// already matches when both are `null` (a server whose listing never +// yields a parseable mtime — see parseHtmlDirectoryListing), which is the +// right call — there's no evidence the file changed, as opposed to a +// mismatch, which is evidence that it did. Treating "unknown" as "assume +// changed" would defeat the cache on every single crawl instead of just +// when a file actually needs re-hydrating. function resolveStub(author, fileName, mtime, authorRoot, cacheSnapshot) { const path = `${authorRoot}${encodeURIComponent(fileName)}`; const cached = cacheSnapshot.get(path); - if (cached && mtime != null && cached.mtime === mtime && cached.article?.hydrated) return cached.article; + if (cached && cached.mtime === mtime && cached.article?.hydrated) return cached.article; return buildStub(author, fileName, mtime, authorRoot); } -- 2.54.0 From c3d8f0171cab00372d2fced843c2c4f61460dcf2 Mon Sep 17 00:00:00 2001 From: biosias Date: Tue, 22 Sep 2026 16:02:38 +0200 Subject: [PATCH 06/12] Fix blank line silently dropping YAML block arrays (AI) A blank line between a block-array key and its "- item" lines matched neither the item nor the key regex, resetting arrayKey to null and causing the following items to be silently skipped. Blank lines are now ignored instead of ending the array. Co-Authored-By: Claude Sonnet 5 --- js/frontmatter.js | 1 + 1 file changed, 1 insertion(+) diff --git a/js/frontmatter.js b/js/frontmatter.js index 8a7c643..de50486 100644 --- a/js/frontmatter.js +++ b/js/frontmatter.js @@ -39,6 +39,7 @@ export function parseFrontmatter(raw) { const meta = {}; let arrayKey = null; for (const line of match[1].split(/\r?\n/)) { + if (!line.trim()) continue; // blank line — doesn't close a block array const item = line.match(/^\s*-\s+(.*)$/); if (item && arrayKey) { meta[arrayKey].push(parseScalar(item[1])); continue; } const field = line.match(/^([A-Za-z][A-Za-z0-9_-]*):(?:\s*(.*))?$/); -- 2.54.0 From 6d424ffbc9db280e2cf4c0ecc50edcb7e1664935 Mon Sep 17 00:00:00 2001 From: biosias Date: Tue, 22 Sep 2026 16:04:10 +0200 Subject: [PATCH 07/12] Stop crawl-health warning false-positiving on non-article files (AI) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The loaded/total ratio check treated a low filename-match rate as a sign of network/CORS breakage, but indexing does no per-article fetch at all — a low ratio just means many non-conforming .md files coexist with real articles, which is normal and not a health problem. Removed that heuristic and its now-unused crawlHealthArticlesFailed string; real breakage is still fully caught by the authors-failed check. Co-Authored-By: Claude Sonnet 5 --- README.md | 12 ++++++------ js/views.js | 24 +++++++++++++----------- strings/cs.json | 1 - strings/en.json | 1 - strings/sk.json | 1 - 5 files changed, 19 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index fa15122..f02b0de 100644 --- a/README.md +++ b/README.md @@ -517,12 +517,12 @@ of content is ever fetched per page view. a large site to a handful of requests instead of hydrating the entire article set up front. The page count is exact once the whole set's been scanned, and a lower bound (at least one more page) until then. -- **Crawl health** — if the root author-listing fetch fails outright, if every individual - author's own directory listing fails, or if very few of the known candidate files parsed - successfully (a malformed file name, or a total breakage like the wrong origin or CORS - misconfigured), a small warning banner is shown above the article list instead of silently - rendering an empty page (see `getCrawlStats()` / `renderCrawlHealthWarning()` in - `js/discovery.js` / `js/views.js`). +- **Crawl health** — if the root author-listing fetch fails outright, or if every individual + author's own directory listing fails (wrong origin, CORS misconfigured), a small warning banner + is shown above the article list instead of silently rendering an empty page (see + `getCrawlStats()` / `renderCrawlHealthWarning()` in `js/discovery.js` / `js/views.js`). + Non-conforming `.md` file names aren't treated as a health problem — only as a per-file skip — + since indexing does no per-article fetch that could actually fail. - **Design rationale** — the original crawler design (wave-based crawling with a per-article listing fetch, recency prioritisation, HTML-autoindex fallback, long-term "deploy and forget" constraints) is documented in diff --git a/js/views.js b/js/views.js index 2a69883..bd29d62 100644 --- a/js/views.js +++ b/js/views.js @@ -172,27 +172,29 @@ function renderCard(article) { // autoindex disabled) should be visible rather than silently rendering an // empty/near-empty list forever — see .claude/nginx-sitemap-crawler-plan.md L3. // Three independent signals feed this: the root author-listing fetch itself -// failing outright (rootFailed), every individual author's own listing +// failing outright (rootFailed), or every individual author's own listing // failing (checked regardless of site size, since discovery.js now silently // skips individual author failures — see the wave-2 comment in -// discovery.js), and, for larger sites, a near-zero loaded/total ratio at -// the article level. Only meaningful once the crawl has actually finished -// (`done`) — mid-crawl stats are incomplete and would misreport a healthy -// crawl as broken. +// discovery.js). Unlike an older version of this check, a low loaded/total +// ratio (candidate .md files that didn't match the article naming +// convention) is deliberately *not* treated as a health problem: indexing +// does no per-article fetch at all, so a non-conforming file name is a +// filename issue, not a network/CORS one — and legitimately non-conforming +// .md files (notes, leftovers) coexisting with real articles shouldn't +// trigger a false "something's broken" banner. Only meaningful once the +// crawl has actually finished (`done`) — mid-crawl stats are incomplete and +// would misreport a healthy crawl as broken. function renderCrawlHealthWarning(view, done) { if (!done) return; - const { total, loaded, authorsTotal, authorsFailed, rootFailed } = getCrawlStats(); + const { authorsTotal, authorsFailed, rootFailed } = getCrawlStats(); // rootFailed means the very first listing fetch (the author directories // themselves) couldn't be read at all — a total breakage same in spirit // as every author's own listing failing, just one level higher up. const authorsBroken = rootFailed || (authorsTotal > 0 && authorsFailed === authorsTotal); - const articlesBroken = total >= 5 && loaded / total < 0.1; - if (!authorsBroken && !articlesBroken) return; + if (!authorsBroken) return; const warning = document.createElement('p'); warning.className = 'error'; - warning.textContent = authorsBroken - ? t('crawlHealthAuthorsFailed') - : t('crawlHealthArticlesFailed', { loaded, total }); + warning.textContent = t('crawlHealthAuthorsFailed'); view.appendChild(warning); } diff --git a/strings/cs.json b/strings/cs.json index 0fd8971..d82ed90 100644 --- a/strings/cs.json +++ b/strings/cs.json @@ -16,7 +16,6 @@ "readingTime": "{minutes} min čtení", "crawlHealthAuthorsFailed": "Pozor: nepodařilo se načíst žádného autora — zkontrolujte nastavení serveru.", - "crawlHealthArticlesFailed": "Pozor: načtena jen {loaded} z {total} položek — zkontrolujte nastavení serveru.", "loadingArticles": "Načítám články…", "loadingGeneric": "Načítám…", diff --git a/strings/en.json b/strings/en.json index d2954b4..02f9964 100644 --- a/strings/en.json +++ b/strings/en.json @@ -16,7 +16,6 @@ "readingTime": "{minutes} min read", "crawlHealthAuthorsFailed": "Warning: couldn't load any authors — check your server configuration.", - "crawlHealthArticlesFailed": "Warning: only loaded {loaded} of {total} items — check your server configuration.", "loadingArticles": "Loading articles…", "loadingGeneric": "Loading…", diff --git a/strings/sk.json b/strings/sk.json index 40f7d6f..ff81b7b 100644 --- a/strings/sk.json +++ b/strings/sk.json @@ -16,7 +16,6 @@ "readingTime": "{minutes} min čítania", "crawlHealthAuthorsFailed": "Pozor: nepodarilo sa načítať žiadneho autora — skontrolujte nastavenie servera.", - "crawlHealthArticlesFailed": "Pozor: načítaných len {loaded} z {total} položiek — skontrolujte nastavenie servera.", "loadingArticles": "Načítavam články…", "loadingGeneric": "Načítavam…", -- 2.54.0 From cf2733368f2576f8b26136f0a3a85330839c5f2d Mon Sep 17 00:00:00 2001 From: biosias Date: Tue, 22 Sep 2026 16:04:49 +0200 Subject: [PATCH 08/12] Catch render() errors from the progress-listener callback (AI) The onArticlesProgress listener invoked async render() without awaiting or catching it, unlike the final render() call, which is inside a try/catch. A mid-crawl render failure was an unhandled rejection that left the list stuck on its loading state forever instead of showing articlesLoadError. Co-Authored-By: Claude Sonnet 5 --- js/views.js | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/js/views.js b/js/views.js index bd29d62..5300b8c 100644 --- a/js/views.js +++ b/js/views.js @@ -284,12 +284,19 @@ export async function loadArticleList(filter = null, page = 1) { } }; - const unsubscribe = onArticlesProgress((partialArticles, done) => { render(partialArticles, done); }); + // Same error handling as the final render below (not just the same + // message — an error thrown here, e.g. a malformed template, would + // otherwise be an unhandled rejection that leaves the list stuck on its + // loading state instead of ever showing articlesLoadError). + const showRenderError = error => { + if (generation === currentGeneration) view.innerHTML = `

${t('articlesLoadError', { message: error.message })}

`; + }; + const unsubscribe = onArticlesProgress((partialArticles, done) => { render(partialArticles, done).catch(showRenderError); }); try { const allArticles = await discoverArticles(); await render(allArticles, true); } catch (error) { - if (generation === currentGeneration) view.innerHTML = `

${t('articlesLoadError', { message: error.message })}

`; + showRenderError(error); } finally { unsubscribe(); } -- 2.54.0 From 057f6673cbc6f0920a8ad28a48766719452a3ce2 Mon Sep 17 00:00:00 2001 From: biosias Date: Tue, 22 Sep 2026 16:05:54 +0200 Subject: [PATCH 09/12] Cap concurrent per-author listing fetches during indexing (AI) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Wave 2 of the crawl fetched every author's directory listing via a plain Promise.all, with no limit — a site with hundreds of authors fired one simultaneous request per author on every cold-cache load. Routed through mapWithConcurrency/MAX_CONCURRENT_REQUESTS, the same cap hydrateArticles() already respects for content fetches. Co-Authored-By: Claude Sonnet 5 --- js/discovery.js | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/js/discovery.js b/js/discovery.js index e879fbe..a7ebc2f 100644 --- a/js/discovery.js +++ b/js/discovery.js @@ -4,13 +4,16 @@ import { mapWithConcurrency } from './concurrency.js'; import { getAllCacheEntries, setCacheEntry, pruneCache } from './cache.js'; import { t } from './i18n.js'; -// Caps simultaneous markdown content fetches during hydration (see -// hydrateArticle()/hydrateArticles()) — the only per-article requests left, -// since indexing no longer needs a per-article listing fetch at all (every -// stub field is encoded in its file's own name — see buildStub()). Raise it -// for a fast/local nginx, lower it for a slow or shared/production instance -// — see .claude/nginx-sitemap-crawler-plan.md §2a for the original rationale -// (written when this also governed indexing requests). +// Caps simultaneous requests in the two places this module still fires one +// per item: markdown content fetches during hydration (see +// hydrateArticle()/hydrateArticles()) and each author's own directory +// listing during wave 2 of indexing (see runCrawl()) — indexing no longer +// needs a per-*article* fetch at all (every stub field is encoded in its +// file's own name — see buildStub()), but still fetches one listing per +// author. Raise it for a fast/local nginx, lower it for a slow or shared/ +// production instance — see .claude/nginx-sitemap-crawler-plan.md §2a for +// the original rationale (written when this also governed per-article +// indexing requests). const MAX_CONCURRENT_REQUESTS = 10; const MONTH_ABBREVIATIONS = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec']; @@ -333,13 +336,15 @@ async function runCrawl() { // Wave 2 — always fetched live, in full, for every author (not just the // current page): each author's own directory listing, with per-file - // mtimes. This is what builds the flat, cross-author list that drives - // global sorting and pagination — see the plan doc's "keep the flat - // feed" decision. It's also the *only* per-author network request - // indexing ever needs: every stub field (slug, date, tags) is parsed - // straight out of each article file's own name (see - // buildStub/parseArticleFileName) — there's no per-article fetch at - // index time. The `type === 'file'` filter below also means any + // mtimes. Capped by MAX_CONCURRENT_REQUESTS like hydrateArticles() below + // — a site with many authors would otherwise fire one listing request + // per author simultaneously, with no limit. This is what builds the + // flat, cross-author list that drives global sorting and pagination — + // see the plan doc's "keep the flat feed" decision. It's also the *only* + // per-author network request indexing ever needs: every stub field + // (slug, date, tags) is parsed straight out of each article file's own + // name (see buildStub/parseArticleFileName) — there's no per-article + // fetch at index time. The `type === 'file'` filter below also means any // subdirectory (the author's shared `media/` folder, a `draft/` folder — // see "Adding an article" in the README) is skipped without ever being // descended into. The author's own bio file (`.md`) is filtered @@ -350,7 +355,7 @@ async function runCrawl() { // origin, CORS) remains visible via getCrawlStats() instead of just // silently producing an empty article list. let authorsFailed = 0; - const authorFileLists = await Promise.all(authors.map(async author => { + const authorFileLists = await mapWithConcurrency(authors, MAX_CONCURRENT_REQUESTS, async author => { const authorRoot = `${ARTICLES_URL}${encodeURIComponent(author.name)}/`; const bioFileName = `${author.name}.md`; try { @@ -362,7 +367,7 @@ async function runCrawl() { authorsFailed += 1; return []; } - })); + }); const allFiles = authorFileLists.flat(); // Every stub is parsed synchronously from data already in hand — no -- 2.54.0 From 2b2cdb1d3c8638c85eb365b77483854fee7e4e1d Mon Sep 17 00:00:00 2001 From: biosias Date: Tue, 22 Sep 2026 16:07:45 +0200 Subject: [PATCH 10/12] Translate the optional #site-title header link at startup (AI) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit document.title was already set from the siteName string, but a visible #site-title header link stayed hardcoded in whatever language it was typed in — a translated site showed a translated tab title next to an untranslated header. Set once in router.js's init() rather than showListView(), so a page that loads straight into an article route still gets it. Documented #site-title as optional embedding markup and corrected README's claim that isn't JS-templated. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --- README.md | 14 +++++++++++--- js/router.js | 7 +++++++ 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index f02b0de..d68f8c3 100644 --- a/README.md +++ b/README.md @@ -271,9 +271,12 @@ The resolved locale also drives date formatting (`Intl.DateTimeFormat`), search tag-label sorting (`localeCompare`), via `getLocale()` in the same module — so a translation file doesn't just relabel text, it affects how dates and sorting behave too. -Site-specific branding that lives in the embedding page itself (the page `<title>`, the header -search placeholder, the sidebar section headings) is plain static HTML, not JS-templated — -translate it by editing that page directly (`demo/index.html` for this repo's own demo). +The page `<title>` and an optional `#site-title` header link (see "Embedding in an existing +page" below) are both set from the `siteName` string at startup, so they stay in sync with the +resolved locale automatically — translate `siteName` in `strings/xx.json` rather than editing +the page. Other site-specific branding in the embedding page itself (the header search +placeholder, the sidebar section headings) is plain static HTML, not JS-templated — translate +that by editing the page directly (`demo/index.html` for this repo's own demo). ## Embedding in an existing page @@ -350,6 +353,11 @@ demo does. ```html <nav id="tag-filters" aria-label="Filter by tag"></nav> ``` +- **Site title link — `#site-title`.** Optional; its text is set to the `siteName` string once at + startup, so it stays in sync with the resolved locale without editing the page per language. + ```html + <a href="#" id="site-title">My Site</a> + ``` `#tag-filters` is optional — omit it and nothing breaks, the article list and search still work standalone. diff --git a/js/router.js b/js/router.js index bcdf633..e9fa7e1 100644 --- a/js/router.js +++ b/js/router.js @@ -78,6 +78,13 @@ function init() { if (!document.getElementById('list-view') || !document.getElementById('article-view')) { throw new Error('Missing required #list-view and/or #article-view element — see the "Embedding in an existing page" section of the README.'); } + // Optional embedding-page markup (see "Embedding in an existing + // page" in the README) — set once here rather than in + // showListView()/loadArticle() so a page that loads straight into + // an article route (a direct link) still gets it, not just one that + // starts on the list view. + const siteTitle = document.getElementById('site-title'); + if (siteTitle) siteTitle.textContent = t('siteName'); }); // Show a top-level error if dependencies fail to load (or a mount point -- 2.54.0 From 4e2229fed9defff340a8d0aadd1fafff7931b5bf Mon Sep 17 00:00:00 2001 From: biosias <jakub@gdos.sk> Date: Tue, 22 Sep 2026 16:08:17 +0200 Subject: [PATCH 11/12] Log hydration failures instead of swallowing them silently (AI) hydrateArticleNow()'s catch-all discarded network, HTTP, and parsing errors alike with no diagnostic, using the same fallback for all of them. A real bug in parseFrontmatter()/deriveContentFields() would silently leave an article title-less forever with no way to tell it apart from a transient fetch blip. Now logged via console.error. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --- js/discovery.js | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/js/discovery.js b/js/discovery.js index a7ebc2f..8e1466a 100644 --- a/js/discovery.js +++ b/js/discovery.js @@ -294,7 +294,14 @@ async function hydrateArticleNow(stub) { const authoritative = { ...stub }; Object.assign(stub, meta, authoritative, deriveContentFields(meta, content, stub.path), { hydrated: true }); await setCacheEntry(stub.path, { mtime: stub.mtime, article: stub }); - } catch { /* leave stub unhydrated — retried next time it's rendered */ } + } catch (error) { + // Leave stub unhydrated — retried next time it's rendered. Logged + // (not surfaced in the UI, which just shows the title-less card) + // so a persistent failure — a real bug in parseFrontmatter() or + // deriveContentFields(), not just a transient fetch blip — is still + // visible somewhere instead of silently retrying forever. + console.error(`Failed to hydrate article ${stub.path}:`, error); + } return stub; } -- 2.54.0 From 96cefba24dd28555966042a4a12dec1101d0add1 Mon Sep 17 00:00:00 2001 From: biosias <jakub@gdos.sk> Date: Tue, 22 Sep 2026 16:18:03 +0200 Subject: [PATCH 12/12] Cache per-query search matches so repeat searches skip unchanged articles (AI) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a searchQueries IndexedDB store (js/cache.js) keyed by the exact search text, holding a { articlePath: { mtime, matched } } map per query. loadSearch() loads it once per new query, trusts a cached result whenever an article's mtime still matches instead of running matchesQuery() again, and writes back whatever's newly checked each scan round. Repeat searches for the same term — even after a page reload — now only need to actually check articles that are new or have changed since, not the whole site again. Capped at MAX_CACHED_SEARCHES (30) distinct queries with least-recently-used eviction so trying many different terms over time doesn't grow the cache unbounded. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --- README.md | 21 +++++++++--- js/cache.js | 92 ++++++++++++++++++++++++++++++++++++++++++++++------- js/views.js | 50 +++++++++++++++++++++++++++-- 3 files changed, 144 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index d68f8c3..39f4066 100644 --- a/README.md +++ b/README.md @@ -507,11 +507,14 @@ of content is ever fetched per page view. content, so a code change to how articles are derived (in `js/discovery.js`) can make old cached entries stale in *structure*, not just content — and a file's `mtime` won't catch that. Bump `CACHE_SCHEMA_VERSION` in `js/cache.js` whenever that shape changes; it's baked into - the IndexedDB database name, so a bump invalidates every cached entry on the next deploy. -- **Concurrency** — `MAX_CONCURRENT_REQUESTS` in `js/discovery.js` (default `10`) caps how - many hydration (content) fetches run at once — the only per-article requests left, since - indexing no longer needs one; raise it for a fast/local nginx, lower it for a slow or shared/ - production instance. + the IndexedDB database name, so a bump invalidates every cached entry on the next deploy — + articles and cached searches alike. A change to the search-match record shape instead, or a new + store entirely, only needs `DB_VERSION` bumped (IndexedDB's own upgrade counter): `onupgradeneeded` + creates whatever store is missing without touching the other one's existing data. +- **Concurrency** — `MAX_CONCURRENT_REQUESTS` in `js/discovery.js` (default `10`) caps both how + many hydration (content) fetches run at once and how many authors' directory listings are + fetched at once during indexing; raise it for a fast/local nginx, lower it for a slow or + shared/production instance. - **Prefetching** — once the current page has fully rendered, the next page's articles are hydrated in the background (fire-and-forget), so clicking "next" is usually an instant cache hit rather than a fresh round-trip. @@ -525,6 +528,14 @@ of content is ever fetched per page view. a large site to a handful of requests instead of hydrating the entire article set up front. The page count is exact once the whole set's been scanned, and a lower bound (at least one more page) until then. +- **Persistent search-match cache** — alongside the resumable in-session scan above, each query's + per-article match results (`{ mtime, matched }`, keyed by article path) are cached in IndexedDB + too (`searchQueries` store in `js/cache.js`, keyed by the query text itself) and reused across + page reloads and browser sessions, not just while paging through one visit. Searching the same + term again only re-fetches and re-checks articles that are new or whose `mtime` has changed + since — everything else is trusted from the cache without a content re-fetch at all. Capped at + `MAX_CACHED_SEARCHES` (default `30`) distinct queries, evicting the least-recently-searched one + first, so trying many different search terms over time doesn't grow the cache unbounded. - **Crawl health** — if the root author-listing fetch fails outright, or if every individual author's own directory listing fails (wrong origin, CORS misconfigured), a small warning banner is shown above the article list instead of silently rendering an empty page (see diff --git a/js/cache.js b/js/cache.js index 2994674..6d52d38 100644 --- a/js/cache.js +++ b/js/cache.js @@ -1,10 +1,11 @@ -// Persistent cache for discovered articles, keyed by each article file's own -// path. Raw IndexedDB (no wrapper library — see +// Persistent cache for discovered articles (keyed by each article file's own +// path) and for per-search-query match results (keyed by the query text — +// see setSearchCacheEntry() below). Raw IndexedDB (no wrapper library — see // .claude/nginx-sitemap-crawler-plan.md L4/L5 on the zero-dependency, // long-stable-API requirement). If IndexedDB is unavailable (private // browsing, old browser, blocked), every operation below silently becomes a -// no-op so discovery.js gets live-fetch behaviour for free without needing -// its own availability branching (plan doc R6). +// no-op so discovery.js/views.js get live-fetch/live-scan behaviour for free +// without needing their own availability branching (plan doc R6). // Bump this whenever discovery.js's derived article record shape changes. // The version is baked into the DB name so a code deploy invalidates every @@ -18,7 +19,19 @@ const CACHE_SCHEMA_VERSION = 3; // this module degrades to live-fetch behaviour whenever the cache is empty // or unavailable). Do this deliberately, not casually. const DB_NAME = `cmkiss-cache-v${CACHE_SCHEMA_VERSION}`; -const STORE_NAME = 'articleFiles'; +const ARTICLE_STORE = 'articleFiles'; +const SEARCH_STORE = 'searchQueries'; +// IndexedDB's own upgrade counter — separate from CACHE_SCHEMA_VERSION, +// which only governs the *article* record shape (and busts the cache by +// renaming the whole database). Bump this instead whenever a store or index +// is added/changed; onupgradeneeded below creates only what's missing, so +// existing article-cache data survives the upgrade. +const DB_VERSION = 2; +// Caps how many distinct queries' match results stay cached — otherwise +// every unique term ever typed into search would keep its own per-article +// map forever, growing unbounded on a long-lived site. Evicted +// least-recently-searched-first — see setSearchCacheEntry(). +const MAX_CACHED_SEARCHES = 30; let cacheAvailable = true; let dbPromise = null; @@ -27,9 +40,13 @@ function openDb() { if (!dbPromise) { dbPromise = new Promise((resolve, reject) => { if (typeof indexedDB === 'undefined') { reject(new Error('IndexedDB unavailable')); return; } - const request = indexedDB.open(DB_NAME, 1); + const request = indexedDB.open(DB_NAME, DB_VERSION); request.onupgradeneeded = () => { - request.result.createObjectStore(STORE_NAME, { keyPath: 'path' }); + const db = request.result; + if (!db.objectStoreNames.contains(ARTICLE_STORE)) db.createObjectStore(ARTICLE_STORE, { keyPath: 'path' }); + if (!db.objectStoreNames.contains(SEARCH_STORE)) { + db.createObjectStore(SEARCH_STORE, { keyPath: 'query' }).createIndex('lastUsed', 'lastUsed'); + } }; request.onsuccess = () => resolve(request.result); request.onerror = () => reject(request.error); @@ -42,7 +59,7 @@ function openDb() { // Opens a store and resolves/rejects with the given IDBRequest's own outcome // — the standard pattern, so result timing never depends on transaction // lifecycle details. -async function runRequest(mode, makeRequest) { +async function runRequest(storeName, mode, makeRequest) { if (!cacheAvailable) return undefined; let db; try { @@ -51,7 +68,7 @@ async function runRequest(mode, makeRequest) { return undefined; // openDb() itself already flipped cacheAvailable } try { - const store = db.transaction(STORE_NAME, mode).objectStore(STORE_NAME); + const store = db.transaction(storeName, mode).objectStore(storeName); return await new Promise((resolve, reject) => { const request = makeRequest(store); request.onsuccess = () => resolve(request.result); @@ -72,12 +89,12 @@ async function runRequest(mode, makeRequest) { // reads this once up front rather than doing a point lookup per file during // indexing. Empty Map on a miss or if the cache is unavailable. export async function getAllCacheEntries() { - const records = await runRequest('readonly', store => store.getAll()); + const records = await runRequest(ARTICLE_STORE, 'readonly', store => store.getAll()); return new Map((records || []).map(record => [record.path, record])); } export async function setCacheEntry(path, entry) { - await runRequest('readwrite', store => store.put({ ...entry, path })); + await runRequest(ARTICLE_STORE, 'readwrite', store => store.put({ ...entry, path })); } // Removes any cached entry whose path is no longer in the live directory @@ -92,7 +109,7 @@ export async function pruneCache(validPaths) { return; // openDb() itself already flipped cacheAvailable } try { - const store = db.transaction(STORE_NAME, 'readwrite').objectStore(STORE_NAME); + const store = db.transaction(ARTICLE_STORE, 'readwrite').objectStore(ARTICLE_STORE); await new Promise((resolve, reject) => { const cursorRequest = store.openCursor(); cursorRequest.onsuccess = event => { @@ -108,3 +125,54 @@ export async function pruneCache(validPaths) { // mean IndexedDB itself is unavailable. } } + +// Per-query search-match cache: `entries` is a { [articlePath]: { mtime, +// matched } } map — see loadSearch() in views.js, which trusts an entry +// (skipping a content re-fetch entirely) whenever its mtime still matches +// the live article, and only re-checks what's new or changed. Keyed by the +// exact (locale-lowercased) text matchesQuery() itself compares against, so +// "Linux" and "linux" share one cached entry. +export async function getSearchCacheEntry(query) { + return runRequest(SEARCH_STORE, 'readonly', store => store.get(query)); +} + +export async function setSearchCacheEntry(query, entries) { + await runRequest(SEARCH_STORE, 'readwrite', store => store.put({ query, entries, lastUsed: Date.now() })); + await pruneSearchCache(); +} + +// Keeps the search-query cache bounded to MAX_CACHED_SEARCHES entries, +// dropping the least-recently-searched ones first — the same "don't grow +// unbounded" concern pruneCache() addresses for the article cache, just +// against searches instead of renamed/deleted files. +async function pruneSearchCache() { + if (!cacheAvailable) return; + let db; + try { + db = await openDb(); + } catch { + return; + } + try { + const store = db.transaction(SEARCH_STORE, 'readwrite').objectStore(SEARCH_STORE); + let excess = await new Promise((resolve, reject) => { + const request = store.count(); + request.onsuccess = () => resolve(request.result - MAX_CACHED_SEARCHES); + request.onerror = () => reject(request.error); + }); + if (excess <= 0) return; + await new Promise((resolve, reject) => { + const cursorRequest = store.index('lastUsed').openCursor(); + cursorRequest.onsuccess = event => { + const cursor = event.target.result; + if (!cursor || excess <= 0) { resolve(); return; } + cursor.delete(); + excess -= 1; + cursor.continue(); + }; + cursorRequest.onerror = () => reject(cursorRequest.error); + }); + } catch { + // Same reasoning as runRequest() above. + } +} diff --git a/js/views.js b/js/views.js index 5300b8c..2bedb1d 100644 --- a/js/views.js +++ b/js/views.js @@ -1,5 +1,6 @@ import { templates } from './templates.js'; import { deriveContentFields, discoverArticles, fetchAuthorProfile, findArticle, getCrawlStats, hydrateArticles, onArticlesProgress, resolveMediaUrl } from './discovery.js'; +import { getSearchCacheEntry, setSearchCacheEntry } from './cache.js'; import { parseFrontmatter } from './frontmatter.js'; import { t, getLocale } from './i18n.js'; @@ -374,6 +375,16 @@ const SEARCH_SCAN_BATCH = PAGE_SIZE; // same page) skip scanning entirely, since `matches` only ever grows and // stub order never changes. Reset whenever the query text changes; holds // only the most recent query, so a new search simply replaces it. +// +// `entries` is this same session's in-memory mirror of the persistent +// per-query cache (see cache.js's getSearchCacheEntry()/ +// setSearchCacheEntry()): a { [articlePath]: { mtime, matched } } map, +// loaded once when the query first changes and written back whenever a scan +// round adds something new to it (`dirty`). Reusing it — instead of only +// `cursor`/`matches` — means even a *first* search after a page reload can +// skip re-fetching and re-checking every article whose content hasn't +// changed since it was last searched for, no matter how this query's own +// scan happened to leave off last time. let searchState = null; export async function loadSearch(query, page = 1) { @@ -399,7 +410,16 @@ export async function loadSearch(query, page = 1) { renderFilterNavigation(stubs); if (!searchState || searchState.query !== query || searchState.stubs !== stubs) { - searchState = { query, stubs, cursor: 0, matches: [] }; + // Loaded once per new query. Pruned to the current stub set's + // paths so a renamed/deleted article's old cached result can't + // linger — cheap, since `stubs` (unlike hydrating them) is + // already in hand with no further fetch. + const cached = await getSearchCacheEntry(needle); + if (generation !== currentGeneration) return; + const validPaths = new Set(stubs.map(article => article.path)); + const entries = {}; + for (const [path, entry] of Object.entries(cached?.entries || {})) if (validPaths.has(path)) entries[path] = entry; + searchState = { query, stubs, cursor: 0, matches: [], entries, dirty: false }; } const state = searchState; @@ -451,12 +471,38 @@ export async function loadSearch(query, page = 1) { while (state.cursor < state.stubs.length && state.matches.length < target) { if (generation !== currentGeneration) return; const batch = state.stubs.slice(state.cursor, state.cursor + SEARCH_SCAN_BATCH); + // Still hydrates the whole batch — a no-op per already-hydrated + // stub (see hydrateArticle() in discovery.js), which a cache hit + // below always is: it can only exist for a path this exact + // query already matched content against, at this same mtime, + // which means it's already sitting in the article-level cache + // too. Rendering pageArticles further down never re-hydrates, + // so this stays unconditional rather than trusting that chain + // of reasoning to never drift out of sync. await hydrateArticles(batch); if (generation !== currentGeneration) return; state.cursor += batch.length; - batch.forEach(article => { if (matchesQuery(article)) state.matches.push(article); }); + batch.forEach(article => { + // A cached result only counts as known if its mtime still + // matches the live stub — an edited-since-last-search + // article is re-checked, same as one with no entry at all. + const known = state.entries[article.path]; + const isKnown = known?.mtime === article.mtime; + const matched = isKnown ? known.matched : matchesQuery(article); + if (!isKnown) { state.entries[article.path] = { mtime: article.mtime, matched }; state.dirty = true; } + if (matched) state.matches.push(article); + }); renderResults(); } + // Persist whatever's new this round — fire-and-forget, since a + // failed/slow write should never hold up rendering (cache.js + // already swallows its own storage errors). Next time this exact + // query is searched, anything covered here needs no re-fetch at + // all, only what's new or changed since. + if (state.dirty) { + state.dirty = false; + setSearchCacheEntry(needle, state.entries); + } } catch (error) { if (generation === currentGeneration) view.innerHTML = `<p class="error">${t('searchError', { message: error.message })}</p>`; } -- 2.54.0