diff --git a/README.md b/README.md index b1a3fdc..39f4066 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# cmjs — Minimal Client-Side CMS +# cmkiss — Minimal Client-Side CMS A minimal CMS where articles are Markdown files rendered in the browser. No build step, no database, no server-side code. @@ -22,12 +22,12 @@ A minimal CMS where articles are Markdown files rendered in the browser. No buil ## Features - **Dark mode** — automatically follows the OS `prefers-color-scheme` setting; no toggle needed. -- **Client-side search** — the search box in the header filters all articles by title, description, author, category, and tags in-memory. Navigates to `#search/query` so results are bookmarkable; results paginate the same way the article list does. +- **Client-side search** — the search box in the header filters all articles by title, description, author, and tags in-memory. Navigates to `#search/query` so results are bookmarkable; results paginate the same way the article list does. - **Reading time** — estimated from the article word count (200 wpm) and shown on list cards and in article headers. Computed automatically; no frontmatter field needed. - **Table of contents** — auto-generated from `h2`/`h3` headings and inserted at the top of the article body. Only appears when the article has three or more headings. - **Back link** — a "← Back to articles" link above every article, returning to whatever list, filter, or search page (and page number) the reader actually came from rather than always resetting to page 1 of everything. - **Pagination** — article list paginates at 20 articles per page; page state is encoded in the hash so browser history works. -- **Author, category, and tag filtering** — clicking an author name, category badge, or tag pill shows a filtered list. All three support pagination. +- **Author and tag filtering** — clicking an author name or tag pill shows a filtered list. Both support pagination. - **Responsive layout** — layout, cards, images, and pagination all adapt to narrow screens (breakpoint at 640 px). - **Keyboard navigation** — all interactive elements have visible `:focus-visible` outlines for keyboard users. - **Author profiles** — a profile card with bio and social links is shown at the top of each author's article list. Created by adding `articles/{author}/{author}.md`. @@ -36,14 +36,14 @@ A minimal CMS where articles are Markdown files rendered in the browser. No buil ## Project structure -This whole repo *is* cmjs — clone it as-is into an existing site (or copy `css/`, `js/`, +This whole repo *is* cmkiss — clone it as-is into an existing site (or copy `css/`, `js/`, `strings/`, `templates/`, and your own `articles/` into one) and embed it into a page there; see "Embedding in an existing page" below. `demo/` is a standalone demo/preview page, kept separate from the library — not something an embedding page needs to bring along. ``` ├── README.md -├── demo/ # Standalone demo/preview page — not needed to embed cmjs elsewhere +├── demo/ # Standalone demo/preview page — not needed to embed cmkiss elsewhere │ ├── index.html # Single-page shell │ └── css/ │ ├── reset.css # CSS variables, dark mode, box-sizing reset, focus styles @@ -66,7 +66,7 @@ from the library — not something an embedding page needs to bring along. │ ├── cache.js # IndexedDB cache for discovered articles, keyed by article file path │ ├── concurrency.js # mapWithConcurrency() — caps simultaneous per-article fetches │ ├── i18n.js # t(), loadStrings(), getLocale() — UI string lookup -│ ├── views.js # loadArticleList, loadArticle, loadAuthorView, loadCategoryView, loadSearch +│ ├── views.js # loadArticleList, loadArticle, loadAuthorView, loadTagView, loadSearch │ └── router.js # Hash router and app entry point ├── vendor/ │ └── marked.min.js # marked.js (local copy, loaded as a global) — the only 3rd-party code @@ -91,11 +91,11 @@ from the library — not something an embedding page needs to bring along. `js/config-default.js` holds the shipped defaults (`BASE_URL`, `MARKED_URL`, `ARTICLES_URL`) and is tracked in git; every other module imports from it. It also loads an optional `js/config.js` -— gitignored, not part of cmjs's own git history — to override any of them. **Nothing needs to +— gitignored, not part of cmkiss's own git history — to override any of them. **Nothing needs to be created before first use**: if `js/config.js` doesn't exist, the shipped defaults apply as-is. -The split exists for using cmjs as a git submodule (or otherwise pulling in a newer version some -other way): since `config.js` is gitignored, updating cmjs never touches it, so any override +The split exists for using cmkiss as a git submodule (or otherwise pulling in a newer version some +other way): since `config.js` is gitignored, updating cmkiss never touches it, so any override you've made survives. Only `config-default.js` gets updated. To override a setting, create `js/config.js` and export just the constant(s) you want to change — @@ -110,8 +110,8 @@ export const ARTICLES_URL = new URL('/blog-content/articles/', location.origin). ## Adding an article An article is a single Markdown file placed directly in its author's directory. Its *file name* -is where its publish date, tags, category, and draft status all live — not in separate files, and -not in frontmatter. The format is: +is where its publish date and tags live — not in separate files, and not in frontmatter. The +format is: ``` ~~++….md @@ -121,18 +121,21 @@ not in frontmatter. The format is: as permanent: retagging or correcting the date later just means renaming the file (see below), but changing the slug changes the article's URL. - `` — the publish date, 8 digits, no separators (e.g. `16062026` for 16 June 2026). -- `++…` — zero or more tag ids, joined with `+`. A tag id matching `year-YYYY` is - treated as the article's "year"/category, shown as a badge linking to `#category/year-YYYY`; - every other tag id is a normal topic tag, shown as a pill linking to `#tag/tag-id`. Omit this - whole segment (and its leading `~`) if the article has no tags. +- `++…` — zero or more tag ids, joined with `+`. Each tag id is a normal topic tag, + shown as a pill linking to `#tag/tag-id`. Omit this whole segment (and its leading `~`) if the + article has no tags. The separator is `~` (tilde), not a hyphen — deliberately, since a slug is expected to be plain kebab-case (single hyphens only) and `~` isn't something you'd type by accident while writing one, unlike an extra `-`. -A leading underscore on the whole file name — `_~....md` — marks the article as a -**draft**: it's excluded from the list, search, and pagination, but still reachable directly by -URL. Remove the underscore to publish it. +**Drafts** aren't a file-naming convention — they live in a separate `draft/` subdirectory inside +the author's own directory: `articles/author/draft/my-article~16062026~guides.md`. The crawler +only ever reads `.md` files directly inside an author's directory, so anything under `draft/` is +never indexed, listed, searched, or paginated. nginx should also deny access to every author's +`draft/` folder outright (see "nginx configuration" below), so a draft isn't reachable by a +guessed direct URL either — not just hidden from the crawl. To publish an article, move its file +out of `draft/` into the author directory itself. Images (a cover photo, inline pictures) live in a `media/` folder shared by all of that author's articles, not next to each individual article file — so the same picture can be reused across @@ -143,14 +146,15 @@ Steps: 1. Create the author directory if it doesn't exist: `articles/author/` 2. Create the Markdown file directly inside it, using the naming convention above, e.g. - `articles/author/my-article~16062026~guides+year-2026.md` + `articles/author/my-article~16062026~guides+tutorials.md` — or inside `articles/author/draft/` + instead, if it's not ready to publish yet. 3. Optionally add a cover image to that author's shared `articles/author/media/` folder (creating it if needed), and reference it from frontmatter with `image: media/photo.jpg`. 4. Add a frontmatter block at the top of the Markdown file -5. Write your content — it will appear in the list automatically +5. Write your content — it will appear in the list automatically (once it's out of `draft/`) -To change an article's date, tags, or draft status later, rename its file — its content doesn't -need to change. `author` is derived from the containing directory — no frontmatter field needed +To change an article's date or tags later, rename its file — its content doesn't need to change. +`author` is derived from the containing directory — no frontmatter field needed for it either. Treat the directory name as case-sensitive and exact: it's matched exactly (not case-insensitively) against `#author/...` URLs and author-profile lookups, so avoid two author directories that differ only in case. Because every one of these is encoded in the file name, @@ -180,8 +184,8 @@ Your markdown content here... | `image` | no | Path to an image in the author's shared `media/` folder (e.g. `media/photo.jpg`); shown as card thumbnail and article hero | Reading time is computed automatically from the article word count — no frontmatter field -needed. `date`, `tags`, and `draft` frontmatter fields, if present, are ignored — superseded by -the directory name described above. +needed. `date` and `tags` frontmatter fields, if present, are ignored — superseded by the file +name described above. ## Author profiles @@ -227,13 +231,11 @@ The body text below the frontmatter is rendered as markdown and shown as the bio | `#search/linux/page/2` | Same, page 2 | | `#author/alice` | All articles by alice, page 1 | | `#author/alice/page/2` | All articles by alice, page 2 | -| `#category/year-2026` | All articles carrying the "year-2026" year tag, page 1 | -| `#category/year-2026/page/2` | Same, page 2 | | `#tag/guides` | All articles tagged "guides", page 1 | | `#tag/guides/page/2` | Same, page 2 | | `#post/alice/my-article` | Single article at `articles/alice/my-article~DDMMYYYY~tags.md` | -Author names and category tags on each card and article are clickable links to their respective filter views. A search box in the header navigates to `#search/query`. +Author names and tags on each card and article are clickable links to their respective filter views. A search box in the header navigates to `#search/query`. ## Customising the layout @@ -269,13 +271,16 @@ The resolved locale also drives date formatting (`Intl.DateTimeFormat`), search tag-label sorting (`localeCompare`), via `getLocale()` in the same module — so a translation file doesn't just relabel text, it affects how dates and sorting behave too. -Site-specific branding that lives in the embedding page itself (the page ``, the header -search placeholder, the sidebar section headings) is plain static HTML, not JS-templated — -translate it by editing that page directly (`demo/index.html` for this repo's own demo). +The page `<title>` and an optional `#site-title` header link (see "Embedding in an existing +page" below) are both set from the `siteName` string at startup, so they stay in sync with the +resolved locale automatically — translate `siteName` in `strings/xx.json` rather than editing +the page. Other site-specific branding in the embedding page itself (the header search +placeholder, the sidebar section headings) is plain static HTML, not JS-templated — translate +that by editing the page directly (`demo/index.html` for this repo's own demo). ## Embedding in an existing page -To add cmjs to any HTML page, three things are needed: the CSS, the two mount-point `<div>`s, and the script. +To add cmkiss to any HTML page, three things are needed: the CSS, the two mount-point `<div>`s, and the script. ### 1. CSS — in `<head>` @@ -303,7 +308,7 @@ The site-level CSS files (`reset.css`, `header.css`, `layout.css`) are optional <div id="article-view" hidden></div> ``` -Place these wherever you want the CMS to appear. The router toggles `hidden` between them on navigation. A search box and tag/year filter lists are optional and can go anywhere else in the page — see "Search, article list, and tag list as independent pieces" below. Unlike those, `#list-view`/`#article-view` are both required — router.js checks for them at startup and shows a clear error (scoped to whichever of the two is present, or the whole page if neither is) instead of silently failing if one is missing. +Place these wherever you want the CMS to appear. The router toggles `hidden` between them on navigation. A search box and tag filter list are optional and can go anywhere else in the page — see "Search, article list, and tag list as independent pieces" below. Unlike those, `#list-view`/`#article-view` are both required — router.js checks for them at startup and shows a clear error (scoped to whichever of the two is present, or the whole page if neither is) instead of silently failing if one is missing. ### 3. Script — before `</body>` @@ -313,10 +318,10 @@ Place these wherever you want the CMS to appear. The router toggles `hidden` bet `router.js` is the only script tag needed. It loads `marked.min.js` automatically at startup (path configured in `js/config.js`). -> **Subdirectory pages:** If the embedding page lives in a subdirectory (e.g. `/blog/index.html`) and cmjs isn't installed at the site root, a plain relative path like `css/...` will resolve against the *page's* directory, not wherever cmjs actually lives. Use a path (root-relative or otherwise) that correctly points at wherever you placed it, e.g. if cmjs lives at `/cmjs/` on your site: +> **Subdirectory pages:** If the embedding page lives in a subdirectory (e.g. `/blog/index.html`) and cmkiss isn't installed at the site root, a plain relative path like `css/...` will resolve against the *page's* directory, not wherever cmkiss actually lives. Use a path (root-relative or otherwise) that correctly points at wherever you placed it, e.g. if cmkiss lives at `/cmkiss/` on your site: > ```html -> <link rel="stylesheet" href="/cmjs/css/components/card.css"> -> <script type="module" src="/cmjs/js/router.js"></script> +> <link rel="stylesheet" href="/cmkiss/css/components/card.css"> +> <script type="module" src="/cmkiss/js/router.js"></script> > ``` > All internal fetches (articles, templates, marked.js) resolve relative to the module files via `import.meta.url` and work automatically — only the HTML tags need a path pointing at the right place. @@ -348,17 +353,14 @@ demo does. ```html <nav id="tag-filters" aria-label="Filter by tag"></nav> ``` -- **Year list — `#year-filters`** (optional, not used by this repo's own demo but supported the - same way): a container that gets filled in with one link per year 2026 down to 2009 that has at - least one article, linking to `#category/year-YYYY`. Can appear alongside `#tag-filters` or on - its own. +- **Site title link — `#site-title`.** Optional; its text is set to the `siteName` string once at + startup, so it stays in sync with the resolved locale without editing the page per language. ```html - <nav id="year-filters" aria-label="Filter by year"></nav> + <a href="#" id="site-title">My Site</a> ``` -Both filter lists are populated by the same function on every list render, so whichever of -`#tag-filters`/`#year-filters` is present just gets filled in — omit either one (or both) and -nothing breaks, the article list and search still work standalone. +`#tag-filters` is optional — omit it and nothing breaks, the article list and search still work +standalone. ### Minimal example @@ -386,19 +388,30 @@ nothing breaks, the article list and search still work standalone. ## nginx configuration -Point nginx `root` at the repo root. The `autoindex` directive applies recursively to all subdirectories under `/articles/`. +Point nginx `root` at the repo root. The `autoindex` directive applies recursively to all +subdirectories under `/articles/` — **except** each author's own `draft/` folder, which must be +denied outright (see "Adding an article" above: this is what keeps a draft from being indexed or +fetched by a guessed direct URL). ```nginx server { listen 80; server_name example.com; - root /var/www/cmjs; + root /var/www/cmkiss; # Serve the demo page location /demo/ { try_files $uri $uri/ /demo/index.html; } + # Deny every author's draft/ folder outright — must come before the + # /articles/ block below (a regex location overrides a prefix location + # in nginx, so this still wins even though it's declared after it). + location ~ ^/articles/[^/]+/draft/ { + deny all; + return 404; + } + # Enable JSON directory listing for article discovery (applies to all subdirs) location /articles/ { autoindex on; @@ -414,15 +427,21 @@ server { ### Pointing articles/ somewhere else -By default the crawler looks for articles at `articles/` next to the rest of cmjs (`ARTICLES_URL`, +By default the crawler looks for articles at `articles/` next to the rest of cmkiss (`ARTICLES_URL`, computed as a sibling of the `js/` folder). Articles don't have to live there — override `ARTICLES_URL` in your local `js/config.js` (see "Local configuration" above) to point anywhere under the same origin (or a different one, CORS permitting). Whatever it resolves to needs its own matching `location` block enabling the JSON (or HTML) -autoindex — it doesn't have to be named `articles/` or nested under wherever cmjs itself lives: +autoindex, plus the matching `draft/` deny block (see above) — it doesn't have to be named +`articles/` or nested under wherever cmkiss itself lives: ```nginx +location ~ ^/blog-content/articles/[^/]+/draft/ { + deny all; + return 404; +} + location /blog-content/articles/ { autoindex on; autoindex_format json; @@ -433,7 +452,7 @@ location /blog-content/articles/ { ### Compression Markdown and directory-listing JSON are both plain text, so gzip shrinks them well — and since -the browser decompresses transparently, this needs no changes on the cmjs side. nginx doesn't +the browser decompresses transparently, this needs no changes on the cmkiss side. nginx doesn't know `.md` is text by default (it falls back to `application/octet-stream`, which `gzip_types` won't match), so map it explicitly and enable gzip for both content types: @@ -459,12 +478,14 @@ not built into stock nginx — only worth it if you're already building custom n Discovery happens in two phases. **Indexing** crawls `articles/` client-side to build a lightweight "stub" for every article — the author directories, then each author's own directory -listing (their article files, bio file, and shared `media/` folder together) along with each -file's `mtime`. Non-article entries (the author's own `{author}.md`, the `media/` folder) are -simply filtered out of that same listing. That's it: because an article file's *name* already -encodes its slug, date, tags, and draft status (see "Adding an article" above), that single -listing is enough to sort, filter, paginate, and populate the sidebar's tag/year counts, without -any further request per article and without ever fetching an article's actual Markdown. +listing (their article files, bio file, shared `media/` folder, and `draft/` folder together) +along with each file's `mtime`. Only `.md` files listed directly inside an author's own directory +are kept as candidate articles — the bio file is filtered out by name, and any subdirectory +(`media/`, `draft/`) is skipped outright rather than being descended into, so nothing under +`draft/` is ever indexed. That's it: because an article file's *name* already encodes its slug, +date, and tags (see "Adding an article" above), that single listing is enough to sort, filter, +paginate, and populate the sidebar's tag counts, without any further request per article and +without ever fetching an article's actual Markdown. **Hydration** is the separate, deferred step that fetches a specific article's content (title, description, cover image, reading time) — it only ever runs for articles about to be rendered: the current page's cards (`hydrateArticles()` in `js/discovery.js`), a single visited @@ -486,11 +507,14 @@ of content is ever fetched per page view. content, so a code change to how articles are derived (in `js/discovery.js`) can make old cached entries stale in *structure*, not just content — and a file's `mtime` won't catch that. Bump `CACHE_SCHEMA_VERSION` in `js/cache.js` whenever that shape changes; it's baked into - the IndexedDB database name, so a bump invalidates every cached entry on the next deploy. -- **Concurrency** — `MAX_CONCURRENT_REQUESTS` in `js/discovery.js` (default `10`) caps how - many hydration (content) fetches run at once — the only per-article requests left, since - indexing no longer needs one; raise it for a fast/local nginx, lower it for a slow or shared/ - production instance. + the IndexedDB database name, so a bump invalidates every cached entry on the next deploy — + articles and cached searches alike. A change to the search-match record shape instead, or a new + store entirely, only needs `DB_VERSION` bumped (IndexedDB's own upgrade counter): `onupgradeneeded` + creates whatever store is missing without touching the other one's existing data. +- **Concurrency** — `MAX_CONCURRENT_REQUESTS` in `js/discovery.js` (default `10`) caps both how + many hydration (content) fetches run at once and how many authors' directory listings are + fetched at once during indexing; raise it for a fast/local nginx, lower it for a slow or + shared/production instance. - **Prefetching** — once the current page has fully rendered, the next page's articles are hydrated in the background (fire-and-forget), so clicking "next" is usually an instant cache hit rather than a fresh round-trip. @@ -504,12 +528,20 @@ of content is ever fetched per page view. a large site to a handful of requests instead of hydrating the entire article set up front. The page count is exact once the whole set's been scanned, and a lower bound (at least one more page) until then. -- **Crawl health** — if the root author-listing fetch fails outright, if every individual - author's own directory listing fails, or if very few of the known candidate files parsed - successfully (a malformed file name, or a total breakage like the wrong origin or CORS - misconfigured), a small warning banner is shown above the article list instead of silently - rendering an empty page (see `getCrawlStats()` / `renderCrawlHealthWarning()` in - `js/discovery.js` / `js/views.js`). +- **Persistent search-match cache** — alongside the resumable in-session scan above, each query's + per-article match results (`{ mtime, matched }`, keyed by article path) are cached in IndexedDB + too (`searchQueries` store in `js/cache.js`, keyed by the query text itself) and reused across + page reloads and browser sessions, not just while paging through one visit. Searching the same + term again only re-fetches and re-checks articles that are new or whose `mtime` has changed + since — everything else is trusted from the cache without a content re-fetch at all. Capped at + `MAX_CACHED_SEARCHES` (default `30`) distinct queries, evicting the least-recently-searched one + first, so trying many different search terms over time doesn't grow the cache unbounded. +- **Crawl health** — if the root author-listing fetch fails outright, or if every individual + author's own directory listing fails (wrong origin, CORS misconfigured), a small warning banner + is shown above the article list instead of silently rendering an empty page (see + `getCrawlStats()` / `renderCrawlHealthWarning()` in `js/discovery.js` / `js/views.js`). + Non-conforming `.md` file names aren't treated as a health problem — only as a per-file skip — + since indexing does no per-article fetch that could actually fail. - **Design rationale** — the original crawler design (wave-based crawling with a per-article listing fetch, recency prioritisation, HTML-autoindex fallback, long-term "deploy and forget" constraints) is documented in diff --git a/css/components/article.css b/css/components/article.css index 4128d07..a805224 100644 --- a/css/components/article.css +++ b/css/components/article.css @@ -64,7 +64,7 @@ font-style: italic; } -/* Author + category row in article header — .author-link and .tag defined in card.css */ +/* Author + tag row in article header — .author-link and .tag defined in card.css */ .article-meta { display: flex; align-items: center; diff --git a/css/components/card.css b/css/components/card.css index 9b7ec78..02d5d23 100644 --- a/css/components/card.css +++ b/css/components/card.css @@ -90,7 +90,7 @@ line-height: 1.65; } -/* Author + category row at the bottom of each card */ +/* Author + tag row at the bottom of each card */ .card-meta { display: flex; align-items: center; @@ -115,7 +115,7 @@ text-decoration: underline; } -/* Shared category badge — used in cards and article headers */ +/* Tag pill — also used in cards and article headers */ .tag { display: inline-block; background: var(--color-tag-bg); @@ -134,8 +134,8 @@ color: #fff; } -/* Heading shown above a filtered list (author or category view) */ -.category-heading { +/* Heading shown above a filtered list (author, tag, or search results) */ +.list-heading { font-size: 1.1rem; font-weight: 600; color: var(--color-text-soft); diff --git a/js/cache.js b/js/cache.js index e9b293f..6d52d38 100644 --- a/js/cache.js +++ b/js/cache.js @@ -1,10 +1,11 @@ -// Persistent cache for discovered articles, keyed by each article file's own -// path. Raw IndexedDB (no wrapper library — see +// Persistent cache for discovered articles (keyed by each article file's own +// path) and for per-search-query match results (keyed by the query text — +// see setSearchCacheEntry() below). Raw IndexedDB (no wrapper library — see // .claude/nginx-sitemap-crawler-plan.md L4/L5 on the zero-dependency, // long-stable-API requirement). If IndexedDB is unavailable (private // browsing, old browser, blocked), every operation below silently becomes a -// no-op so discovery.js gets live-fetch behaviour for free without needing -// its own availability branching (plan doc R6). +// no-op so discovery.js/views.js get live-fetch/live-scan behaviour for free +// without needing their own availability branching (plan doc R6). // Bump this whenever discovery.js's derived article record shape changes. // The version is baked into the DB name so a code deploy invalidates every @@ -12,8 +13,25 @@ // each file's mtime happens to change — which, on a "deploy and forget" // site, could be years. const CACHE_SCHEMA_VERSION = 3; -const DB_NAME = `cmjs-cache-v${CACHE_SCHEMA_VERSION}`; -const STORE_NAME = 'articleFiles'; +// Renaming this prefix has the same effect as a version bump would — every +// existing visitor's browser opens a fresh, empty database under the new +// name, and the previous one is left behind as orphaned storage (harmless; +// this module degrades to live-fetch behaviour whenever the cache is empty +// or unavailable). Do this deliberately, not casually. +const DB_NAME = `cmkiss-cache-v${CACHE_SCHEMA_VERSION}`; +const ARTICLE_STORE = 'articleFiles'; +const SEARCH_STORE = 'searchQueries'; +// IndexedDB's own upgrade counter — separate from CACHE_SCHEMA_VERSION, +// which only governs the *article* record shape (and busts the cache by +// renaming the whole database). Bump this instead whenever a store or index +// is added/changed; onupgradeneeded below creates only what's missing, so +// existing article-cache data survives the upgrade. +const DB_VERSION = 2; +// Caps how many distinct queries' match results stay cached — otherwise +// every unique term ever typed into search would keep its own per-article +// map forever, growing unbounded on a long-lived site. Evicted +// least-recently-searched-first — see setSearchCacheEntry(). +const MAX_CACHED_SEARCHES = 30; let cacheAvailable = true; let dbPromise = null; @@ -22,9 +40,13 @@ function openDb() { if (!dbPromise) { dbPromise = new Promise((resolve, reject) => { if (typeof indexedDB === 'undefined') { reject(new Error('IndexedDB unavailable')); return; } - const request = indexedDB.open(DB_NAME, 1); + const request = indexedDB.open(DB_NAME, DB_VERSION); request.onupgradeneeded = () => { - request.result.createObjectStore(STORE_NAME, { keyPath: 'path' }); + const db = request.result; + if (!db.objectStoreNames.contains(ARTICLE_STORE)) db.createObjectStore(ARTICLE_STORE, { keyPath: 'path' }); + if (!db.objectStoreNames.contains(SEARCH_STORE)) { + db.createObjectStore(SEARCH_STORE, { keyPath: 'query' }).createIndex('lastUsed', 'lastUsed'); + } }; request.onsuccess = () => resolve(request.result); request.onerror = () => reject(request.error); @@ -37,7 +59,7 @@ function openDb() { // Opens a store and resolves/rejects with the given IDBRequest's own outcome // — the standard pattern, so result timing never depends on transaction // lifecycle details. -async function runRequest(mode, makeRequest) { +async function runRequest(storeName, mode, makeRequest) { if (!cacheAvailable) return undefined; let db; try { @@ -46,7 +68,7 @@ async function runRequest(mode, makeRequest) { return undefined; // openDb() itself already flipped cacheAvailable } try { - const store = db.transaction(STORE_NAME, mode).objectStore(STORE_NAME); + const store = db.transaction(storeName, mode).objectStore(storeName); return await new Promise((resolve, reject) => { const request = makeRequest(store); request.onsuccess = () => resolve(request.result); @@ -67,12 +89,12 @@ async function runRequest(mode, makeRequest) { // reads this once up front rather than doing a point lookup per file during // indexing. Empty Map on a miss or if the cache is unavailable. export async function getAllCacheEntries() { - const records = await runRequest('readonly', store => store.getAll()); + const records = await runRequest(ARTICLE_STORE, 'readonly', store => store.getAll()); return new Map((records || []).map(record => [record.path, record])); } export async function setCacheEntry(path, entry) { - await runRequest('readwrite', store => store.put({ ...entry, path })); + await runRequest(ARTICLE_STORE, 'readwrite', store => store.put({ ...entry, path })); } // Removes any cached entry whose path is no longer in the live directory @@ -87,7 +109,7 @@ export async function pruneCache(validPaths) { return; // openDb() itself already flipped cacheAvailable } try { - const store = db.transaction(STORE_NAME, 'readwrite').objectStore(STORE_NAME); + const store = db.transaction(ARTICLE_STORE, 'readwrite').objectStore(ARTICLE_STORE); await new Promise((resolve, reject) => { const cursorRequest = store.openCursor(); cursorRequest.onsuccess = event => { @@ -103,3 +125,54 @@ export async function pruneCache(validPaths) { // mean IndexedDB itself is unavailable. } } + +// Per-query search-match cache: `entries` is a { [articlePath]: { mtime, +// matched } } map — see loadSearch() in views.js, which trusts an entry +// (skipping a content re-fetch entirely) whenever its mtime still matches +// the live article, and only re-checks what's new or changed. Keyed by the +// exact (locale-lowercased) text matchesQuery() itself compares against, so +// "Linux" and "linux" share one cached entry. +export async function getSearchCacheEntry(query) { + return runRequest(SEARCH_STORE, 'readonly', store => store.get(query)); +} + +export async function setSearchCacheEntry(query, entries) { + await runRequest(SEARCH_STORE, 'readwrite', store => store.put({ query, entries, lastUsed: Date.now() })); + await pruneSearchCache(); +} + +// Keeps the search-query cache bounded to MAX_CACHED_SEARCHES entries, +// dropping the least-recently-searched ones first — the same "don't grow +// unbounded" concern pruneCache() addresses for the article cache, just +// against searches instead of renamed/deleted files. +async function pruneSearchCache() { + if (!cacheAvailable) return; + let db; + try { + db = await openDb(); + } catch { + return; + } + try { + const store = db.transaction(SEARCH_STORE, 'readwrite').objectStore(SEARCH_STORE); + let excess = await new Promise((resolve, reject) => { + const request = store.count(); + request.onsuccess = () => resolve(request.result - MAX_CACHED_SEARCHES); + request.onerror = () => reject(request.error); + }); + if (excess <= 0) return; + await new Promise((resolve, reject) => { + const cursorRequest = store.index('lastUsed').openCursor(); + cursorRequest.onsuccess = event => { + const cursor = event.target.result; + if (!cursor || excess <= 0) { resolve(); return; } + cursor.delete(); + excess -= 1; + cursor.continue(); + }; + cursorRequest.onerror = () => reject(cursorRequest.error); + }); + } catch { + // Same reasoning as runRequest() above. + } +} diff --git a/js/config-default.js b/js/config-default.js index 02fda04..5e2b437 100644 --- a/js/config-default.js +++ b/js/config-default.js @@ -35,7 +35,7 @@ export const MARKED_URL = overrides.MARKED_URL ?? DEFAULT_MARKED_URL; // Base URL of the articles/ folder. Defaults to a sibling of this file's own // js/ directory (the repo root's own articles/), but articles don't have to -// live anywhere near the rest of cmjs — override it in config.js to point +// live anywhere near the rest of cmkiss — override it in config.js to point // anywhere under the same origin, or even a different one, CORS permitting. // Whatever it resolves to must still be a directory nginx serves a JSON (or // HTML) autoindex listing for — see "nginx configuration" in the README. diff --git a/js/discovery.js b/js/discovery.js index 4632ef6..8e1466a 100644 --- a/js/discovery.js +++ b/js/discovery.js @@ -4,13 +4,16 @@ import { mapWithConcurrency } from './concurrency.js'; import { getAllCacheEntries, setCacheEntry, pruneCache } from './cache.js'; import { t } from './i18n.js'; -// Caps simultaneous markdown content fetches during hydration (see -// hydrateArticle()/hydrateArticles()) — the only per-article requests left, -// since indexing no longer needs a per-article listing fetch at all (every -// stub field is encoded in its file's own name — see buildStub()). Raise it -// for a fast/local nginx, lower it for a slow or shared/production instance -// — see .claude/nginx-sitemap-crawler-plan.md §2a for the original rationale -// (written when this also governed indexing requests). +// Caps simultaneous requests in the two places this module still fires one +// per item: markdown content fetches during hydration (see +// hydrateArticle()/hydrateArticles()) and each author's own directory +// listing during wave 2 of indexing (see runCrawl()) — indexing no longer +// needs a per-*article* fetch at all (every stub field is encoded in its +// file's own name — see buildStub()), but still fetches one listing per +// author. Raise it for a fast/local nginx, lower it for a slow or shared/ +// production instance — see .claude/nginx-sitemap-crawler-plan.md §2a for +// the original rationale (written when this also governed per-article +// indexing requests). const MAX_CONCURRENT_REQUESTS = 10; const MONTH_ABBREVIATIONS = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec']; @@ -98,10 +101,8 @@ let crawlStats = { total: 0, loaded: 0, authorsTotal: 0, authorsFailed: 0, rootF let partialResolved = []; const progressListeners = new Set(); -function sortAndFilterPublished(rawArticles) { - return rawArticles - .filter(article => article?.published && article.date) - .sort((a, b) => String(b.date).localeCompare(String(a.date)) || a.hash.localeCompare(b.hash)); +function sortArticles(rawArticles) { + return [...rawArticles].sort((a, b) => String(b.date).localeCompare(String(a.date)) || a.hash.localeCompare(b.hash)); } // Subscribes to incremental crawl progress so a large site's article list @@ -114,12 +115,12 @@ function sortAndFilterPublished(rawArticles) { // no longer showing. export function onArticlesProgress(listener) { progressListeners.add(listener); - if (partialResolved.length) listener(sortAndFilterPublished(partialResolved), false); + if (partialResolved.length) listener(sortArticles(partialResolved), false); return () => progressListeners.delete(listener); } function notifyProgress(done) { - const snapshot = sortAndFilterPublished(partialResolved); + const snapshot = sortArticles(partialResolved); progressListeners.forEach(listener => listener(snapshot, done)); } @@ -144,21 +145,23 @@ function excerpt(content) { } // An article file's own name is the sole authoritative source of its slug, -// publish date, tags/category, and draft status — see the README's "Adding -// an article" section. Format: `<slug>~<DDMMYYYY>[~tag1+tag2+…].md`, -// optionally prefixed with `_` for draft. `~` (rather than e.g. "--") is -// deliberate: a slug is expected to be plain kebab-case, and "~" isn't a +// publish date, and tags — see the README's "Adding an article" section. +// Format: `<slug>~<DDMMYYYY>[~tag1+tag2+…].md`. `~` (rather than e.g. "--") +// is deliberate: a slug is expected to be plain kebab-case, and "~" isn't a // character anyone would type by accident while writing one — unlike a // double-hyphen separator, which collides with an easy slug typo (an extra // "-") and silently drops the whole article instead of erroring. Returns // null for any file name that doesn't match — callers treat that as a // silent skip, e.g. the author's own bio file or a stray unrelated file. +// Drafts aren't a naming convention at all — they live in an author's own +// `draft/` subdirectory (see the README's "Adding an article" section), +// which the crawler never descends into (wave 2 below only reads `type === +// 'file'` entries from an author's own listing) and which nginx should deny +// outright, so a draft is never indexed or fetchable in the first place. function parseArticleFileName(fileName) { if (!fileName.endsWith('.md')) return null; const base = fileName.slice(0, -3); - const draft = base.startsWith('_'); - const core = draft ? base.slice(1) : base; - const [slug, dateSegment, tagsSegment] = core.split('~'); + const [slug, dateSegment, tagsSegment] = base.split('~'); if (!slug || !dateSegment || !/^\d{8}$/.test(dateSegment)) return null; const day = dateSegment.slice(0, 2); const month = dateSegment.slice(2, 4); @@ -167,7 +170,6 @@ function parseArticleFileName(fileName) { slug, date: `${year}-${month}-${day}`, tags: tagsSegment ? tagsSegment.split('+').filter(Boolean) : [], - draft, }; } @@ -185,11 +187,8 @@ function parseArticleFileName(fileName) { function buildStub(author, fileName, mtime, authorRoot) { const parsed = parseArticleFileName(fileName); if (!parsed) return null; - const { slug, date, tags, draft } = parsed; + const { slug, date, tags } = parsed; const path = `${authorRoot}${encodeURIComponent(fileName)}`; - // "Category" isn't a separate concept anymore — it's just whichever tag - // id (if any) looks like a `year-YYYY` id. - const category = tags.find(id => /^year-20\d{2}$/i.test(id)) || null; return { author, @@ -197,11 +196,8 @@ function buildStub(author, fileName, mtime, authorRoot) { hash: `${author}/${slug}`, path, mtime, - category, - categoryYear: category ? Number(category.replace(/^year-/i, '')) : null, date, tags, - published: !draft, hydrated: false, }; } @@ -224,11 +220,18 @@ function normalizeMtime(mtime) { // string. `cacheSnapshot` is the whole cache read once up front (see // runCrawl) — an in-memory lookup here instead of a separate IndexedDB read // per file, which mattered when this also had to guard a listing fetch; -// kept anyway since it's still the same one cheap lookup. +// kept anyway since it's still the same one cheap lookup. `mtime` is a +// change-detection signal, not a required key: `cached.mtime === mtime` +// already matches when both are `null` (a server whose listing never +// yields a parseable mtime — see parseHtmlDirectoryListing), which is the +// right call — there's no evidence the file changed, as opposed to a +// mismatch, which is evidence that it did. Treating "unknown" as "assume +// changed" would defeat the cache on every single crawl instead of just +// when a file actually needs re-hydrating. function resolveStub(author, fileName, mtime, authorRoot, cacheSnapshot) { const path = `${authorRoot}${encodeURIComponent(fileName)}`; const cached = cacheSnapshot.get(path); - if (cached && mtime != null && cached.mtime === mtime && cached.article?.hydrated) return cached.article; + if (cached && cached.mtime === mtime && cached.article?.hydrated) return cached.article; return buildStub(author, fileName, mtime, authorRoot); } @@ -291,7 +294,14 @@ async function hydrateArticleNow(stub) { const authoritative = { ...stub }; Object.assign(stub, meta, authoritative, deriveContentFields(meta, content, stub.path), { hydrated: true }); await setCacheEntry(stub.path, { mtime: stub.mtime, article: stub }); - } catch { /* leave stub unhydrated — retried next time it's rendered */ } + } catch (error) { + // Leave stub unhydrated — retried next time it's rendered. Logged + // (not surfaced in the UI, which just shows the title-less card) + // so a persistent failure — a real bug in parseFrontmatter() or + // deriveContentFields(), not just a transient fetch blip — is still + // visible somewhere instead of silently retrying forever. + console.error(`Failed to hydrate article ${stub.path}:`, error); + } return stub; } @@ -327,28 +337,32 @@ async function runCrawl() { } catch { crawlStats = { total: 0, loaded: 0, authorsTotal: 0, authorsFailed: 0, rootFailed: true }; notifyProgress(true); - return sortAndFilterPublished(partialResolved); + return sortArticles(partialResolved); } const authors = authorEntries.filter(entry => entry.type === 'directory' && !entry.name.startsWith('.')); // Wave 2 — always fetched live, in full, for every author (not just the // current page): each author's own directory listing, with per-file - // mtimes. This is what builds the flat, cross-author list that drives - // global sorting and pagination — see the plan doc's "keep the flat - // feed" decision. It's also the *only* per-author network request - // indexing ever needs: every stub field (slug, date, tags, category, - // draft status) is parsed straight out of each article file's own name - // (see buildStub/parseArticleFileName) — there's no per-article fetch - // at index time. The author's own bio file (`<author>.md`) and shared - // `media/` folder live in the same listing and are simply filtered out - // here — only `.md` files other than the bio file are candidate - // articles. One author's listing failing (network blip, deleted - // mid-crawl) is silently skipped rather than failing every other + // mtimes. Capped by MAX_CONCURRENT_REQUESTS like hydrateArticles() below + // — a site with many authors would otherwise fire one listing request + // per author simultaneously, with no limit. This is what builds the + // flat, cross-author list that drives global sorting and pagination — + // see the plan doc's "keep the flat feed" decision. It's also the *only* + // per-author network request indexing ever needs: every stub field + // (slug, date, tags) is parsed straight out of each article file's own + // name (see buildStub/parseArticleFileName) — there's no per-article + // fetch at index time. The `type === 'file'` filter below also means any + // subdirectory (the author's shared `media/` folder, a `draft/` folder — + // see "Adding an article" in the README) is skipped without ever being + // descended into. The author's own bio file (`<author>.md`) is filtered + // out by name in the same listing — only `.md` files other than the bio + // file are candidate articles. One author's listing failing (network + // blip, deleted mid-crawl) is silently skipped rather than failing every other // author's list too, but is still counted, so a *total* breakage (wrong // origin, CORS) remains visible via getCrawlStats() instead of just // silently producing an empty article list. let authorsFailed = 0; - const authorFileLists = await Promise.all(authors.map(async author => { + const authorFileLists = await mapWithConcurrency(authors, MAX_CONCURRENT_REQUESTS, async author => { const authorRoot = `${ARTICLES_URL}${encodeURIComponent(author.name)}/`; const bioFileName = `${author.name}.md`; try { @@ -360,7 +374,7 @@ async function runCrawl() { authorsFailed += 1; return []; } - })); + }); const allFiles = authorFileLists.flat(); // Every stub is parsed synchronously from data already in hand — no @@ -379,7 +393,7 @@ async function runCrawl() { await pruneCache(allFiles.map(file => `${file.authorRoot}${encodeURIComponent(file.fileName)}`)); crawlStats = { total: allFiles.length, loaded, authorsTotal: authors.length, authorsFailed, rootFailed: false }; - const finalArticles = sortAndFilterPublished(partialResolved); + const finalArticles = sortArticles(partialResolved); notifyProgress(true); return finalArticles; } diff --git a/js/frontmatter.js b/js/frontmatter.js index 8a7c643..de50486 100644 --- a/js/frontmatter.js +++ b/js/frontmatter.js @@ -39,6 +39,7 @@ export function parseFrontmatter(raw) { const meta = {}; let arrayKey = null; for (const line of match[1].split(/\r?\n/)) { + if (!line.trim()) continue; // blank line — doesn't close a block array const item = line.match(/^\s*-\s+(.*)$/); if (item && arrayKey) { meta[arrayKey].push(parseScalar(item[1])); continue; } const field = line.match(/^([A-Za-z][A-Za-z0-9_-]*):(?:\s*(.*))?$/); diff --git a/js/router.js b/js/router.js index f5895cb..e9fa7e1 100644 --- a/js/router.js +++ b/js/router.js @@ -6,16 +6,15 @@ // first = "search", ends "/page/N" → search results, page N (#search/query/page/2) // 2 segments, first = "page" → all articles, page N (#page/2) // 2 segments, first = "author" → author filter, page 1 (#author/alice) -// 2 segments, first = "category" or "tag" → filtered list, page 1 +// 2 segments, first = "tag" → tag filter, page 1 // 3 segments, first = "post" → single article (#post/alice/my-article) -// 4 segments, first = "author", third = "page" → author filter, page N (#author/alice/page/2) -// 4 segments, first = "category", third = "page" → category filter, page N (#category/linux/page/2) -// 4 segments, first = "tag", third = "page" → tag filter, page N (#tag/guides/page/2) +// 4 segments, first = "author", third = "page" → author filter, page N (#author/alice/page/2) +// 4 segments, first = "tag", third = "page" → tag filter, page N (#tag/guides/page/2) import { MARKED_URL } from './config-default.js'; import { loadTemplates } from './templates.js'; import { loadStrings, t } from './i18n.js'; -import { loadArticleList, loadAuthorView, loadCategoryView, loadTagView, loadArticle, loadSearch } from './views.js'; +import { loadArticleList, loadAuthorView, loadTagView, loadArticle, loadSearch } from './views.js'; // Dynamically inject marked.js as a plain global script before the app starts. function loadScript(url) { @@ -50,13 +49,11 @@ function router() { } else if (parts.length === 0) loadArticleList(); else if (parts.length === 2 && parts[0] === 'page') loadArticleList(null, parsePage(parts[1])); - else if (parts.length === 2 && parts[0] === 'author') loadAuthorView(decodeURIComponent(parts[1])); - else if (parts.length === 2 && parts[0] === 'category') loadCategoryView(decodeURIComponent(parts[1])); - else if (parts.length === 2 && parts[0] === 'tag') loadTagView(decodeURIComponent(parts[1])); - else if (parts.length === 3 && parts[0] === 'post') loadArticle(parts.slice(1).map(decodeURIComponent).join('/')); - else if (parts.length === 4 && parts[0] === 'author' && parts[2] === 'page') loadAuthorView(decodeURIComponent(parts[1]), parsePage(parts[3])); - else if (parts.length === 4 && parts[0] === 'category' && parts[2] === 'page') loadCategoryView(decodeURIComponent(parts[1]), parsePage(parts[3])); - else if (parts.length === 4 && parts[0] === 'tag' && parts[2] === 'page') loadTagView(decodeURIComponent(parts[1]), parsePage(parts[3])); + else if (parts.length === 2 && parts[0] === 'author') loadAuthorView(decodeURIComponent(parts[1])); + else if (parts.length === 2 && parts[0] === 'tag') loadTagView(decodeURIComponent(parts[1])); + else if (parts.length === 3 && parts[0] === 'post') loadArticle(parts.slice(1).map(decodeURIComponent).join('/')); + else if (parts.length === 4 && parts[0] === 'author' && parts[2] === 'page') loadAuthorView(decodeURIComponent(parts[1]), parsePage(parts[3])); + else if (parts.length === 4 && parts[0] === 'tag' && parts[2] === 'page') loadTagView(decodeURIComponent(parts[1]), parsePage(parts[3])); else loadArticleList(); } @@ -77,16 +74,23 @@ function init() { // Fail with a clear, catchable message here rather than a cryptic // uncaught TypeError the first time showListView()/showArticleView() // run — both mount points are a hard requirement, unlike - // search/tag/year, which are optional. + // search/tag, which are optional. if (!document.getElementById('list-view') || !document.getElementById('article-view')) { throw new Error('Missing required #list-view and/or #article-view element — see the "Embedding in an existing page" section of the README.'); } + // Optional embedding-page markup (see "Embedding in an existing + // page" in the README) — set once here rather than in + // showListView()/loadArticle() so a page that loads straight into + // an article route (a direct link) still gets it, not just one that + // starts on the list view. + const siteTitle = document.getElementById('site-title'); + if (siteTitle) siteTitle.textContent = t('siteName'); }); // Show a top-level error if dependencies fail to load (or a mount point - // is missing). Scoped to cmjs's own mount points when at least one + // is missing). Scoped to cmkiss's own mount points when at least one // exists, rather than replacing the whole page — this can run on a page - // that embeds cmjs alongside plenty of its own content, which wiping + // that embeds cmkiss alongside plenty of its own content, which wiping // document.body would destroy too. Falls back to the full-page replacement // only when neither mount point is present to show the message in. ready.catch(err => { diff --git a/js/views.js b/js/views.js index 003fbae..2bedb1d 100644 --- a/js/views.js +++ b/js/views.js @@ -1,5 +1,6 @@ import { templates } from './templates.js'; import { deriveContentFields, discoverArticles, fetchAuthorProfile, findArticle, getCrawlStats, hydrateArticles, onArticlesProgress, resolveMediaUrl } from './discovery.js'; +import { getSearchCacheEntry, setSearchCacheEntry } from './cache.js'; import { parseFrontmatter } from './frontmatter.js'; import { t, getLocale } from './i18n.js'; @@ -48,7 +49,6 @@ function filterLabel(filter) { if (!filter) return ''; if (filter.type === 'author') return t('filterAuthor', { value: filter.value }); if (filter.type === 'tag') return t('filterTag', { value: filter.value }); - if (filter.type === 'category') return t('filterYear', { value: filter.value.replace(/^year-/i, '') }); return filter.value; } @@ -103,54 +103,22 @@ function renderFilterLink(label, href, count, active = false) { return link; } -// Years and tags render independently — either `<nav>` can be absent from -// the page (e.g. a site that only wants tag filtering, like this project's -// own index.html) without the other silently stopping too. Whichever -// section renders first gets the "all articles" reset link, so it's never -// lost just because the years nav isn't on the page. function renderFilterNavigation(articles, activeFilter = null) { - const years = document.getElementById('year-filters'); const tags = document.getElementById('tag-filters'); - if (!years && !tags) return; - const allArticlesLink = () => renderFilterLink(t('filterAllArticles'), '#', articles.length, !activeFilter); - - if (years) { - years.innerHTML = ''; - const yearCounts = new Map(); - articles.forEach(article => { - const year = Number(article.categoryYear); - if (year) yearCounts.set(year, (yearCounts.get(year) || 0) + 1); + if (!tags) return; + tags.innerHTML = ''; + const tagCounts = new Map(); + articles.forEach(article => (article.tags || []).forEach(tag => tagCounts.set(tag, (tagCounts.get(tag) || 0) + 1))); + const tagList = document.createElement('div'); + tagList.className = 'filter-list'; + tagList.appendChild(renderFilterLink(t('filterAllArticles'), '#', articles.length, !activeFilter)); + [...tagCounts.keys()] + .sort((a, b) => a.localeCompare(b, getLocale())) + .forEach(id => { + const active = activeFilter?.type === 'tag' && activeFilter.value === id; + tagList.appendChild(renderFilterLink(id, `#tag/${encodeURIComponent(id)}`, tagCounts.get(id), active)); }); - const yearList = document.createElement('div'); - yearList.className = 'filter-list'; - yearList.appendChild(allArticlesLink()); - for (let year = 2026; year >= 2009; year -= 1) { - const count = yearCounts.get(year) || 0; - const active = activeFilter?.type === 'category' && activeFilter.value.toLowerCase() === `year-${year}`; - yearList.appendChild(renderFilterLink(t('yearLabel', { year }), `#category/year-${year}`, count, active)); - } - years.appendChild(yearList); - } - - if (tags) { - tags.innerHTML = ''; - const tagCounts = new Map(); - articles.forEach(article => (article.tags || []).forEach(tag => tagCounts.set(tag, (tagCounts.get(tag) || 0) + 1))); - const tagList = document.createElement('div'); - tagList.className = 'filter-list'; - if (!years) tagList.appendChild(allArticlesLink()); - // A `year-YYYY` id is also a normal entry in `article.tags` (that's how - // it's recognised as the category — see buildStub() in discovery.js), - // but it belongs in the years nav above, not duplicated here. - [...tagCounts.keys()] - .filter(id => !/^year-20\d{2}$/i.test(id)) - .sort((a, b) => a.localeCompare(b, getLocale())) - .forEach(id => { - const active = activeFilter?.type === 'tag' && activeFilter.value === id; - tagList.appendChild(renderFilterLink(id, `#tag/${encodeURIComponent(id)}`, tagCounts.get(id), active)); - }); - tags.appendChild(tagList); - } + tags.appendChild(tagList); } // Builds the #post/author/slug hash for an article, encoding each segment @@ -164,17 +132,11 @@ function postHref(article) { return `#post/${encodeURIComponent(article.author)}/${encodeURIComponent(article.slug)}`; } -// Shared between renderCard() and loadArticle() — date, reading-time, and -// the category/year badge are populated identically in both places. -function applyArticleMetaFields(field, { date, readingTime, category, categoryYear }) { +// Shared between renderCard() and loadArticle() — date and reading-time are +// populated identically in both places. +function applyArticleMetaFields(field, { date, readingTime }) { setField(field('date'), date, (time, value) => { time.dateTime = String(value); time.textContent = formatDate(value); }); setField(field('reading-time'), readingTime, (time, value) => { time.textContent = t('readingTime', { minutes: value }); }); - // A post only counts as a "category" when one of its directory-name - // tags happens to name a year — not every article has one. - setField(field('category'), category, (categoryEl, value) => { - categoryEl.textContent = t('yearLabel', { year: categoryYear }); - categoryEl.href = `#category/${encodeURIComponent(value)}`; - }); } function renderCard(article) { @@ -211,27 +173,29 @@ function renderCard(article) { // autoindex disabled) should be visible rather than silently rendering an // empty/near-empty list forever — see .claude/nginx-sitemap-crawler-plan.md L3. // Three independent signals feed this: the root author-listing fetch itself -// failing outright (rootFailed), every individual author's own listing +// failing outright (rootFailed), or every individual author's own listing // failing (checked regardless of site size, since discovery.js now silently // skips individual author failures — see the wave-2 comment in -// discovery.js), and, for larger sites, a near-zero loaded/total ratio at -// the article level. Only meaningful once the crawl has actually finished -// (`done`) — mid-crawl stats are incomplete and would misreport a healthy -// crawl as broken. +// discovery.js). Unlike an older version of this check, a low loaded/total +// ratio (candidate .md files that didn't match the article naming +// convention) is deliberately *not* treated as a health problem: indexing +// does no per-article fetch at all, so a non-conforming file name is a +// filename issue, not a network/CORS one — and legitimately non-conforming +// .md files (notes, leftovers) coexisting with real articles shouldn't +// trigger a false "something's broken" banner. Only meaningful once the +// crawl has actually finished (`done`) — mid-crawl stats are incomplete and +// would misreport a healthy crawl as broken. function renderCrawlHealthWarning(view, done) { if (!done) return; - const { total, loaded, authorsTotal, authorsFailed, rootFailed } = getCrawlStats(); + const { authorsTotal, authorsFailed, rootFailed } = getCrawlStats(); // rootFailed means the very first listing fetch (the author directories // themselves) couldn't be read at all — a total breakage same in spirit // as every author's own listing failing, just one level higher up. const authorsBroken = rootFailed || (authorsTotal > 0 && authorsFailed === authorsTotal); - const articlesBroken = total >= 5 && loaded / total < 0.1; - if (!authorsBroken && !articlesBroken) return; + if (!authorsBroken) return; const warning = document.createElement('p'); warning.className = 'error'; - warning.textContent = authorsBroken - ? t('crawlHealthAuthorsFailed') - : t('crawlHealthArticlesFailed', { loaded, total }); + warning.textContent = t('crawlHealthAuthorsFailed'); view.appendChild(warning); } @@ -250,7 +214,7 @@ function renderArticleListBody(view, { total, pageArticles, page, pages, filter, renderCrawlHealthWarning(view, done); if (heading) { const headingElement = document.createElement('h2'); - headingElement.className = 'category-heading'; + headingElement.className = 'list-heading'; headingElement.textContent = heading; view.appendChild(headingElement); } @@ -283,19 +247,16 @@ export async function loadArticleList(filter = null, page = 1) { const applyFilter = allArticles => { renderFilterNavigation(allArticles, filter); if (!filter) return allArticles; - return allArticles.filter(article => { - if (filter.type === 'tag') return article.tags.includes(filter.value); - // Author names are exact, case-sensitive directory names (used - // as-is for paths and profile lookups elsewhere) — comparing - // them case-insensitively here could merge two distinctly-named - // author directories that differ only in case. - if (filter.type === 'author') return article.author === filter.value; - return String(article[filter.type] || '').toLowerCase() === filter.value.toLowerCase(); - }); + if (filter.type === 'tag') return allArticles.filter(article => article.tags.includes(filter.value)); + // Author names are exact, case-sensitive directory names (used + // as-is for paths and profile lookups elsewhere) — comparing them + // case-insensitively here could merge two distinctly-named author + // directories that differ only in case. + return allArticles.filter(article => article.author === filter.value); }; - // Sorting/filtering/pagination all work on stubs alone (tags, date, - // category — none of that needs a content fetch). Only the current + // Sorting/filtering/pagination all work on stubs alone (tags, date — + // none of that needs a content fetch). Only the current // page's cards need real content, so only that slice gets hydrated — // this is what keeps a large site's list fast: the rest of the tree's // markdown is never fetched until the user actually pages to it. @@ -324,12 +285,19 @@ export async function loadArticleList(filter = null, page = 1) { } }; - const unsubscribe = onArticlesProgress((partialArticles, done) => { render(partialArticles, done); }); + // Same error handling as the final render below (not just the same + // message — an error thrown here, e.g. a malformed template, would + // otherwise be an unhandled rejection that leaves the list stuck on its + // loading state instead of ever showing articlesLoadError). + const showRenderError = error => { + if (generation === currentGeneration) view.innerHTML = `<p class="error">${t('articlesLoadError', { message: error.message })}</p>`; + }; + const unsubscribe = onArticlesProgress((partialArticles, done) => { render(partialArticles, done).catch(showRenderError); }); try { const allArticles = await discoverArticles(); await render(allArticles, true); } catch (error) { - if (generation === currentGeneration) view.innerHTML = `<p class="error">${t('articlesLoadError', { message: error.message })}</p>`; + showRenderError(error); } finally { unsubscribe(); } @@ -393,7 +361,6 @@ export async function loadAuthorView(author, page = 1) { } } -export const loadCategoryView = (category, page = 1) => loadArticleList({ type: 'category', value: category }, page); export const loadTagView = (tag, page = 1) => loadArticleList({ type: 'tag', value: tag }, page); // How many stubs to hydrate-and-check per scanning round (see loadSearch @@ -408,6 +375,16 @@ const SEARCH_SCAN_BATCH = PAGE_SIZE; // same page) skip scanning entirely, since `matches` only ever grows and // stub order never changes. Reset whenever the query text changes; holds // only the most recent query, so a new search simply replaces it. +// +// `entries` is this same session's in-memory mirror of the persistent +// per-query cache (see cache.js's getSearchCacheEntry()/ +// setSearchCacheEntry()): a { [articlePath]: { mtime, matched } } map, +// loaded once when the query first changes and written back whenever a scan +// round adds something new to it (`dirty`). Reusing it — instead of only +// `cursor`/`matches` — means even a *first* search after a page reload can +// skip re-fetching and re-checking every article whose content hasn't +// changed since it was last searched for, no matter how this query's own +// scan happened to leave off last time. let searchState = null; export async function loadSearch(query, page = 1) { @@ -425,7 +402,7 @@ export async function loadSearch(query, page = 1) { const filter = { type: 'search', value: query }; try { const needle = query.toLocaleLowerCase(getLocale()); - const matchesQuery = article => [article.title, article.description, article.author, article.authorName, article.category, ...article.tags] + const matchesQuery = article => [article.title, article.description, article.author, article.authorName, ...article.tags] .some(value => String(value || '').toLocaleLowerCase(getLocale()).includes(needle)); const stubs = await discoverArticles(); @@ -433,7 +410,16 @@ export async function loadSearch(query, page = 1) { renderFilterNavigation(stubs); if (!searchState || searchState.query !== query || searchState.stubs !== stubs) { - searchState = { query, stubs, cursor: 0, matches: [] }; + // Loaded once per new query. Pruned to the current stub set's + // paths so a renamed/deleted article's old cached result can't + // linger — cheap, since `stubs` (unlike hydrating them) is + // already in hand with no further fetch. + const cached = await getSearchCacheEntry(needle); + if (generation !== currentGeneration) return; + const validPaths = new Set(stubs.map(article => article.path)); + const entries = {}; + for (const [path, entry] of Object.entries(cached?.entries || {})) if (validPaths.has(path)) entries[path] = entry; + searchState = { query, stubs, cursor: 0, matches: [], entries, dirty: false }; } const state = searchState; @@ -485,12 +471,38 @@ export async function loadSearch(query, page = 1) { while (state.cursor < state.stubs.length && state.matches.length < target) { if (generation !== currentGeneration) return; const batch = state.stubs.slice(state.cursor, state.cursor + SEARCH_SCAN_BATCH); + // Still hydrates the whole batch — a no-op per already-hydrated + // stub (see hydrateArticle() in discovery.js), which a cache hit + // below always is: it can only exist for a path this exact + // query already matched content against, at this same mtime, + // which means it's already sitting in the article-level cache + // too. Rendering pageArticles further down never re-hydrates, + // so this stays unconditional rather than trusting that chain + // of reasoning to never drift out of sync. await hydrateArticles(batch); if (generation !== currentGeneration) return; state.cursor += batch.length; - batch.forEach(article => { if (matchesQuery(article)) state.matches.push(article); }); + batch.forEach(article => { + // A cached result only counts as known if its mtime still + // matches the live stub — an edited-since-last-search + // article is re-checked, same as one with no entry at all. + const known = state.entries[article.path]; + const isKnown = known?.mtime === article.mtime; + const matched = isKnown ? known.matched : matchesQuery(article); + if (!isKnown) { state.entries[article.path] = { mtime: article.mtime, matched }; state.dirty = true; } + if (matched) state.matches.push(article); + }); renderResults(); } + // Persist whatever's new this round — fire-and-forget, since a + // failed/slow write should never hold up rendering (cache.js + // already swallows its own storage errors). Next time this exact + // query is searched, anything covered here needs no re-fetch at + // all, only what's new or changed since. + if (state.dirty) { + state.dirty = false; + setSearchCacheEntry(needle, state.entries); + } } catch (error) { if (generation === currentGeneration) view.innerHTML = `<p class="error">${t('searchError', { message: error.message })}</p>`; } @@ -546,7 +558,7 @@ export async function loadArticle(hash) { field('title').textContent = meta.title || article.slug; // Publish date comes from the article file's own name (already // resolved onto `article.date`), not the raw frontmatter. - applyArticleMetaFields(field, { date: article.date, readingTime, category: article.category, categoryYear: article.categoryYear }); + applyArticleMetaFields(field, { date: article.date, readingTime }); setField(field('description'), meta.description, (description, value) => { description.textContent = value; }); const image = field('image'); if (coverUrl) { image.src = coverUrl; image.alt = meta.title || article.slug; } else image.closest('figure')?.remove(); diff --git a/strings/cs.json b/strings/cs.json index 6cda7cb..d82ed90 100644 --- a/strings/cs.json +++ b/strings/cs.json @@ -5,9 +5,7 @@ "filterAuthor": "Autor: {value}", "filterTag": "Štítek: {value}", - "filterYear": "Ročník: {value}", "filterAllArticles": "Všechny články", - "yearLabel": "Ročník {year}", "paginationLabel": "Stránkování", "paginationPrevious": "Předchozí", @@ -18,7 +16,6 @@ "readingTime": "{minutes} min čtení", "crawlHealthAuthorsFailed": "Pozor: nepodařilo se načíst žádného autora — zkontrolujte nastavení serveru.", - "crawlHealthArticlesFailed": "Pozor: načtena jen {loaded} z {total} položek — zkontrolujte nastavení serveru.", "loadingArticles": "Načítám články…", "loadingGeneric": "Načítám…", diff --git a/strings/en.json b/strings/en.json index 9b14a42..02f9964 100644 --- a/strings/en.json +++ b/strings/en.json @@ -5,9 +5,7 @@ "filterAuthor": "Author: {value}", "filterTag": "Tag: {value}", - "filterYear": "Year: {value}", "filterAllArticles": "All articles", - "yearLabel": "Year {year}", "paginationLabel": "Pagination", "paginationPrevious": "Previous", @@ -18,7 +16,6 @@ "readingTime": "{minutes} min read", "crawlHealthAuthorsFailed": "Warning: couldn't load any authors — check your server configuration.", - "crawlHealthArticlesFailed": "Warning: only loaded {loaded} of {total} items — check your server configuration.", "loadingArticles": "Loading articles…", "loadingGeneric": "Loading…", diff --git a/strings/sk.json b/strings/sk.json index 25ccfaa..ff81b7b 100644 --- a/strings/sk.json +++ b/strings/sk.json @@ -5,9 +5,7 @@ "filterAuthor": "Autor: {value}", "filterTag": "Štítok: {value}", - "filterYear": "Ročník: {value}", "filterAllArticles": "Všetky články", - "yearLabel": "Ročník {year}", "paginationLabel": "Stránkovanie", "paginationPrevious": "Predchádzajúca", @@ -18,7 +16,6 @@ "readingTime": "{minutes} min čítania", "crawlHealthAuthorsFailed": "Pozor: nepodarilo sa načítať žiadneho autora — skontrolujte nastavenie servera.", - "crawlHealthArticlesFailed": "Pozor: načítaných len {loaded} z {total} položiek — skontrolujte nastavenie servera.", "loadingArticles": "Načítavam články…", "loadingGeneric": "Načítavam…", diff --git a/templates/article.html b/templates/article.html index 8d3f308..cdcd6d8 100644 --- a/templates/article.html +++ b/templates/article.html @@ -8,7 +8,6 @@ <p class="description" data-field="description"></p> <div class="article-meta"> <a class="author-link" data-field="author" href=""></a> - <a class="tag" data-field="category" href=""></a> <span class="tag-list" data-field="tags"></span> </div> </header> diff --git a/templates/card.html b/templates/card.html index 57b88f6..d25af6d 100644 --- a/templates/card.html +++ b/templates/card.html @@ -11,7 +11,6 @@ <p data-field="description"></p> <div class="card-meta"> <a class="author-link" data-field="author" href=""></a> - <a class="tag" data-field="category" href=""></a> <span class="tag-list" data-field="tags"></span> </div> </div>