558 lines
31 KiB
Markdown
558 lines
31 KiB
Markdown
# cmkiss — Minimal Client-Side CMS
|
|
|
|
A minimal CMS where articles are Markdown files rendered in the browser. No build step, no database, no server-side code.
|
|
|
|
## How it works
|
|
|
|
1. The browser traverses `articles/` via a server directory index — discovering authors and
|
|
their article files; nginx JSON autoindex is preferred, with HTML autoindexes supported as a
|
|
fallback.
|
|
2. Sorting, filtering, and publish status all come from the article file's own *name*
|
|
(`slug~DDMMYYYY~tags.md`) — knowable straight from the listing that discovers it, with no
|
|
further request per article. A `.md` file's actual content (title, description, image) is
|
|
only fetched once that article is about to be shown — the current page's cards, a single
|
|
article, or a search — not for the whole site up front. See "Adding an article" and "Article
|
|
discovery & caching" below.
|
|
3. Clicking an article fetches the Markdown file and renders it with [marked.js](https://marked.js.org/).
|
|
4. Hash-based routing handles navigation — back/forward works without page reloads.
|
|
5. Once an article's content has been fetched, it's cached in IndexedDB, keyed by the article
|
|
file's own path and invalidated by that file's `mtime`, so repeat visits skip re-fetching
|
|
unchanged content — see "Article discovery & caching" below.
|
|
|
|
## Features
|
|
|
|
- **Dark mode** — automatically follows the OS `prefers-color-scheme` setting; no toggle needed.
|
|
- **Client-side search** — the search box in the header filters all articles by title, description, author, and tags in-memory. Navigates to `#search/query` so results are bookmarkable; results paginate the same way the article list does.
|
|
- **Reading time** — estimated from the article word count (200 wpm) and shown on list cards and in article headers. Computed automatically; no frontmatter field needed.
|
|
- **Table of contents** — auto-generated from `h2`/`h3` headings and inserted at the top of the article body. Only appears when the article has three or more headings.
|
|
- **Back link** — a "← Back to articles" link above every article, returning to whatever list, filter, or search page (and page number) the reader actually came from rather than always resetting to page 1 of everything.
|
|
- **Pagination** — article list paginates at 20 articles per page; page state is encoded in the hash so browser history works.
|
|
- **Author and tag filtering** — clicking an author name or tag pill shows a filtered list. Both support pagination.
|
|
- **Responsive layout** — layout, cards, images, and pagination all adapt to narrow screens (breakpoint at 640 px).
|
|
- **Keyboard navigation** — all interactive elements have visible `:focus-visible` outlines for keyboard users.
|
|
- **Author profiles** — a profile card with bio and social links is shown at the top of each author's article list. Created by adding `articles/{author}/{author}.md`.
|
|
- **Offline-friendly caching** — discovered articles are cached in IndexedDB, so repeat visits skip re-fetching content that hasn't changed.
|
|
- **Translatable UI** — every UI string is looked up from a locale JSON file (English, Czech, and Slovak ship by default); adding a language needs no code changes.
|
|
|
|
## Project structure
|
|
|
|
This whole repo *is* cmkiss — clone it as-is into an existing site (or copy `css/`, `js/`,
|
|
`strings/`, `templates/`, and your own `articles/` into one) and embed it into a page there; see
|
|
"Embedding in an existing page" below. `demo/` is a standalone demo/preview page, kept separate
|
|
from the library — not something an embedding page needs to bring along.
|
|
|
|
```
|
|
├── README.md
|
|
├── demo/ # Standalone demo/preview page — not needed to embed cmkiss elsewhere
|
|
│ ├── index.html # Single-page shell
|
|
│ └── css/
|
|
│ ├── reset.css # CSS variables, dark mode, box-sizing reset, focus styles
|
|
│ ├── header.css # Site header, title link, search form
|
|
│ └── layout.css # Main content wrapper, state messages, responsive
|
|
├── css/
|
|
│ └── components/ # Component styles — needed by any embedding page
|
|
│ ├── card.css # Article list cards and filter heading
|
|
│ ├── article.css # Single article view, back link, hero image
|
|
│ ├── content.css # Rendered markdown styles (.content *)
|
|
│ ├── pagination.css # Prev/next page navigation
|
|
│ ├── toc.css # Auto-generated table of contents
|
|
│ └── author-profile.css # Author profile card (avatar, bio, social links)
|
|
├── js/
|
|
│ ├── config-default.js # Shipped defaults for BASE_URL, MARKED_URL, ARTICLES_URL
|
|
│ ├── config.js # Optional local overrides — gitignored; see "Local configuration" below
|
|
│ ├── frontmatter.js # parseFrontmatter() — pure YAML-front parser
|
|
│ ├── templates.js # loadTemplates(), templates cache
|
|
│ ├── discovery.js # discoverArticles(), fetchJson(), article cache
|
|
│ ├── cache.js # IndexedDB cache for discovered articles, keyed by article file path
|
|
│ ├── concurrency.js # mapWithConcurrency() — caps simultaneous per-article fetches
|
|
│ ├── i18n.js # t(), loadStrings(), getLocale() — UI string lookup
|
|
│ ├── views.js # loadArticleList, loadArticle, loadAuthorView, loadTagView, loadSearch
|
|
│ └── router.js # Hash router and app entry point
|
|
├── vendor/
|
|
│ └── marked.min.js # marked.js (local copy, loaded as a global) — the only 3rd-party code
|
|
├── strings/
|
|
│ ├── en.json # UI strings (default/fallback locale) — see "Translations" below
|
|
│ ├── cs.json # Czech translation
|
|
│ └── sk.json # Slovak translation
|
|
├── templates/
|
|
│ ├── card.html # Article list card layout
|
|
│ ├── article.html # Full article view layout
|
|
│ └── author-profile.html # Author profile card layout
|
|
└── articles/
|
|
└── author/ # One directory per author
|
|
├── author.md # Author profile (optional)
|
|
├── media/ # Shared images for this author — see "Adding an article"
|
|
│ ├── avatar.jpg # Referenced by author.md's `avatar:` field
|
|
│ └── cover.jpg # Referenced by one or more articles' `image:` field
|
|
└── slug~DDMMYYYY~tag1+tag2.md # One file per article — see "Adding an article"
|
|
```
|
|
|
|
## Local configuration
|
|
|
|
`js/config-default.js` holds the shipped defaults (`BASE_URL`, `MARKED_URL`, `ARTICLES_URL`) and
|
|
is tracked in git; every other module imports from it. It also loads an optional `js/config.js`
|
|
— gitignored, not part of cmkiss's own git history — to override any of them. **Nothing needs to
|
|
be created before first use**: if `js/config.js` doesn't exist, the shipped defaults apply as-is.
|
|
|
|
The split exists for using cmkiss as a git submodule (or otherwise pulling in a newer version some
|
|
other way): since `config.js` is gitignored, updating cmkiss never touches it, so any override
|
|
you've made survives. Only `config-default.js` gets updated.
|
|
|
|
To override a setting, create `js/config.js` and export just the constant(s) you want to change —
|
|
anything you don't export falls back to `config-default.js`. For example, to move where articles
|
|
live (see "Pointing articles/ somewhere else" below for the matching nginx-side change this
|
|
needs):
|
|
|
|
```js
|
|
export const ARTICLES_URL = new URL('/blog-content/articles/', location.origin).href;
|
|
```
|
|
|
|
## Adding an article
|
|
|
|
An article is a single Markdown file placed directly in its author's directory. Its *file name*
|
|
is where its publish date and tags live — not in separate files, and not in frontmatter. The
|
|
format is:
|
|
|
|
```
|
|
<slug>~<DDMMYYYY>~<tag1>+<tag2>+….md
|
|
```
|
|
|
|
- `<slug>` — the stable identifier used in the article's URL (`#post/author/slug`). Treat this
|
|
as permanent: retagging or correcting the date later just means renaming the file (see below),
|
|
but changing the slug changes the article's URL.
|
|
- `<DDMMYYYY>` — the publish date, 8 digits, no separators (e.g. `16062026` for 16 June 2026).
|
|
- `<tag1>+<tag2>+…` — zero or more tag ids, joined with `+`. Each tag id is a normal topic tag,
|
|
shown as a pill linking to `#tag/tag-id`. Omit this whole segment (and its leading `~`) if the
|
|
article has no tags.
|
|
|
|
The separator is `~` (tilde), not a hyphen — deliberately, since a slug is expected to be plain
|
|
kebab-case (single hyphens only) and `~` isn't something you'd type by accident while writing
|
|
one, unlike an extra `-`.
|
|
|
|
**Drafts** aren't a file-naming convention — they live in a separate `draft/` subdirectory inside
|
|
the author's own directory: `articles/author/draft/my-article~16062026~guides.md`. The crawler
|
|
only ever reads `.md` files directly inside an author's directory, so anything under `draft/` is
|
|
never indexed, listed, searched, or paginated. nginx should also deny access to every author's
|
|
`draft/` folder outright (see "nginx configuration" below), so a draft isn't reachable by a
|
|
guessed direct URL either — not just hidden from the crawl. To publish an article, move its file
|
|
out of `draft/` into the author directory itself.
|
|
|
|
Images (a cover photo, inline pictures) live in a `media/` folder shared by all of that author's
|
|
articles, not next to each individual article file — so the same picture can be reused across
|
|
several articles without keeping a duplicate copy per article. Reference one from frontmatter or
|
|
markdown with a path relative to the author directory, e.g. `image: media/photo.jpg`.
|
|
|
|
Steps:
|
|
|
|
1. Create the author directory if it doesn't exist: `articles/author/`
|
|
2. Create the Markdown file directly inside it, using the naming convention above, e.g.
|
|
`articles/author/my-article~16062026~guides+tutorials.md` — or inside `articles/author/draft/`
|
|
instead, if it's not ready to publish yet.
|
|
3. Optionally add a cover image to that author's shared `articles/author/media/` folder
|
|
(creating it if needed), and reference it from frontmatter with `image: media/photo.jpg`.
|
|
4. Add a frontmatter block at the top of the Markdown file
|
|
5. Write your content — it will appear in the list automatically (once it's out of `draft/`)
|
|
|
|
To change an article's date or tags later, rename its file — its content doesn't need to change.
|
|
`author` is derived from the containing directory — no frontmatter field needed
|
|
for it either. Treat the directory name as case-sensitive and exact: it's matched exactly (not
|
|
case-insensitively) against `#author/...` URLs and author-profile lookups, so avoid two author
|
|
directories that differ only in case. Because every one of these is encoded in the file name,
|
|
discovering an article's metadata never requires fetching anything beyond the directory listing
|
|
that already reveals it — no further per-article request at all, which is what makes the "Article
|
|
discovery & caching" section below possible.
|
|
|
|
> **A note on trust:** article and author-bio Markdown is rendered directly via marked.js with no
|
|
> HTML-sanitization pass — anyone who can write a `.md` file here can include raw HTML, including
|
|
> `<script>` tags. Only let people you trust add or edit content.
|
|
|
|
### Frontmatter format
|
|
|
|
```yaml
|
|
---
|
|
title: My Article Title
|
|
description: A short summary shown in the article list.
|
|
---
|
|
|
|
Your markdown content here...
|
|
```
|
|
|
|
| Field | Required | Notes |
|
|
|-------|----------|-------|
|
|
| `title` | yes | Shown in the list and article header |
|
|
| `description` | no | Short summary shown under the title in the list |
|
|
| `image` | no | Path to an image in the author's shared `media/` folder (e.g. `media/photo.jpg`); shown as card thumbnail and article hero |
|
|
|
|
Reading time is computed automatically from the article word count — no frontmatter field
|
|
needed. `date` and `tags` frontmatter fields, if present, are ignored — superseded by the file
|
|
name described above.
|
|
|
|
## Author profiles
|
|
|
|
To add a profile for an author, create a file named `{author}.md` directly inside the author's directory:
|
|
|
|
```
|
|
articles/bob/bob.md
|
|
```
|
|
|
|
The profile appears as a card above the article list when a visitor filters by that author. It is optional — authors without a profile file work normally.
|
|
|
|
### Profile frontmatter
|
|
|
|
```yaml
|
|
---
|
|
title: Bob Johnson
|
|
avatar: bob-avatar.jpg
|
|
github: https://github.com/bob
|
|
twitter: https://twitter.com/bob
|
|
linkedin: https://linkedin.com/in/bob
|
|
mastodon: https://mastodon.social/@bob
|
|
website: https://bob.example.com
|
|
---
|
|
|
|
Short bio rendered as markdown.
|
|
```
|
|
|
|
| Field | Notes |
|
|
|-------|-------|
|
|
| `title` | Display name shown in the profile card. Falls back to the author directory name. |
|
|
| `avatar` | Path to an image, relative to the author directory — e.g. `media/avatar.jpg` if kept in the shared media folder alongside article images, or a bare filename if kept directly beside `{author}.md`. Displayed as a circular thumbnail. |
|
|
| `github` / `twitter` / `linkedin` / `mastodon` / `website` | Any subset of these renders as pill links below the bio. |
|
|
|
|
The body text below the frontmatter is rendered as markdown and shown as the bio.
|
|
|
|
## Routing
|
|
|
|
| URL hash | View |
|
|
|----------|------|
|
|
| *(empty)* | All articles, page 1 |
|
|
| `#page/2` | All articles, page 2 |
|
|
| `#search/linux` | Search results for "linux", page 1 |
|
|
| `#search/linux/page/2` | Same, page 2 |
|
|
| `#author/alice` | All articles by alice, page 1 |
|
|
| `#author/alice/page/2` | All articles by alice, page 2 |
|
|
| `#tag/guides` | All articles tagged "guides", page 1 |
|
|
| `#tag/guides/page/2` | Same, page 2 |
|
|
| `#post/alice/my-article` | Single article at `articles/alice/my-article~DDMMYYYY~tags.md` |
|
|
|
|
Author names and tags on each card and article are clickable links to their respective filter views. A search box in the header navigates to `#search/query`.
|
|
|
|
## Customising the layout
|
|
|
|
The HTML structure for the article list card, the full article view, and the author profile
|
|
card live in `templates/`:
|
|
|
|
- `templates/card.html` — the card shown in the article list
|
|
- `templates/article.html` — the full article view (header, hero image, content)
|
|
- `templates/author-profile.html` — the profile card (avatar, bio, social links)
|
|
|
|
Edit these files directly to change layout — no JS changes needed. Elements are identified by `data-field` attributes, which is how the JS modules find and populate them. Keep existing `data-field` values when rearranging; add new ones if you add new fields.
|
|
|
|
## Translations
|
|
|
|
Every UI string the JS renders (loading/error states, pagination, filter labels, the table of
|
|
contents heading, etc.) is looked up through `t(key, params)` in `js/i18n.js` rather than
|
|
hardcoded — `strings/en.json` is the default/fallback string table.
|
|
|
|
Available locales: `en` (default), `cs` (Czech), `sk` (Slovak).
|
|
|
|
To add another translation:
|
|
|
|
1. Set `<html lang="xx">` on the embedding page (or `demo/index.html` for this repo's own demo)
|
|
to the target language code (e.g. `cs`).
|
|
2. Add `strings/xx.json` with the same keys as `en.json` (copy it as a starting point) —
|
|
you only need to include the keys you're translating; anything missing quietly falls back to
|
|
English rather than breaking.
|
|
3. `{placeholder}` tokens in a string (e.g. `"{page} of {totalPages}"`) are substituted at
|
|
render time — keep them in your translated string, in whatever order/position reads
|
|
naturally in that language.
|
|
|
|
The resolved locale also drives date formatting (`Intl.DateTimeFormat`), search matching, and
|
|
tag-label sorting (`localeCompare`), via `getLocale()` in the same module — so a translation
|
|
file doesn't just relabel text, it affects how dates and sorting behave too.
|
|
|
|
cmkiss never names the page: the `<title>`, the site header, and any other branding (the header
|
|
search placeholder, the sidebar section headings) belong to the embedding page and are plain
|
|
static HTML — translate them by editing the page directly (`demo/index.html` for this repo's own
|
|
demo). The only change cmkiss makes to the tab title is in the article view, where it shows
|
|
`<article title> | <page's own title>`; list views restore the page's original `<title>` exactly.
|
|
|
|
## Embedding in an existing page
|
|
|
|
To add cmkiss to any HTML page, three things are needed: the CSS, the two mount-point `<div>`s, and the script.
|
|
|
|
### 1. CSS — in `<head>`
|
|
|
|
```html
|
|
<!-- Site-level styles (or replace with your own) -->
|
|
<link rel="stylesheet" href="demo/css/reset.css">
|
|
<link rel="stylesheet" href="demo/css/header.css">
|
|
<link rel="stylesheet" href="demo/css/layout.css">
|
|
|
|
<!-- Component styles -->
|
|
<link rel="stylesheet" href="css/components/card.css">
|
|
<link rel="stylesheet" href="css/components/article.css">
|
|
<link rel="stylesheet" href="css/components/content.css">
|
|
<link rel="stylesheet" href="css/components/pagination.css">
|
|
<link rel="stylesheet" href="css/components/toc.css">
|
|
<link rel="stylesheet" href="css/components/author-profile.css">
|
|
```
|
|
|
|
The site-level CSS files (`reset.css`, `header.css`, `layout.css`) are optional — they style the shell around the CMS. If your page already has a layout and reset, you can skip them and only load the component stylesheets. Note that `reset.css` also defines the CSS custom properties (colours, radius) used throughout — if you skip it, define the variables yourself. `author-profile.css` is only needed if any of your authors have a profile file (see "Author profiles" above) — safe to skip otherwise.
|
|
|
|
### 2. Mount points — in `<body>`
|
|
|
|
```html
|
|
<div id="list-view"></div>
|
|
<div id="article-view" hidden></div>
|
|
```
|
|
|
|
Place these wherever you want the CMS to appear. The router toggles `hidden` between them on navigation. A search box and tag filter list are optional and can go anywhere else in the page — see "Search, article list, and tag list as independent pieces" below. Unlike those, `#list-view`/`#article-view` are both required — router.js checks for them at startup and shows a clear error (scoped to whichever of the two is present, or the whole page if neither is) instead of silently failing if one is missing.
|
|
|
|
### 3. Script — before `</body>`
|
|
|
|
```html
|
|
<script type="module" src="js/router.js"></script>
|
|
```
|
|
|
|
`router.js` is the only script tag needed. It loads `marked.min.js` automatically at startup (path configured in `js/config.js`).
|
|
|
|
> **Subdirectory pages:** If the embedding page lives in a subdirectory (e.g. `/blog/index.html`) and cmkiss isn't installed at the site root, a plain relative path like `css/...` will resolve against the *page's* directory, not wherever cmkiss actually lives. Use a path (root-relative or otherwise) that correctly points at wherever you placed it, e.g. if cmkiss lives at `/cmkiss/` on your site:
|
|
> ```html
|
|
> <link rel="stylesheet" href="/cmkiss/css/components/card.css">
|
|
> <script type="module" src="/cmkiss/js/router.js"></script>
|
|
> ```
|
|
> All internal fetches (articles, templates, marked.js) resolve relative to the module files via `import.meta.url` and work automatically — only the HTML tags need a path pointing at the right place.
|
|
|
|
### Search, article list, and tag list as independent pieces
|
|
|
|
The mount points above don't have to sit next to each other, or even all be present — each is a
|
|
plain element the router looks up by `id`, wherever it happens to be in your page. This lets you
|
|
scatter search, the article list, and the tag list across a custom layout (search box in the site
|
|
header, tag list in a sidebar, articles in the main column) instead of stacking them the way the
|
|
demo does.
|
|
|
|
- **Article list + single article — `#list-view` / `#article-view`.** These two are the only
|
|
required pair; the router toggles `hidden` between them and always needs both present somewhere
|
|
on the page. This pair is also where **search results** render — there's no separate results
|
|
container, so any page using the search box below still needs `#list-view` on it somewhere to
|
|
actually show what was searched for.
|
|
- **Search box — `#search-form` / `#search-input`.** Fully optional and independent; place it
|
|
anywhere (e.g. a site-wide header, not necessarily near the list). Submitting it navigates to
|
|
`#search/query`; matches render into `#list-view`.
|
|
```html
|
|
<form id="search-form" role="search">
|
|
<input type="search" id="search-input" placeholder="Search articles…" autocomplete="off">
|
|
</form>
|
|
```
|
|
- **Tag list — `#tag-filters`.** Optional; put a container (a `<nav>` works well) anywhere, e.g. a
|
|
sidebar. It's automatically filled in with one link per tag in use (alphabetical, with an
|
|
article count) every time a list view renders — no manual updates needed. Clicking a tag
|
|
navigates to `#tag/tag-id`, rendering the filtered list into `#list-view`.
|
|
```html
|
|
<nav id="tag-filters" aria-label="Filter by tag"></nav>
|
|
```
|
|
|
|
`#tag-filters` is optional — omit it and nothing breaks, the article list and search still work
|
|
standalone.
|
|
|
|
### Minimal example
|
|
|
|
```html
|
|
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<title>My Site</title>
|
|
<link rel="stylesheet" href="css/components/card.css">
|
|
<link rel="stylesheet" href="css/components/article.css">
|
|
<link rel="stylesheet" href="css/components/content.css">
|
|
<link rel="stylesheet" href="css/components/pagination.css">
|
|
<link rel="stylesheet" href="css/components/toc.css">
|
|
</head>
|
|
<body>
|
|
<div id="list-view"></div>
|
|
<div id="article-view" hidden></div>
|
|
|
|
<script type="module" src="js/router.js"></script>
|
|
</body>
|
|
</html>
|
|
```
|
|
|
|
## nginx configuration
|
|
|
|
Point nginx `root` at the repo root. The `autoindex` directive applies recursively to all
|
|
subdirectories under `/articles/` — **except** each author's own `draft/` folder, which must be
|
|
denied outright (see "Adding an article" above: this is what keeps a draft from being indexed or
|
|
fetched by a guessed direct URL).
|
|
|
|
```nginx
|
|
server {
|
|
listen 80;
|
|
server_name example.com;
|
|
root /var/www/cmkiss;
|
|
|
|
# Serve the demo page
|
|
location /demo/ {
|
|
try_files $uri $uri/ /demo/index.html;
|
|
}
|
|
|
|
# Deny every author's draft/ folder outright — must come before the
|
|
# /articles/ block below (a regex location overrides a prefix location
|
|
# in nginx, so this still wins even though it's declared after it).
|
|
location ~ ^/articles/[^/]+/draft/ {
|
|
deny all;
|
|
return 404;
|
|
}
|
|
|
|
# Enable JSON directory listing for article discovery (applies to all subdirs)
|
|
location /articles/ {
|
|
autoindex on;
|
|
autoindex_format json;
|
|
autoindex_localtime off; # keep mtimes in GMT — the crawler's cache relies on consistent, unambiguous timestamps
|
|
}
|
|
}
|
|
```
|
|
|
|
> **Note:** `autoindex_format json` requires nginx 1.7.9+. Check with `nginx -v`. If
|
|
> `autoindex_format json` is ever lost (config drift, an older nginx), the crawler falls back
|
|
> to parsing nginx's default HTML directory listing instead of breaking outright.
|
|
|
|
### Pointing articles/ somewhere else
|
|
|
|
By default the crawler looks for articles at `articles/` next to the rest of cmkiss (`ARTICLES_URL`,
|
|
computed as a sibling of the `js/` folder). Articles don't have to live there — override
|
|
`ARTICLES_URL` in your local `js/config.js` (see "Local configuration" above) to point anywhere
|
|
under the same origin (or a different one, CORS permitting).
|
|
|
|
Whatever it resolves to needs its own matching `location` block enabling the JSON (or HTML)
|
|
autoindex, plus the matching `draft/` deny block (see above) — it doesn't have to be named
|
|
`articles/` or nested under wherever cmkiss itself lives:
|
|
|
|
```nginx
|
|
location ~ ^/blog-content/articles/[^/]+/draft/ {
|
|
deny all;
|
|
return 404;
|
|
}
|
|
|
|
location /blog-content/articles/ {
|
|
autoindex on;
|
|
autoindex_format json;
|
|
autoindex_localtime off;
|
|
}
|
|
```
|
|
|
|
### Compression
|
|
|
|
Markdown and directory-listing JSON are both plain text, so gzip shrinks them well — and since
|
|
the browser decompresses transparently, this needs no changes on the cmkiss side. nginx doesn't
|
|
know `.md` is text by default (it falls back to `application/octet-stream`, which `gzip_types`
|
|
won't match), so map it explicitly and enable gzip for both content types:
|
|
|
|
```nginx
|
|
# in http {} or a shared mime block
|
|
types {
|
|
text/markdown md;
|
|
}
|
|
|
|
# in http {} or the server block above
|
|
gzip on;
|
|
gzip_types text/markdown text/plain application/json; # json covers the autoindex_format listings
|
|
gzip_min_length 256;
|
|
gzip_vary on;
|
|
```
|
|
|
|
Pre-compressing with `gzip_static` is possible but needs a `.md.gz` alongside every article, which
|
|
is a build step this project deliberately has none of — plain runtime `gzip on` is the better fit
|
|
here. Brotli usually compresses text better still, but needs the separate `ngx_brotli` module,
|
|
not built into stock nginx — only worth it if you're already building custom nginx modules.
|
|
|
|
## Article discovery & caching
|
|
|
|
Discovery happens in two phases. **Indexing** crawls `articles/` client-side to build a
|
|
lightweight "stub" for every article — the author directories, then each author's own directory
|
|
listing (their article files, bio file, shared `media/` folder, and `draft/` folder together)
|
|
along with each file's `mtime`. Only `.md` files listed directly inside an author's own directory
|
|
are kept as candidate articles — the bio file is filtered out by name, and any subdirectory
|
|
(`media/`, `draft/`) is skipped outright rather than being descended into, so nothing under
|
|
`draft/` is ever indexed. That's it: because an article file's *name* already encodes its slug,
|
|
date, and tags (see "Adding an article" above), that single listing is enough to sort, filter,
|
|
paginate, and populate the sidebar's tag counts, without any further request per article and
|
|
without ever fetching an article's actual Markdown.
|
|
**Hydration** is the separate, deferred step that fetches a specific article's content (title,
|
|
description, cover image, reading time) — it only ever runs for articles about to be rendered:
|
|
the current page's cards (`hydrateArticles()` in `js/discovery.js`), a single visited
|
|
article, or (since full-text search needs real content) as many search results as the page being
|
|
viewed actually needs. A large site's list is therefore fast to show regardless of how many
|
|
articles it has — indexing costs one request per author, full stop, and only ~20 articles' worth
|
|
of content is ever fetched per page view.
|
|
|
|
- **Cache** — hydrated articles are cached per article file in IndexedDB (`js/cache.js`),
|
|
keyed by the file's own path and invalidated by comparing its live `mtime` against the cached
|
|
one — once *any* visit renders an article, every later visit skips its content fetch too, until
|
|
the file is renamed (a rename changes the cache key, so a retagged/redated article is treated as
|
|
new — see "Adding an article" above) or its own mtime changes. Indexing reads the entire cache
|
|
once up front (`getAllCacheEntries()`, kicked off in parallel with the author/directory-listing
|
|
requests) so a warm cache doesn't cost an IndexedDB read per article. If IndexedDB is unavailable
|
|
(private browsing, disabled storage), discovery transparently falls back to fetching everything
|
|
live on every visit.
|
|
- **Cache schema version** — the cached records store the full derived article shape, not raw
|
|
content, so a code change to how articles are derived (in `js/discovery.js`) can make old
|
|
cached entries stale in *structure*, not just content — and a file's `mtime` won't catch that.
|
|
Bump `CACHE_SCHEMA_VERSION` in `js/cache.js` whenever that shape changes; it's baked into
|
|
the IndexedDB database name, so a bump invalidates every cached entry on the next deploy —
|
|
articles and cached searches alike. A change to the search-match record shape instead, or a new
|
|
store entirely, only needs `DB_VERSION` bumped (IndexedDB's own upgrade counter): `onupgradeneeded`
|
|
creates whatever store is missing without touching the other one's existing data.
|
|
- **Concurrency** — `MAX_CONCURRENT_REQUESTS` in `js/discovery.js` (default `10`) caps both how
|
|
many hydration (content) fetches run at once and how many authors' directory listings are
|
|
fetched at once during indexing; raise it for a fast/local nginx, lower it for a slow or
|
|
shared/production instance.
|
|
- **Prefetching** — once the current page has fully rendered, the next page's articles are
|
|
hydrated in the background (fire-and-forget), so clicking "next" is usually an instant cache
|
|
hit rather than a fresh round-trip.
|
|
- **Paused/resumable search** — search needs real text (title, description), unlike the list
|
|
view, so it hydrates articles rather than just reading stubs — but only as many as the page
|
|
being viewed needs, in the same newest-first order as everything else (`loadSearch()` in
|
|
`js/views.js`). It scans stubs in small batches, stops as soon as it's found enough matches
|
|
to fill the current page (plus one, just to know whether a next page exists), and resumes
|
|
exactly where it left off — no re-checking — if the user pages further into the *same* query.
|
|
Paging back needs no scanning at all, since matches only ever accumulate. This keeps a search on
|
|
a large site to a handful of requests instead of hydrating the entire article set up front. The
|
|
page count is exact once the whole set's been scanned, and a lower bound (at least one more
|
|
page) until then.
|
|
- **Persistent search-match cache** — alongside the resumable in-session scan above, each query's
|
|
per-article match results (`{ mtime, matched }`, keyed by article path) are cached in IndexedDB
|
|
too (`searchQueries` store in `js/cache.js`, keyed by the query text itself) and reused across
|
|
page reloads and browser sessions, not just while paging through one visit. Searching the same
|
|
term again only re-fetches and re-checks articles that are new or whose `mtime` has changed
|
|
since — everything else is trusted from the cache without a content re-fetch at all. Capped at
|
|
`MAX_CACHED_SEARCHES` (default `30`) distinct queries, evicting the least-recently-searched one
|
|
first, so trying many different search terms over time doesn't grow the cache unbounded.
|
|
- **Crawl health** — if the root author-listing fetch fails outright, or if every individual
|
|
author's own directory listing fails (wrong origin, CORS misconfigured), a small warning banner
|
|
is shown above the article list instead of silently rendering an empty page (see
|
|
`getCrawlStats()` / `renderCrawlHealthWarning()` in `js/discovery.js` / `js/views.js`).
|
|
Non-conforming `.md` file names aren't treated as a health problem — only as a per-file skip —
|
|
since indexing does no per-article fetch that could actually fail.
|
|
- **Design rationale** — the original crawler design (wave-based crawling with a per-article
|
|
listing fetch, recency prioritisation, HTML-autoindex fallback, long-term "deploy and forget"
|
|
constraints) is documented in
|
|
[`.claude/nginx-sitemap-crawler-plan.md`](.claude/nginx-sitemap-crawler-plan.md), which ships
|
|
alongside the code. The per-article listing fetch it describes no longer exists (metadata moved
|
|
into the file name instead, and articles no longer live in a directory of their own), but the
|
|
rest of that reasoning — the caching model, the longevity constraints, the HTML fallback — still
|
|
applies.
|
|
|
|
## Running locally
|
|
|
|
Using nginx locally (example for Linux):
|
|
```bash
|
|
# Symlink or copy the repo to your nginx webroot, then reload:
|
|
sudo nginx -s reload
|
|
```
|
|
Then open `/demo/` in a browser to preview.
|
|
|
|
> Python's `http.server` returns HTML directory listings, not JSON, so article discovery won't work with it. Use nginx.
|