Rewrite article discovery for author/slug layout with IndexedDB cache (AI)
Drops the expa-YYYY category directory in favor of author/slug/slug.md with .tag and .date marker files; adds mtime-based IndexedDB caching, concurrency-limited crawling, an HTML-autoindex mtime fallback, and the #post/<author>/<slug> route this layout change requires. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AXSVPS3UmmASiiprPfcnUd
This commit is contained in:
AIandClaude Sonnet 5 committed
1 parent
b1017f29c2
commit
b92bbc7129
8 files changed
+322
-85
No files matched your search
@@ -4,10 +4,11 @@ A minimal CMS where articles are Markdown files rendered in the browser. No buil
|
||||
|
||||
## How it works
|
||||
|
||||
1. The browser traverses `cmjs/articles/` via a server directory index — discovering authors, year categories, and article files. Articles may be direct Markdown files or use the nested `author/category/post/post.md` layout; nginx JSON autoindex is preferred, with HTML autoindexes supported as a fallback.
|
||||
2. Each `.md` file's YAML frontmatter is parsed to build the article list.
|
||||
1. The browser traverses `cmjs/articles/` via a server directory index — discovering authors and their post directories (`author/slug/slug.md`); nginx JSON autoindex is preferred, with HTML autoindexes supported as a fallback.
|
||||
2. Each `.md` file's YAML frontmatter is parsed for title/description/image; publish date and tags/category come from empty `.date`/`.tag` marker files beside it (see "Adding an article" below).
|
||||
3. Clicking an article fetches the Markdown file and renders it with [marked.js](https://marked.js.org/).
|
||||
4. Hash-based routing handles navigation — back/forward works without page reloads.
|
||||
5. Discovered articles are cached in IndexedDB, keyed by post directory and invalidated by directory `mtime`, so repeat visits skip re-fetching unchanged content — see "Article discovery & caching" below.
|
||||
|
||||
## Features
|
||||
|
||||
@@ -47,35 +48,45 @@ cmjs/
|
||||
│ ├── frontmatter.js # parseFrontmatter() — pure YAML-front parser
|
||||
│ ├── templates.js # loadTemplates(), templates cache
|
||||
│ ├── discovery.js # discoverArticles(), fetchJson(), article cache
|
||||
│ ├── cache.js # IndexedDB cache for discovered articles, keyed by post directory
|
||||
│ ├── concurrency.js # mapWithConcurrency() — caps simultaneous per-article fetches
|
||||
│ ├── views.js # loadArticleList, loadArticle, loadAuthorView, loadCategoryView, loadSearch
|
||||
│ └── router.js # Hash router and app entry point
|
||||
├── templates/
|
||||
│ ├── card.html # Article list card layout
|
||||
│ └── article.html # Full article view layout
|
||||
├── articles/
|
||||
│ └── author/ # One directory per author
|
||||
│ └── expa-YYYY/ # Main expedition-year category
|
||||
│ └── slug/ # One directory per post
|
||||
│ └── slug.md
|
||||
│ └── author/ # One directory per author
|
||||
│ ├── author.md # Author profile (optional)
|
||||
│ └── slug/ # One directory per post
|
||||
│ ├── slug.md # Article content + frontmatter
|
||||
│ ├── tag-id.tag # Zero or more empty tag/category markers
|
||||
│ └── DD-MM-YYYY.date # Publish date marker (exactly one)
|
||||
└── pictures/ # Article images (referenced via image: frontmatter field)
|
||||
```
|
||||
|
||||
## Adding an article
|
||||
|
||||
1. Create the author, year, and post directories if they don't exist: `cmjs/articles/author/expa-YYYY/my-article/`
|
||||
2. Create the Markdown file inside: `cmjs/articles/author/expa-YYYY/my-article/my-article.md`
|
||||
3. Optional tag markers can sit beside the Markdown file as `tag-name.tag`.
|
||||
4. Add a frontmatter block at the top
|
||||
5. Write your content — it will appear in the list automatically
|
||||
1. Create the author and post directories if they don't exist: `cmjs/articles/author/my-article/`
|
||||
2. Create the Markdown file inside: `cmjs/articles/author/my-article/my-article.md`
|
||||
3. Add a publish-date marker beside it: `cmjs/articles/author/my-article/DD-MM-YYYY.date` (e.g.
|
||||
`16-06-2026.date`) — an empty file, exactly one per article. This is the sole source of the
|
||||
publish date; a `date:` frontmatter field is no longer read.
|
||||
4. Add tag/category markers as needed: `cmjs/articles/author/my-article/tag-id.tag` (e.g.
|
||||
`kometa.tag`, `expa-2026.tag`) — empty files, any number. A tag id matching `expa-YYYY` is
|
||||
treated as the article's "year"/category and shown in the Ročníky sidebar; every other tag
|
||||
id is a normal topic tag.
|
||||
5. Add a frontmatter block at the top of the Markdown file
|
||||
6. Write your content — it will appear in the list automatically
|
||||
|
||||
`author` and `category` are derived from the directory path — no frontmatter fields needed for them.
|
||||
`author` is derived from the directory path — no frontmatter field needed for it. Publish date,
|
||||
tags, and category are derived from the `.date`/`.tag` marker files, not frontmatter.
|
||||
|
||||
### Frontmatter format
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: My Article Title
|
||||
date: 2026-06-16
|
||||
description: A short summary shown in the article list.
|
||||
draft: false
|
||||
---
|
||||
@@ -86,12 +97,13 @@ Your markdown content here...
|
||||
| Field | Required | Notes |
|
||||
|-------|----------|-------|
|
||||
| `title` | yes | Shown in the list and article header |
|
||||
| `date` | yes | ISO format `YYYY-MM-DD`, used for sorting newest-first |
|
||||
| `description` | no | Short summary shown under the title in the list |
|
||||
| `image` | no | Filename of the preview image in the `pictures/` directory (e.g. `photo.jpg`); shown as card thumbnail and article hero |
|
||||
| `draft` | no | Set to `true`, `yes`, or `1` to hide from the list |
|
||||
|
||||
Reading time is computed automatically from the article word count — no frontmatter field needed.
|
||||
Reading time is computed automatically from the article word count — no frontmatter field
|
||||
needed. `date` and `tags` frontmatter fields, if present, are ignored — superseded by the
|
||||
`.date`/`.tag` marker files above.
|
||||
|
||||
## Author profiles
|
||||
|
||||
@@ -136,10 +148,11 @@ The body text below the frontmatter is rendered as markdown and shown as the bio
|
||||
| `#search/linux` | Search results for "linux" |
|
||||
| `#author/alice` | All articles by alice, page 1 |
|
||||
| `#author/alice/page/2` | All articles by alice, page 2 |
|
||||
| `#category/linux` | All articles in "linux", page 1 |
|
||||
| `#category/linux/page/2` | All articles in "linux", page 2 |
|
||||
| `#alice/linux/my-article` | Single legacy article at `cmjs/articles/alice/linux/my-article.md` |
|
||||
| `#alice/expa-2024/my-article/my-article` | Single nested article at `cmjs/articles/alice/expa-2024/my-article/my-article.md` |
|
||||
| `#category/expa-2026` | All articles carrying the "expa-2026" year tag, page 1 |
|
||||
| `#category/expa-2026/page/2` | Same, page 2 |
|
||||
| `#tag/kometa` | All articles tagged "kometa", page 1 |
|
||||
| `#tag/kometa/page/2` | Same, page 2 |
|
||||
| `#post/alice/my-article` | Single article at `cmjs/articles/alice/my-article/my-article.md` |
|
||||
|
||||
Author names and category tags on each card and article are clickable links to their respective filter views. A search box in the header navigates to `#search/query`.
|
||||
|
||||
@@ -249,11 +262,33 @@ server {
|
||||
location /cmjs/articles/ {
|
||||
autoindex on;
|
||||
autoindex_format json;
|
||||
autoindex_localtime off; # keep mtimes in GMT — the crawler's cache relies on consistent, unambiguous timestamps
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
> **Note:** `autoindex_format json` requires nginx 1.7.9+. Check with `nginx -v`.
|
||||
> **Note:** `autoindex_format json` requires nginx 1.7.9+. Check with `nginx -v`. If
|
||||
> `autoindex_format json` is ever lost (config drift, an older nginx), the crawler falls back
|
||||
> to parsing nginx's default HTML directory listing instead of breaking outright.
|
||||
|
||||
## Article discovery & caching
|
||||
|
||||
Articles are discovered by crawling `cmjs/articles/` client-side: fetch the author
|
||||
directories, then each author's post directories along with their `mtime`, then — only for
|
||||
directories whose `mtime` has changed since the last visit — that directory's own listing (to
|
||||
read its `.tag`/`.date` markers) and its Markdown content.
|
||||
|
||||
- **Cache** — results are cached per post directory in IndexedDB (`cmjs/js/cache.js`), keyed
|
||||
by directory path and invalidated by comparing the live `mtime` against the cached one. If
|
||||
IndexedDB is unavailable (private browsing, disabled storage), the crawler transparently
|
||||
falls back to fetching everything live on every visit.
|
||||
- **Concurrency** — `MAX_CONCURRENT_REQUESTS` in `cmjs/js/discovery.js` (default `10`) caps how
|
||||
many post directories are crawled at once; raise it for a fast/local nginx, lower it for a
|
||||
slow or shared/production instance.
|
||||
- **Design rationale** — the full reasoning (wave-based crawling, recency prioritisation,
|
||||
HTML-autoindex fallback, and long-term "deploy and forget" constraints) is documented in
|
||||
[`.claude/nginx-sitemap-crawler-plan.md`](.claude/nginx-sitemap-crawler-plan.md), which ships
|
||||
alongside the code.
|
||||
|
||||
## Running locally
|
||||
|
||||
|
||||
Whitespace-only changes.
-2
@@ -1,6 +1,5 @@
|
||||
---
|
||||
title: Hello World
|
||||
date: 2026-06-16
|
||||
description: A sample article to demonstrate the CMS.
|
||||
image: hello-world-preview.webp
|
||||
draft: false
|
||||
@@ -15,7 +14,6 @@ Articles are plain Markdown files stored in the `articles/` directory. Each file
|
||||
```yaml
|
||||
---
|
||||
title: My Article Title
|
||||
date: 2026-06-16
|
||||
description: A short summary shown in the article list.
|
||||
draft: false
|
||||
---
|
||||
@@ -0,0 +1,84 @@
|
||||
// Persistent cache for discovered articles, keyed by slug-directory path.
|
||||
// Raw IndexedDB (no wrapper library — see .claude/nginx-sitemap-crawler-plan.md
|
||||
// L4/L5 on the zero-dependency, long-stable-API requirement). If IndexedDB is
|
||||
// unavailable (private browsing, old browser, blocked), every operation below
|
||||
// silently becomes a no-op so discovery.js gets live-fetch behaviour for free
|
||||
// without needing its own availability branching (plan doc R6).
|
||||
|
||||
const DB_NAME = 'cmjs-cache';
|
||||
const STORE_NAME = 'slugDirs';
|
||||
|
||||
let cacheAvailable = true;
|
||||
let dbPromise = null;
|
||||
|
||||
function openDb() {
|
||||
if (!dbPromise) {
|
||||
dbPromise = new Promise((resolve, reject) => {
|
||||
if (typeof indexedDB === 'undefined') { reject(new Error('IndexedDB unavailable')); return; }
|
||||
const request = indexedDB.open(DB_NAME, 1);
|
||||
request.onupgradeneeded = () => {
|
||||
request.result.createObjectStore(STORE_NAME, { keyPath: 'path' });
|
||||
};
|
||||
request.onsuccess = () => resolve(request.result);
|
||||
request.onerror = () => reject(request.error);
|
||||
request.onblocked = () => reject(new Error('IndexedDB open blocked'));
|
||||
}).catch(error => { cacheAvailable = false; throw error; });
|
||||
}
|
||||
return dbPromise;
|
||||
}
|
||||
|
||||
// Opens a store and resolves/rejects with the given IDBRequest's own outcome
|
||||
// — the standard pattern, so result timing never depends on transaction
|
||||
// lifecycle details.
|
||||
async function runRequest(mode, makeRequest) {
|
||||
if (!cacheAvailable) return undefined;
|
||||
try {
|
||||
const db = await openDb();
|
||||
const store = db.transaction(STORE_NAME, mode).objectStore(STORE_NAME);
|
||||
return await new Promise((resolve, reject) => {
|
||||
const request = makeRequest(store);
|
||||
request.onsuccess = () => resolve(request.result);
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
} catch {
|
||||
cacheAvailable = false;
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
export function isCacheAvailable() {
|
||||
return cacheAvailable;
|
||||
}
|
||||
|
||||
// Returns the cached { path, mtime, article } record, or undefined on a miss
|
||||
// (or if the cache is unavailable).
|
||||
export async function getCacheEntry(path) {
|
||||
return runRequest('readonly', store => store.get(path));
|
||||
}
|
||||
|
||||
export async function setCacheEntry(path, entry) {
|
||||
await runRequest('readwrite', store => store.put({ ...entry, path }));
|
||||
}
|
||||
|
||||
// Removes any cached entry whose path is no longer in the live directory
|
||||
// listing (renamed/deleted slugs), so the cache doesn't grow unbounded.
|
||||
export async function pruneCache(validPaths) {
|
||||
if (!cacheAvailable) return;
|
||||
const valid = new Set(validPaths);
|
||||
try {
|
||||
const db = await openDb();
|
||||
const store = db.transaction(STORE_NAME, 'readwrite').objectStore(STORE_NAME);
|
||||
await new Promise((resolve, reject) => {
|
||||
const cursorRequest = store.openCursor();
|
||||
cursorRequest.onsuccess = event => {
|
||||
const cursor = event.target.result;
|
||||
if (!cursor) { resolve(); return; }
|
||||
if (!valid.has(cursor.value.path)) cursor.delete();
|
||||
cursor.continue();
|
||||
};
|
||||
cursorRequest.onerror = () => reject(cursorRequest.error);
|
||||
});
|
||||
} catch {
|
||||
cacheAvailable = false;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
// Runs `fn` over `items` with at most `limit` calls in flight at once.
|
||||
// A worker-pool: each worker pulls the next index off a shared cursor until
|
||||
// the list is exhausted, so slow items don't stall workers behind them.
|
||||
export async function mapWithConcurrency(items, limit, fn) {
|
||||
const results = new Array(items.length);
|
||||
let cursor = 0;
|
||||
async function worker() {
|
||||
while (cursor < items.length) {
|
||||
const index = cursor++;
|
||||
results[index] = await fn(items[index], index);
|
||||
}
|
||||
}
|
||||
await Promise.all(Array.from({ length: Math.min(limit, items.length) }, worker));
|
||||
return results;
|
||||
}
|
||||
+136
-52
@@ -1,10 +1,27 @@
|
||||
import { parseFrontmatter } from './frontmatter.js';
|
||||
import { BASE_URL, PICTURES_URL } from './config.js';
|
||||
import { mapWithConcurrency } from './concurrency.js';
|
||||
import { getCacheEntry, setCacheEntry, pruneCache } from './cache.js';
|
||||
|
||||
// Caps simultaneous per-article fetches (see the "Wave 3+4" step below) so a
|
||||
// large site doesn't burst hundreds of requests at once. Raise it for a
|
||||
// fast/local nginx, lower it for a slow or shared/production instance — see
|
||||
// .claude/nginx-sitemap-crawler-plan.md §2a for the full rationale.
|
||||
const MAX_CONCURRENT_REQUESTS = 10;
|
||||
|
||||
const MONTH_ABBREVIATIONS = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'];
|
||||
|
||||
// A slug directory's own `<DD-MM-YYYY>.date` marker file (see README).
|
||||
const DATE_FILE_PATTERN = /^(\d{2})-(\d{2})-(\d{4})\.date$/;
|
||||
|
||||
function parseHtmlDirectoryListing(raw) {
|
||||
const entries = [];
|
||||
const seen = new Set();
|
||||
const anchors = /<a\b[^>]*\bhref\s*=\s*["']([^"']+)["'][^>]*>/gi;
|
||||
const anchors = /<a\b[^>]*\bhref\s*=\s*["']([^"']+)["'][^>]*>[\s\S]*?<\/a>/gi;
|
||||
// nginx's default autoindex puts "DD-Mon-YYYY HH:MM" right after the </a>
|
||||
// on the same line — used as a best-effort mtime when JSON autoindex
|
||||
// isn't available (see the "mtime" note in the README).
|
||||
const datePattern = /(\d{2})-([A-Za-z]{3})-(\d{4})\s+(\d{2}):(\d{2})/;
|
||||
let match;
|
||||
while ((match = anchors.exec(raw))) {
|
||||
const href = match[1].replace(/&/g, '&').replace(/"/g, '"').replace(/'/g, "'");
|
||||
@@ -13,7 +30,16 @@ function parseHtmlDirectoryListing(raw) {
|
||||
const name = decodeURIComponent(clean.replace(/\/$/, '')).split('/').pop();
|
||||
if (!name || name === '.' || name === '..' || seen.has(name)) continue;
|
||||
seen.add(name);
|
||||
entries.push({ name, type: clean.endsWith('/') ? 'directory' : 'file' });
|
||||
|
||||
const lineEnd = raw.indexOf('\n', anchors.lastIndex);
|
||||
const tail = raw.slice(anchors.lastIndex, lineEnd === -1 ? raw.length : lineEnd);
|
||||
const dateMatch = datePattern.exec(tail);
|
||||
const monthIndex = dateMatch ? MONTH_ABBREVIATIONS.indexOf(dateMatch[2]) : -1;
|
||||
const mtime = monthIndex >= 0
|
||||
? new Date(Date.UTC(+dateMatch[3], monthIndex, +dateMatch[1], +dateMatch[4], +dateMatch[5])).toISOString()
|
||||
: null;
|
||||
|
||||
entries.push({ name, type: clean.endsWith('/') ? 'directory' : 'file', mtime });
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
@@ -31,12 +57,6 @@ export async function fetchJson(url) {
|
||||
throw new Error(`Expected a directory listing at ${url}`);
|
||||
}
|
||||
|
||||
export function asList(value) {
|
||||
if (Array.isArray(value)) return value.map(String).filter(Boolean);
|
||||
if (value == null || value === '') return [];
|
||||
return String(value).split(',').map(item => item.trim()).filter(Boolean);
|
||||
}
|
||||
|
||||
export function resolveMediaUrl(value, articlePath = '') {
|
||||
if (!value) return '';
|
||||
let path = String(value).trim();
|
||||
@@ -68,6 +88,15 @@ export async function fetchTagRegistry() {
|
||||
}
|
||||
|
||||
let articleCache = null;
|
||||
let crawlStats = { total: 0, loaded: 0 };
|
||||
|
||||
// How many of the last crawl's slug directories actually produced an article
|
||||
// vs. how many were candidates — lets the UI show a "something's badly wrong"
|
||||
// indicator instead of silently rendering an empty/near-empty list forever.
|
||||
export function getCrawlStats() {
|
||||
return { ...crawlStats };
|
||||
}
|
||||
|
||||
function isPublished(meta) {
|
||||
if (meta.status != null) return ['published', 'publish'].includes(String(meta.status).toLowerCase());
|
||||
return !['true', 'yes', '1'].includes(String(meta.draft ?? '').toLowerCase().trim());
|
||||
@@ -85,56 +114,111 @@ function excerpt(content) {
|
||||
return `${text.slice(0, 277).replace(/\s+\S*$/, '')}…`;
|
||||
}
|
||||
|
||||
// A single `<DD-MM-YYYY>.date` marker file is the sole authoritative source
|
||||
// of an article's publish date; a missing/ambiguous one falls back to the
|
||||
// (deprecated) frontmatter `date:` field so older, unmigrated content still
|
||||
// works, resolved by the caller.
|
||||
function parseDateFromFiles(files) {
|
||||
const matches = files.map(file => file.name.match(DATE_FILE_PATTERN)).filter(Boolean);
|
||||
if (matches.length !== 1) return null;
|
||||
const [, day, month, year] = matches[0];
|
||||
return `${year}-${month}-${day}`;
|
||||
}
|
||||
|
||||
// Fetches a slug directory's own listing, resolves its markdown file and
|
||||
// `.tag` marker files, and builds the full article record. Returns null on
|
||||
// any failure (missing/unreadable markdown, network error) — the caller
|
||||
// treats that as a silent skip, matching the rest of this module's approach.
|
||||
async function buildArticle(dir) {
|
||||
try {
|
||||
const entries = await fetchJson(dir.root);
|
||||
const files = entries.filter(entry => entry.type === 'file');
|
||||
const tagIds = files.filter(entry => entry.name.endsWith('.tag')).map(entry => entry.name.replace(/\.tag$/, ''));
|
||||
const mdFiles = files.filter(entry => entry.name.endsWith('.md') && !entry.name.startsWith('_'));
|
||||
const mdFile = mdFiles.find(entry => entry.name === `${dir.slug}.md`)
|
||||
|| mdFiles.find(entry => entry.name === 'index.md')
|
||||
|| (mdFiles.length === 1 ? mdFiles[0] : null);
|
||||
if (!mdFile) return null;
|
||||
|
||||
const path = `${dir.root}${encodeURIComponent(mdFile.name)}`;
|
||||
const response = await fetch(path);
|
||||
if (!response.ok) throw new Error(`HTTP ${response.status}`);
|
||||
const raw = await response.text();
|
||||
const { meta, content } = parseFrontmatter(raw);
|
||||
const words = content.trim().split(/\s+/).filter(Boolean).length;
|
||||
// "Category" isn't a separate concept anymore — it's just whichever
|
||||
// tag id (if any) looks like an expedition year.
|
||||
const category = tagIds.find(id => /^expa-20\d{2}$/i.test(id)) || null;
|
||||
|
||||
return {
|
||||
...meta,
|
||||
author: dir.author,
|
||||
category,
|
||||
expa: category ? Number(category.replace(/^expa-/i, '')) : null,
|
||||
slug: dir.slug,
|
||||
path,
|
||||
hash: `${dir.author}/${dir.slug}`,
|
||||
date: parseDateFromFiles(files) || meta.date || null,
|
||||
readingTime: Math.max(1, Math.round(words / 200)),
|
||||
description: meta.description || excerpt(content),
|
||||
tags: tagIds,
|
||||
coverUrl: resolveMediaUrl(meta.cover || meta.image, path),
|
||||
published: isPublished(meta),
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export async function discoverArticles() {
|
||||
if (articleCache) return articleCache;
|
||||
|
||||
// Wave 1 (root) — always fetched live: the author directories.
|
||||
const authorEntries = await fetchJson(`${BASE_URL}articles/`);
|
||||
const authors = authorEntries.filter(entry => entry.type === 'directory' && !entry.name.startsWith('.'));
|
||||
const categoryLists = await Promise.all(authors.map(async author => {
|
||||
const entries = await fetchJson(`${BASE_URL}articles/${encodeURIComponent(author.name)}/`);
|
||||
const directories = entries.filter(entry => entry.type === 'directory' && entry.name !== '_history');
|
||||
const years = directories.filter(entry => /^expa-20\d{2}$/i.test(entry.name));
|
||||
return (years.length ? years : directories).map(category => ({ author: author.name, category: category.name }));
|
||||
}));
|
||||
|
||||
const sources = (await Promise.all(categoryLists.flat().map(async ({ author, category }) => {
|
||||
const root = `${BASE_URL}articles/${encodeURIComponent(author)}/${encodeURIComponent(category)}/`;
|
||||
const entries = await fetchJson(root);
|
||||
const direct = entries.filter(entry => entry.type === 'file' && entry.name.endsWith('.md'))
|
||||
.map(entry => ({ author, category, slug: entry.name.replace(/\.md$/, ''), name: entry.name, root }));
|
||||
const nested = await Promise.all(entries.filter(entry => entry.type === 'directory' && entry.name !== '_history').map(async post => {
|
||||
const postRoot = `${root}${encodeURIComponent(post.name)}/`;
|
||||
const files = (await fetchJson(postRoot)).filter(entry => entry.type === 'file' && entry.name.endsWith('.md') && !entry.name.startsWith('_'));
|
||||
const selected = files.find(entry => entry.name === 'index.md') || files.find(entry => entry.name === `${post.name}.md`) || (files.length === 1 ? files[0] : null);
|
||||
return selected ? { author, category, slug: post.name, name: selected.name, root: postRoot } : null;
|
||||
}));
|
||||
return [...direct, ...nested.filter(Boolean)];
|
||||
}))).flat();
|
||||
|
||||
const articles = await Promise.all(sources.map(async source => {
|
||||
const path = `${source.root}${encodeURIComponent(source.name)}`;
|
||||
try {
|
||||
const response = await fetch(path);
|
||||
if (!response.ok) throw new Error(`HTTP ${response.status}`);
|
||||
const raw = await response.text();
|
||||
const { meta, content } = parseFrontmatter(raw);
|
||||
const words = content.trim().split(/\s+/).filter(Boolean).length;
|
||||
return {
|
||||
...meta,
|
||||
author: source.author,
|
||||
category: source.category,
|
||||
expa: Number(meta.expa || source.category.replace(/^expa-/i, '')) || null,
|
||||
slug: source.slug,
|
||||
path,
|
||||
hash: `${source.author}/${source.category}/${source.slug}`,
|
||||
readingTime: Math.max(1, Math.round(words / 200)),
|
||||
description: meta.description || excerpt(content),
|
||||
tags: asList(meta.tags),
|
||||
coverUrl: resolveMediaUrl(meta.cover || meta.image, path),
|
||||
published: isPublished(meta),
|
||||
};
|
||||
} catch { return null; }
|
||||
// Wave 2 — always fetched live, in full, for every author (not just the
|
||||
// current page): each author's slug directories, with their mtimes. This
|
||||
// is what builds the flat, cross-author list that drives global sorting
|
||||
// and pagination — see the plan doc's "keep the flat feed" decision.
|
||||
const slugDirLists = await Promise.all(authors.map(async author => {
|
||||
const authorRoot = `${BASE_URL}articles/${encodeURIComponent(author.name)}/`;
|
||||
const entries = await fetchJson(authorRoot);
|
||||
return entries
|
||||
.filter(entry => entry.type === 'directory' && entry.name !== '_history')
|
||||
.map(entry => ({
|
||||
author: author.name,
|
||||
slug: entry.name,
|
||||
mtime: entry.mtime ?? null,
|
||||
root: `${authorRoot}${encodeURIComponent(entry.name)}/`,
|
||||
}));
|
||||
}));
|
||||
articleCache = articles.filter(article => article?.published).sort((a, b) => String(b.date || '').localeCompare(String(a.date || '')));
|
||||
const slugDirs = slugDirLists.flat();
|
||||
|
||||
// Wave 3+4 — per slug directory, cache-aware and concurrency-limited: a
|
||||
// directory whose mtime matches its cached entry skips both the listing
|
||||
// fetch (for .tag/.date files) and the markdown content fetch entirely.
|
||||
let loaded = 0;
|
||||
const articles = await mapWithConcurrency(slugDirs, MAX_CONCURRENT_REQUESTS, async dir => {
|
||||
const cached = await getCacheEntry(dir.root);
|
||||
if (cached && dir.mtime != null && cached.mtime === dir.mtime) {
|
||||
if (cached.article) loaded += 1;
|
||||
return cached.article;
|
||||
}
|
||||
const article = await buildArticle(dir);
|
||||
if (article) {
|
||||
loaded += 1;
|
||||
await setCacheEntry(dir.root, { mtime: dir.mtime, article });
|
||||
}
|
||||
return article;
|
||||
});
|
||||
|
||||
await pruneCache(slugDirs.map(dir => dir.root));
|
||||
crawlStats = { total: slugDirs.length, loaded };
|
||||
|
||||
articleCache = articles
|
||||
.filter(article => article?.published && article.date)
|
||||
.sort((a, b) => String(b.date).localeCompare(String(a.date)) || a.hash.localeCompare(b.hash));
|
||||
return articleCache;
|
||||
}
|
||||
|
||||
|
||||
+3
-3
@@ -6,9 +6,10 @@
|
||||
// 2 segments, first = "page" → all articles, page N (#page/2)
|
||||
// 2 segments, first = "author" → author filter, page 1 (#author/alice)
|
||||
// 2 segments, first = "category" or "tag" → filtered list, page 1
|
||||
// 3 segments → single article (#author/category/slug)
|
||||
// 3 segments, first = "post" → single article (#post/alice/my-article)
|
||||
// 4 segments, first = "author", third = "page" → author filter, page N (#author/alice/page/2)
|
||||
// 4 segments, first = "category", third = "page" → category filter, page N (#category/linux/page/2)
|
||||
// 4 segments, first = "tag", third = "page" → tag filter, page N (#tag/kometa/page/2)
|
||||
|
||||
import { MARKED_URL } from './config.js';
|
||||
import { loadTemplates } from './templates.js';
|
||||
@@ -42,11 +43,10 @@ function router() {
|
||||
else if (parts.length === 2 && parts[0] === 'author') loadAuthorView(decodeURIComponent(parts[1]));
|
||||
else if (parts.length === 2 && parts[0] === 'category') loadCategoryView(decodeURIComponent(parts[1]));
|
||||
else if (parts.length === 2 && parts[0] === 'tag') loadTagView(decodeURIComponent(parts[1]));
|
||||
else if (parts.length === 3) loadArticle(parts.map(decodeURIComponent).join('/'));
|
||||
else if (parts.length === 3 && parts[0] === 'post') loadArticle(parts.slice(1).map(decodeURIComponent).join('/'));
|
||||
else if (parts.length === 4 && parts[0] === 'author' && parts[2] === 'page') loadAuthorView(decodeURIComponent(parts[1]), parsePage(parts[3]));
|
||||
else if (parts.length === 4 && parts[0] === 'category' && parts[2] === 'page') loadCategoryView(decodeURIComponent(parts[1]), parsePage(parts[3]));
|
||||
else if (parts.length === 4 && parts[0] === 'tag' && parts[2] === 'page') loadTagView(decodeURIComponent(parts[1]), parsePage(parts[3]));
|
||||
else if (parts.length === 4) loadArticle(parts.map(decodeURIComponent).join('/'));
|
||||
else loadArticleList();
|
||||
}
|
||||
|
||||
|
||||
+29
-8
@@ -1,5 +1,5 @@
|
||||
import { templates } from './templates.js';
|
||||
import { discoverArticles, fetchAuthorProfile, fetchTagRegistry, findArticle, resolveMediaUrl } from './discovery.js';
|
||||
import { discoverArticles, fetchAuthorProfile, fetchTagRegistry, findArticle, getCrawlStats, resolveMediaUrl } from './discovery.js';
|
||||
import { parseFrontmatter } from './frontmatter.js';
|
||||
import { PICTURES_URL } from './config.js';
|
||||
|
||||
@@ -141,7 +141,7 @@ function renderCard(article) {
|
||||
image.src = value;
|
||||
image.alt = article.title || article.slug;
|
||||
if (imageLink) {
|
||||
imageLink.href = `#${article.hash}`;
|
||||
imageLink.href = `#post/${article.hash}`;
|
||||
imageLink.removeAttribute('aria-hidden');
|
||||
imageLink.removeAttribute('tabindex');
|
||||
}
|
||||
@@ -150,7 +150,7 @@ function renderCard(article) {
|
||||
imageLink?.remove();
|
||||
element.classList.add('card--no-image');
|
||||
}
|
||||
field('link').href = `#${article.hash}`;
|
||||
field('link').href = `#post/${article.hash}`;
|
||||
field('title').textContent = article.title || article.slug;
|
||||
setField(field('date'), article.date, (time, value) => { time.dateTime = String(value); time.textContent = czechDate(value); });
|
||||
setField(field('description'), article.description, (description, value) => { description.textContent = value; });
|
||||
@@ -158,13 +158,28 @@ function renderCard(article) {
|
||||
const author = field('author');
|
||||
author.textContent = article.authorName || article.author;
|
||||
author.href = `#author/${encodeURIComponent(article.author)}`;
|
||||
const category = field('category');
|
||||
category.textContent = `Expa ${article.expa || article.category.replace(/^expa-/i, '')}`;
|
||||
category.href = `#category/${encodeURIComponent(article.category)}`;
|
||||
// A slug directory only counts as a "category" when one of its .tag
|
||||
// markers happens to name an expedition year — not every article has one.
|
||||
setField(field('category'), article.category, (category, value) => {
|
||||
category.textContent = `Expa ${article.expa}`;
|
||||
category.href = `#category/${encodeURIComponent(value)}`;
|
||||
});
|
||||
renderTags(field('tags'), article.tags);
|
||||
return element;
|
||||
}
|
||||
|
||||
// Near-total crawl failure (wrong origin after a migration, CORS broken,
|
||||
// autoindex disabled) should be visible rather than silently rendering an
|
||||
// empty/near-empty list forever — see .claude/nginx-sitemap-crawler-plan.md L3.
|
||||
function renderCrawlHealthWarning(view) {
|
||||
const { total, loaded } = getCrawlStats();
|
||||
if (total < 5 || loaded / total >= 0.1) return;
|
||||
const warning = document.createElement('p');
|
||||
warning.className = 'error';
|
||||
warning.textContent = `Pozor: načtena jen ${loaded} z ${total} položek — zkontrolujte nastavení serveru.`;
|
||||
view.appendChild(warning);
|
||||
}
|
||||
|
||||
export async function loadArticleList(filter = null, page = 1) {
|
||||
showListView();
|
||||
window.scrollTo(0, 0);
|
||||
@@ -181,6 +196,7 @@ export async function loadArticleList(filter = null, page = 1) {
|
||||
return String(article[filter.type] || '').toLowerCase() === filter.value.toLowerCase();
|
||||
});
|
||||
view.innerHTML = '';
|
||||
renderCrawlHealthWarning(view);
|
||||
if (filter) {
|
||||
const heading = document.createElement('h2');
|
||||
heading.className = 'category-heading';
|
||||
@@ -289,13 +305,18 @@ export async function loadArticle(hash) {
|
||||
const element = document.importNode(templates.article, true);
|
||||
const field = name => element.querySelector(`[data-field="${name}"]`);
|
||||
field('title').textContent = meta.title || article.slug;
|
||||
setField(field('date'), meta.date, (time, value) => { time.dateTime = String(value); time.textContent = czechDate(value); });
|
||||
// Publish date comes from the slug directory's `.date` marker file
|
||||
// (already resolved onto `article.date`), not the raw frontmatter.
|
||||
setField(field('date'), article.date, (time, value) => { time.dateTime = String(value); time.textContent = czechDate(value); });
|
||||
setField(field('description'), meta.description, (description, value) => { description.textContent = value; });
|
||||
setField(field('reading-time'), article.readingTime, (time, value) => { time.textContent = `${value} min čtení`; });
|
||||
const image = field('image');
|
||||
if (article.coverUrl) { image.src = article.coverUrl; image.alt = meta.title || article.slug; } else image.closest('figure')?.remove();
|
||||
const author = field('author'); author.textContent = meta.authorName || article.author; author.href = `#author/${encodeURIComponent(article.author)}`;
|
||||
const category = field('category'); category.textContent = `Expa ${article.expa}`; category.href = `#category/${encodeURIComponent(article.category)}`;
|
||||
setField(field('category'), article.category, (category, value) => {
|
||||
category.textContent = `Expa ${article.expa}`;
|
||||
category.href = `#category/${encodeURIComponent(value)}`;
|
||||
});
|
||||
renderTags(field('tags'), article.tags);
|
||||
const contentElement = field('content');
|
||||
contentElement.innerHTML = marked.parse(content);
|
||||
|
||||
Reference in new issue
Block a user