Skip to content

How a note is made

A source note is written so that you can skip the original: read the note, know what the source says, decide whether it deserves your time.

What a note looks like

You save a link or a paper. A few minutes later there is a note like this (abridged, from a Wikipedia article, written by llama3.2 on a laptop):

## Executive summary
The Zettelkasten is a method of note-taking and knowledge management developed by Niklas Luhmann,
which involves creating a network of interconnected notes and ideas. It emphasizes the importance of
linking and connecting ideas, rather than simply collecting or storing them.

## Key ideas
- **The Zettelkasten system emphasizes the use of index cards to organize and connect ideas.** This
  approach allows users to create a network of interconnected notes, rather than a linear sequence of thoughts.

## Key terms
- **Zettelkasten**: A system of note-taking and personal knowledge management that consists of small
  items of information stored on paper slips or cards.

## Key quotes
> "Every one admits nowadays that it is advisable to collect materials on separate cards or slips of paper."

## Concepts
- [[Index card file]]

The language is a setting, [notes].language (default en). With es the same kind of note reads like this (abridged, from a paper on the Transformer, written by an earlier version of the pipeline):

## Resumen ejecutivo
El artículo describe el modelo Transformer, que usa mecanismos de atención para la
traducción de secuencias y supera a los modelos publicados en calidad y coste de entrenamiento.

## Ideas clave
- **El Transformer se basa exclusivamente en atención.** Es una alternativa a los modelos
  recurrentes y convolucionales, dominantes hasta entonces.

## Términos clave
- **Self-attention**: Un mecanismo que relaciona posiciones de una secuencia para calcular su representación.

## Frases clave
> "We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely"

Quotes always stay in the language of the source. Section headings, the daily index and the other text the program writes follow the setting; see Language of the notes.

Quotes are checked against the source word for word, glossary terms must appear in it, diagrams are drawn by code, and PDF figures are copied next to the note. The model never touches your files: it only returns JSON that the program validates and then writes.

How it is made

Images and scanned PDFs first. With [llm.ocr] set, a vision model reads the text printed in an image (or in each page of a PDF that has no text layer, up to ocr_max_pages), and that text is the source from here on: everything below, including the checks that quotes appear verbatim, is against what the OCR model read. The picture itself is kept in raw/ and shown in the note.

The language. The instructions to the model are in English for every output language: they name the language to write in (Write ALL text in Spanish) and the request for the note repeats it after the source, where a small model listens. If the answer comes back in the wrong language the worker asks once more, and keeps the second answer with a warning if it is still wrong. A language the program has no word list for is never judged.

The steps:

  1. The fence. Every prompt that carries a source puts it between tags (<source>, <chunk>...), and first makes any look-alike tag or chat marker inside the source inert (fence.py), so a source cannot close the tag and start giving orders in the prompt's own voice. The title and URL that go into an instruction are quoted for the same reason.
  2. Reading. A source that fits in max_source_chars goes to the model whole. A longer one (most papers) is cut into chunks at paragraph boundaries; the references section is dropped. The reading model (llm.summarize) takes notes on every chunk, however many there are: points, defined terms, verbatim quotes, and relations between concepts. A chunk it cannot read is read again in two halves, which a small model often manages; only a chunk that stays unreadable is skipped, with a warning, not a failure. A source of more chunks than max_chunks has the notes of each group of five chunks merged into a section note, level by level, until max_chunks or fewer remain, and the next step works from those (see Long sources). A cap on the model calls for one source (max_calls_per_source) keeps the cost bounded; when it stops the reading early the note says so (below).
  3. Synthesis. The synthesis model (llm.synthesize, or the same one) receives the notes, not the raw text (a source short enough to go whole is written in one call from its text), and writes the digest below. For a chunked source the detailed summary is a call of its own, asked for as 3 to 4 short paragraphs and 4 to 6 key ideas: a list is something a small model can be held to, where one long string ran on until it hit its limit.
  4. Connections. A separate call shows the model the pages you already have (with their one-line summaries) and asks how the new source relates to them, with the reason. It is a separate call on purpose: showing a small model existing pages while it summarizes makes it copy their wording into the new note.
  5. Scrub, checks, then writing. First everything the model wrote is scrubbed as plain prose (scrub.py): no HTML, no image syntax, no control characters; a link or address stays only if the source contains it; a [[link]] stays only if the page exists and a note on this side of the email line may point at it. The run says how many addresses were removed. Then the worker verifies before it writes: quotes must appear verbatim in the source, glossary terms and entities must appear in the source text, invented links are dropped, glossary trivia (duplicates, generic words such as "data", entries that say they have no definition, a lone letter) is dropped, and the diagram is drawn by code from the extracted relations (so it is always valid Mermaid), keeping only relations whose two ends appear in the source or in the note.
  6. Later, on the concept pages. Once a concept or entity page has several sources, a short ## Summary on top of its per-source sections can be written from them: see Consolidated summaries. It is a separate step, off in the nightly run by default.

What the note contains (sections with nothing to say are left out; the table gives the English heading and, in brackets, the Spanish one):

Section What it is
Not read (a warning callout under the title, only when it applies) > [!warning] Not read: 12 of 52 parts of the source (the last 23% of the text, starting at "...") because the limit of 60 model calls per source ([run].max_calls_per_source) was reached. (No leído: ... in a Spanish note). Present only when the call cap stopped the reading before the end of a very long source: how many parts and what share of the text were left out, and where they start
Executive summary (Resumen ejecutivo) The 2-3 sentence answer to "what is this and why does it matter"
Detailed summary (Resumen detallado) The abstract, in 3 to 5 paragraphs (3 to 4 for a chunked source): the argument in order, with the data that supports it
Key ideas (Ideas clave) 4 to 8 ideas (4 to 6 for a chunked source), each with what follows from it (or plain key points if the model gave none)
Key terms (Términos clave) A glossary of terms the source defines, linked to your concept pages
Key quotes (Frases clave) Verbatim quotes worth keeping
Diagram (Diagrama) A Mermaid concept map of the relations found
Figures (Figuras) PDF figures (up to 8, captioned ones first) copied into attachments/; for web pages, images are linked, not downloaded
Connections to your wiki (Conexiones con tu wiki) Pages you already have this relates to, with the relation and why
Open questions (Preguntas abiertas) What the source leaves unanswered
Concepts / Entities / Related (Conceptos, Entidades, Relacionado) Links to the pages the note created or updated

PDFs

A PDF has no paragraphs, headings or tables of its own: it is characters placed on pages, each with a font size and weight. The worker (extract/pdf_text.py, on PDFium) turns them into plain Markdown the model can read:

  • Lines are reflowed into paragraphs by the gaps between lines and by indents. A hyphen at the end of a line is dropped inside a word (en-ables becomes enables) and kept in a compound (feed-forward). The test uses the document's own vocabulary, but not the halves of words that a line break cut (plicative is no word of the paper, so it cannot make multi look like one). The hyphen goes when the joined word is a word of the document, or a word of it with an ending (leverage gives leveraged, omit gives omitted); it stays when the first half is a short joining word (of, in, non...) or a word of the document, usually a long one (sequence-aligned, mixture-of-experts).
  • Running headers, footers and page numbers (a line repeated in the top or bottom margin of at least three pages) are dropped. A References heading is kept as a heading, which is what lets the reader drop the references section later.
  • Headings are found by font size, weight and numbering: a short line (at most 100 characters, not ending in a comma, period or semicolon) that is at least 12% bigger than the body text, or that is bold and starts with a number or one of A., Abstract, Appendix. Numbered ones take their level from the number (1 Introduction is ##, 3.1 Method is ###); the others rank by size. A document can have only so many headings: the cap is the larger of 40 and 8 per page, and when a file has more heading-like lines than that (the labels of a chart set in big type), the numbered ones and the biggest ones are kept and the rest become plain text. Before, more than 200 such lines switched heading detection off for the whole file.
  • Tables are not Markdown tables. A line that is mostly numbers stays one row per line with its numbers intact (Base 65 27.3 3.3); the worker does not guess columns.
  • Text inside images and charts is dropped: text over a picture, and three kinds of chart text. A run of at least 12 label-like lines (axes, legends, bar values) goes. A short line that the document repeats 8 times or more goes too: prose does not repeat a line, and a figure drawn in layers can repeat its labels by the hundred (one test paper had its labels printed 80 times and, before this rule, came out with more than twice the text it has). And short lines packed into a small region of a page (40 or more within about 150 points) are the text of a chart, not of a page, so they go, with the neighbouring cells. A symbol inside a line of prose stays: the V of where V is, or a superscript on the row of its text, is not a label, even when the paper repeats it. Emphasis markers (**, _x_) are not written, so a quote the model copies matches the PDF word for word more often.
  • Figures (extract/pdf_figures.py) are embedded images of a reasonable size, and also a vector drawing (a diagram, a flow chart) that has a Figure N caption next to it. The panels under one caption are one figure: parts up to 120 points apart join, for drawings and for pictures alike, so the left panel of a two-panel figure or the top row of a six-panel one is no longer lost. A table of numbers is not a panel, a figure keeps out of the column of another caption beside it (a table drawn next to a figure, and its text, are not taken into the figure), and the caption of another figure ends the search upwards, so two stacked figures stay two. A PDF printed several pages to a sheet (2 pages per sheet, a handout, a booklet) places each page as a form, and its pictures and drawings are found where the sheet puts them (two identical copies are kept once). At most 8 per PDF, captioned ones first, rendered at 150 dpi and no more than 3,000 pixels a side. A page with more than 20,000 drawing operations, counting those inside forms, gets no figures: it would take minutes to render, and a hostile PDF could keep a run busy that way. A scanned page is rendered at 130 dpi for OCR.

Compared with the engine used before (PyMuPDF with a layout model), this one needs about 8 MB instead of 214 MB, reads a paper of 15 to 50 pages in about a second or less instead of 8 to 60 seconds, and uses about a third of the memory (measured on 10 papers of 8 to 51 pages). The price is the table and chart behaviour above. A note made from an older version is rebuilt from the new text with sb reingest (the PDF is read again from raw/).

Known limits of the PDF reader: the thresholds above (8 repeats, 40 short lines, 120 points, the heading cap) were tuned on a small corpus of test papers, about ten, so a layout unlike them can still go wrong in either direction. The hyphen rule is a heuristic on the document's own vocabulary: a compound whose first half is short and whose second half is not a word of the paper (gpt-researcher) can be joined into one word, and a broken word whose long first half is itself a word keeps its hyphen (multi-plicative).

Known limits of notes in general: a source that needs more than about 45 chunks is read only as far as max_calls_per_source allows, and the note discloses the unread tail (raise the cap for books; a long source costs about two to three times the tokens and time it did when it was sampled); a page that is mostly a list of links (a GitHub README of links) has little to summarize; quality tracks the synthesis model.