Get sources in

A source is anything you want a note of. Every way in ends in the same place: the queue, which sb run works through (see Manage the queue and old notes). The basics are links and PDFs; everything else is optional.

Source	How	Notes
Web page	sb add URL, or clip it with Obsidian Web Clipper into inbox/	Clipping is best for pages behind a login
PDF or paper	Drop it in inbox/, or sb add path.pdf	A PDF with no text layer (a scan) needs OCR; without it, it is rejected with a message saying so
A folder of PDFs	sb add folder/*.pdf	Recipe
Screenshot, photo of a slide, scanned page	Drop the image in inbox/, or sb add image.png	Needs OCR (optional): png, jpg, webp, tiff
LinkedIn, X, Reddit, YouTube	Clip with Web Clipper	Never scraped by the worker; the clipped text is the source
Email	Forward to your +esbi-cli address (Email)	The mail body is the source; PDF and image attachments come with it. Links inside are followed only if you turn on [email].follow_links
Old vault	sb import-legacy (once)	Reads its Links/ and PDFs/

A Markdown clip is ingested from its own body; its source: (or url:) frontmatter becomes the source URL, title: its title and kind: (article, paper, email or video) its kind. Every URL the worker fetches itself is checked first: only http(s), only public addresses, on every redirect.

Add a link

sb add https://en.wikipedia.org/wiki/Zettelkasten

Queued 1 (0 already known). `sb run` processes them.

Nothing is downloaded yet. Add as many as you like in one command: sb add URL1 URL2 URL3. Put a link that has ? or & in quotes, or the shell reads it: sb add "https://example.com/post?id=7". Then:

sb run

To do one right now, without the queue, use sb ingest URL (add --dry-run to print the model's plan and write nothing). A page that needs a login, or is mostly scripts, fails with No readable article content found: clip it with the Web Clipper instead. A GitHub repository link is read through its README.

Add a PDF

Either copy it into the vault's inbox/ folder and scan:

cp ~/Downloads/paper.pdf ~/Documents/Obsidian/esbi/inbox/

sb scan

Queued 1 sources from the inbox.

or give its path to sb add, which leaves the file where it is:

sb add ~/Downloads/paper.pdf

The difference: a file in inbox/ is moved to raw/inbox/, which keeps it for good. A file added by path is read from where it is when the run starts, so do not move it before then. Either way a copy of the PDF is saved in raw/ when the note is written. sb run scans inbox/ itself, so sb scan is only for seeing the queue early.

How the text of a PDF is read. The text is rebuilt from the characters in the PDF's text layer (with PDFium, the PDF engine of Chrome), as plain Markdown: lines are joined into paragraphs, a hyphen at the end of a line is dropped inside a word and kept in a compound (feed-forward), running headers and page numbers are dropped, and headings are found by font size, weight and numbering (## 1 Introduction, ### 3.1 Method; a file keeps at most 8 per page, at least 40, the numbered and the biggest first). Three things to know:

• Tables are not Markdown tables. Each row stays on one line with its numbers intact (Base 65 27.3 3.3), so the model can read the values but the note does not show a grid.
• Text inside images and charts is not read. Text printed over a picture, and the labels of a chart (a label that the file repeats 8 times or more, and short lines packed into one spot of a page), are left out of the text (a scanned PDF is the exception: with OCR its pages are read as images).
• Figures are the embedded pictures and, when a drawing has a Figure N caption next to it, that vector drawing too (diagrams and flow charts); the panels under one caption count as one figure; at most 8 per PDF, rendered at a bounded size.

A PDF that needs a password is rejected with Could not open PDF: .... The notes keep emphasis out of the text (no ** or _x_), which is why a quote copied from it matches the PDF word for word more often. Why it is built this way, and what it cannot do: How a note is made.

How a PDF's note gets its title. The title stored in the PDF (its metadata Title) is used only when the first two pages show it, because a slide deck often keeps the title of the template it was made from. Otherwise the note is named after the file: _ reads as a space, so does - in a name with no spaces (spacing-effect.pdf), extra spaces are dropped, and the .pptx, .docx, .key or .odp that an export leaves (Slides.pptx.pdf) is removed. Notes made before 0.4.0 keep their names: rename one in Obsidian, which updates the links to it.

Add a folder of PDFs

Let the shell list the files:

sb add ~/Papers/*.pdf

Queued 2 (0 already known). `sb run` processes them.

If the PDFs are in subfolders, copy them all into inbox/ first:

find ~/Papers -name '*.pdf' -exec cp {} ~/Documents/Obsidian/esbi/inbox/ \;

Two different files with the same name would overwrite each other during the copy (the worker only protects files after they reach inbox/). The worker skips a PDF it already has (same bytes, even under another name) and keeps a different PDF with the same name as name (2).pdf. A big folder takes a long time with a local model: sb run --limit 5 processes five at a time, and the nightly job does batches of 20 (max_sources_per_run), up to 6 a day (max_batches_per_day).

Add a Markdown clip

A clip is a Markdown file with title: and source: (the page URL) at the top. The Web Clipper makes them; so can any tool that saves a page as Markdown.

---
title: Notes on the spacing effect
source: https://example.org/spacing
kind: article
---

The spacing effect says that learning is better when study sessions are spread out...

Drop it in inbox/ (or sb add clip.md). The text of the file is the source; the page is not fetched again. A clip with fewer than 40 characters of text is rejected as empty. Setting the Clipper up: Web Clipper and YouTube.

Page chrome is removed first. A clip of a GitHub, LinkedIn, X (Twitter), Reddit or YouTube page carries the site's own furniture along with the text: menus, "Sign in", cookie and consent lines, GitHub's repository sidebar (Languages, Releases, Contributors), reaction and share bars, "People also viewed", related-post lists, footers. The worker removes the lines it knows to be that furniture before the model reads the clip, so the note is about the article and not about the menu. The rule is conservative on purpose:

• Only a whole line goes, and only when the whole line is known chrome; a sentence that merely contains such words is kept, and so are code blocks, plain bullets and YouTube transcript rows.
• The site is known from the clip's source: address. Without one, the page's own footer text identifies it. A clip from any other site is left exactly as it is.
• The original clip stays in raw/inbox/ untouched. A clip dropped in inbox/ is moved there; a clip you add by path from anywhere else (sb add clip.md, sb ingest clip.md) is copied there, unchanged, when its note is written (a taken name becomes name (2).md: nothing in raw/ is overwritten, and a copy that is already there is not made again). The snapshot raw/<date>-<slug>.md, which is what the note was made from, holds the stripped text.
• The note records how many lines went as stripped_lines: N in its frontmatter (only when N is more than 0), and the run prints a warning: Removed N lines of page chrome from the clip.
• The 40-character minimum is applied after stripping, so a clip that was only chrome (a login wall, a cookie page) is rejected with has almost no text.

Add a YouTube video

Clip the video page with the Web Clipper's YouTube template: it needs a video that has a transcript, and it is described in Web Clipper and YouTube. The note is written from the transcript, and quotes and glossary terms get a link that opens the video at that second. The worker never downloads video or audio.

Without the Clipper, a file with kind: video, source: set to the video's URL and a transcript with one line per row, each written as **2:20** · text of the line, is read the same way.

Add an image or a scanned PDF

This needs OCR turned on once (below). Then it is the same as a PDF:

sb add ~/Pictures/slide.png

Queued 1 (0 already known). `sb run` processes them.

Without OCR, sb add says why instead of queueing:

error: not an http(s) URL or an existing .pdf/.md file:
  ~/Pictures/slide.png
  Images: reading images and scanned PDFs is off: `sb ocr enable` sets up a local vision model ([llm.ocr], https://rubenamaury.github.io/esbi-cli/docs/reference/configuration/)

Add an email

Set up the mailbox once (sb setup email); after that, forward anything to your you+esbi-cli@gmail.com address and the next sb run makes a note. See Capture email.

Import an older vault

If you have an older folder of links and PDFs (Links/DD-MM-YYYY.md with one [url] per line, and PDFs/), set legacy_vault under [paths] in the config and run it once:

sb import-legacy

Queued 5 sources (0 already known).

It never changes the old folder, and is safe to repeat.

Images and scanned PDFs (OCR)

Off by default, and a choice you can make, see and change. See where you stand:

sb ocr status

Turn it on, and have the model downloaded now:

sb ocr enable --pull

(sb init asks the same question the first time, default No; sb init --ocr [--ocr-model NAME] [--pull-ocr-model] does it without questions. Nothing is downloaded without --pull, a typed yes, or ollama pull by hand.) The recommended model is qwen3-vl:2b-instruct:

ollama pull qwen3-vl:2b-instruct     # 1.9 GB download, about 3.4 GB in memory while it reads

sb ocr disable turns it off and keeps the model choice (the same as enabled = false under [llm.ocr]). deepseek-ocr:3b is a choice for machines with 16 GB of memory or more; on an 8 GB Mac it is far too slow: see Choosing the OCR model.

A vision model served by Ollama on your machine reads the text printed in the image, and that text becomes the source: the note is made like any other (summary, glossary, quotes checked against the text). The original image is kept in raw/ and shown in the note under "Figures".

• What it reads well: screenshots, slides, and pages of text, in English and Spanish, including a photo taken at a slight angle with some noise. Measured numbers are in Choosing the OCR model.
• What it cannot do: a photo or a diagram with no text has nothing to read. The source then fails with no readable text, and the reason is kept in the queue (sb status); no empty note is written. It does not describe pictures, because a description is the model's guess and none of it can be checked against the source. It does not read handwriting reliably, and it can drop or swap a word in dense small print: check anything that matters against the original.
• Scanned PDFs: a PDF whose pages have no text layer is rendered page by page and read the same way. At most [run].ocr_max_pages pages (default 10) are read; when a PDF is longer the run says how many were read.
• Time: expect from a few seconds (a screenshot) to under a minute per page on a small Mac, since the reading model and the writing model take turns in Ollama.
• Private by construction: the OCR model must be an Ollama model running on this machine ([llm.ocr] refuses anything else), so an image, and the text read from it, is never sent to a cloud model. If a scanned PDF or an image came attached to an email, the text read from it is treated as email (only your local [llm.private] model writes it).
• Not supported (reported by sb scan and sb run, never ignored): HEIC (convert it to JPG first), Word, and any other file type. They stay in inbox/.

If a source already exists

The worker never ingests the same thing twice, and never destroys an earlier copy.

You add	What happens
The same link again (tracking junk, case or trailing slash differ)	sb add says already known; nothing is queued twice
A link that is already a note in the wiki	sb add skips it and names the page (already in the wiki as [[...]])
A link already ingested, but never seen by sb add (for example through sb ingest)	Queued, then skipped at run time with no model call (skipped); the page is fetched once more first
The same PDF or clip dropped in inbox/ again, even renamed	Recognised by its bytes, removed from the inbox, reported as a duplicate
A different PDF with the same file name	Both kept: the new one is stored as name (2).pdf and processed
The same text under another path or URL	Caught when ingested, by a hash of its text: skipped
A page whose content changed since you ingested it	Skipped (same URL). To redo it: sb ingest URL --force
A link you deleted from the wiki and want again	The queue still remembers it as done: sb drop URL, then sb add URL
The same email again	Recognised by its Message-ID and marked seen, not saved twice