Readable by agents and people alike

Agents are readers now. A Papyrus site is built so that a person with a browser and an agent with an HTTP client get the same published content, each in the form that suits it. This page says what ships today, shows where to see it, and says plainly what does not exist yet.

Open to reading agents by default

A Papyrus site's entire published content is open to reading agents unless you decide otherwise. In production, robots.txt allows every crawler, AI crawlers included, and points to the sitemap. Papyrus blocks no crawler by default; a site that wants to can add lines with robots.extra. Staging and review copies are the opposite: closed to every crawler and marked noindex.

Every page has a Markdown twin

Each published page is also written as a Markdown file beside the HTML: x.html has x.md, x/index.html has x/index.md, and the home page has index.md. An article's twin carries its title, summary, authors, dates and canonical URL, then its text. Listings and archives are Markdown link lists that point at the other twins. The HTML head advertises its twin with <link rel="alternate" type="text/markdown">, and the file is served as text/markdown; charset=utf-8.

Only published pages get one. Pages marked noindex and the 404 page do not.

The twin is Markus source

Since Papyrus 5.19.0 the twin is the page's Markus source as it was written, not a flattened copy. A Markus directive stays in the file with all its attributes: an ::image{src="..." alt="..." layout="full" caption="..."} keeps its layout and caption, and a marketing page keeps its ::band, ::column and ::button lines. Markus is a superset of Markdown, so the file is still plain text that any reader or agent can use, and it keeps the extension .md and the text/markdown type. An interactive section (an island, which is a chart or a carousel that needs JavaScript) carries a summary text: the HTML shows it where the island will appear until the script loads, and the twin keeps it on the ::island line, so a reader without JavaScript and an agent both get the point of the section.

Every Markdown output is Markus source

Wherever a Papyrus site serves or copies Markdown, what you get is the Markus source as written, with every directive intact. These outputs all do:

• the .md twin of every article, page written as a file, and home page;
• llms-full.txt, which is those twins in one file;
• the text that Copy Markdown puts on the clipboard, and the file that View as Markdown opens;
• the response to a request with Accept: text/markdown, which is the same twin file;
• the .md twins of the Markus documentation sites we host on GitHub Pages;
• the Markus files that papyrus ops content export writes.

Three outputs are not page text, so there is nothing to flatten: llms.txt and the listing, archive and tag twins are link lists that lead to the twins above; the Open in ChatGPT, Claude and Perplexity links hand over the page's address, never its text; and the RSS and Atom feeds carry each entry's summary as plain text. The home-page twin and Copy Markdown reached this behavior in the release after Papyrus 5.19.0, which removed a second title they used to add in front of a page written as bands.

Pages written as files

A page that a site's author wrote as a file (a pages entry with a source, such as a topic page or a landing page) is read the way an article is. It has a twin, it is listed under Pages in llms.txt with its title and description, its text is part of llms-full.txt, and it carries the same page actions as an article. A noindex page has none of these.

Ask for the twin by name

On a site whose pages live at directory addresses (/decisions/some-page/), /decisions/some-page.md returns the twin too, so an agent that guesses the address from the page's name finds it. The edge function rewrites the request to /decisions/some-page/index.md; the guess works on sites that have picked up Papyrus 5.19.0 and redeployed their edge function.

See it on Pilobol.us, or on one of its articles: the page and its twin.

llms.txt and llms-full.txt

Each site publishes /llms.txt in the llmstxt.org shape: the site name, its description, then links to the Markdown twins, grouped by section, with an Optional list that holds the feeds, the sitemap and the full-content file. /llms-full.txt is the whole site's published content in one file. We generate it only when the site has at most 200 pages, and we cut it off at one megabyte, so it stays something an agent can actually load.

Both are rebuilt when content is published. A site that is not the production site (staging, a review build) publishes neither.

See Pilobol.us llms.txt and llms-full.txt.

Ask for Markdown, get Markdown

An agent does not have to know the twin's address. A request with Accept: text/markdown for a page gets the twin back from the same URL:

curl -H 'Accept: text/markdown' https://pilobol.us/articles/the-end-of-the-rave.html

A browser asking for HTML gets HTML from that same address. The edge function that does this chooses by the Accept header's quality values, and the two versions are cached separately. It runs on every site we serve through CloudFront, including tactus.anth.us and distinctive-mark.com. It does not run on the documentation sites we host on GitHub Pages, such as markus.anth.us, or on the Biased-Decisions and Hard-Decisions sites. Those still publish Markdown twins and llms.txt, and you fetch a twin by its own address.

Copy Markdown, View as Markdown

Article pages and pages written as files carry a small actions group. Copy Markdown puts the page, with its title and canonical URL, on your clipboard, ready to paste into an assistant. View as Markdown opens the twin. Without JavaScript the group is a plain list of links. Open in ChatGPT, Open in Claude and Open in Perplexity links sit in the same group on the sites listed below; a site turns them on in its settings. Those links carry a short prompt and the page's address, never the page's text, and nothing happens until someone clicks.

The actions appear on articles, and on guide and documentation pages where a site turns them on. You can see the group at the top of this article on Pilobol.us, on the Kanbus getting-started guide, the Prospectus start guide and the Markus documentation. The Open in ChatGPT, Claude and Perplexity links are live on pages of Pilobol.us, Anth.us, Threat Intelligence, the Papyrus reader, Kanbus, Apricity, Prospectus, Tactus, Chattic, Cyclotron, Distinctive Mark, the Markus, Biblicus, Kanbus and Cyclotron documentation, and this site. Listings and home pages do not carry them.

Feeds, sitemap and structured data

Sites with dated content publish RSS at /feed.xml and Atom at /atom.xml, with stable identifiers and canonical links. Every site publishes a sitemap, a robots.txt that points to it, canonical URLs, Open Graph and Twitter tags, and Article JSON-LD with publication and modification dates. All of it comes from the same published content as the pages, so an agent that prefers feeds or structured data does not get a different story from a person reading the page.

See the Pilobol.us RSS feed and Atom feed.

Agents running the newsroom

The reading side is only half of it. The newsroom itself is built to be operated by agents: the Papyrus command-line tool and its API cover newsroom operations, including publishing, so an agent can run those steps. Editors decide what is ready. Automation works for the desk, not instead of it.

What is not built

MCP, a remote server that lets an agent search and read a site through tools, and WebMCP, which does the same for agents inside the browser, are on the roadmap. Neither exists yet. Today an agent reads a Papyrus site through the files and headers above, and operates a newsroom through the command line and API.

Documentation

The full reference, with every projection.json switch, is Agent-readable sites in the Papyrus repository.

Where it runs

This is the standard Papyrus front end, which the Anthus sites run on. How it is put together is documented, and the Built with Papyrus page lists the sites.