Writerly

Writerly is a lightweight, indentation-based markup language for authoring tree-structured documents that resemble ordinary prose. Its syntax is descended from Elm-Markup.

This Gleam package constitutes Writerly’s reference implementation. It can:

• convert between Writerly and VXML
• assemble a document distributed across a directory of .wly files, according to Writerly’s own multi-file document specification

Writerly is deliberately tag- and attribute-agnostic: it assigns no built-in meaning to a document’s vocabulary. Each project typically defines that vocabulary through a process known as desugaring, which transforms parsed VXML into target-specific structure. The VXML Pipeline package provides desugaring support.

The Writerly extension for VS Code provides syntax highlighting, diagnostics, and document navigation.

Example

|> Chapter
    title=The observatory on the hill
    handle=observatory

    The road to the observatory follows the river for several
    miles before turning uphill. In summer, tall grass hides
    the stones that once marked the route.

    At the summit stands a small white building with a copper
    dome. The telescope inside was installed in 1927, but its
    clockwork drive still follows the stars across the sky.

    |> Section
        title=Recording an observation
        handle=recording-observations

        An observation begins with the time, the position of
        the telescope, and a short description of the weather.
        Blank lines separate paragraphs, while consecutive
        lines remain part of the same paragraph.

        A project may give _underscores_ a meaning such as
        emphasis, although Writerly itself leaves them as text.

        |> Note
            class=field-note

            Clouds near the horizon can make a bright star
            appear to fade and return several times.

        The measurements can then be processed by a program:

        ```gleam&id=record
        pub fn record(name, brightness) {
          #(name, brightness)
        }
        ```

    !! Recheck the installation date before publication.

The pipe symbol |> starts a new node, with the tag name and nothing else on the same line. Its attributes and children are indented by four spaces. Ordinary prose needs no marker. Consecutive text lines form a paragraph, while blank lines are represented explicitly rather than discarded.

Beginning-of-line escape sequences

Indentation, !!, and triple backticks have structural meaning at the start of a line. Prefix them with a backslash when they should instead be text:

\    text beginning with spaces
\!! text, not a comment
\``` text, not a code fence

Writerly removes that one escape backslash and preserves the following text. Inside a fenced code block, use the same technique for a content line that would otherwise look like the closing fence. The serializer adds these escapes when required.

Use from Gleam

Add the package to a Gleam project:

gleam add writerly

Parse a source string with one non-blank top-level element directly to VXML:

import writerly
import vxml/io_lines

pub fn parse(source: String) {
  source
  |> io_lines.string_to_input_lines("example.wly", 0)
  |> writerly.input_lines_to_vxml
}

input_lines_to_vxml preserves source provenance through VXML Blame values. The lower-level string_to_writerlys and input_lines_to_writerlys functions are available when a source contains multiple top-level nodes; convert their results with writerlys_to_vxmls.

Multi-file documents

assemble_input_lines accepts either a .wly file or a directory. A directory may use __parent.wly as its root file:

book/
├── __parent.wly
├── 01-introduction.wly
├── 02-examples.wly
└── appendix/
    ├── __parent.wly
    └── 01-tables.wly

Sibling files are processed in lexicographic path order. Their contents are indented beneath the contents of __parent.wly if a __parent.wly file is present. Otherwise they are concatenated at their native level of indentation. Nested directories are assembled recursively. Files and directories whose names start with # or do not end with .wly are ignored.

Use assemble_input_lines_with_path_selector to select source paths, or path_selector_from_only_paths to construct a selector from a list of paths. Assembly returns InputLine values so that filenames and line numbers remain available to later parsing and VXML processing.

VXML representation

The conversion to VXML uses ordinary VXML element and text nodes, with three reserved element names:

• WriterlyBlankLine represents a semantic blank line;
• WriterlyCodeBlock represents a fenced code block;
• WriterlyComment represents a comment block.

The synthetic attribute WriterlyCodeBlockInfoString stores the leading info string from a code fence. Structured fence annotations become ordinary VXML attributes. Commented-out element attributes use keys of the form WriterlyCommentedAttribute<N>Spaces, where N records their original spacing. Applications should use the public helper functions for recognizing and constructing these keys rather than assembling them by hand.

Cross-document references

Writerly has an official convention for cross-document references based on handles. A handle connects a definition site to one or more usage sites. Handle syntax is interpreted by editor tooling and desugaring rather than by the core parser. A definition may be an ordinary element attribute:

|> Theorem
    handle=abel-inversion

or an in-text marker:

abel-inversion##<<

A usage places >> immediately before the handle name:

The result now follows from Theorem >>abel-inversion.

A single backslash suppresses either kind of in-text recognition:

abel-inversion\##<< is ordinary text
\>>abel-inversion is ordinary text

The full handle grammar at the end of this README describes additional definition forms, escaping rules, and other capabilities.

In parsed VXML, handle remains an ordinary attribute and in-text handle forms remain text. Projects interpret them later through desugaring. The VXML Pipeline package supplies shared handle desugarers.

The Writerly extension for VS Code provides navigation, completion, rename support, and diagnostics for handles that follow the same grammar.

Inline formatting

Unlike some other lightweight markup languages, Writerly assigns no special meaning to underscores, asterisks, tildes, or similar inline delimiters. They remain ordinary text. A project may introduce rich-text formatting with its own delimiters and interpret them during desugaring.

Writerly provides structural elements, attributes, paragraphs, comments, and fenced code blocks without prescribing a built-in rich-text vocabulary. This is one respect in which Writerly may differ from syntax familiar from Elm-Markup and other lightweight markup languages.

Syntax specification

Handle grammar

Handles use the following character classes:

name interior: Unicode letters, numbers, marks, _ . : - ' ^
name final:    Unicode letters, numbers, marks, _ ' ^
decorator:     # followed by one or more Unicode letters, numbers, marks, _ : ' -

Thus a name cannot end with ., :, or -. Decorators follow the name and are metadata rather than part of the indexed handle.

An attribute definition is handle=name in an element’s attribute zone. An in-text definition has either of these forms:

<boundary>name<decorators>##<<
##name<decorators>##<<

<boundary> is the beginning of line content, a space, {, (, [, or the position immediately after an active value closer. The second form uses ## as an explicit opener and may occur anywhere. ##<< must be directly adjacent to the name or final decorator; any intervening backslash prevents recognition, regardless of backslash count.

An in-text definition optionally has a same-line value:

name##<<value>>##

The value extends from ##<< to the first active >>##. Everything inside a confirmed value, including spaces, brackets, pipes, and ##<<, is value text. If no active closer occurs on the same line, the value is empty and text after ##<< remains ordinary text.

An odd-length run of consecutive backslashes immediately before >>## escapes the closer; an even-length run, including zero, leaves it active. An active closer is consumed eagerly, but the position after it becomes a definition boundary, allowing chaining:

##first##<<first value>>##second##<<second value>>##

The complete definition grammar applies inside fenced code-block content, but not on fence lines. Handle usages are not interpreted inside code blocks.

A usage is >>name. Outside code blocks it may occur without a left boundary. An odd-length run of consecutive backslashes immediately before >> escapes the usage; an even-length run leaves it active:

>>name       active
\>>name      escaped
\\>>name     active
\\\>>name    escaped

Escape backslashes are preserved in parsed source. Removing them, interpreting definition values, and replacing definitions or usages are responsibilities of later desugaring stages.

Source and indentation

A Writerly document is a sequence of lines. Its structural indentation is made of spaces and must be a multiple of four. A child is indented exactly four spaces farther than its parent. Unexpected deeper indentation is an error.

Writerly documents conventionally use the .wly filename extension.

Elements

An element begins with |> followed by a tag name:

|> article

Whitespace surrounding the name after |> is trimmed. A tag name must match:

[A-Za-z_:][-A-Za-z0-9._:]*

The result is an ordinary VXML element with the same tag name. Attributes and children occur on subsequent lines, indented four more spaces. An element ends when the indentation returns to that element’s level or lower. Closing tags are not written.

Attributes

Immediately after an element line, lines of the following form are parsed as attributes:

|> figure
    id=overview
    class=wide diagram

The first = separates the key and value. An attribute key must match:

[A-Za-z_][-A-Za-z0-9._:]*

The key and value become an ordinary VXML attribute. Attribute values are trimmed at both ends. More than 100 leading spaces after the = are rejected. Empty values are allowed.

Attribute parsing stops at the first line that is not a valid attribute. If an element’s first text line resembles an attribute, insert a blank line before it:

|> Example

    equation=a+b

Paragraphs and text lines

Consecutive ordinary lines at the same indentation become the Line values of one VXML text node. After removing structural indentation, Writerly preserves the remaining text, including trailing whitespace.

Leading spaces in text content cannot be written directly because leading spaces normally express structural indentation. To preserve them, place a backslash immediately after the line’s structural indentation. The backslash is removed while the spaces following it are retained. The same escape allows text to begin with !! or triple backticks:

\    text beginning with spaces
\!! text, not a comment
\``` text, not a code fence

Exactly one escape backslash is removed. Further backslashes remain part of the text. The serializer inserts one escape when required.

Blank lines

Every empty source line becomes an empty VXML element named WriterlyBlankLine. Blank lines are therefore semantic nodes: adding or removing one changes the VXML tree. Their meaning, if any, is assigned by later processing.

Comments

At ordinary child position, consecutive lines beginning with !! become one WriterlyComment VXML element containing a text node. The !! marker is removed from the stored line content:

!! first comment line
!! second comment line

In the attribute region immediately following an element, an !! line is instead stored as a synthetic VXML attribute. The number of spaces after !! is preserved in its key, up to a maximum of 100:

|> figure
    !! class=temporarily-disabled
    src=figure.svg

Fenced code blocks

A line beginning with three backticks opens a code block. A line containing three backticks at the same indentation closes it. The result is a WriterlyCodeBlock VXML element:

```gleam
pub fn answer() {
  42
}
```

The closing fence may have trailing whitespace but no annotation. Lines inside the block retain their relative indentation. Prefix a content line that would look like a closing fence with a backslash; parsing removes exactly that one backslash.

The text immediately following the opening backticks is the info string. It must not begin with a space. An info string may be followed by structured annotations written as &key=value:

```gleam&id=answer&class=example
```

Annotation keys follow the attribute-key grammar. Values are trimmed and obey the same 100-leading-space limit as element attributes. Write \& for a literal ampersand and \\ for a literal backslash in the info string or an annotation. Parsing and serialization preserve the distinction between the leading info string and structured annotations.

Top-level cardinality

A Writerly source may contain any number of top-level nodes, but parsing it as one VXML tree requires exactly one non-blank root. input_lines_to_vxml returns MissingRoot or NonUniqueRoot when that requirement is not met. For multiple roots, use string_to_writerlys or input_lines_to_writerlys, then convert the result to a list of VXML nodes with writerlys_to_vxmls.