Writerly

Writerly is a lightweight, indentation-based markup language for authoring tree-structured documents that resemble ordinary prose. Its syntax is descended from Elm-Markup.

This Gleam package constitutes Writerly’s reference implementation. It can:

• convert between Writerly and VXML
• assemble a document distributed across a directory of .wly files, according to Writerly’s own multi-file document specification

Writerly is deliberately tag- and attribute-agnostic: it assigns no built-in meaning to a document’s vocabulary. Each project typically defines that vocabulary through a process known as desugaring, which transforms parsed VXML into target-specific structure. The VXML Pipeline package provides desugaring support.

The Writerly extension for VS Code provides syntax highlighting, diagnostics, and document navigation.

Example

|> Chapter
    title=The observatory on the hill
    handle=observatory

    The road to the observatory follows the river for
    several miles before turning uphill. In summer, tall
    grass hides the stones that once marked the route.

    At the summit stands a small white building with a
    copper dome. The telescope inside was installed in
    1927, but its clockwork drive still follows the stars
    across the sky.

    |> Section
        title=Recording an observation
        handle=recording-observations

        An observation begins with the time, the position
        of the telescope, and a short description of the
        weather. Blank lines separate paragraphs, while
        consecutive lines remain part of the same paragraph.

        A project may give _underscores_ a meaning such
        as emphasis, although Writerly itself leaves them
        as text.

        |> Note
            class=field-note

            Clouds near the horizon can make a bright
            star appear to fade and return several times.

        The measurements can then be processed by a program:

        ```gleam&id=record
        pub fn record(name, brightness) {
          #(name, brightness)
        }
        ```

    !! Recheck the installation date before publication.

The pipe symbol |> starts a new node, with the tag name and nothing else on the same line. Its attributes and children are indented by four spaces. Ordinary prose needs no marker. Consecutive text lines form a paragraph, while blank lines are represented by their own node type in the parsed AST.

Beginning-of-line escape sequences

Indentation, !!, and triple backticks have structural meaning at the start of a line. Prefix them with a backslash when they should instead be text:

\   text beginning with spaces
\!! text, not a comment
\```text, not a code fence

Writerly removes that one escape backslash and preserves the following text. Inside a fenced code block, use the same technique for a content line that would otherwise look like the closing fence. The serializer adds these escapes when required.

Use from Gleam

Add the package to a Gleam project:

gleam add writerly

Parse a source string with one non-blank top-level element directly to VXML:

import writerly
import vxml/io_lines

pub fn parse(source: String) {
  source
  |> io_lines.string_to_input_lines("example.wly", 0)
  |> writerly.input_lines_to_vxml
}

input_lines_to_vxml preserves source provenance through VXML Blame values. The lower-level string_to_writerlys and input_lines_to_writerlys functions are available when a source contains multiple top-level nodes; convert their results with writerlys_to_vxmls.

Multi-file documents

assemble_input_lines accepts either a .wly file or a directory. A directory may use __parent.wly as its root file:

book/
├── __parent.wly
├── 01-introduction.wly
├── 02-examples.wly
└── appendix/
    ├── __parent.wly
    └── 01-tables.wly

Sibling files are processed in lexicographic path order. Their contents are indented beneath the contents of __parent.wly if a __parent.wly file is present. Otherwise they are concatenated at their native level of indentation. Nested directories are assembled recursively. Files and directories whose names start with # or do not end with .wly are ignored.

Use assemble_input_lines_with_path_selector to select source paths, or path_selector_from_only_paths to construct a selector from a list of paths. Assembly returns InputLine values so that filenames and line numbers remain available to later parsing and VXML processing.

VXML representation

The conversion to VXML uses ordinary VXML element and text nodes, with three reserved element names:

• WriterlyBlankLine represents a semantic blank line;
• WriterlyCodeBlock represents a fenced code block;
• WriterlyComment represents a comment block.

The synthetic attribute WriterlyCodeBlockInfoStringPrefix stores the nonempty, unmarked prefix of a code fence’s info string: the part before its first unescaped &key=value annotation. The attribute is absent when that prefix is empty. Structured fence annotations become ordinary VXML attributes. Commented-out element attributes use keys of the form WriterlyCommentedAttribute<N>Spaces, where N records their original spacing. Applications should use the public helper functions for recognizing and constructing these keys rather than assembling them by hand.

VXML-to-Writerly conversion and Writerly serialization return Result values. Malformed reserved elements, empty text nodes, and malformed manually constructed Writerly nodes are reported as SerializationError values.

Cross-document references

Writerly has an official convention for cross-document references based on handles. A handle connects a definition site to one or more usage sites. Handle syntax is interpreted by editor tooling and desugaring rather than by the core parser. A definition may be an ordinary element attribute:

|> Theorem
    handle=odd-order-theorem

or an in-text marker:

odd-order-theorem##<<

A usage places >> immediately before the handle name:

The result now follows from Theorem >>odd-order-theorem.

A single backslash suppresses either kind of in-text recognition:

odd-order-theorem\##<< is ordinary text
\>>odd-order-theorem is ordinary text

Inside fenced code-block content, in-text handle definitions remain active, but >>handle usages are ordinary code text. Neither form is recognized on the opening or closing fence line.

The full handle grammar at the end of this README describes additional definition forms, escaping rules, and other capabilities.

In parsed VXML, handle remains an ordinary attribute and in-text handle forms remain text. Projects interpret them later through desugaring. The VXML Pipeline package supplies shared handle desugarers.

The Writerly extension for VS Code provides navigation, completion, rename support, and diagnostics for handles that follow the same grammar.

Inline formatting

Unlike some other lightweight markup languages, Writerly assigns no special meaning to underscores, asterisks, tildes, or similar inline delimiters. They remain ordinary text. A project may introduce rich-text formatting with its own delimiters and interpret them during desugaring.

Writerly provides structural elements, attributes, paragraphs, comments, and fenced code blocks without prescribing a built-in rich-text vocabulary. This is one respect in which Writerly may differ from syntax familiar from Elm-Markup and other lightweight markup languages.

Syntax Reference

Source and indentation

A Writerly document is a sequence of lines. Its structural indentation is made of spaces and must be a multiple of four. A child is indented exactly four spaces farther than its parent. Unexpected deeper indentation is an error.

Writerly documents conventionally use the .wly filename extension.

Line classification

Parsing proceeds linearly. Structural indentation first determines the current parent and the line’s position within it. A line is then interpreted according to that position:

1. An empty line is a semantic blank line.
2. In an element’s initial attribute region, a valid key=value line is an attribute and an !! line is a commented attribute.
3. At ordinary child position, |> opens an element, !! begins a comment, and three initial backticks open a fenced code block.
4. Any other nonempty line is paragraph text.

A beginning-of-line escape suppresses recognition of comment and code-fence markers and permits content to begin with otherwise structural spaces.

Elements

An element begins with |> followed by a tag name:

|> article

Whitespace surrounding the name after |> is trimmed. A tag name must match:

[A-Za-z_:][-A-Za-z0-9._:]*

The result is an ordinary VXML element with the same tag name. Attributes and children occur on subsequent lines, indented four more spaces. An element ends when the indentation returns to that element’s level or lower. Closing tags are not written.

Attributes

Immediately after an element line, lines of the following form are parsed as attributes:

|> figure
    id=overview
    class=wide diagram

The first = separates the key and value. An attribute key must match:

[A-Za-z_][-A-Za-z0-9._:]*

The key and value become an ordinary VXML attribute. Leading spaces and tabs after the = are preserved as part of the value, up to a maximum of 100. More than 100 leading spaces or tabs is a parsing error. Trailing spaces and tabs are trimmed. Empty values are allowed.

Attribute parsing stops at the first line that is not a valid attribute. If an element’s first text line resembles an attribute, insert a blank line before it:

|> Example

    equation=a+b

Paragraphs and text lines

Consecutive ordinary lines at the same indentation become the Line values of one VXML text node. After removing structural indentation, Writerly preserves the remaining text, including trailing whitespace.

Leading spaces in text content cannot be written directly because leading spaces normally express structural indentation. To preserve them, place a backslash immediately after the line’s structural indentation. The backslash is removed while the spaces following it are retained. The same escape allows text to begin with !! or triple backticks:

\    text beginning with spaces
\!! text, not a comment
\``` text, not a code fence

Exactly one escape backslash is removed. Further backslashes remain part of the text. The serializer inserts one escape when required.

Blank lines

Every empty source line becomes an empty VXML element named WriterlyBlankLine. Blank lines are therefore semantic nodes: adding or removing one changes the VXML tree. Their meaning, if any, is assigned by later processing.

Comments

At ordinary child position, consecutive lines beginning with !! become one WriterlyComment VXML element containing a text node. The !! marker is removed from the stored line content:

!! first comment line
!! second comment line

In the attribute region immediately following an element, an !! line is instead stored as a synthetic VXML attribute. The number of spaces after !! is preserved in its key, up to a maximum of 100. More than 100 spaces is a parsing error:

|> figure
    !! class=temporarily-disabled
    src=figure.svg

Fenced code blocks

A line beginning with three backticks opens a code block. A line containing three backticks at the same indentation closes it. The result is a WriterlyCodeBlock VXML element:

```gleam
pub fn answer() {
  42
}
```

The closing fence may have trailing whitespace but no annotation. Lines inside the block retain their relative indentation. Prefix a content line that would look like a closing fence with a backslash; parsing removes exactly that one backslash.

The text immediately following the opening backticks is the info string. It must not begin with a space. An info string may be followed by structured annotations written as &key=value:

```gleam&id=answer&class=example
```

Annotation keys follow the attribute-key grammar. Their values preserve leading spaces and tabs, trim trailing spaces and tabs, and obey the same 100-character leading-whitespace limit as element attributes; exceeding that limit is a parsing error. Write \& for a literal ampersand and \\ for a literal backslash in the info string or an annotation. Parsing and serialization preserve the distinction between the leading info string and structured annotations.

Top-level cardinality

A Writerly source may contain any number of top-level nodes, but parsing it as one VXML tree requires exactly one non-blank root. input_lines_to_vxml returns MissingRoot or NonUniqueRoot when that requirement is not met. For multiple roots, use string_to_writerlys or input_lines_to_writerlys, then convert the result to a list of VXML nodes with writerlys_to_vxmls.

Handle grammar

Handles are an official Writerly convention interpreted by editor tooling and desugaring. The core Writerly parser preserves their source syntax as ordinary attributes or text.

Handle names and decorators use these character classes:

name interior: Unicode letters, numbers, marks, _ . : - ' ^
name final:    Unicode letters, numbers, marks, _ ' ^
decorator:     # followed by one or more Unicode letters, numbers, marks, _ : ' -

A name contains at least one character and cannot end with ., :, or -. Zero or more decorators may follow it. Decorators are metadata and are not part of the indexed handle name.

Attribute definitions

Within an element’s attribute region, handle defines a handle. Its value has one of these forms:

name
name <<value>>

The bare form assigns an empty value. In the assigned form, the first << and final >> delimit the value. The value may itself contain << or >>.

In-text definitions

An in-text definition has one of these forms:

<boundary>name<decorators>##<<
##name<decorators>##<<

<boundary> is the beginning of text content, a space, {, (, [, or the position immediately after an active value closer. The boundary is retained. The second form uses an explicit ## opener, which is consumed, and may occur at any position.

The ##<< terminator must be directly adjacent to the name or final decorator. Any intervening backslash prevents recognition, regardless of the number of backslashes.

An in-text definition may carry a same-line value:

name##<<value>>##

The value begins after ##<< and ends before the first active >>## on the same line. Within a confirmed value, all other characters—including spaces, brackets, pipes, and ##<<—are value text. If the line contains no active closer, the definition has an empty value and scanning restarts after ##<< as ordinary text; that remaining text may contain another definition.

A run of consecutive backslashes immediately before >>## controls whether the closer is active. An even count, including zero, is active; an odd count escapes the closer. The escaping backslash is removed from the value. An active closer is consumed eagerly, and its ending position is a new definition boundary. Definitions can therefore be chained:

##first##<<first value>>##second##<<second value>>##

The complete in-text definition grammar applies to fenced code-block content, but not to opening or closing fence lines.

Usages

A usage is >>name. Outside fenced code blocks it has no left-boundary requirement. Usages are not recognized inside code-block content or on fence lines.

A run of consecutive backslashes immediately before >> controls whether the usage is active. An even count, including zero, is active; an odd count escapes the usage:

>>name       active
\>>name      escaped
\\>>name     active
\\\>>name    escaped

The Writerly parser preserves these backslashes. Removing usage escapes, materializing definitions, interpreting definition values, and resolving usages belong to later desugaring stages.

Example

Given this Writerly source, patterned after a project chapter:

|> Chapter
    title=An illustrative chapter

    This opening paragraph has three lines,
    the third of which starts with an  
    \ escaped space, and the second of which has two trailing spaces.


    |> Example
        label=A small program
        !! class=draft
        style=color:red
        ```gleam&id=answer&listing=true
        pub fn answer() {
          42
        }
        ```

    !! This comment block remains
    !! available to later processing.
    \```text, not a code fence

Parsing and conversion produce the following serialized VXML:

<> Chapter
  title=An illustrative chapter
  <> WriterlyBlankLine
  <>
    'This opening paragraph has three lines,'
    'the third of which starts with an  '
    ' escaped space, and the second of which has two trailing spaces.'
  <> WriterlyBlankLine
  <> WriterlyBlankLine
  <> Example
    label=A small program
    WriterlyCommentedAttribute1Spaces=class=draft
    style=color:red
    <> WriterlyCodeBlock
      WriterlyCodeBlockInfoStringPrefix=gleam
      id=answer
      listing=true
      <>
        'pub fn answer() {'
        '  42'
        '}'
  <> WriterlyBlankLine
  <> WriterlyComment
    <>
      ' This comment block remains'
      ' available to later processing.'
  <>
    '```text, not a code fence'

The in-memory VXML additionally associates a Blame value with every element, attribute, text node, and text line. The serialized form above omits those values.