Skip to content

Users' files ​

From the spec

This page is spec/user-files.md, the specification the compiler and the backends are written from. It is rebuilt from the spec with every deploy (how).

How users' own files stay working while the app under them changes: the API an app exports to user code and its compatibility, the index of what each tree uses, history, drafts and bases, migrations, deprecations and the compiled cache. The design is docs/editor.md (Where users' edits live; Keeping users' files working when the app changes); trees, stores and running are std/sandbox.md, serving them app.md (User trees). It is built in weft-trees (core/crates/weft-trees), used by std/sandbox on the server and by weft api and weft trees.

The exported API ​

User code sees only what the app gives it: each run of user code imports some of the app's components, functions and modules under labels (std/sandbox's run(tree, { imports: { Card: FancyCard, money: cash } }), app.serve's sandbox: req => ({ imports: { Card: FancyCard } }); std/sandbox.md, Imports). The label is the name user code sees, the value the app's own binding. The exported API is all the labels, found by the compiler in the app's code (weft_check::sandbox_imports); nothing is exported otherwise.

weft
import FancyCard from "components/ui/FancyCard"
import { cash } from "lib/ui/format"
let page = try await run(tree, { imports: { Card: FancyCard, money: cash }, caps: { user } })
  • What a label exports: a component, itself and its props; a function, value or type, that export; a module imported whole (import * as format), every export of it. A type a signature names is written out unless a label gives it.

  • Internal renames never break users' files. The app renames a function or a type, renames a component's file or moves a module, and the labels stay (the edit API's rename rewrites only the value): the API's version doesn't change and no tree is touched. Only a label or a type is an API change, and that is what the migrations and deprecations below are for.

  • Keys are labels. A component is its label (Card), one of its props Card.title; a function, value or type is app:<label> (app:money), an export of a module given whole app/<label>:<name> (app/dates:today). Server functions, globals and contexts aren't exported to user code.

  • The description (weft api --json): per key its kind, its file, its type (a function's signature, a value's type, a type's definition, an attribute's parameters) and, for a component, each prop's type, whether it is required and whether it can be bound, with any deprecation. Types are the checker's, made stable: a type a label gives stays that name (app:Theme, described as an export of its own), the standard library's are std:<Name>, any other is written out; record fields and union members are sorted.

  • Its version is a hash of every label's signature (kind, type, props). A rename or a move in the app's own code, a deprecation, a comment or any change inside a function or component leave it as it is.

  • The tree a user edits sees the exported files and what they need (their imports, the components they use); a tree naming any other component, export or module of the app is a problem of the tree (E1004: "<Secret> isn't given to user code"). Each run then sees only its own imports (std/sandbox.md, Imports): a file using more falls back.

Compatibility ​

weft api diff <old> <new> classifies each change per key:

changecompatiblebreaking
exportsa new exportan export removed, or its kind changed
a component's propsa new optional prop; a required prop made optional; a wider type (every value it took still fits); a prop made bindablea prop removed; a new required prop, or an optional one made required (key: the component); a narrower type; bind: taken away
a functiona new optional parameter, wider parameters, a narrower resulta new required parameter, a narrower parameter, a wider result, sync ↔ async
a valuea narrower typea wider one
a type, an attribute's parametersa wider type (an added optional field)any other change; and every export and prop whose type names a broken type breaks too
any body, a deprecationalwaysnever

The same classifier enforces semver for packages: publishing refuses a version whose change from the one before needs a bigger bump (docs/packages.md, Semver enforcement).

"Fits" is the language's assignability (language.md, Structural typing): literals to their base type, members to unions, lists, maps, sets, tuples and results covariantly, parameters contravariantly, records by their fields (more fields only at the top level, where the value is projected).

The index ​

Every save records what each file of the tree uses (uses in its manifest, uses.rs), read from its syntax: a tag naming one of the app's components, and not one of the tree's own, uses it and each prop it passes (<Card {title} bind:open> uses Card, Card.title, Card.open); a component named as a value uses it; an import from an exported module uses each name it imports (* as m: every export of that module). The store keeps one entry per key and tree (the published version's and the draft's together), updated on every save from the difference with what the tree used before: new entries before the manifest, gone ones after it, so a lookup never misses a tree that uses a key.

store.trees_using(keys) (using in Weft) lists the entries under each key: it reads only what it finds, never the trees. A deploy with no breaking change looks up nothing it could find, and touches nothing.

Stores ​

UserFiles (store.rs) is the interface every store implements: the published tree and the draft, saving (each changed file a new version in its history; served at once or kept as the draft), writing a manifest's state without a new version, history and revisions, deleting, the index, the app's files by revision, the app's API by version, compiled trees. Skein's built-in module for users' files implements it natively (versioned, content-addressed files, the index, compiled artifacts, migration state); every other store is KvFiles over a key-value store with listing by prefix (Kv): this process's memory, a folder, an S3 bucket, a SQL table (weft_user_files (k, v), SQLite, Postgres and MySQL through std/db's drivers; a prefix is a range of the primary key).

text
<name>/manifest.json          what is served
<name>/draft.json             the draft, while there is one
<name>/f/<id>/<rev>.weft      every revision of a file (`rev`: a hash of its source)
<name>/h/<id>.json            a file's history: [{ rev, at, by?, note?, kind }], oldest first
<name>/v/<version>/<id>.weft  trees saved before revisions (still read)
$index/<key>/<name>           the index: one empty entry per key a tree uses
$app/<id>/<rev>.weft          the app's files by revision (what copies' bases name)
$api/<version>.json           the app's API by version (what trees' `api` names)
$deploy.json                  the last deploy this store saw: { api, migration, day }
$compiled/<name>/<key>        compiled trees

Keys and names inside $index and $compiled escape % and / (%25, %2F). A tree's name can't start with $.

Copies, the source itself, history and drafts ​

  • A copy per file (a tree's kind is "copies", the default): store.copy(name, "components/ui/Card") copies the app's file into the tree's draft by its file name, recording its base: the app's file and its revision. Every other file stays the app's. A component the tree has of its own replaces the app's of that name for that tree, the app's own files included.
  • The source itself (kind: "source"): the user's own project, its files edited in place; no file has a base.
  • History: every save of a file whose source changed is a version in its history (kind draft, published, migrated); store.revision(name, file, rev) gives one back, and going back is saving it again.
  • Drafts: saveDraft keeps a tree apart from what is served (checked, kept even with problems, which come back), publish checks the draft and serves it, discard drops it. save is both at once.
  • The app's change since the base: store.changes(name) gives, per copy whose app file changed, the app's diff (base → now), the user's (base → theirs), both merged three-way and whether that was clean. The editor offers keep theirs (the base moves to now), take the app's (theirs stays in the history) or bring it over (the merge as the first proposal; the agent refines it as an edit to review). Nothing changes by itself.

Migrations ​

An app ships each breaking change with a migration: user/migrations/<id>.json, run in the order of their ids.

json
{ "note": "Card's `title` is `heading` now", "ops": [{ "op": "renameProp", "component": "Card", "from": "title", "to": "heading" }] }
  • The format is declarative: the edit API's operations (spec/edit.md), as data. breaks lists the keys whose trees it migrates; left out, its operations' keys. The operations: renameProp { component, from, to } (also bind: and {title}), renameComponent { from, to } (tags, closing tags, the component as a value), replaceComponent { from, to, props } (its props renamed old → new), setProp { component, prop, value } (a new required prop, given where it is missing; value a Weft expression), removeProp { component, prop }, addArgument { function, at, value }, renameExport { module, from, to }, moveExport { from, to, name }, and edit { edit }, any operation of the edit API as it serializes one. They are patches over the lossless syntax tree: comments and formatting stay.
  • The seam: EditOps (ops.rs) applies an operation to one file. The edit API (weft-edit) implements it once it lands; until then Builtin does all but edit, which fails the migration with that reason.
  • Proposed next: migrations written in Weft, for transforms data can't say: user/migrations/<id>.weft exporting fn migrate(tree: Tree) over a std/migrate module that exposes the edit API's operations to migration code (the tree's files, their syntax trees, renameProp, wrap, setProp, …), run in a sandbox like user code. The declarative form stays the way to write the common ones.
  • The chain (migrate.rs): a tree's state is in its manifest, migrations: { at, failed? }: the last migration it was brought to, and the one that failed (its reason, and the app's API version it failed against). The migrations after at run in order, each only when the tree (as the ones before left it) uses one of its keys; every operation goes over every file; the result is checked once against the app. If it checks, it is saved and published as a new version (migrated in the history) at the last migration's id. If an operation fails or the result doesn't check, nothing changes: the tree keeps its version, failed records why, and it isn't tried again until the app's API changes.
  • Where it runs: in the app server, after a deploy (eager, below) and when a tree is served or opened (lazy); and in weft trees migrate <folder> for a folder store.

Deploys ​

The first time a process makes a store (dir, database, a bucket's trees) in an app that exports to user code, it compares the app with the store's $deploy.json and, when they differ:

  1. keeps the app's API by version ($api/) and its exported files by revision ($app/);
  2. plans: the API's change from the last deploy's (ApiDiff), and the keys it affects: the breaking changes', the new migrations' (after the last deploy's), the deprecations whose date has passed since the last deploy's day; one index lookup per key;
  3. brings the affected trees saved in the last 30 days (active) up to the app in the background, one per turn of the server, so requests go on between them; the others when they are next served or opened;
  4. writes $deploy.json.

Bringing a tree up (deploy::decide, the same eager and lazy): its pending migrations run; then its fallbacks are computed (below); what changed is written, and nothing else. A tree whose API version differs from the app's but uses nothing that changed is left as it is: no read beyond its manifest, no write, no compile.

Deprecations ​

@deprecated(until: "2027-06-01", use: "heading") goes before an exported declaration (a function, value, type, attribute, a component's prop) or, written before a file's imports, its --- or at its end, on the file (a component). Both arguments are optional strings, until a YYYY-MM-DD date (language.md, Attributes).

  • A deprecation is a compatible change: the old export keeps working, so trees whose migration failed keep compiling against it.
  • The warning goes only to the users the index finds: store.using(<the deprecated keys>) (api().deprecated); store.status(name).warnings lists each file, key, date and replacement, for the editor's "fix it" by the agent.
  • After the date, a file still using the export (or one the app no longer has) falls back to the app's: a copy is served as the app's file, a page of the user's own isn't served (its route falls through to the app's). The reason is recorded in the tree's manifest (fallback: [{ file, key, until, use, reason }]), for the editor to show.

The compiled cache ​

A compiled tree is cached by its version and the version of what it uses of the API (cache::key: a hash of the signatures of the keys in its uses, a component's only its name and required props). A deploy that changes nothing a tree uses keeps its key, so serving finds it compiled; a tree compiles again only when its version changes (a save, a migration) or a signature it uses does. Today compiled trees are kept per process (bytecode modules serialize, a component's template doesn't yet), so a new process compiles each tree once when it first serves it; put_compiled/compiled in the store interface are where persisted artifacts go once templates serialize.

Commands ​

  • weft api [--json] [project] prints the exported API; weft api diff <old> [new] the change (a project or an API's .json), exiting 1 when it breaks; weft api snapshot writes user/api.json (the API as deployed, as std/db's db/snapshot.json); weft api check fails when a breaking change since the snapshot has no migration.
  • weft trees migrate [--all] [--project p] <folder> brings the trees of a folder store that the index finds affected (--all: every tree) up to the app; weft trees status <folder> <name> shows one.

Scale ​

cargo run --release -p weft-trees --example scale (in core/): 100,000 synthetic trees sized like examples/sites' (38 lines, 2.3 KB of source and 1.6 files per tree on average, 13 exported components), 1% of them using Card's title, in a memory store; the real checker. On the sandbox's 4 cores (2026-11-17):

saving a tree (sources, history, manifest, index)61 µs; what it uses, read from its syntax: 0.5 ms
the store1.43 M keys, 338 MB; 1.0 M index entries (10 per tree); 513 MB resident
a deploy with no breaking changeAPI and diff 49 ms; plan 0.5 ms; 0 trees affected, 0 reads, 0 writes; every compiled-cache key unchanged; a tree served lazily: 4.3 µs to decide, no check
a deploy breaking Card (1% of trees)the index lookup 1.5 ms, 1,000 trees found; eager migration 15–18 ms a tree on one core (≈ 60 trees/s, nearly all of it the check), 108 trees/s on 4 threads; 621 MB at the peak

Decided here ​

  1. User code sees only what its runs import, under labels; the exported API is all of them, found by the compiler (owner, 2026-11-17: no list of exports in configuration). Keys are labels: Card, Card.prop, app:money, app/dates:today. The API's version is over labels and types, so the app's own names are free to change.
  2. The API is described from the checker's types and versioned by its signatures only, so bodies never change it.
  3. The index is one entry per key and tree, kept on every save, so a deploy reads only the affected trees.
  4. One store interface (UserFiles) for Skein's module and for key-value stores; the layout is shared by memory, folders, buckets and SQL.
  5. Migrations are declarative edit-API operations in user/migrations/*.json, a chain per tree checked once; Weft migrations over std/migrate are the proposed next step.
  6. Eager for trees saved in the last 30 days, lazy for the rest; a failed migration isn't retried until the API changes.
  7. @deprecated(until:, use:) before any exported declaration; past the date, files fall back to the app's with the reason recorded.
  8. Compiled trees are keyed by tree version and the API they use.

Weft is in development. The reference is the spec as it stands today.