Skip to content

std/http ​

From the spec

This page is spec/std/http.md, the specification the compiler and the backends are written from. It is rebuilt from the spec with every deploy (how).

An HTTP server and client (core/crates/weft-check/src/std/http.weft declares it; the server runtime implements it, server.md). Records whose fields are functions (Router, Session, FetchResponse) are values like any other: a router is built by value, every method giving a new one.

weft
import { router, json, text, serve, local, session, cors } from "std/http"
import type { Request } from "std/http"

type User = { id: string, name: string }
let user = local<User>("user")

let api = router()
  .use(cors())
  .use(async (req, next) => try await next(user.set(req, { id: "1", name: "Ann" })))
  .get("/todos/:id", async req => text(`todo ${req.params.id} for ${user.get(req)?.name ?? "?"}`))
  .post("/echo", async req => json(try req.json()))      // a body that isn't JSON: the handler fails, a 500
let server = try await serve({ port: 8080 }, api)       // at the top level: a failure fails the program's load

Requests ​

  • Request = { method, url, path, query, headers, cookies, params, body, text, json, locals, remote }: path is percent-decoded and has no query; query the first value per key; headers lowercase names, a repeated header's values joined with ", "; cookies from cookie; body the whole body as Bytes, read before the handler runs (up to the server's maxBody, 1 MiB by default: past it the answer is a 413); req.text() the body as UTF-8 (lossy), req.json(): json ! SyntaxError parsed (std/json's error; check the value with as); req.chunk() the body's next piece (Streaming, below); remote the peer's address.
  • params is a route's path parameters. Typed by the route: a route method called with a literal path gives its handler a Request whose params is a record of the path's parameters, all strings (/todos/:id gives { id: string }; /files/*path gives path, the rest of the path joined by /; * alone is rest). A path that isn't a string literal (or a template without substitutions) is an error (route-path). Elsewhere (middleware, a plain handler) params is json: an empty object.
  • request(): Request? is the request the running code serves: a route's handler and its middleware, a server function or typed endpoint called over the network, a page being rendered, through every await of theirs; null outside one (server.md, The async context). It is how a server function knows who calls: try await session(request()!).user().

Responses ​

  • Response = { status, headers: [string, string][], body: Bytes, file?: string, stream?: number }: headers keep their order and may repeat (set-cookie); file, when set, streams that file as the body; stream is a streamed body's id (stream, sse below make it).
  • json(value, init?) (application/json), text(s, init?), html(s, init?), bytes(b, contentType, init?), status(code, body?), redirect(route, status = 302) (location is the Route's href: redirect(routes.todos()), redirect(routes.home()), redirect(Route.url("https://example.com/"))), file(path, init?) (its type from the extension; a 404 when it doesn't exist). init = { status?, headers? }.
  • header(res, name, value), setCookie(res, name, value, options?), clearCookie(res, name, options?) give the response with a header added. Cookies are Path=/, HttpOnly and SameSite=Lax unless options says otherwise ({ maxAge?: seconds, path?, domain?, httpOnly?, secure?, sameSite? }).

Streaming ​

weft
import { router, serve, sse, fetch } from "std/http"
import type { Events } from "std/http"

type Token = { text: string }

let api = router().post("/chat", async req => sse<Token>(async (events: Events<Token>) => {
  let res = try await fetch("https://llm.example/v1/stream", { method: "POST", json: try req.json() })
  let upstream = res.events()
  let mut e = try await upstream.next()
  while e != null && !events.closed() {
    await events.send({ text: e.data })
    e = try await upstream.next()
  }
}))
  • stream(produce, init?): Response: a response whose body produce writes as it goes. The head (init's status and headers; content-type: application/octet-stream unless they give one, cache-control: no-cache) is sent when the handler returns it; then produce runs as an async function of its own with a Writer = { write(chunk: Bytes | string): Task<boolean>, closed(): boolean }. write settles once the connection has taken the chunk (text as UTF-8; at most 16 chunks wait, which is the backpressure), with false once the client has gone: stop then (closed() says so too). The body ends when produce's task settles; produce may fail (=> void ! json), and an error, a panic or a stop in it is logged and ends the body (the status is already sent). The request's timeout bounds the head only.
  • sse<T: data>(produce, init?): Response: server-sent events (text/event-stream, cache-control: no-cache, x-accel-buffering: no, what EventSource reads). produce gets Events<T> = { send(value: T, options?: { event?, id? }): Task<boolean>, comment(text): Task<boolean>, closed(): boolean }: send writes one event whose data is value as JSON (event: and id: lines when given), comment a : line (a keep-alive); both as write.
  • Request bodies as they arrive. serve's streamBody: string[] names paths (each one and what is under it) whose bodies aren't read before the handler: their req.body is empty and req.chunk(): Task<Bytes?> gives the body's next piece as it arrives (null at the end), up to maxStreamBody bytes (1 GiB; past it chunk fails with PayloadTooLarge { message }, and with a NetworkError when the body breaks off) rather than maxBody. Elsewhere chunk() gives the whole body once, then null. Read the body before answering: it goes with the request.
  • Reading events. res.events(): ServerEvents reads a fetched response's body as server-sent events (the HTML standard's parser: data: lines joined with \n, event:, id:, comments and unknown fields skipped, CR, LF or CRLF line ends): ServerEvents = { next(): Task<ServerEvent?> }, ServerEvent = { event, data, id } (event is message unless named; id the last id given), null when the body ends; a read that breaks off fails with a NetworkError. events(url, options?) does the GET itself (accept: text/event-stream, options.headers), and works in the browser too, with the engine's fetch: how a page reads its server's sse stream (try (try json.parse(e.data)) as Token). It is not EventSource: it doesn't reconnect. Await each next() before the next.

Routers ​

  • router(); .get/.post/.put/.patch/.delete/.all(path, handler); .use(middleware); .mount(prefix, router); .handle(req).
  • Handler = (req: Request) => Task<Response ! json>; Middleware = (req: Request, next: Handler) => Task<Response ! json>. A handler may fail with any error (every error type is data, so it fits json): the router passes it on, and a middleware sees it as the failure of next (try await next(req) passes it on).
  • A request runs the router's middleware in the order added (each calls next(req), with the same request or a new one, or answers itself), then the first route, in the order added, whose path and method match. A GET route answers HEAD too. A path some route matches with another method is a 405; no match is a 404. A mounted router's routes match below its prefix (by whole segments) and see path without it (url keeps the whole); its own middleware runs inside.
  • Paths: /todos/:id (one segment), /files/*path (the rest; may be empty), static segments; a trailing / doesn't matter. Parameters are percent-decoded.
  • A request whose handler failed is answered with a 500, and the error is printed (error: and its JSON, on stderr); a panic or a stop in its turns is a 500 too, printed with its name and message. In development (weft dev) the body says the error (its JSON) or the panic's name and message. std/auth's AuthError answers with its own status and message (auth.md): Unauthorized 401, Forbidden 403, RateLimited 429 (with retry-after); the server knows it by its kind.

Request-scoped values ​

local<T>(name) gives { name, get(req): T?, set(req, value): Request }: middleware sets a value on the request it passes on (next(user.set(req, u))), and handlers read it (user.get(req)). The values live with the request (req.locals is a handle on them) and go when it is answered. Values are values: set gives a new request; the old one doesn't see it.

Sessions ​

  • session(req): Session, from the request's signed cookie (weft.sid: the session's id and its HMAC-SHA256 signature, compared in constant time; a cookie whose signature doesn't hold is no session). Session = { id: string?, user(), get(key), set(key, value, res), login(user, res), logout(res) }.
  • user(): Task<Principal? ! StoreError>: who signed in. Principal = { id: string, name?: string, roles?: string[], data?: json }: what login stored.
  • login(user, res): a new session (a new id on every sign-in: no fixation; the old one is deleted) holding user, and res with its cookie set. logout(res): the session deleted, res with the cookie cleared. get(key) reads a value of the session; set(key, value, res) writes one (starting a session, and setting its cookie, when there is none or it ended). std/auth (auth.md) signs in and out on top of these, with the cookie on whatever answers the request.
  • A session's record is { user, values, flows, created, seen }. It ends maxAge after it started or idle after its last use (seen, written back at most once a minute, or a tenth of idle when that is shorter); an ended one is deleted when read.
  • sessions({ secret?, store?, cookie?, maxAge?, idle?, secure? }) configures them for the process (call it in a module's top level). The secret is options.secret, else WEFT_SESSION_SECRET, else random per process (sessions then end with it). maxAge (30 days) and idle (7 days) are in seconds. Cookies are Secure when the request came over HTTPS (x-forwarded-proto, forwarded or the origin say so) unless secure says otherwise. store is any { get(id), set(id, value, maxAge), delete(id), deleteUser?(userId) } returning tasks that may fail with StoreError (std/db's sessionStore(db) is one: db.md, Sessions); the default keeps them in the process's memory and can't fail. A session function whose store failed fails with that StoreError: user(), get and set, login and logout. deleteUser deletes every session whose user.id is userId (std/auth's sign-out everywhere needs it). The store also keeps std/auth's link tokens (keys auth:link:…).

Middleware in the box ​

  • cors(options?): answers preflight requests (OPTIONS with access-control-request-method: a 204 with the allowed methods, headers and maxAge) and adds access-control-allow-origin to the rest: every origin (*) unless origins lists them (then the request's origin when listed, nothing otherwise); with credentials, the request's origin and access-control-allow-credentials: true.
  • files(dir, options?): GET and HEAD of a file under prefix (default /) in dir, with an ETag (304 for a matching if-none-match) and cache-control: public, max-age=<maxAge seconds>; a folder serves its index.html. Paths can't climb out of dir (.., encoded or not, and hidden files are refused). Anything else goes on to next.

Serving ​

serve(options, handler): listens on options.host (else HOST, else 0.0.0.0) and options.port (else PORT, else 8080; 0 picks a free port) and serves every request with handler (a Router or a Handler); gives Server = { port, url, close() } once it listens, or fails with a NetworkError when it can't. maxBody (bytes) and timeout (ms, 30,000: a 504 past it) bound each request. The program runs while a server is open. close() stops accepting and settles once in-flight requests are answered (at most 10 s). SIGINT and SIGTERM close every server that way, and the program then ends.

The client ​

fetch(url, options?): Task<FetchResponse ! NetworkError> (options = { method?, headers?, body?: Bytes | string, json?, timeout? }; json sends JSON with its content type). It settles when the response's head has arrived: FetchResponse = { status, ok, url, headers, text(), json(), bytes(), chunk() }; any status is a response (ok for 200–299). text, json and bytes read the rest of the body; chunk() reads the next piece of it (null at the end), for streaming; a body can be read once (a second read panics). A request that can't be made (a bad URL included), or takes longer than timeout (30,000 ms by default, the whole exchange), fails with a NetworkError, and so does a body read that breaks off; json() fails with std/json's SyntaxError too. HTTPS uses the web's root certificates.

Errors ​

type
NetworkError = NetworkError { message: string, status?: number }a request that couldn't be made or answered: fetch and its body reads, serve that can't listen; a server function's call from the client (server.md): no connection, an answer that isn't a value, the server's 4xx or 500 (status)
StoreError = StoreError { message: string }a SessionStore that failed

Decided here ​

  1. Route handlers get typed params from the literal path, by a checker rule on Router's route methods, and a non-literal path is an error; elsewhere params is json.
  2. Bodies are read whole before the handler (bounded by maxBody), except on the paths streamBody names, which req.chunk() reads as they arrive (bounded by maxStreamBody); a response streams through a producer function (stream, sse) with awaited writes as backpressure.
  3. local<T>() keys, with values held host-side for the request's life, as the per-request store; request() for code that has no req (server functions, typed endpoints).
  4. Sessions are signed-id cookies over a pluggable store, with a fixed Principal record (id, and data for the rest) instead of a type parameter; they end by age and by idleness, and the store can delete a user's sessions at once.
  5. Cookie maxAge in seconds, as the attribute; every other duration in milliseconds.
  6. Handlers and middleware may fail with any error (! json): the request is answered with a 500 and the error printed, except std/auth's AuthError, which answers with its status. Handlers needn't map every error to a response themselves; one that wants another answer handles the error (try … else).

Weft is in development. The reference is the spec as it stands today.