Native background processes for WASI extensions

Native background processes for WASI extensions

Native background processes for WASI extensions

Function

The optional Rust SDK host-process feature exposes

bt_extension_sdk::host_process::request(json) to extension implementations.

It starts native programs using argument arrays and returns immediately. Polling

returns bounded output and state without waiting for process completion. This

requires the updated BT host; older 1.1.4 binaries do not provide the import.

This is a native process permission, outside the WASI sandbox. Only trusted

extensions and executables should receive it. Path declarations validate listed

files, but cannot constrain arbitrary behavior of a native program. FFmpeg is

selected by the video extension; the host has no media-specific logic.

Syntax

The manifest declares "permissions": ["process", "fs_read", "fs_write"].

The SDK imports bts_host.process_request(i32, i32, i32, i32) -> i32 with UTF-8

JSON requests and {ok: value} or {error: message} response envelopes. The SDK

unwraps the envelope into Result<String, String>. This is an extension-author

API; BT applications normally use video jobs.

Request fields

FieldTypeRequiredDefaultRange / meaning
opStringYesNonespawn, poll, cancel, close.
programStringFor spawnNoneNonempty executable path or PATH name; no implicit shell.
argsArray[String]For spawnNoneAt most 256 arguments; all request JSON at most 64 KiB.
timeout_msIntNo600001–300000; includes startup and automatically enforced by the worker.
read_pathsArray[String]No[]Existing project-relative paths; requires fs_read.
write_pathsArray[String]No[]Project-relative outputs, existing parent; requires fs_write.
cleanup_pathsArray[String]No[]Caller-owned empty ordinary files reserved with create-new semantics; removed after failure, cancellation or timeout and process reaping; requires fs_write.
idIntExcept spawnNoneExisting positive task ID in this extension instance.
discard_outputBoolNofalseFor close only: atomically arrange deletion of owned cleanup_paths, even if the worker just succeeded. Ordinary close retains successful output.

Listed paths reject parent traversal and canonical paths outside the project root.

The executable's working directory is the project root. Native execution retains

the operating-system account's authority. Process and requested filesystem

permissions must also be enabled by the running BT process.

Return value

spawn returns {id: Int}. close removes the handle immediately and returns

{closed: true}; cancellation and process reaping continue in the worker. Poll

and cancel return these fields:

FieldTypePresentDefaultMeaning / range
stateStringAlwaysNonequeued, running, succeeded, failed, cancelled, timed_out.
stdout, stderrStringAlwaysEmptyLast 1 MiB of each stream; invalid UTF-8 uses replacement characters.
stdout_truncated, stderr_truncatedBoolAlwaysfalseEarlier bytes exceeded the retained tail.
exit_codeInt or JSON nullAlwaysnullProcess exit code when available; null while running or without a numeric code.
elapsed_msIntAlways0Milliseconds since submission; frozen at terminal state.

BT example

Resource and platform notes

Each instance allows four active processes and 32 retained handles; the host

process allows 32 active processes across instances. Full capacity rejects new

jobs immediately. Closing a running handle does not free its active reservation

until the worker has reaped its process. There is no unbounded waiting queue.

Each stream retains at most 1 MiB; the WASM SDK reuses a 16 MiB response buffer

to cover JSON escaping. Closing all handles releases their output storage.

Windows uses hidden child processes and a private kill-on-close Job Object;

Unix uses a separate process group for normal cancellation. Native programs must

not deliberately escape their process group/job. Abrupt Unix host termination

does not guarantee descendant cleanup. Windows x64 is exercised locally; Unix

code requires validation on its target. Dropping a store requests cancellation of

its children. Successful outputs remain available after close.

Web request scripts already run in BT's bounded blocking pool. Media commands

run in dedicated bounded workers; applications poll once per request rather than

sleeping in a request. No extra work is added to unrelated VM instructions.