Native background processes for WASI extensions

Native background processes for WASI extensions

Native background processes for WASI extensions

Function

The optional Rust SDK host-process feature exposes

bt_extension_sdk::host_process::request(json) to extension implementations.

It starts native programs using argument arrays and returns immediately. Polling

returns bounded output and state without waiting for process completion. This

requires the updated BT host; older 1.1.4 binaries do not provide the import.

This host capability runs outside the WASI sandbox and is available uniformly to

extensions when the BT build includes it. Native programs inherit the authority

of the operating-system account. Path declarations validate listed files for

ownership and cleanup, but cannot constrain arbitrary native-program behavior.

Installing an extension therefore means trusting its code. FFmpeg is selected by

the video extension; the host has no media-specific logic.

Syntax

No manifest permission declaration is required. The SDK imports

bts_host.process_request(i32, i32, i32, i32) -> i32 with UTF-8

JSON requests and {ok: value} or {error: message} response envelopes. The SDK

unwraps the envelope into Result<String, String>. This is an extension-author

API; BT applications normally use video jobs.

Request fields

FieldTypeRequiredDefaultRange / meaning
opStringYesNonespawn, poll, cancel, close.
programStringFor spawnNoneNonempty executable path or PATH name; no implicit shell.
argsArray[String]For spawnNoneAt most 256 arguments; all request JSON at most 64 KiB.
timeout_msIntNo600001–300000; includes startup and automatically enforced by the worker.
read_pathsArray[String]No[]Existing project-relative paths retained as task ownership metadata.
write_pathsArray[String]No[]Project-relative outputs with an existing parent.
cleanup_pathsArray[String]No[]Caller-owned empty ordinary files reserved with create-new semantics; removed after failure, cancellation or timeout and process reaping.
idIntExcept spawnNoneExisting positive task ID in this extension instance.
discard_outputBoolNofalseFor close only: atomically arrange deletion of owned cleanup_paths, even if the worker just succeeded. Ordinary close retains successful output.

Listed paths reject parent traversal and canonical paths outside the project root.

The executable's working directory is the project root. Native execution retains

the operating-system account's authority. The optional process-wide BT policy

must allow process execution and filesystem use for requests listing paths.

Return value

spawn returns {id: Int}. close removes the handle immediately and returns

{closed: true}; cancellation and process reaping continue in the worker. Poll

and cancel return these fields:

FieldTypePresentDefaultMeaning / range
stateStringAlwaysNonequeued, running, succeeded, failed, cancelled, timed_out.
stdout, stderrStringAlwaysEmptyLast 1 MiB of each stream; invalid UTF-8 uses replacement characters.
stdout_truncated, stderr_truncatedBoolAlwaysfalseEarlier bytes exceeded the retained tail.
exit_codeInt or JSON nullAlwaysnullProcess exit code when available; null while running or without a numeric code.
elapsed_msIntAlways0Milliseconds since submission; frozen at terminal state.

BT example

Resource and platform notes

Each instance allows four active processes and 32 retained handles; the host

process allows 32 active processes across instances. Full capacity rejects new

jobs immediately. Closing a running handle does not free its active reservation

until the worker has reaped its process. There is no unbounded waiting queue.

Each stream retains at most 1 MiB; the WASM SDK reuses a 16 MiB response buffer

to cover JSON escaping. Closing all handles releases their output storage.

Windows uses hidden child processes and a private kill-on-close Job Object;

Unix uses a separate process group for normal cancellation. Native programs must

not deliberately escape their process group/job. Abrupt Unix host termination

does not guarantee descendant cleanup. Windows x64 is exercised locally; Unix

code requires validation on its target. Dropping a store requests cancellation of

its children. Successful outputs remain available after close.

Web request scripts already run in BT's bounded blocking pool. Media commands

run in dedicated bounded workers; applications poll once per request rather than

sleeping in a request. No extra work is added to unrelated VM instructions.