Phone Fabric

Computer use with a leash. Phone Fabric lets an agent operate a real device UI — launch apps, tap, type, read the screen — but only inside a capability token its owner minted: every action permission-checked, metered against a budget, and written to an append-only audit trail.

Why It Matters

The last mile of agent autonomy is software that has no API. An enormous amount of the world is reachable only through an app's user interface — and the naive way to give an agent that reach is to hand it an open-ended screen and hope. That is exactly the failure mode this platform is built to avoid: an unattended agent with unscoped access to a general-purpose device is unscoped, full stop.

Phone Fabric is device operation rebuilt on the platform's authorization spine. An agent does not "get a phone" — it gets a session: a short-lived lease on a device from the registry's pool, opened under an IronKey capability token that names exactly which apps it may touch, how many actions it may take, whether it may see pixels at all, and how long the lease lives. The agent perceives the device as a structured accessibility tree, acts one audited step at a time, and loses everything — instantly, mid-session — the moment its token is revoked, exhausted, or expired.

The design bet is simple: computer use becomes safe the same way money movement became safe here — not by trusting the agent, but by making every unit of authority explicit, bounded, and revocable.

Authority: a Capability, Not a Credential

Phone access is never a property of the agent's own JWT. Every Phone Fabric call carries two credentials: the agent's ordinary JWT (who is acting) and an att_ capability token in the X-Capability-Token header (what it is allowed to do right now). The token must carry the phone.use permission and rides the full Layer-4 machinery documented in Agent Access Control — which is precisely why the model is trustworthy:

• Issuing is deliberate. Minting a capability token requires the authz.delegate meta-permission, which no ordinary role grants. Someone with authority chose to give this agent a phone lease.
• Authority is an intersection. On every call, the token's effective permissions are caveats ∩ the issuer's current authority. Clamp or suspend the issuer and every phone token it ever issued goes empty on the next request.
• Revocation cascades — and kills sessions live. Revoking a token (or any ancestor in its delegation chain) revokes its whole subtree in one operation, and any active phone session authorized by a token in that subtree is flipped to revoked in the same instant. There is no cleanup lag to exploit: the next action simply fails.
• Budgets are arithmetic. The token carries a spend budget and an optional use count; phone activity draws both down. When either runs out the token is exhausted and the session cannot continue.

The phone-specific caveats on the token define the shape of the lease:

Caveat	Effect
phone_allowed_apps	App allowlist — launching (or tapping into) any package outside it is denied and audited
phone_max_actions	Hard ceiling on the number of actions in the session
phone_no_screenshot	Tree-only observation — screenshot requests are refused, structure is all the agent gets
phone_ttl_seconds	Session lifetime; absent, a registry default applies (ten minutes out of the box)

A parent agent can attenuate a phone capability down to a sub-agent exactly like any other capability — fewer apps, fewer actions, shorter life — and the child token can never be looser than its parent on any axis.

Session Lifecycle

Start resolves the token, checks the pool against the registry's concurrency cap (a full pool answers 429 rather than queueing), leases a device instance, and opens the session in a known home state — optionally launching a first app in the same call if the allowlist permits. Every subsequent call re-proves the lease: the session must still be active, the token must still be valid and unexpired, and the TTL must not have passed. A session ends one of four ways — the agent ends it, the TTL lapses (expired), the token dies (revoked), or an operator kills it — and in every case the device returns to the pool.

Acting

An agent drives the device one action per call. Each action names a type, an optional target from the observed tree, and text where relevant:

Action type	Meaning
launch	Open an app by package name (allowlist-checked)
tap	Tap a node by its tree id (tapping an app icon is allowlist-checked too)
type	Enter text into an editable node
swipe · key	Scroll/gesture, or press a hardware key
back · home	Navigate back, or return to the home screen

The checks run in a fixed order before anything touches the device: the max_actions ceiling, then the app allowlist for anything that would enter a new package. A denied action costs nothing but is still audited with its reason (max_actions, app_not_allowed:<package>) — the trail records what the agent tried, not just what it did. An allowed action executes on the device, debits one metering unit, advances the session's action counter, and returns the resulting package, screen, and a sequence number that matches its audit row.

Seeing: the Accessibility Tree

Observation is tree-first. An observe call returns the current screen as a structured tree of UI nodes — each with an id, role (button, text, input, list, …), visible text, owning package, and whether it is tappable or editable — plus the current package and screen name. That structure is what the agent reasons over and what action targets refer to.

Screenshots are the exception, not the default: they must be requested explicitly per call, and a lease minted with phone_no_screenshot refuses them outright (the response says so with screenshot_blocked: true). This is a deliberate privacy and containment posture — a tree-only lease lets an agent operate an app without ever being able to exfiltrate raw pixels, and it keeps the audit trail meaningful, because targets are named nodes rather than blind coordinates. Observation is metered against the token budget like any action (looking spends authority too), though only mutating actions count toward the max_actions ceiling.

Metering & Limits

Every session start and every action or observation debits the capability token's budget — one unit each by default, operator-tunable. This is the standard Layer-4 consume debit: a counter on the token, checked before anything happens, flipping the token to exhausted when the budget or use count runs out.

It is worth being precise about what this metering is not: it is not money movement. Phone Fabric sits entirely on the deny-only side of the platform — budgets are bookkeeping about authority, evaluated pre-escrow, and no value ever moves as a result of a phone action. Like the rest of IronKey, the fabric can stop things; it can never spend. The supply invariant cannot see it.

The limits stack, from narrowest to widest:

Scope	Limit
Per session	TTL · max_actions · app allowlist · screenshot policy
Per token	Spend budget · max uses · expiry · issuer's live authority (clamp the issuer, the token empties)
Per registry	Master enable switch · enforcement mode · concurrent-session cap · per-action cost

Endpoint Reference

All agent endpoints require the agent JWT and the X-Capability-Token header; act and observe additionally require the presented token to be the same one that opened the session — a token cannot hijack another token's lease.

Endpoint	What it does
POST /api/v1/phone/sessions	Start a session: lease a device, arm the TTL, optionally launch a first app
POST /api/v1/phone/sessions/{id}/act	Perform one metered, caveat-checked action
GET /api/v1/phone/sessions/{id}/observe	Read the accessibility tree (?screenshot=true honored only if the lease allows)
POST /api/v1/phone/sessions/{id}/end	End the session and release the device
GET /api/v1/phone/sessions/mine	Your recent sessions with status and end reason

POST /api/v1/phone/sessions
Authorization: Bearer <agent-jwt>
X-Capability-Token: att_…
{ "launch": "com.example.notes" }

POST /api/v1/phone/sessions/{id}/act
Authorization: Bearer <agent-jwt>
X-Capability-Token: att_…
{ "type": "tap", "target": "node-42" }

A successful start returns the session id, its expiry, the caveats in force (allowed apps, action ceiling), and the token budget remaining — the agent knows the exact shape of its leash from the first response.

The Audit Trail

Every session writes an append-only, per-action trace: sequence number, action type, target, metering cost, outcome (ok, error, or denied), and detail. Denials are first-class entries — an agent probing the edges of its allowlist leaves the same quality of record as one doing its job. Session boundaries additionally emit lifecycle events (PhoneSessionStarted, PhoneSessionEnded) that flow to the owner's notification channel, so a human can watch their agent pick a device up and put it down in near-real time.

Operator Safety Rails

The operator's view assumes the worst day, not the best one. Admin endpoints (platform-admin credential):

Endpoint	What it does
GET /api/v1/admin/phone/pool	Device pool state: instances, leases, active sessions vs the concurrency cap
GET /api/v1/admin/phone/sessions	Every session on the registry, plus the fabric's enabled/backend/mode status
GET /api/v1/admin/phone/sessions/{id}/actions	The full action trace for one session — the live action viewer's data source
POST /api/v1/admin/phone/sessions/{id}/kill	Surgically end one active session and release its device
POST /api/v1/admin/phone/global-halt	Kill every active session and release every device, in one call

Kill and halt are status-only operations: they end authority and free hardware, and touch nothing else — no balance, no token budget rewrite, no side effects to reconcile. Between the owner's cascade revoke (IronKey), the operator's per-session kill, and the global halt, there are three independent hands on the brake.

Control Plane vs Data Plane

Be clear about what is protocol and what is provisioning. The control plane is the protocol surface — sessions, capability gating, caveats, metering, the audit trail, the admin rails — and it is what this chapter documents; it ships in the registry and behaves identically everywhere. The data plane is the devices themselves, and that is a registry-operator choice: a deterministic mock device (the permanent test surface — a scripted phone that behaves identically on every run, ideal for developing and evaluating agents), or emulated Android devices driven over the debug bridge, where action targets are real resource identifiers instead of mock node ids.

The driver contract is the same either way — tree-first observation, one action at a time, with all capability checks, metering, and audit owned by the registry rather than the device layer. But device capacity, device type, and whether the fabric is enabled at all vary by registry operator. Check before you build: the fabric answers 503 where it is disabled, and the admin pool view states the backend and capacity where it is not.

Status

Phone Fabric's control plane ships in the registry today, dark by default. Two switches stage it: a master enable (PHONE_FABRIC_ENABLED, default off — the endpoints refuse service until an operator turns it on) and an enforcement mode for the phone.use permission check (PHONE_USE_MODE: off → shadow → enforce, the same staged-rollout discipline as the rest of IronKey). A valid capability token is required in every mode — staging only governs whether a token lacking phone.use is logged or refused. The capability path — issue, attenuate, meter, exhaust, cascade-revoke with live session kill — is exercised end-to-end against the deterministic mock backend; emulated-device data planes are provisioned per registry, so availability varies by operator.

See also Agent Access Control for the capability and delegation model the fabric rides, Agents & Identity for the credentials underneath it, Security Architecture for the zero-trust posture it extends to devices, and The Event Ledger for why deny-only systems never touch the supply invariant.