# Architecture — how a command reaches the phone

> How one command travels from your AI to the phone, and the three transports Aster speaks: remote WebSocket, on-device Ktor MCP server, and Binder IPC.

Aster here is the open-source Model Context Protocol server and Android companion app for AI-driven phone control, published on npm as aster-mcp. It is unrelated to Aster DM Healthcare, Aster Data Systems or the ASTER instrument on NASA’s Terra satellite.

Source: https://aster.matterwardlabs.com/architecture/ · Part of Aster (https://github.com/satyajiit/aster-mcp) · Last generated 2026-09-13

## How one command reaches the phone

1. **Your AI client** — The assistant picks a tool and calls it by its full name — aster_take_screenshot, not take_screenshot — over Streamable HTTP to http://localhost:5988/mcp. Arguments travel as JSON. Nothing in this hop knows a phone exists.
2. **Aster server — admit** — The MCP handler parses the arguments against that tool's schema and resolves the target device. A device that was never approved, or that has no live socket in this process, is refused here. A rejected call never reaches the phone and never touches a permission.
3. **Aster server — dispatch** — sendCommand mints a UUID, drops the aster_ prefix and pushes a single JSON frame down the device WebSocket on port 5987. The promise it returns is held open against that id. (30 s timeout)
4. **Android companion** — The command handler on the phone dispatches the action to whichever subsystem owns it — the accessibility service, MediaStore, telephony, CameraX. The kill-switch notification and the fail-closed package policy apply before any screen control runs, in every transport. (~90 ms to ~10 s)
5. **Result frame** — The phone answers on the same socket, carrying the id it was handed. The server matches that id to the waiting promise and clears the timeout. An answer that arrives late, or for a command nobody is waiting on, is dropped rather than resolved.
6. **Back to the model** — The handler shapes the payload into MCP content — text for structured results, a base64 image for screenshots and photos — and returns it as the tool result. It goes to your client and nowhere else; the server keeps no copy off your machine.

## Five commands, traced

Five real commands through the hops above, each with the tools involved and a round-trip figure. The prompt is what a person types; everything after it is what the machinery does.

### Take a screenshot of my phone

- **Tool call:** aster_take_screenshot with the device id — the only required argument.
- **Server:** Resolves the approved device and sends the action take_screenshot down the socket.
- **Device:** The accessibility service grabs the current frame and encodes it as a JPEG at quality 75.
- **Result:** The capture comes back as base64 image content the model can actually look at. Large frames are written to device storage first and fetched with a follow-up read, so a big screenshot cannot blow up the WebSocket frame.
- **Tools:** `aster_take_screenshot`, `aster_get_screen_hierarchy`

### Find all my beach photos from last December

- **Tool call:** aster_search_media with the sentence as written; the server parses it into a date window plus keywords.
- **Server:** Sends search_media with the parsed filter rather than the raw sentence, so the phone does no language work.
- **Device:** A MediaStore query, then an EXIF and location pass over the matches.
- **Result:** Matching photos come back as metadata with timestamps and locations intact. The images themselves stay on the phone until you ask for one.
- **Tools:** `aster_index_media_metadata`, `aster_search_media`

### Vibrate my phone, I dropped it behind the couch

- **Tool call:** aster_vibrate with a waveform pattern of [0, 500, 200, 500].
- **Server:** Nothing to marshal beyond the pattern array — the shortest path through the handler.
- **Device:** Vibrator.vibrate with a waveform effect built from the pattern.
- **Result:** Two 500 ms pulses with a 200 ms gap. The fastest trace on this page, about 90 ms round trip.
- **Tools:** `aster_vibrate`, `aster_play_audio`

### Call Mom and tell her I will be about 20 minutes late

- **Tool call:** aster_make_call_with_voice with the number, the sentence to speak, and waitSeconds: 8.
- **Server:** Routes make_call_with_voice and then waits. The eight seconds are spent on the device while the call connects, not in the server.
- **Device:** A call intent, speakerphone forced on, then text-to-speech says the sentence once the line is up.
- **Result:** The message is delivered on speakerphone after the wait. About 10 s end to end, nearly all of it call setup. This is also the tool a dedicated AI phone uses to ring you unprompted — same call, nobody typed the prompt.
- **Tools:** `aster_make_call_with_voice`, `aster_make_call`, `aster_speak_tts`

### Anything urgent on my phone?

- **Tool call:** aster_read_notifications with a priority filter.
- **Server:** Sends read_notifications; the filtering happens against what the phone reports, not a cached copy.
- **Device:** The notification listener reads the shade and ranks what is there — 14 active at the time of this call.
- **Result:** Two urgent messages, one missed call, one delivery window. About 150 ms round trip, which is why this is the one people leave running.
- **Tools:** `aster_read_notifications`, `aster_post_notification`

## Transports

| Mode | Where the MCP server runs | Tool namespace | When to use it |
|---|---|---|---|
| Remote WebSocket | Node server on your machine. Phone dials out to port 5987; clients speak MCP over HTTP on port 5988. | aster_* — 49 MCP tools | The default. You run the server on a laptop or a home box and point any MCP client at it. The only mode with the server-side device-approval gate. |
| On-device MCP server | Ktor plus the MCP Kotlin SDK, embedded in the app and running on the phone itself. Default port 8080. No Node server in the middle. | unprefixed actions — 77 in the on-device catalog | You want a client to reach the phone directly, on the LAN or over a private mesh. Trust is whatever your own network controls give you. |
| Binder IPC | Same device, no network hop at all. An app on the phone — OpenAlly, for example — binds Aster's service directly. | unprefixed actions — 77 in the on-device catalog | An agent already running on the phone drives it locally: a 32-character token checked in constant time plus an on-device approval prompt. Lowest latency, works with the radio off. |

## Ports

Running `aster start` binds three ports on the machine you run it on. The phone only needs to reach the first one; you only need to reach the other two. Open nothing to the public internet — put the link on your LAN or a private mesh.

- `5987` — Device WebSocket. The Android companion connects here.
- `5988` — API + MCP HTTP. Streamable-HTTP MCP endpoint at /mcp; REST API at /api.
- `5989` — Web dashboard. Device registry, approvals, live screen control, logs.

The on-device MCP server is the exception: it runs inside the app on the phone and listens on port 8080 by default, so none of the three ports above exist in that mode. Binder IPC opens no port at all.

## Addresses

- **MCP endpoint for clients:** `http://localhost:5988/mcp`
- **Health check:** `http://localhost:5988/api/health`
- **Web dashboard:** `http://localhost:5989`
- **Device WebSocket:** `ws://<server-ip>:5987`