Skip to content

Tool Pick|android-agent — let an agent drive your Android phone, ~10x cheaper than screenshots

Nov 20, 20261 min
TL;DRandroid-agent is an MCP server that lets an agent drive an Android phone by reading the accessibility tree as a text list instead of sending screenshots. Install: clone the repo and run python setup.py. It solves the token bill phone-automation agents rack up every time they look at the screen.

Look up & save

Loading dictionary…

中文版

Tool info

FieldValue
Nameandroid-agent
TypeCLI + MCP server
GitHubnebigoktug/android-agent
Stars4 (at review time; repo created 2026-11-18, version v0.4)
LanguagePython 3.10+
LicenseMIT (the optional ADB Keyboard component is GPL-2.0)
Installgit clone, then python setup.py

What problem it solves

You want an agent to drive your phone — test a flow in an app, check a setting, look something up on a page that only exists in the mobile version. Most phone-automation agents today work by taking a screenshot, deciding on an action, and taking another screenshot to confirm — and every single look at the screen means handing the model an image, at roughly 1,500 tokens a pop. Driving a phone usually means looking dozens of times in a row, so the token bill and the latency both climb linearly with every screenshot.

android-agent reads the screen a different way: instead of a screenshot, it reads Android's native accessibility tree and turns the screen into a numbered text list — something like [5] button "Wi-Fi · HomeNetwork" — and the agent just replies "tap 5". The project's own BENCHMARK.md gives the numbers: across 6 everyday tasks (Settings, a YouTube search, Calculator, Clock, Play Store, Maps), 19 screen reads cost about 2,700 tokens total as text lists, versus about 28,500 tokens for the same 19 looks as screenshots — nearly a 10x difference. It only falls back to a screenshot or OCR when the text list genuinely doesn't capture what's on screen (games, some Flutter apps, maps).

Good fit for: building a test, support, or task agent for a phone app without burning most of your context budget just looking at the screen; or running an agent that drives the phone itself on an old handset with no PC around, using Termux alone.

android-agent's blind-tap guard
android-agent's blind-tap guardThe agent reads the screen to get a numbered list, picks an item to tap, and android-agent re-reads the screen right before tapping to confirm nothing changed before it acts. read current screen numbered text list decide to tap item 5 re-read screen before tap screen matches, tap sent Agent android-agent Android phone
  1. Agent requests the current screen
  2. android-agent returns a numbered text list
  3. Agent decides to tap item 5
  4. android-agent re-reads the screen right before tapping
  5. Screen still matches, so the tap is sent

Getting started

Install

From a computer (easiest):

# install adb first
winget install Google.PlatformTools    # Windows
brew install android-platform-tools     # macOS
sudo apt install adb                    # Linux

# on the phone: Settings → About phone → tap "Build number" 7 times to unlock Developer options
# turn on "USB debugging" in Developer options, plug in a USB cable, and tap Allow on the phone

git clone https://github.com/nebigoktug/android-agent
cd android-agent
python setup.py

Directly on the phone, no PC needed (Termux):

# install Termux from F-Droid (not the outdated Play Store version)
pkg install -y git python android-tools
git clone https://github.com/nebigoktug/android-agent && cd android-agent
python setup.py

Register it as a Claude Code MCP server:

claude mcp add phone -- python /path/to/android-agent/mcp_server.py

Basic usage

python phone.py screen                 # read the current screen (then only what changed)
python phone.py tap 5                  # tap by the number from the last screen read
python phone.py tap "Bluetooth"        # or tap by label, matched against the live screen
python phone.py find "Developer options"  # scroll down automatically until it's on screen
python phone.py type "hello" , key enter  # chain commands with a comma

Through MCP, an agent calls the same capabilities directly as tools: screen, tap, type, swipe, key, open_app, open_link, list_apps, wait, screenshot, rotation_lock.

Advanced usage

Skip the agent harness entirely and let agent.py run the whole loop itself from a single natural-language instruction:

pip install anthropic
export ANTHROPIC_API_KEY=...
python agent.py "turn on Bluetooth and tell me which devices are paired"

Before anything that sends a message, pays, deletes, installs an app, or changes an account setting, agent.py stops and asks you in the terminal first.

Compared to the alternatives

android-agentscreenshot-based phone agentsAppium
How the screen reaches the modelaccessibility tree → text listscreenshot → image tokensUI element tree (not LLM-oriented)
Token cost per screen readlow (~150 tokens, per the project's own benchmark)high (~1,500 tokens, per the same benchmark)not applicable
Native MCP supportdepends on the project needs a bridge you build
Runs standalone on the phone (no PC) via Termuxusually needs a computer needs an Appium server
Re-checks the screen right before tapping built independs on the projectleft to your own test script
Built forAI agents driving a phoneAI agents driving a phoneQA test automation

Things to watch for

  • Still early (v0.4): the author has only tested it by hand on one Xiaomi phone running Android 16 (Termux, wireless). CI only runs device-free tests on Windows/macOS/Linux, so it hasn't been exercised across the UI quirks of other Android brands — run a full pass by hand before wiring it into anything automated.
  • Enabling USB/wireless debugging is itself a risk: it's an Android developer setting that exposes real control over the device. Turn it on only for a computer you trust, and turn it back off when you're done.
  • The optional companion app needs the Accessibility permission: it cuts screen-read latency from about 3 seconds to about 0.4 seconds, but accessibility access can read almost everything on screen — the README says so itself. Only install it from a source you trust.
  • Mixed licenses: the main project is MIT, but the optional ADB Keyboard component is GPL-2.0 (a third-party package by senzhk) — worth checking the boundary before any commercial integration.

Today's takeaway

Most tools for "letting AI operate a screen" default to treating it as a computer-vision problem — screenshot, guess the coordinates, screenshot again to confirm. android-agent asks the opposite question: the device already carries a structured-text representation of the screen for screen readers, so why re-encode it into an image just to have the model decode it back into structure? Keeping "reading the screen" inside the data structure the system already exposes natively — instead of routing it through an image round-trip — is the real reason it saves roughly 10x in tokens; the saving isn't about a smarter model, it's about skipping an unnecessary encode/decode detour.

References