A local alternative to real-device clouds for Android QA automation with YAML

In short: BenchFleet runs Android QA workflows on your own phones from a desktop app. Each workflow is a YAML file you can keep in version control, built in a visual editor or recorded by doing the task once, and runs on one phone or every phone, on a schedule or from CI. Maestro's YAML flows also run locally. Pro costs £12 a month + VAT.

Writing tests as YAML keeps them readable for the whole QA team and easy to review in a pull request. Running them on your own phones instead of a real-device cloud keeps builds and test accounts in-house and the cost flat. This page compares the tools that do both, with their websites. Checked on 23 October 2026 on each tool's own pages. We make BenchFleet.

Free for up to 3 phones, with no account. Pro adds workflows, schedules and AI for up to 25 phones from £12 a month + VAT.

Download free See pricing

The options compared

Option	Test format	Where it runs	Price as seen	Website
BenchFleet	YAML workflows, also built in a visual editor or recorded on a phone	Your computer and the phones connected to it; other computers through gateways (Team)	Free for 3 phones; workflows on Pro, £12 a month + VAT	benchfleet.com
Maestro	YAML flows (`launchApp`, `tapOn`, `assertVisible`)	Its CLI on your computer, with emulators and physical Android phones; Maestro Cloud for hosted devices	Free locally; Maestro Cloud $250 per device a month	maestro.dev
Appium	Test code in Java, Python, JavaScript and other languages	A local Appium server with your phones	Free, open source	appium.io
DeviceFarmer (STF)	None: browser control and remote adb for devices	Linux or BSD servers you run	Free, open source	github.com/DeviceFarmer/stf
DeviceLab	Runs your Maestro, Appium, Espresso, XCUITest or WebdriverIO suites	Devices you own, coordinated by its hosted service	Free for one device; then $99 per device a month	devicelab.dev
Kobiton	Appium, plus scriptless tests captured from a manual session	Its cloud, or on-premises	Startup from $83 a month; on-premises on request	kobiton.com
HeadSpin	Its automation through the Automation+ add-on on Go, included on Pro	Its cloud; private or on-premise on Pro	Go from $83 to $100 a month; Pro per project	headspin.io
AWS Device Farm	Test suites you upload	AWS data centres; private devices	$0.17 per device minute; private devices from $200 a month	aws.amazon.com/device-farm

What the YAML looks like

A Maestro flow lists commands against the app:

yaml

appId: com.android.settings
---
- launchApp
- tapOn: "About phone"
- assertVisible: "Android version"

A BenchFleet workflow names each step, finds elements by their text or ID, and checks what should appear after each tap. This is part of the example that opens Settings and reads the Android version:

yaml

schema: controlhub/workflow@1
id: about-phone
name: About phone check
defaults:
  timeout: 8s
  retry: {max_attempts: 2, backoff: 1s}
steps:
  - id: open_settings
    action: app.restart
    with:
      package: com.android.settings
      post: {foreground: com.android.settings, timeout_s: 8}
  - id: open_about
    action: ui.tap
    with: {selector: {text_in: [About phone, About device]}}
    post: {appears: {text_in: [Model name, Model]}, timeout_s: 8}
  - id: read_version
    action: ui.read_text
    with:
      selector:
        resource_id: android:id/summary
        below: {text_in: [Android version]}
    save_as: shown

You rarely type this yourself: record the task once on a phone, or add steps in the visual editor, and edit the file when you want to.

How BenchFleet runs YAML workflows Pro

On one phone, a group or every phone that matches, after confirming how many phones a run will reach. A dry run shows what would happen without touching the phones.
On a schedule: at a set time, every few minutes or hours, or once, with a choice of what happens to a run the computer missed.
From CI: a signed webhook starts a workflow after each build.
New versions safely: publish a new version to a few canary phones first, promote it to every phone after enough successful runs and a person's approval, or roll back in one click.
Every run on record: a timeline with screenshots, and a replay of each step with the screens before and after it. See replaying an Android automation step by step.
Steps that survive small changes: elements are found by their text and role first, with image and text recognition on the screen as a fallback.

The same phones stay available for testing by hand and for AI agents over MCP. See Android app testing on real phones and the self-hosted lab and device cloud comparison.

Maestro, Appium, DeviceFarmer, DeviceLab, Kobiton, HeadSpin and AWS Device Farm are the names of their owners' products and projects. BenchFleet is not affiliated with any of them.

Questions

Can I write Android UI tests as YAML and run them on my own phones?

Yes. BenchFleet workflows are YAML files that run on the phones connected to your computer, and Maestro flows are YAML files that its free, open-source CLI runs on emulators and physical Android phones.

How is a BenchFleet workflow different from a Maestro flow?

Both describe steps such as opening an app, tapping an element and checking text. A BenchFleet workflow also runs inside a fleet app, on one phone or many, with schedules, staged roll-outs of new versions, approvals, a replay of every run and phones a person can take over. Maestro is a test framework you run from the command line or Maestro Studio.

Do I have to write the YAML by hand?

No. In BenchFleet you can record a workflow by doing the task once on a phone, or build it in the visual editor, and the YAML is written for you. You can still edit the file and keep it in version control.

Can my CI pipeline start the workflows?

Yes, on Pro and Team. A signed webhook starts a workflow from your pipeline, for example after each build, and each run keeps a timeline with screenshots.

Which cloud alternatives run on-premise instead?

Kobiton offers full on-premises set-ups, HeadSpin deploys on-premise on its Pro plan, and AWS Device Farm offers private devices in its cloud. Their prices are by quote or on their pricing pages.

Start with the Free plan