The Device Truth Audit

The problem

In every fleet I’ve looked at, including my own, a third to half of QA time goes to test maintenance: re-running flakes, chasing device-only failures, arguing with red CI.

AI coding assistants make it worse. Code merges faster than anyone can verify it on a real device.

I use AI agents for everything - and trust none of it until it runs on a real phone. That’s the whole business.

What you get

Week 1: triage. Your failure history (CI logs, test reports) clustered by failure signature, separating true regressions, environment flakes, device-specific failures, and test-authoring debt. The method is built on the machine-learning test-triage system I built at Amazon Prime Video: 500K+ test results a month, 98% clustering accuracy.

Week 2: root cause on real devices. Top offenders reproduced and root-caused on physical hardware with runtime debugging (breakpoint-level, not log-guessing) and full evidence capture: screen video, device logs, UI-tree diffs.

Deliverable. A ranked-fix report: each top failure with root cause, evidence, effort estimate, and a prioritized order that maximizes suite stability per engineering hour. Plus a flake-taxonomy breakdown your team keeps using after I’m gone. I diagnose and rank. Your team implements.

Read a sample report → The real method, run against my own fleet (50+ repos) first.

Price

Fixed price, from $7,500: one app, up to two platforms (e.g. iOS + Android from one codebase). More platforms or more apps are scoped in the statement of work (SOW).

No retainer, no license, no commitment beyond the two weeks. If I find nothing worth fixing, the report says so.

This isn’t for pre-launch teams or teams without a CI failure history yet - there’d be nothing to diagnose. It’s for teams that already ship, don’t trust their suite, and want to know why before the next release.

You’d be an early client. That’s why the price starts here, and why the sample report runs on my own repos: check every claim in it before you pay for one on yours.

Non-disclosure agreement (NDA) on request. Your CI history and builds are reviewed locally for the two-week engagement. Every artifact is delivered to you and deleted from my systems after handoff.

The bench

The audit runs on the same stack that builds, tests, and ships my own four-platform app (Android, iOS, Wear OS, watchOS). AI agents drive physical phones, watches, and tablets (install, drive, record, debug), with a runtime debugger attached when it matters.

It exists because I use it daily, not because a client asked for it.

My background: ten years of mobile and test infrastructure at Amazon (Prime Video), Paramount, and Ticketmaster. Device farms, test frameworks, release platforms, ML test triage. AWS triple-certified, PMP.