Every desktop flow forces the same decision at some point: do you target the application through Power Automate Desktop selectors, or do you fall back to image recognition and click on pixels? I have built flows both ways, and I have babysat flows built both ways. The short answer is that selectors break less often and are cheaper to fix. But the honest answer needs numbers, because there are situations where image recognition is genuinely the only option, and pretending otherwise sets teams up for pain.
One caveat before the comparison. If the target system has an API or a documented connector, neither of these is the right answer. I wrote about that in Stop Reaching for Desktop Flows. This post assumes you have already lost that argument and the UI is all you have.
What Breaks Power Automate Desktop Selectors
Selectors read the application’s accessibility tree. They break when the tree changes: a developer renames a control, an update reshuffles the window hierarchy, or the selector was recorded with a fragile ordinal like Button[3] instead of a stable attribute like AutomationId. The failure is loud and specific. You get an error naming the exact UI element that could not be found, and the flow stops at that step.
Image recognition breaks when pixels change. A theme update, a new font, Windows display scaling moving from 100% to 125%, a notification toast overlapping the target, or an unattended machine resolving at 1024×768 instead of the 1920×1080 you recorded at. None of these are code changes. Most of them are things IT does without telling you. The failure is often not even a failure: the match lands on the wrong lookalike button and the flow keeps running with wrong data.
Maintenance Cost
When a selector breaks, I open the UI element repository, repair one attribute, and I am done in ten minutes. The fix is surgical because the error tells me exactly which element died. Better yet, editing the custom selector to use wildcards on volatile attributes (window titles that include document names, for example) prevents whole categories of future breaks.
When an image match breaks, I have to reproduce the exact visual state of the machine to re-capture the image. If the flow runs unattended, that means connecting to the runner, matching its resolution and scaling, and re-recording. I have seen a single Windows font scaling change take out six image-based steps across three flows in one morning. Each one needed a new capture. There is no surgical fix for pixels.
Environment Sensitivity
Selectors are largely immune to resolution, scaling, and theming. The accessibility tree does not care whether the window is maximized or what DPI the session runs at. This matters enormously for unattended runs, where the RDP session Power Automate creates may not match your development machine at all.
Image recognition is hostage to the environment. You can loosen the tolerance parameter (it defaults to 10 in PAD’s image actions) to survive minor rendering differences, but every point of tolerance you add increases the chance of a false positive match. Tight tolerance breaks on scaling changes. Loose tolerance clicks the wrong thing. There is no setting that gives you both.
Debuggability
A failed selector produces an error message with the element name and the step. You can read the run history in the Power Automate Desktop portal and know what happened without touching the machine. A failed or mismatched image gives you “image not found” at best and silent wrong-clicks at worst. Diagnosing it means screenshots, session recordings, and guesswork. The same principle that makes AI agents fail in production when they worked fine in testing applies here: silent failures and poor observability compound until something important breaks without warning.
| Dimension | Selectors | Image recognition |
|---|---|---|
| Breaks on | UI tree changes | Any pixel change |
| Fix time | Minutes, one attribute | Re-capture on matching environment |
| DPI and resolution | Mostly immune | Highly sensitive |
| Failure mode | Loud, named element | Silent wrong-clicks possible |
Where Image Recognition Actually Wins
Citrix and RDP sessions where the remote application streams as a bitmap and exposes no accessibility tree. Legacy Java apps without the access bridge configured. Canvas-rendered UIs that paint everything into a single element. In these cases selectors return nothing and image recognition, ideally combined with OCR to verify state before clicking, is the legitimate tool. In my experience, that covers maybe one flow in ten. The rest use images out of convenience, and convenience at build time is fragility at run time.
The Decision Rule
Choose selectors if the accessibility tree exposes the element at all, even if you have to hand-edit the selector with wildcards to make it stable. Choose image recognition only when the tree is genuinely empty, and when you do, pin the runner’s resolution and scaling, verify with OCR before destructive clicks, and document the capture environment. If you are routing outputs from these flows into Power Platform agents, the same measurement discipline covered in How to Measure Useful Work Per Dollar for Your Power Platform AI Agents applies: you need to know whether the automation is actually completing useful work, not just running without errors. Drag and drop fast. Plan even faster.
Frequently Asked Questions
When should I use Power Automate Desktop selectors instead of image recognition?
Power Automate Desktop selectors are the better default choice because they read the application’s accessibility tree rather than pixels, making them resistant to resolution changes, theme updates, and display scaling. Image recognition is only worth considering when a UI element has no accessible properties for a selector to target.
Why do Power Automate Desktop selectors break after application updates?
Selectors break when the underlying accessibility tree of the application changes, such as when a developer renames a control or a software update reshuffles the window hierarchy. Using stable attributes like AutomationId instead of positional references like Button[3] reduces how often this happens.
Why does image recognition in Power Automate Desktop click the wrong element?
Image recognition matches on pixels, so changes like a new font, a notification covering the target, or a different screen resolution can cause the match to land on the wrong button without throwing an error. This makes failures harder to detect because the flow continues running with incorrect actions.
How do I reduce maintenance work when desktop flow UI elements keep breaking?
Switch from image-based clicks to selectors wherever possible, since selector errors identify the exact element that failed and repairs are usually quick attribute fixes. Using wildcards on volatile parts of a selector, such as window titles that include document names, can also prevent entire categories of recurring breaks.
Leave a Reply