It struck me that using OpenCV to get the coordinates of a button might actually be _simpler_ (from a code writing perspective...) than many of the solutions to these sorts of problems I’ve used in the past. This is a pretty straightforward, maybe even 1:1 pixel mapped match you are looking for, returning the coordinates to click could be accomplished in very few lines of code with OpenCV. You could build a pretty sweet little automation library that just consumes screen captures of the things you want to click. Would greatly reduce the tedium often found in automation coding. This strikes me as so obvious I’m guessing such a library probably already exists...
With these fancy tools bringing their own UI toolkits to the table, it's possible that tab was broken— when you roll your own, accessibility is usually the first thing out the window.
I love this. Clever to the Max. Reminds me of a time I remember vividly. Needed to build a high stakes click automation script I once wrote to fix some billing amounts before customers would be incorrectly charged. API doesn’t exist so what to do write a script to extract the visual location and automated click. So dirty but so satisfying when it worked.
Photoshop and Illustrator seem to have fairly comprehensive AppleScript dictionaries at least (nothing for Premiere Pro though, but that's not surprising as it that was a Windows-first product)
I have doubts that it still exists, at least in a complete state. I guess that whoever reverse-engineered the file format for the original sequence editing tool would have preferred to read the original source code instead.
