Automating a Test Interface With 24 Lines of Python
Repetition is where bad workflows become obvious. Clicking through question after question on a rigid interface felt less like problem solving and more like waiting for a human to imitate a machine. That was the context for this project. Not to build something sophisticated. To see how little code it took to automate something painfully repetitive.
Using Vision Instead of APIs
The whole project was built around template matching. PyAutoGUI scanned the screen for answer choices and navigation buttons, then clicked through a fixed loop of one hundred questions. There was no browser integration. No scraping. No internal hooks. Just screenshots, confidence thresholds, and mouse events.
One small detail I found interesting was tuning confidence values differently for different UI elements. The question area and next button were stable, so I used stricter matching. Answer choices were less reliable, so they used lower thresholds. That tiny decision mattered more than most of the script.
What Simplicity Hides
The code was short, but it encoded a lot of assumptions.
- The question count was hardcoded to one hundred
- Answer selection followed a fixed B then C then D priority
- The whole bot assumed one monitor setup and specific text scaling
That made the project fragile, but it also made the tradeoff obvious. I was not building a general automation framework. I was building a narrowly scoped tool and learning through constraints.
What Broke First
Screen based automation fails in boring ways. A small UI shift can break everything. A popup can throw off the click sequence. Even timing mattered. Fixed sleeps that seemed fine on one run could be too fast on another.
If I rebuilt it, I would add state checks before every action. I would detect whether a question actually advanced. I would log failures. At the time I skipped all of that because I wanted to get something working end to end. That was the right call for a first pass, but it also showed where prototypes stop being reliable.
What I Actually Learned
I started this as a small automation experiment. What stayed with me was not the bot itself but the lesson that brittle systems often work until one hidden assumption moves. This project made that concrete.
Sometimes twenty four lines of code are enough to expose a much bigger engineering problem.