SEOClaw: Building an Orchestration Layer for Search, Not Another SEO Tool

SEO tooling often fragments the workflow. Audits live in one place. Rankings live somewhere else. Content planning sits in a spreadsheet. Reporting becomes manual glue. That fragmentation was the problem I cared about more than any single optimization feature.

SEOClaw started as an attempt to treat SEO and GEO as orchestration. Not just analysis. A system where audits, agents, queues, reporting, and notifications could behave like one pipeline.

Building Around Orchestration

The core architectural decision was separating the system into three parts. A Next.js dashboard for control and visibility. A FastAPI agent service for running analysis graphs. A worker and queue layer to connect them.

That separation mattered because long running analysis does not belong inside a request cycle. Once audits moved through BullMQ into workers and then into the agent service, the architecture started feeling like a system instead of a dashboard with extra scripts attached.

• An SEO graph for scraping, keyword analysis, and quick win generation
• A GEO graph for generative visibility analysis and schema recommendations
• An orchestrator that combined both workflows with retries and structured outputs

One technical detail I liked was forcing LLM outputs into structured JSON shaped responses. That solved more problems than prompt tuning ever did.

The Hard Part Was Reliability

The interesting engineering problem was not calling an LLM. It was handling failure. Jobs stall. APIs rate limit. Scrapes break. Model outputs drift. Most of the useful work was around retries, backoff, queue behavior, and deciding what happened when one component failed but the system should keep moving.

That is where the worker model helped. The queue made the system less fragile. It also made future scheduling feel natural instead of bolted on.

Some Things Were More Prototype Than Product

Parts of the project were intentionally ahead of themselves. Some dashboard data was sample data. Google OAuth was scaffolded more than complete. Some persistence paths were thinner than I wanted. Those are real limitations.

I also built around one LLM provider. That made iteration faster, but I would add provider abstraction much earlier if I rebuilt it. Dependency risk becomes architecture risk surprisingly fast.

Another thing I would change is testing. There was careful thought around workflows and policies, but not enough explicit test coverage around graph execution and failure cases.

What I Learned From the System Shape

The non obvious insight was that the graphs mattered less than the boundaries around them. Multi tenancy, role policies, sandbox tokens, webhook delivery, and queue isolation were not supporting details. They were part of the product.

That shifted how I think about AI systems. The model is one component. Coordination is the real engineering.

I also learned that hardcoded sample data can be dangerous. It makes interfaces look mature before the feedback loops are mature.

Where It Pointed

This project was where I started thinking less in features and more in runtimes. Less in prompts and more in workflows.

Good systems do not just generate insight. They move work.