Talha Riaz
← All work
Production AI

Agent platform

AI gets interesting when it has to run without you.

WeiBlocks · 2024 to present

24automated pipelinesRunning on a schedule and on events, not on someone pressing a button
8specialized agentsEach with a narrow job rather than one general purpose prompt
6h → 15mmanual work per weekResearch and content workflows that used to be done by hand

This is the internal platform that took a set of manual research and content workflows, roughly six hours a week of somebody's attention, down to under fifteen minutes of review.

The interesting part is not the model calls. It is that the whole thing runs unattended, which means it is subject to exactly the same questions as any other production system: what happens when a step fails, how would anyone know, and can it recover without a human noticing.

What I owned

Designed and built the platform: the agent orchestration, the retrieval ingestion, the model routing layer, the event driven scheduling, and the health monitoring around all of it.

Scale

24 automated pipelines across 8 specialized agents, ingesting from 13+ data sources, with cost aware routing between models.

An agent is another production dependency

A model call is not a function call. Its output can be incomplete or malformed. Retrieval can come back empty or stale. A tool can time out. A router can send a task to the wrong model. None of those raise an exception in any useful sense, they just produce something plausible and wrong.

So the workflow is built the way I would build any event driven pipeline: explicit states rather than a chain of awaits, validation at the boundary where output enters the next step, and health monitoring that treats a pipeline going quiet as a signal rather than as success.

Model routing is the part that pays for itself. Task complexity decides which model runs, so the cheap model handles the volume and the expensive one is reserved for work that actually needs it. That is a cost decision, but it is also a latency and a reliability decision, because a smaller model failing fast is easier to recover from than a large one timing out slowly.

How a run is shaped

Every step has a state, so a failure has a place to be

24 automated pipelines
8 specialized agents
RAG-style ingestion
Model routing
Event-driven scheduling
Health monitoring

What it already handles

  • Empty retrieval

    Source returns nothing relevant

    RETRIEVE
    EMPTY
    HALTED
    No confident answer from an empty context
  • Routing by complexity

    Task arrives with unknown cost profile

    TASK
    ROUTE
    AGENT
    The expensive model is the exception, not the default
  • Tool timeout

    External tool stops responding

    TOOL
    TIMEOUT
    FAILED STATE
    A partial result never passes as a whole one
  • Pipeline goes quiet

    Scheduled run produces nothing

    SCHEDULED
    NO OUTPUT
    MONITOR
    Silence is a signal, not a success

Failure modes

  • Failure

    Retrieval returns nothing useful

    How it is caught

    Validation on the retrieval step before it feeds the agent

    What happens next

    The step fails explicitly rather than letting an agent answer confidently from an empty context, which is the failure that is hardest to spot downstream.

  • Failure

    A tool call times out

    How it is caught

    Bounded timeout per tool invocation

    What happens next

    Retry within the step's budget, then the pipeline lands in a failed state that is visible to monitoring rather than silently producing a partial result.

  • Failure

    Model output does not match the expected shape

    How it is caught

    Validation at the step boundary

    What happens next

    Rejected and retried rather than passed to the next step. A malformed output that gets forwarded becomes someone else's confusing bug three steps later.

  • Failure

    A scheduled pipeline stops running

    How it is caught

    Automated health monitoring on pipeline execution

    What happens next

    Silence is treated as a signal. A pipeline that produces nothing is indistinguishable from one that was never triggered unless something is watching for the absence.

Why I chose what I chose

  • Why route between models instead of using one good one?

    Most tasks in these pipelines are not hard. Sending all of them to the most capable model is paying a premium on volume to avoid thinking about which work is actually difficult. Routing on task complexity means the cheap model carries the load and the expensive one is reserved for the cases that need it.

  • Why explicit states rather than a chain of awaits?

    A chain of awaits gives you one failure mode, the whole thing threw, and no idea where. Explicit states mean a pipeline that fails at step six is visibly a pipeline that failed at step six, which is the difference between a fix and an investigation.

  • Why treat silence as a failure signal?

    An unattended pipeline that produces nothing looks exactly like one that was never triggered. If nothing is watching for absence, the failure mode of the whole platform is that it quietly stops working and everyone assumes it is fine.

What I would reconsider

  • This is internal tooling rather than a customer facing product, so it has not been pushed on the axes the other systems have. It has not needed to survive a traffic spike or a multi tenant isolation requirement, and I would not claim it has.
  • Validation at step boundaries catches malformed output but not confidently wrong output. That is the harder problem and it is not solved here, it is bounded by keeping a human on the review step rather than removing them entirely.
Stack

LangChain · LangGraph · RAG · Model Routing · Multi-Agent Orchestration · Event-Driven Scheduling · Node.js · TypeScript