AI & Automation

Ads Review Automation

Rules flag every pause; a model explains, and may argue to keep an ad but never pause one

2026Production pipeline

Credit

Primary engineering: Prada Dipa (LinkedIn, opens in a new tab)
Maintenance: Hady Satya
Directed by: Denandro YusufManager and supervisor · Technical & product direction · Requirements direction · Guardrail and review direction · QA and operational rollout

Problem: Weekly ad reviews were manual, and the pause thresholds lived in one person's head.
What I did: I directed a rules engine that flags each pause, and a model that explains it and may argue to keep an ad.
Outcome: The same numbers now raise the same flags every week; a person decides.

Explore

The component graph, from the source repository. Follow the arrows: left to right, and down where two steps share a column. The burgundy bars are where decisions are made; the numbers on the lines are the key flows, written out under the graph.

Key flows

Connector to Gzip JSON cache: active frames
Connector to Transform + metrics: raw daily rows
Gzip JSON cache to Transform + metrics: cache hit
Transform + metrics to Rules engine: ad-level metrics
thresholds.json to Rules engine: pause caps
thresholds.json to Versioned prompts: rendered as table
Rules engine to Report orchestrator: status + reasons
Product coverage index to Report orchestrator: remaining count
Report orchestrator to Versioned prompts: bucket payload
LLM narrator to Report orchestrator: verdicts merged by ad id
Report orchestrator to Sheets + Drive export: guarded hook

Go deeper

The full account

Problem: The weekly paid-media review was a manual pass through Ads Manager: read ROAS, CPA, CPM and CPV per ad, decide what to pause, re-type the conclusions into a document. The thresholds lived in someone's head, so the same numbers produced different decisions on different weeks.
What was built: A staged Python pipeline that fetches ad performance, scores every ad against thresholds held in one JSON file, prints beside each pause how many other live ads still serve that product, and ships the review in four formats.
Role: Manager and technical lead, not the implementing engineer — see the credits for who built it. Defined the system, translated the media buyer's remembered judgement into a threshold table and a decision order, reviewed the prompt's output contract and cited-evidence rules before they shipped, and directed QA and the operational rollout.
What changed: Turned a remembered judgement into a reproducible one: the thresholds that drive the decision now feed the engine, the spreadsheet templates and the LLM prompt from a single file.

Primary engineering by Prada Dipa, with Hady Satya maintaining the system. I manage and supervise the project, define technical and product direction, review the implementation and its guardrails, and direct QA and operational rollout.

Context

A direct-to-consumer apparel brand reviewed paid media weekly across three funnel stages. The work was mechanical — open each stage, read the metrics per ad, decide keep or pause — but the judgement was inconsistent between reviews, because the caps were recalled rather than written down.

There was also a failure mode nobody was systematically checking: pausing an underperforming ad can leave a product with no live ad at all. That is a merchandising question disguised as a media-buying decision, and the review now puts it in front of the reader.

Architecture and the system

A staged pipeline: connector → cache → transform → metrics → rules → coverage → report. The threshold file is the pivot — it feeds the rules engine, the spreadsheet column templates and the LLM prompt from one place, which is what keeps the three from drifting apart.

The engine flags. The model explains, and may argue back.

This is the load-bearing decision of the whole project. A pluggable rules engine loads pause thresholds from a single JSON config, applies them per funnel stage and sub-stage, and enforces a seven-day grace period so a new ad is never flagged. Every pause starts there: the engine flags it, and the model is told that flag is the default.

The LLM is a narrative analyst layered on top. It receives the batch summary, the ads the engine flagged, the ads paused recently, each ad's product and remaining coverage count, and the engine's one-line verdict. It writes the review, and it may argue to keep a flagged ad — the holiday override below, low spend, or a cited number that disagrees — and the prompt requires it to label that as an override. It can never pause an ad the engine kept, and it cannot set a threshold.

The system prompt is generated at import time by rendering the threshold table directly out of the same JSON the engine reads, so the caps the model is given cannot drift from the ones enforced. That makes the correct number available to the model; it does not stop the model writing a wrong one, and nothing in code checks for that — see below.

What the prompt asks for, and what code enforces

The prompt sets the output contract: a fixed verdict/action/stance matrix, three bullets per ad, the breach bullet and the reasoning bullet citing two different metrics, and no hedging language or invented numbers. Those are instructions to the model, not checks. No code counts the bullets, checks which metrics they cite or compares a number with the data, so a person reads every review before acting on it — the pipeline writes a report and never pauses an ad itself.

What code does enforce is the model's reach. It only sees the ads the engine flagged and the ads paused recently, and its answers are merged back onto the engine's own list by ad ID, so it cannot add an ad to the pause list. For ads that are already paused, the recommended next action is computed deterministically in Python from rules status, age and spend, and any model bullet containing an action marker is stripped. That strip is the one code-level check on the model's words.

A missing API key or a failed call degrades to an empty analysis section rather than breaking the report. The deterministic review still ships.

Prompts as versioned source

Prompt versions v4 through v7 are retained in-tree as source files. v7 adds a holiday override, as a prompt rule: the model may turn a flagged pause into a keep, labelled as an override, only when three conditions hold at once: a curated shopping holiday falls inside a 30-day window, the breach is at most 10% over cap, and the model can quote a theme token from the ad name tying the product to that holiday. Code computes the holiday calendar and the days-to-holiday line the model reads, and has tests; the three-condition gate itself is applied by the model, and nothing checks it afterwards.

That is what a prompt looks like when it is treated as production logic rather than a text box: versioned, and reviewable in a diff. It is still a prompt, which is why the review stays with a person.

Coverage, printed rather than enforced

A product-coverage index parses the base product out of each ad name and counts how many other live ads still serve it. That count is printed beside each pause, so a reader can see when a pause would take a product's last ad offline. It is information, not a veto, and it withholds nothing: an earlier prompt let the model hold a pause on coverage grounds, and the v6 prompt records that rule being removed.

What was hard

Stopping the model from being persuasive about the wrong thing

An LLM asked to justify a pause will justify a pause — convincingly, and sometimes with a number it invented. The fix was structural rather than prompt-tuning: render the caps into the prompt from the engine's own config, compute the action for paused ads in Python, and strip any action marker the model emits. The model may still argue to keep an ad, but it has to label that as an override. Constrain the surface, not the wording.

One config, three consumers

The thresholds feed the rules engine, the spreadsheet column templates and the prompt. Any one of those drifting makes the review quietly wrong. Making all three read the same JSON at import time was the cheapest possible guarantee that they agree.

Recovering ads that were paused just before the review

An ad paused two days before the review disappears from the default insights window, taking its context with it. A separate two-step fetch recovers those ads so the report can say what happened to them rather than silently dropping them.

AI and automation

AI

The model is gpt-4o-mini in JSON mode, called once per ad-set bucket. It never sees creative imagery — thumbnails are downloaded for the report, not sent to a model, and there is no vision call anywhere in the repo. The prompt asks it to explain each flag in three cited bullets a media buyer will actually read, and to say so, as a labelled override, when it thinks a flagged ad should keep running. It can never add a pause.

Automation

Replaces the manual weekly pass: pulling per-stage performance, comparing each ad against remembered thresholds, deciding keep or pause, and re-typing the result into a document. The pipeline does fetch, evaluation, coverage count and write-up in one run, and emits the review as Markdown, JSON, self-contained HTML and PDF, plus an optional formatted Google Sheet with creative thumbnails rendered in-cell.

Direction and delivery

My part was direction and review: I set what each feature had to do, and reviewed the implementation and its guardrails before release. The design writing and the code are the engineering team's, with Prada Dipa as primary engineer. Four features each shipped with a dated design document stating locked decisions and component boundaries, paired with a separate step-by-step implementation plan — roughly 5,000 lines of design writing against an 11,700-line codebase. Work landed through feature branches merged by pull request with Conventional Commit subjects; the Sheets export was decomposed into a dozen commits that each ship one component rather than one large drop. Several code comments cite stakeholder review as the reason for a behavioural change.

Size signals

Commits: 98
Pull requests merged: 7
Prompt versions in tree: 4
Report formats: 4
Design-doc lines: ~5,000
Tests in the repository: 66

Counted from the source repository. No impact metric is claimed that the source does not prove.

What I would tell the next person

Let the rules make the first call and the model argue with it in the open. An override the model has to label is auditable; a model that is the rule is not.
If a prompt encodes production logic, version it as source and diff it like source.
Design docs that state what was deliberately excluded are worth more than docs that only describe what was built.

Technologies and access

Python 3
pandas
Meta Marketing API v25
OpenAI API
gpt-4o-mini
gspread
google-api-python-client
Playwright
markdown-it-py
openpyxl
pytest

Integrations

Meta Marketing API (insights, budgets, activity log, creatives)
OpenAI Chat Completions (JSON mode)
Google Sheets API
Google Drive API v3

Internal reporting pipeline. Reports contain live account performance and are not published.

At a glance