# How Indian Opinion is built: a news brief service on WordPress

Indian Opinion is a news brief service that runs on WordPress. It does no original reporting. It reads reports from Indian outlets, groups the reports that describe one event, and publishes one brief for that event. This page describes the shape of the system in general terms, and it uses our open-source utilities to show the kind of problem each layer has to solve.

Everything here about how stories are chosen and written comes from our public policy pages: the [AI use policy](https://indianopinion.org/ai-use-policy/), the [editorial policy](https://indianopinion.org/editorial-policy/) and the [corrections policy](https://indianopinion.org/corrections-policy/).

## What a brief is

Each article has three parts, as the editorial policy sets them out.

- **The Story in Brief** is a factual retelling written from the full text of every report in the group. It carries no fact that is not in those reports.

- **Indian Opinion Analysis** is a short paragraph drafted by comparing how each outlet framed the same story: what one overstates, what another leaves out, and what the full set of reporting supports. It takes no side and ends with something concrete to watch.

- **Coverage** lists every source report, credited by outlet and linked, each with a stance label for how that report framed that story, such as pro-government, government-critical, neutral-report or sensationalist.

Every brief also states its corroboration count: the number of independent outlets it was built from. The [corroboration rule](https://indianopinion.org/editorial-policy/) decides when a story has enough outlets behind it to publish, with a stated exception for service journalism such as exam results and recruitment notices. A [stance label](https://indianopinion.org/data-and-reports/) describes one outlet's framing of one story. It is not a verdict on the outlet.

## The shape of the system

There are two paths. The publishing path turns incoming reports into briefs. The request path serves those briefs to readers, crawlers and agents.

On the publishing path, items arrive from feeds and are filtered. Each one is tied to the outlet that filed it. Reports about one event are grouped, so one event becomes one article instead of a stack of near duplicates. A language model writes the brief, the analysis and the stance labels from the grouped reports. Articles publish automatically.

On the request path, WordPress renders the page. An edge cache sits in front of it for anonymous requests. Media is served from object storage, not from the application server. Every page has a plain-text twin: add .md to the address, or send Accept: text/markdown. There are JSON endpoints for one brief and for recent briefs, listed at [the route index](https://indianopinion.org/wp-json/io/v1/), and an [llms.txt](https://indianopinion.org/llms.txt) that tells automated readers how to use and cite the site. The [for-agents page](https://indianopinion.org/for-agents/) and [/agents/](https://indianopinion.org/agents/) cover the rest.

After publication, a sampled automated quality check reads published briefs and reports what it finds. The editor verifies its findings and approves corrections, and a corrected brief carries a dated correction line.

## About the open-source pieces

Between 11 and 19 November 2026 we released a set of WordPress and PHP utilities under [github.com/IndianOpinion](https://github.com/IndianOpinion). They were specified, built, reviewed and released by language-model coding agents working under the editor's direction, in five separate roles: a spec writer, a spec reviewer, a builder that saw only the spec, an independent code reviewer and a releaser. The editor made the ruling on every open question. [We open-sourced 47 small utilities in nine days](https://indianopinion.org/blog/forty-seven-small-utilities/) lists them all, and [Open source at Indian Opinion](https://indianopinion.org/blog/open-source/) has a note on each.

Some repos are standalone rewrites of fixes we made while running this site; the rest were written from scratch from an idea. None of them is the production code. Every one is at v0.1.0 and has been tested on stubs, mocks and a throwaway database. In the walk below, read each link as "this is the kind of problem that repo solves", not "the site runs this repo".

## The publishing path

### Importing and filtering

A site that imports from many feeds needs its include and exclude rules in one place, and it needs to see what a rule would do before the rule acts. [feed-rules](https://github.com/IndianOpinion/feed-rules) is a clean rewrite of that idea: rules match whole words in any script, and a rule can run in a log-only test state first. A feed that dies quietly is the next problem. [feed-health](https://github.com/IndianOpinion/feed-health) tracks per-feed yield, fetch failures, HTTP status and last-item age, and warns when a feed goes quiet or recovers.

When every import rule runs inside one scheduled hook under one lock, a slow pass delays every other rule. [import-scheduler](https://github.com/IndianOpinion/import-scheduler) gives each rule its own interval on a dedicated cron lane with its own database lock.

Some aggregator feeds link to a wrapper page instead of the article, so every item is credited to the aggregator and URL de-duplication never matches the same story from a direct feed. [gnews-unwrap](https://github.com/IndianOpinion/gnews-unwrap) resolves those wrappers back to publisher URLs. Once you have the page, [readable-extract](https://github.com/IndianOpinion/readable-extract) pulls the body, title, authors, dates and lead image in plain PHP, says how sure it is of each field, and refuses to pass off a biography, a playlist or a section front as an article.

Attribution depends on knowing which newsroom owns a URL. Mobile, AMP, regional and legacy hosts of one outlet should give one key, and the same article reached through a tracking query or a trailing slash should give one article key. That is what [outlet-key](https://github.com/IndianOpinion/outlet-key) does; its explain() method lists every decision it took. A corroboration count is only honest if two hosts of one newsroom count once.

### Grouping

Grouping needs a cheap first pass and a more careful second one. A simhash of a short title is a cheap way to find near duplicates, but much PHP simhash code drops every character outside a-z and 0-9, so a title in an Indian script hashes to zero and appears to match every other such title ([Text is not ASCII](https://indianopinion.org/blog/text-is-not-ascii/) covers this and similar traps). [simhash-unicode](https://github.com/IndianOpinion/simhash-unicode) tokenises letters, digits and combining marks in every script, and refuses to call two texts near on too little signal.

For similarity over embeddings, storing vectors as JSON in database rows runs out of memory and time well before a corpus needs a vector database. [packed-vectors](https://github.com/IndianOpinion/packed-vectors) keeps embeddings in a packed float32 file with a small header, rebuilds it atomically, and scans it by exact cosine similarity in bounded memory.

### Calling models safely

Every model call can fail in more ways than an HTTP status shows: a truncated response, a schema violation, a refusal, an exhausted balance reported inside a success code. [llm-json-php](https://github.com/IndianOpinion/llm-json-php) turns one call into either a PHP array that matches your JSON schema or a typed exception. It retries only what can change, walks a chain of models, and runs a guard over the parsed output that catches text cut off mid-sentence, stray control and zero-width characters, and words that mix Latin with Cyrillic or Greek letters.

Paid generation work in bulk needs a queue with a cap and an off switch. [gen-queue](https://github.com/IndianOpinion/gen-queue) is a WordPress job queue with a daily spend cap, a kill switch, retries with backoff and per-attempt cost records; with no cap set, nothing runs. [spend-meter](https://github.com/IndianOpinion/spend-meter) sits underneath: one ledger and one price table for every plugin on a site, and a decision before each call on whether it may spend. Prompt wording should change without a deploy, and every version should stay on record. [prompt-library](https://github.com/IndianOpinion/prompt-library) stores templates with typed variables as immutable numbered versions, and other plugins fetch them by slug and version. [A queue, a meter and a prompt library](https://indianopinion.org/blog/a-queue-a-meter-and-a-prompt-library/) covers the three in depth.

### Images

Most imported items have no usable first image: a spacer, an icon, or a picture of the wrong size or shape. [featured-from-source](https://github.com/IndianOpinion/featured-from-source) keeps trying candidates from the source page in a fixed order, copies the chosen one into the media library with its credit and provenance, and never replaces an image an editor chose.

WordPress tells the browser that every image is as wide as the screen, so a phone downloads a large file for a small card. [srcset-truth](https://github.com/IndianOpinion/srcset-truth) replaces the sizes value with the real width of the slot. [share-cards](https://github.com/IndianOpinion/share-cards) makes the og:image for posts and archive pages, drawing a card with the headline only when the featured image is missing, too small or the wrong shape, and rendering it the first time a scraper asks.

## The request path

A news site publishes all day, and an uncached WordPress render is slow, so the cache cannot be switched off. The hard part is never caching the wrong page. [wp-edge-cache-headers](https://github.com/IndianOpinion/wp-edge-cache-headers) makes the decision once in PHP per request: anonymous HTML gets a public Cache-Control with s-maxage and stale-while-revalidate, while a logged-in visitor, a preview, a nonce or a response that sets a cookie gets private, no-store, with the reason in a debug header.

Cache misses still have to be fast. A front page built from many card lists runs many secondary WP_Query calls. [query-id-cache](https://github.com/IndianOpinion/query-id-cache) remembers the post IDs and row counts those queries returned for a few minutes, loads the posts fresh by primary key, and drops "latest" lists the moment a publish or term change can alter them.

Comments and a page cache fight each other. A reader whose comment is held for moderation sees it vanish on refresh, and an approved comment waits for the cache to expire. [comments-through-cache](https://github.com/IndianOpinion/comments-through-cache) sets a short-lived cookie at set_comment_cookies so that reader bypasses the cache until moderation is done, and purges only the affected article's URLs when a comment is approved, origin caches before edge caches.

Serving media from object storage takes load off the application server and makes the server easier to replace. [uploads-on-r2](https://github.com/IndianOpinion/uploads-on-r2) is an edge function that answers /wp-content/uploads/ URLs from a bucket and falls back to the origin when a file is not there yet. It denies private folders on the decoded key and reads only media extensions, because an edge function bypasses the origin's own deny rules.

Behind an edge network, the web server sees the edge's address, not the visitor's, so logs and rate limits are wrong. [cloudflare-realip-nginx](https://github.com/IndianOpinion/cloudflare-realip-nginx) keeps the real-IP trust list in step with the network's published ranges and tests the new snippet before installing it. Search engines that accept pushed URL notifications should hear about new pages without slowing the publish path. [indexnow-batch](https://github.com/IndianOpinion/indexnow-batch) sends them in batches on a schedule and re-checks every URL at send time, so drafts and noindexed pages are never announced.

### Mail and accounts

Agent sign-up and reader accounts depend on mail arriving. Some hosts block outbound SMTP. [signed-mail-gateway](https://github.com/IndianOpinion/signed-mail-gateway) is an edge function that sends mail only for requests signed with HMAC-SHA256 over timestamp, nonce and body, refuses replays, and comes with a plugin that routes wp_mail() through it and falls back once, only when the message certainly did not go out. [wp-mail-from](https://github.com/IndianOpinion/wp-mail-from) fixes the smaller problem underneath: WordPress sends from a wordpress@ address that usually has no mailbox, so bounces vanish. The must-use plugin sets a real From address, name and envelope sender.

## What we do not have

- **No newsroom reading every article before it goes live.** As the [AI use policy](https://indianopinion.org/ai-use-policy/) says, articles publish automatically and no person reads them first. The sampled quality check does not reach every article. The editor owns the rules, the sources and every correction.

- **No language editions.** The machine-translated editions were discontinued in September 2026. The site is English only, and the former edition addresses return 410 Gone.

- **No production mileage for the open-source repos.** They are standalone rewrites or new builds at v0.1.0. None has run its CI on GitHub yet, and several were written from an idea and have never run on a live site.

- **No original reporting.** Every fact in a brief comes from the outlet credited at its foot.

Canonical: https://indianopinion.org/blog/how-indian-opinion-is-built/

AI disclosure: Every summary and comment here is written by AI and published automatically, without a person reviewing each article. We do no original reporting. How this works: https://indianopinion.org/ai-use-policy/
