How Indian Opinion is built: a news brief service on WordPress

The shape of the system behind indianopinion.org, from reading reports to serving pages and their plain-text twins, with our open-source utilities as the illustration of each layer.

Indian Opinion is a news brief service that runs on WordPress. It does no original reporting. It reads reports from Indian outlets, groups the reports that describe one event, and publishes one brief for that event. This page describes the shape of the system in general terms, and it uses our open-source utilities to show the kind of problem each layer has to solve.

Everything here about how stories are chosen and written comes from our public policy pages: the AI use policy, the editorial policy and the corrections policy.

What a brief is

Each article has three parts, as the editorial policy sets them out.

  • The Story in Brief is a factual retelling written from the full text of every report in the group. It carries no fact that is not in those reports.
  • Indian Opinion Analysis is a short paragraph drafted by comparing how each outlet framed the same story: what one overstates, what another leaves out, and what the full set of reporting supports. It takes no side and ends with something concrete to watch.
  • Coverage lists every source report, credited by outlet and linked, each with a stance label for how that report framed that story, such as pro-government, government-critical, neutral-report or sensationalist.

Every brief also states its corroboration count: the number of independent outlets it was built from. The corroboration rule decides when a story has enough outlets behind it to publish, with a stated exception for service journalism such as exam results and recruitment notices. A stance label describes one outlet’s framing of one story. It is not a verdict on the outlet.

The shape of the system

There are two paths. The publishing path turns incoming reports into briefs. The request path serves those briefs to readers, crawlers and agents.

On the publishing path, items arrive from feeds and are filtered. Each one is tied to the outlet that filed it. Reports about one event are grouped, so one event becomes one article instead of a stack of near duplicates. A language model writes the brief, the analysis and the stance labels from the grouped reports. Articles publish automatically.

On the request path, WordPress renders the page. An edge cache sits in front of it for anonymous requests. Media is served from object storage, not from the application server. Every page has a plain-text twin: add .md to the address, or send Accept: text/markdown. There are JSON endpoints for one brief and for recent briefs, listed at the route index, and an llms.txt that tells automated readers how to use and cite the site. The for-agents page and /agents/ cover the rest.

After publication, a sampled automated quality check reads published briefs and reports what it finds. The editor verifies its findings and approves corrections, and a corrected brief carries a dated correction line.

About the open-source pieces

Between 11 and 19 November 2026 we released a set of WordPress and PHP utilities under github.com/IndianOpinion. They were specified, built, reviewed and released by language-model coding agents working under the editor’s direction, in five separate roles: a spec writer, a spec reviewer, a builder that saw only the spec, an independent code reviewer and a releaser. The editor made the ruling on every open question. We open-sourced 47 small utilities in nine days lists them all, and Open source at Indian Opinion has a note on each.

Some repos are standalone rewrites of fixes we made while running this site; the rest were written from scratch from an idea. None of them is the production code. Every one is at v0.1.0 and has been tested on stubs, mocks and a throwaway database. In the walk below, read each link as “this is the kind of problem that repo solves”, not “the site runs this repo”.

The publishing path

Importing and filtering

A site that imports from many feeds needs its include and exclude rules in one place, and it needs to see what a rule would do before the rule acts. feed-rules is a clean rewrite of that idea: rules match whole words in any script, and a rule can run in a log-only test state first. A feed that dies quietly is the next problem. feed-health tracks per-feed yield, fetch failures, HTTP status and last-item age, and warns when a feed goes quiet or recovers.

When every import rule runs inside one scheduled hook under one lock, a slow pass delays every other rule. import-scheduler gives each rule its own interval on a dedicated cron lane with its own database lock.

Some aggregator feeds link to a wrapper page instead of the article, so every item is credited to the aggregator and URL de-duplication never matches the same story from a direct feed. gnews-unwrap resolves those wrappers back to publisher URLs. Once you have the page, readable-extract pulls the body, title, authors, dates and lead image in plain PHP, says how sure it is of each field, and refuses to pass off a biography, a playlist or a section front as an article.

Attribution depends on knowing which newsroom owns a URL. Mobile, AMP, regional and legacy hosts of one outlet should give one key, and the same article reached through a tracking query or a trailing slash should give one article key. That is what outlet-key does; its explain() method lists every decision it took. A corroboration count is only honest if two hosts of one newsroom count once.

Grouping

Grouping needs a cheap first pass and a more careful second one. A simhash of a short title is a cheap way to find near duplicates, but much PHP simhash code drops every character outside a-z and 0-9, so a title in an Indian script hashes to zero and appears to match every other such title (Text is not ASCII covers this and similar traps). simhash-unicode tokenises letters, digits and combining marks in every script, and refuses to call two texts near on too little signal.

For similarity over embeddings, storing vectors as JSON in database rows runs out of memory and time well before a corpus needs a vector database. packed-vectors keeps embeddings in a packed float32 file with a small header, rebuilds it atomically, and scans it by exact cosine similarity in bounded memory.

Calling models safely

Every model call can fail in more ways than an HTTP status shows: a truncated response, a schema violation, a refusal, an exhausted balance reported inside a success code. llm-json-php turns one call into either a PHP array that matches your JSON schema or a typed exception. It retries only what can change, walks a chain of models, and runs a guard over the parsed output that catches text cut off mid-sentence, stray control and zero-width characters, and words that mix Latin with Cyrillic or Greek letters.

Paid generation work in bulk needs a queue with a cap and an off switch. gen-queue is a WordPress job queue with a daily spend cap, a kill switch, retries with backoff and per-attempt cost records; with no cap set, nothing runs. spend-meter sits underneath: one ledger and one price table for every plugin on a site, and a decision before each call on whether it may spend. Prompt wording should change without a deploy, and every version should stay on record. prompt-library stores templates with typed variables as immutable numbered versions, and other plugins fetch them by slug and version. A queue, a meter and a prompt library covers the three in depth.

Images

Most imported items have no usable first image: a spacer, an icon, or a picture of the wrong size or shape. featured-from-source keeps trying candidates from the source page in a fixed order, copies the chosen one into the media library with its credit and provenance, and never replaces an image an editor chose.

WordPress tells the browser that every image is as wide as the screen, so a phone downloads a large file for a small card. srcset-truth replaces the sizes value with the real width of the slot. share-cards makes the og:image for posts and archive pages, drawing a card with the headline only when the featured image is missing, too small or the wrong shape, and rendering it the first time a scraper asks.

The request path

A news site publishes all day, and an uncached WordPress render is slow, so the cache cannot be switched off. The hard part is never caching the wrong page. wp-edge-cache-headers makes the decision once in PHP per request: anonymous HTML gets a public Cache-Control with s-maxage and stale-while-revalidate, while a logged-in visitor, a preview, a nonce or a response that sets a cookie gets private, no-store, with the reason in a debug header.

Cache misses still have to be fast. A front page built from many card lists runs many secondary WP_Query calls. query-id-cache remembers the post IDs and row counts those queries returned for a few minutes, loads the posts fresh by primary key, and drops “latest” lists the moment a publish or term change can alter them.

Comments and a page cache fight each other. A reader whose comment is held for moderation sees it vanish on refresh, and an approved comment waits for the cache to expire. comments-through-cache sets a short-lived cookie at set_comment_cookies so that reader bypasses the cache until moderation is done, and purges only the affected article’s URLs when a comment is approved, origin caches before edge caches.

Serving media from object storage takes load off the application server and makes the server easier to replace. uploads-on-r2 is an edge function that answers /wp-content/uploads/ URLs from a bucket and falls back to the origin when a file is not there yet. It denies private folders on the decoded key and reads only media extensions, because an edge function bypasses the origin’s own deny rules.

Behind an edge network, the web server sees the edge’s address, not the visitor’s, so logs and rate limits are wrong. cloudflare-realip-nginx keeps the real-IP trust list in step with the network’s published ranges and tests the new snippet before installing it. Search engines that accept pushed URL notifications should hear about new pages without slowing the publish path. indexnow-batch sends them in batches on a schedule and re-checks every URL at send time, so drafts and noindexed pages are never announced.

Mail and accounts

Agent sign-up and reader accounts depend on mail arriving. Some hosts block outbound SMTP. signed-mail-gateway is an edge function that sends mail only for requests signed with HMAC-SHA256 over timestamp, nonce and body, refuses replays, and comes with a plugin that routes wp_mail() through it and falls back once, only when the message certainly did not go out. wp-mail-from fixes the smaller problem underneath: WordPress sends from a wordpress@ address that usually has no mailbox, so bounces vanish. The must-use plugin sets a real From address, name and envelope sender.

What we do not have

  • No newsroom reading every article before it goes live. As the AI use policy says, articles publish automatically and no person reads them first. The sampled quality check does not reach every article. The editor owns the rules, the sources and every correction.
  • No language editions. The machine-translated editions were discontinued in September 2026. The site is English only, and the former edition addresses return 410 Gone.
  • No production mileage for the open-source repos. They are standalone rewrites or new builds at v0.1.0. None has run its CI on GitHub yet, and several were written from an idea and have never run on a live site.
  • No original reporting. Every fact in a brief comes from the outlet credited at its foot.
Search Trending
Loading

Signing-in 3 seconds...

Signing-up 3 seconds...

All fields are required.