What an independent code review caught in 47 small repos

Every one of our 47 build pull requests reached review with passing tests and clean static analysis. Every review still found real defects. This is the catalogue, by class, with the rule we now apply to each.

Between 11 and 19 November 2026 we released 47 small open-source utilities under github.com/IndianOpinion. Most are WordPress plugins or PHP libraries. Each one went through the same kind of pull request: a “<repo> 0.1.0” build PR that a reviewer read in full before it was merged.

Every one of those 47 pull requests arrived at review with its test suite passing. Static analysis was clean at the level the builder had set, and coding standards checks were clean. Every review still found defects that would have mattered on a live site. Some were security holes. Some lost data or money. Some meant a whole feature never ran.

The work was done by language-model coding agents under the editor’s direction, in five separate roles: a spec writer, a spec reviewer, a builder that saw only the spec, an independent code reviewer, and a releaser. The editor made the rulings on every open question. This post is about the fourth role. We read the review summary posted on every build PR, all 47 of them, and checked the fix commits where a claim needed it. The summary on the indexnow-batch PR repeats the one for query-id-cache, so for indexnow-batch we read its fix commits instead.

The defects fall into nine classes. For each: what the bug looked like, where we found it, the fix, and the rule we now apply.

1. Server-side request forgery through redirects and odd addresses

Several plugins fetch a URL that a user or a feed supplied. The builders trusted WordPress’s safe request functions. Review found the gaps.

In featured-from-source, the core safe-URL check refused only a handful of private IPv4 ranges. A source URL, or a redirect, pointing at the link-local cloud metadata address went straight through. So did carrier-grade NAT space, IPv6 unique local and link-local addresses, and IPv4-mapped IPv6. The fix is an own AddressGuard that resolves A and AAAA records on every hop (page, image and robots.txt), refuses the request if any answer is not public, and pins the cURL connection to the checked address so a rebinding DNS answer cannot move it. featured-from-prompt had the same hole on image downloads and now follows redirects hop by hop with the same check.

The spellings matter too. echo-robots-guard and crosspost both let IPv4-mapped and NAT64 forms of loopback and private addresses pass, and crosspost let 6to4 forms through as well. gnews-unwrap relied on filter_var flags, whose coverage differs between PHP versions, and looked up A records only: a host with nothing but a loopback AAAA record passed.

Rule: every hop is resolved for both address families, checked against explicit range tables including embedded IPv4, and the connection is pinned to the address that was checked.

2. Check-then-act races on caps, locks and counters

This was the most common class. A plugin reads a counter, compares it with a cap, then writes the new count. Two workers both read the old value, both pass, and the cap is exceeded.

The subtle version is WordPress’s own add_option(). It checks whether the option exists, then sends INSERT ... ON DUPLICATE KEY UPDATE. In finetune-export the reviewer showed against MySQL 8.0 that two requests past the existence check both got a truthy result, so both “acquired” the batch lock. The same pattern was fixed in alt-text-fill and tldr-block. audio-to-post took over an expired lock by deleting it and adding a new one, so two processes could both take it.

The fixes share one shape:

-- take a free lock: only one INSERT can succeed
INSERT IGNORE INTO wp_options (option_name, option_value, autoload)
VALUES ('plugin_run_lock', 'owner-token|expiry', 'no');

-- take over an expired lock: compare and swap on the value that was read
UPDATE wp_options SET option_value = 'new-token|expiry'
WHERE option_name = 'plugin_run_lock' AND option_value = 'old-token|expiry';

Counters got the same treatment. post-embeddings kept its daily counters in one option, read and rewritten; each counter is now its own row, changed with one INSERT ... ON DUPLICATE KEY UPDATE. editor-ai-sidebar now takes its daily cap with a single UPDATE ... WHERE hits < cap. newsapi-pull checks and counts its shared quota in one step under a MySQL named lock.

Caches race as well. In spend-meter, a 60-second cache held the whole network’s spend total, and each check added only its own estimate. Inside one minute, a burst on one site could pass the network cap by any amount. The cache now holds only the other sites’ total, and each check adds its own site’s live figure.

Rule: a cap, lock or counter changes in one atomic database statement, and its test runs two real connections or forked processes, not a stub.

3. Protected directories that fail open

Some plugins keep private files in a directory guarded by .htaccess and an empty index.php. Review asked what happens when that guard does not work.

In finetune-export, with WordPress installed in its own subdirectory, the default export directory one level above ABSPATH sat inside the served home directory but counted as outside the web root. It was guarded only by .htaccess, which nginx ignores. Its self-check also reported a redirect as “safe”. Now the home directory and document root both count as web root, and only a 4xx answer counts as blocked. post-embeddings had the same probe bug: a 404 or a redirect was reported as blocked, and is now unknown. audio-to-post never checked whether its guard files were written, so a failed write could leave recordings open. featured-from-prompt now stores nothing when its deny files cannot be written, where before it threw an uncaught exception or left a public file.

Rule: if the guard cannot be written and proven, nothing is stored, and only a 4xx response counts as proof.

4. Integration code written against guessed hook names

Many utilities integrate with three of our own libraries (A queue, a meter and a prompt library describes them): spend-meter for spend caps, gen-queue for background jobs and prompt-library for prompt templates. The builder saw only its own spec, so it guessed the companion APIs. Its tests stubbed the guess. Everything passed, and nothing would have run.

feed-item-brief registered its job handler through a gen_queue_register_handler() function that does not exist, so its jobs had no handler. meta-writer listened on an action gen-queue never fires: queued jobs would never have run. yt-to-post called gen_queue_soft_enqueue and gen_queue_register_job_type, neither of which exists, and treated spend-meter’s “no meter” answer as a deny. wc-product-copy called prompt_library_* functions; the real prefix is promptlib_. post-embeddings sent spend-meter estimated_input_tokens where the contract reads est_input_tokens.

The same failure appears against WordPress core. In query-id-cache the test stub claimed core leaves taxonomy clauses transformed. Core transforms a copy. Term scoping never worked against real core, and a publish into a child category did not invalidate the parent’s “latest” list.

Rule: integration code is checked against the released contract of the other component, and a stub models the real code, not the hoped-for one.

5. Credentials following a redirect or leaking into logs

In newsapi-pull, a site filter on the request arguments could raise the redirect limit, and a redirect carries the API key header to whatever host it names. Redirects are now fixed at zero after the filter runs. readable-extract sent its Authorization header to every hop, including a different host or an https to http downgrade; it now goes only to the requested origin.

Logs leaked in quieter ways. In sigv4-php, with PHP’s built-in defaults a stack trace prints string arguments, so a failing credentials constructor showed the first 15 characters of the secret key. Every secret parameter now carries #[\SensitiveParameter], and the leak test scans full traces. post-embeddings and indexnow-batch both truncated an error message before masking the key, so a key straddling the cut survived in part.

Rule: a credential goes to one origin and never across a redirect, and redaction runs before truncation, with a test that greps the output.

6. Model output or user text reaching markup or a prompt unescaped

In comment-draft-replies, the validator decoded entities, so a model’s &lt;script&gt; became a real <script> in the stored reply. Publishing it as an administrator with unfiltered_html skipped kses. Model text is now encoded on publish. editor-ai-sidebar wrote provider text raw through set_block_attribute, so on a rich-text attribute such as an image caption it became markup.

The most surprising case was share-cards. ImageMagick’s pango: coder runs text through its property interpreter before Pango sees it. A headline containing %m was drawn as “PANGO”, and %[fx:1+1] was evaluated to “2”. Every % and \ is now doubled before the read.

Prompts are markup too. In headline-options, the article could close its own delimited block, because entities were decoded and &lt;/article&gt; became a real tag. In sse-chat-widget, a post title could plant a fake citation marker. Both now strip delimiter look-alikes.

Rule: escape at the boundary where the text changes meaning, after the last decode, and fence every model or reader string as data inside a prompt.

7. Regex backtracking limits turning a deny into an allow

PHP’s preg_* functions return false or null when they hit the PCRE backtrack limit. Code that treats that as “no match” fails open.

echo-robots-guard parsed robots.txt with preg_match_all. A single line padded with 5,000 spaces hit the limit, the parse returned early, and every rule after that line was lost, so the guard allowed fetches the site had disallowed. Lines are now split with strpos, substr and trim, checked by a differential run over 4,000 random inputs. readable-extract compiled robots.txt wildcards to a regex; a hostile rule like /*a*a*...*b dropped the rule. It now uses a linear two-pointer glob match. In feed-rules, a keep rule whose regex failed let a later exclude rule drop the item.

Rule: every regex call that can fail is checked, and a failure is decided explicitly before it can mean “allow”.

8. Paid work recorded as free, or lost on a failed write

Review followed paid calls through their failure paths.

In audio-to-post and meta-writer, a failed call was released as if it had never been made, though the provider may have billed it. A failed ledger insert in audio-to-post lost a paid call outright; the row is now buffered and written later. tldr-block recorded a billed answer that failed validation as ok; it is now error_billed. In prompt-library, a run row that could not be stored hid the paid answer behind a storage error. A caller that threw left a draft in comment-draft-replies, and a run in audio-to-post, in a state that was retried and paid for on every tick.

The same class covers plain data loss. indexnow-batch moved queue rows by deleting them and then inserting them elsewhere without checking the insert. A failed insert after a committed delete lost the rows. Every statement is now checked, and a failure rolls back.

Rule: a billed call is recorded as billed whatever happens next, and a failed write is retried or counted, never dropped.

9. Destructive behaviour on uninstall or cleanup

cloudflare-realip-nginx had an install.sh --purge that ran rm -rf on the state directory named in the config file. A config setting that directory to /var/lib would have wiped it. It now removes only the files the tool writes, then the directory only if it is empty. In share-cards, a multisite uninstall called flush_rewrite_rules() after switch_to_blog(), while the global rewrite object still held the first site’s structure, so it wrote the first site’s rules into every site.

Rule: cleanup removes only what the tool itself wrote, by name, and uninstall is tested across more than one site.

Why the builder and its tests missed these

The tests were written by the same hand that wrote the code, from the same spec. They encode the same assumptions. If the builder believed a hook was called gen_queue_register_handler, its stub defined that hook and its test passed. If it believed core transforms tax clauses in place, the stub did that too.

Stubs cannot race. An in-memory options table run by one PHP process will never show two workers passing the same cap. The race fixes only became testable when the reviewer brought in a throwaway MySQL with two connections, or forked eight processes against one key or budget.

Nobody tests the address they did not think of. The SSRF tests covered the private ranges the builder listed. They did not cover IPv4-mapped IPv6, NAT64, an AAAA-only host or the link-local metadata address, because those were not on the list.

Static analysis does not see any of this. None of the defects above is a type error.

What the review cost and what it did not cover

Each review was a full read of the diff against the spec and the editor’s rulings, followed by fix commits on the build branch, usually two to five per repo. Most fixes came with a regression test, and several reviews state that the new test fails on the old code. Reviewers also put wrong defaults back to the spec and corrected documentation that overstated a figure.

The review did not cover everything. No repo has run against a live WordPress install: plugins say “Tested up to” in readme.txt, and that line still needs confirming on a real site. No live model provider was called; the live tests are opt-in and were skipped. No repo has run its CI on GitHub yet, because the push token lacked the workflow scope, so PHP 8.1 compatibility was checked statically and shellcheck cleanliness by reading. Some limits are documented rather than fixed: DNS rebinding in gnews-unwrap and readable-extract, and a per-slug time budget in ascii-slugs that review measured and found is not met. Every repo is at v0.1.0. Treat each as reviewed code that has not yet earned production mileage.

Search Trending
Loading

Signing-in 3 seconds...

Signing-up 3 seconds...

All fields are required.