post-embeddings: related posts and semantic search for WordPress

A WordPress plugin that embeds each published post in the background, keeps a packed sidecar file for fast similarity scans, and offers related-posts and semantic search blocks under daily caps and a kill switch.

post-embeddings is a WordPress plugin for sites with a few thousand posts that want related-posts lists and search that go beyond matching words. It is MIT licensed, and v0.1.0 was released on 19 November 2026.

It was written from scratch from an idea, as part of a set of utilities built between 11 and 19 November 2026 by language-model coding agents under the editor’s direction. It has not run on a live site. It uses one of the sibling utilities, packed-vectors, for storage and scanning.

The problem

Related-posts lists built from shared tags miss stories filed under different terms. Core search matches words, so a reader who types “fuel price rise” gets nothing from a post headed “Petrol and diesel cost more from Monday”. Embeddings fix both: one vector per post and a cosine comparison between vectors.

The hard part is the work around the maths.

  • An embedding costs money and a network call. Doing it inline on save slows the editor. Doing the whole archive at once can spend a month’s budget in an hour.
  • Vectors stored as JSON in database rows do not scale. With thousands of rows the similarity step takes seconds per request, and one result set can exhaust the memory limit. The same vectors packed as float32 are far cheaper to read, which is what packed-vectors is for.
  • A cosine function that compares only the overlapping part of two vectors of different lengths gives plausible but wrong numbers. After a model change nothing errors. Results just get worse.
  • A zero vector, a failed decode or a provider outage must not take a page down.

What it does

  • Computes one embedding per published post in the background, in its own table. A save only marks the row pending.
  • Keeps a packed float32 sidecar file for fast scans, rebuilt after changes.
  • Provides a Related posts block and a Semantic search block, plus shortcodes, REST routes and a PHP function.
  • Backfills an archive through wp post-embeddings backfill, in capped, resumable runs that start with a dry run.
  • Falls back to keyword results, headed “Keyword results”, when semantic search cannot run.
  • Accepts any embeddings provider through the post_embeddings_provider filter. The reference adapter speaks the common HTTP embeddings request shape.

How it works

On save_post and transition_post_status, Indexer hashes the post text (title, excerpt, content with block comments, shortcodes and tags stripped) together with the model key. Unchanged text with a stored vector does nothing. No network call happens in a save.

Drain runs from cron under a lock, takes pending rows oldest first and stops after 20 seconds. Before each call it checks the kill switch, the daily caps and the optional spend meter. Each returned vector is checked for length, finite numbers and a non-zero norm, then normalised and stored as float32. Failures back off from five minutes to twelve hours. An authentication failure pauses the drain.

When a provider reports no token usage, TokenEstimator divides characters by a per-script ratio, about 4 for Latin and 1.5 for Indic scripts. A flat divide-by-four would undercount non-Latin text, and undercounting a budget is the dangerous direction.

For related posts, one primary-key read fetches the vector and the sidecar scan returns count * overfetch hits. PHP then drops the post itself, ineligible posts, other types, near-copies and, optionally, old posts. If the sidecar is missing, corrupt or of another dimension, a chunked table scan serves that request and a rebuild is queued. For search, the query vector is cached under a hash for seven days, and the query text is never stored.

wp post-embeddings backfill --cap=500 --dry-run
wp post-embeddings backfill --cap=500 --yes
wp post-embeddings rebuild
wp post-embeddings related 123

Guard rails

  • Daily caps by UTC date: 500 posts for indexing, 1,000 fresh search embeddings, and an optional token cap. A backfill refuses to run without --cap.
  • A kill switch, as a setting or the constant POST_EMBEDDINGS_DISABLED. Related posts keep serving from stored vectors when it is on.
  • Only published, public, password-free posts are sent to the provider. Search results are filtered again at request time.
  • Vectors of another length are refused rather than compared.
  • The sidecar directory gets a deny-all file for Apache and IIS, and the plugin refuses to build if it cannot write them. It probes the directory over HTTP and reports a readable file as critical.
  • The adapter URL must be HTTPS to a public host, and remote calls go through the WordPress safe-URL check. Changing the host clears the stored key.
  • Search is rate limited per client, queries over 4 KB are refused, and provider error bodies are redacted before logging.

What review changed

  • Daily counters were a read-modify-write of one option, so racing drains and searches lost updates and overshot caps. Each counter is now its own row per UTC date, changed by one INSERT ... ON DUPLICATE KEY UPDATE, and tested on real MySQL with two connections.
  • A backfill counted only billed posts against its cap, so a run of rejected inputs walked past it. A drain and a backfill could also send the same row twice. The backfill now holds the drain lock for its whole run.
  • The adapter URL was open to server-side request forgery. Private, reserved, malformed, query and fragment URLs are now refused.
  • The sidecar exposure probe treated a 404 or a redirect as blocked. Both are now reported as unknown. A relative sidecar directory is refused.
  • The spend-meter calls used the wrong key name and the wrong call shape. They now follow the released contract.

Limits and what is next

  • One vector per post. Long posts are truncated, so the end of a very long post does not count.
  • The scan is exact and linear. The README gives about 100,000 posts as the practical ceiling, with no approximate index and no external vector database.
  • Search shows one page of results.
  • Related lists update with the page cache, not instantly.
  • It needs 64-bit PHP and a local disk with atomic rename. On a network filesystem, turn the sidecar off.
  • Score thresholds depend on the model. The default of 0.30 is a starting point.

There are no open issues. This is v0.1.0. The suite has 114 tests and 794 assertions, including integration tests against a throwaway MySQL 8.0, and the rest run on stubs and a fake provider whose vectors are not semantic. Static analysis and code style checks pass. No real embeddings provider has been used, and CI has not yet run on GitHub.

Install

Requires WordPress 6.4 or later, 64-bit PHP 8.1 to 8.3 with mbstring, and an embeddings API under your own credentials or a self-hosted server. Packed-vectors is not on Packagist yet, so list both repositories in Composer.

{
    "repositories": [
        {"type": "vcs", "url": "https://github.com/IndianOpinion/packed-vectors"},
        {"type": "vcs", "url": "https://github.com/IndianOpinion/post-embeddings"}
    ],
    "require": {"indianopinion/post-embeddings": "^0.1"}
}

The release zip bundles packed-vectors. A real system cron is recommended, because WP-Cron only runs on page views.

All 47 utilities · post-embeddings on GitHub

Search Trending
Loading

Signing-in 3 seconds...

Signing-up 3 seconds...

All fields are required.