packed-vectors is a PHP library for applications that keep embeddings in database rows and need to find the most similar ones. It stores them in a packed float32 file with a small header, rebuilds that file atomically, and scans it by exact cosine similarity in bounded memory. The code is at https://github.com/IndianOpinion/packed-vectors, under the MIT licence. Version v0.1.0 was released on 14 November 2026.
When embeddings live in database rows as JSON text, similarity search hits two walls as the corpus grows.
memory_limit.unpack() is far cheaper.A vector database is often not available on shared or small hosts, and for tens of thousands of rows it is not needed. Teams that write the packed file by hand then meet the same traps: unpack() arrays that start at index 1, machine-order floats, unchecked writes that rename a truncated file into place, a trailing partial record silently dropped, and code that quietly truncates vectors of the wrong length.
dim little-endian float32 values.minScore, and an optional best-hit-per-ref mode.ChunkedScanner, a fallback with identical ranking over pages your own code reads from your store.info and verify.The build step checks the target directory, takes a non-blocking flock, and creates a temporary file in the same directory with exclusive create. It writes a placeholder header, streams records through a 1 MiB buffer while updating the CRC, writes the real header, calls fsync, checks the size, and calls rename(). It then syncs the directory. Vectors are normalised at build time by default, so a scan is one dot product per record.
The scan opens the file once and takes its size from that handle with fstat. It checks the header and the exact size before reading a record, then reads whole-record chunks and decodes floats with one unpack() per block, using explicit 1-based offsets. The top-k is a bounded min-heap with a deterministic tie-break on score, then id, then ref. Memory stays flat whatever the file size.
A scan that opened the old file keeps reading the old inode, and a later scan sees the whole new file. Scanners never take the lock. The main classes are Builder, Scanner, ChunkedScanner, the immutable BuildOptions and ScanOptions, and Internal\Header, Internal\Vectors and Internal\TopK.
$report = (new Builder($path, $dim))->build($records);
$result = (new Scanner($path, new ScanOptions(timeBudgetSeconds: 0.5)))->top($queryVector, 10);
foreach ($result->hits() as $hit) {
echo $hit->id, ' ', $hit->ref, ' ', $hit->score, "\n";
}
For serving, catch UnavailableException, queue one rebuild, and fall back to the chunked scanner for that request. The repo’s own benchmark script, on synthetic vectors, a loaded two-core box, PHP 8.3 and no JIT, reports 0.84 s for 10,000 vectors of 512 dimensions, with about 8 MiB of added memory.
PackedVectorsError. The “fall back and rebuild” family (missing, unreadable, corrupt, wrong version, wrong dimension) also implements UnavailableException. There is no @ suppression anywhere.DimensionMismatchException.length, non_finite, zero_norm and others), or abort the build with onBadRecord: 'throw'. The build report is meant to be logged.BudgetExceededException.lockWaitSeconds is set.rebuildIfStale() escaped unwrapped. It now raises BuildFailedException with the original as the previous exception.IteratorAggregate whose getIterator() threw escaped unwrapped in the same way. The fix is the same, and a test checks the old file is unchanged and no temporary file remains.The review also noted two behaviours it left alone: a payload bit flip in a float exponent can report non_finite before the checksum, and a flipped version bit reports an unsupported version. Both are still corrupt and still UnavailableException.
Test status: this is v0.1.0. 101 PHPUnit tests (6,620 assertions) pass on PHP 8.3, including the memory-budget group. Two were skipped when run as root. PHPStan level 8 and phpcs are clean. PHP 8.1 and 8.2 are verified only by a CI workflow that has not yet run on GitHub. Timings are printed by the benchmark and not asserted. It has not been run on a live site.
Requires PHP 8.1 or later, 64-bit, with no runtime dependencies.
The package is not on Packagist yet, so first add a Composer vcs repository entry for https://github.com/IndianOpinion/packed-vectors.
composer require indianopinion/packed-vectors
Without Composer, download a release and require bootstrap.php once.