# outlet-key: which newsroom, which article, from a URL

outlet-key is a PHP library for anything that counts sources, corroborates a story or deduplicates a feed. It tells you which newsroom owns a news URL and which article it is. The code is at [https://github.com/IndianOpinion/outlet-key](https://github.com/IndianOpinion/outlet-key), under the MIT licence. Version v0.1.0 was released on 14 November 2026.

## The problem

The host string is the wrong answer in both directions. A mobile host, a regional host and a legacy host of one outlet are one newsroom, but counted by host they are three, so a single-source story looks corroborated three times. Two newsrooms that sit under one parent portal domain are two, but counted by registrable domain they are one, so a story that really is corroborated looks single-source.

Articles have the same problem. One article arrives with and without a trailing slash, under an AMP path, with a tracking query, and wrapped in a Google AMP cache URL. Keyed by raw URL that is six articles, ingested six times.

Without a shared library each consumer writes its own prefix strip. The strips drift apart, one outlet ends up with two identities, and anything keyed on outlet forks silently.

## What it does

- Maps a URL or bare host to an outlet key: mirror, AMP, regional, language and legacy hosts of one outlet give one key, and sibling titles under a portal base stay separate.

- Maps a URL to an article key, removing AMP markers, index files, repeated and trailing slashes, tracking parameters, fragments and AMP cache wrapping.

- Honours per-outlet article id rules, so a key survives a change of section or slug.

- Flags aggregators and the caller's own hosts, and offers sameOutlet() and sameArticle().

- Explains every decision with explain() and a CLI --explain flag, and gives sha256 hashes of keys for storage.

- Validates an outlet map file from the command line.

## How it works

The empty string is the key for "unknown", and it is never an outlet. The outlet key is built in order:

- Prepare the input: trim, decode HTML entities once, drop scheme, user info, port and fragment, percent-decode the host once, convert IDN labels with a built-in Punycode encoder, lowercase ASCII, and unwrap a Google AMP cache URL once.

- Strip mirror labels from the left, never below two labels.

- Keep pass-through hosts whole.

- Take the registrable domain from the map's explicit suffixes, then two-letter country codes with a known second level, then the last two labels.

- For portal bases, keep the one label to the left as a separate publication, unless it is a language label.

- Look up the result once in the alias table.

The article key is the outlet key, then the normalised path, then the kept query. On a slug path only id parameters survive. On a script path every non-tracking parameter survives. Outlets with an id rule key as outlet/a/<id>, and other outlets as outlet/s/<slug key>.

The main classes are OutletResolver, ArticleKeyer, OutletMap (immutable and validated), PublicSuffixList, Trace and KeyHash. The map is a JSON file compiled into hash sets, so there is no I/O and no linear scan after load. Overrides use your own entries:

$map = OutletMap::bundled()->withOverrides([
'aliases' => ['example-mirror.com' => 'example.com'],
]);
$resolver = new OutletResolver($map, ['www.my-site.example']);
$keyer = new ArticleKeyer($resolver);

$key = $resolver->outletKey($url);
if ($key !== '' && !$resolver->isAggregator($key) && !$resolver->isOwnOutlet($key)) {
// a newsroom that can corroborate a story
}

On the command line, vendor/bin/outlet-key --explain <url> shows which step decided. The repo's own benchmark script, run on generated URLs rather than real traffic, reports about 9 microseconds for an outlet key and about 25 for an article key on PHP 8.3, one core, over 100,000 mixed URLs.

## Guard rails

- No network access, ever, and no environment variables. The only global state is the memoised bundled map.

- Input over 8192 bytes raises InputTooLongException from both key functions.

- An override is merged and validated again, so a mistake raises InvalidMapException and cannot quietly change keys.

- Keys never depend on loaded extensions. ext-intl and ext-mbstring are not used, and a test runs the fixtures under php -n and compares the output.

- An unusable host gives the empty key, including an all-digit last label that is not an IPv4 address, and a lone scheme.

- Paths keep their case unless an outlet is listed in caseInsensitivePaths, because lowercasing can merge distinct articles. A missed duplicate costs one item. A false merge loses an article.

- Repeated query keys keep every value, sorted, so parameter order cannot change a key.

## What review changed

- A scheme on its own, such as https:, keyed as an outlet named https. A lone scheme is now unparseable.

- A malformed dotted number with an out-of-range last octet keyed as an outlet named after its last two labels. A host with an all-digit last label that is not a valid IPv4 address is now unparseable.

- An AMP cache path with an upper-case marker, /c/S/..., read S as the host. The s/ marker now matches in either case.

- Percent-encoded hosts were unparseable. They are now decoded once, as browsers do, and then validated.

The review also fuzzed 200,000 random structured inputs, which raised nothing except the length exception, and checked the built-in Punycode encoder against ext-intl on 20,000 random labels.

## Limits and what is next

- Redirect wrappers and link shorteners are not unwrapped. Resolve them first.

- The registrable-domain rule is small and explicit. Pass a Public Suffix List for the full one. Only its ICANN section is used unless you ask for the private section.

- Non-ASCII letters are not case-folded or NFC-normalised.

- A map is code, since its patterns run on every URL. Load maps only from sources you trust.

- There is no outlet counting. That is a caller policy, and the repo has an example.

- Keys move when the code's scheme version or the map version changes. Store both beside the key.

- There are no open issues on the repo. The bundled map is a seed, so outlets it does not list fall back to their registrable domain.

Test status: this is v0.1.0. 177 PHPUnit tests (965 assertions) pass, with PHPStan level 8 and PSR-12 checks clean. CI has not yet run on GitHub, so PHP 8.1 and 8.2 were not run. It has been tested on fixtures and fuzzed inputs, not on a live feed.

## Install

Requires PHP 8.1 or later and no runtime dependencies.

The package is not on Packagist yet, so first add a Composer vcs repository entry for https://github.com/IndianOpinion/outlet-key.

composer require indianopinion/outlet-key

Without Composer, clone the repository and require its bootstrap.php, which registers an autoloader for the IndianOpinion\OutletKey namespace.

[All 47 utilities](https://indianopinion.org/blog/open-source/) · [outlet-key on GitHub](https://github.com/IndianOpinion/outlet-key)

Canonical: https://indianopinion.org/blog/open-source/outlet-key/

AI disclosure: Every summary and comment here is written by AI and published automatically, without a person reviewing each article. We do no original reporting. How this works: https://indianopinion.org/ai-use-policy/
