newsapi-pull: NewsAPI headlines into WordPress drafts

A WordPress plugin that pulls headlines from NewsAPI into drafts only, with URL-hash de-duplication, source attribution, a daily quota meter and a hard refusal to run a free-tier key on a production site.

newsapi-pull is a WordPress plugin that pulls headlines from NewsAPI into drafts, with country, language and source filters. It is for editorial teams who want a steady list of leads to work from. It is MIT licensed, and v0.1.0 was released on 15 November 2026. It is independent of NewsAPI and uses its documented HTTP interface with your own API key.

The problem

An editor who wants current headlines to work from has two poor options. Copying links by hand is slow. Wiring a news API in with a quick script goes wrong in four ways, each of which costs real time.

  • The same article arrives repeatedly under a mobile host, an AMP path, a tracking query or a trailing slash, and the editor sees five drafts of one story.
  • Items skipped as duplicates leave no record, so the next run looks at them again. The oldest end of the queue stops moving.
  • The free tier has a small daily request quota. A script that does not count its own requests burns it early in the day and every later call fails.
  • The free tier is for development use, and the service’s terms forbid hiding where material came from. A careless importer runs the free key on a live site and strips the source.

What it does

  • Fetches top headlines or article search results on a schedule, with country, category, language, source, domain and keyword filters, validated before a request is spent.
  • Creates one draft (or pending item) per article, with the description, an editor’s note and an attribution block naming the publisher and linking the original.
  • Records every rejected article that has a usable URL, so the next run does not look at it again.
  • Counts every request on a daily meter and stops before the quota runs out.
  • Has a settings page under Tools and WP-CLI commands: list, run, validate, quota, check, sources, log, export and import.

A fresh install runs nothing. There are no default pulls, and you must choose a plan before anything is fetched.

How it works

Each run goes through pre-checks in order: kill switch, plan chosen, environment rule, key present, pull valid, pull not paused, no back-off, quota meter allows. The first failure ends the run with its reason and sends no request. Then the plugin takes a one-run-at-a-time lock, an option written with an atomic add that expires after 10 minutes. It sends HTTPS requests to the NewsAPI host only, with the key in the X-Api-Key header and TLS verification always on.

For each article, in API order, it rejects removed placeholders and bad or wrapper URLs, skips articles already seen, then checks the title, deny lists, language, age and an optional accept filter. The draft is built from the description and the attribution block, with all API text escaped and the body passed through wp_kses_post. The key is recorded only after the draft exists, so a failed insert is retried.

De-duplication stores two 16-byte hashes per URL in a table, with no URL text. The canonical key ignores the scheme, drops default ports, lower-cases and converts the host to ASCII, strips delivery labels such as www, m and amp, normalises the path, collapses AMP path forms and drops tracking parameters. Language subdomains are kept, so hindi.example.com is a different article. The plain key is frozen forever, so rows keep matching if the canonical rules change. You can try the keys without WordPress:

php examples/check-keys.php "https://m.example.com/news/story/amp/?utm_source=x"

Typical setup in wp-config.php:

define('NEWSAPI_PULL_API_KEY', 'your-key');
define('NEWSAPI_PULL_PLAN', 'developer'); // or 'paid'
define('WP_ENVIRONMENT_TYPE', 'development'); // developer plan only fetches on local or development

Guard rails

  • Drafts only. The status is forced back to draft or pending after every filter, enforced by a wp_insert_post_data hook at PHP_INT_MAX, and checked again after the insert. The plugin never publishes or schedules.
  • The developer plan fetches only when the environment type is local or development. No setting, filter, constant, CLI flag or button changes that. The environment filter may only turn true into false. Only the paid plan fetches elsewhere.
  • The key is never in a URL, log, notice or CLI output. A redactor replaces it with [redacted], and the settings page shows only the last four characters.
  • A daily limit (100 on developer) with a reserve of 20 requests held back for manual runs. A kill switch constant, NEWSAPI_PULL_DISABLED, wins over the setting.
  • Every admin action checks manage_options and a nonce, and all output is escaped.
  • Uninstall removes the plugin’s own data and never touches the drafts.

What review changed

  • The quota meter was a read-modify-write of one option with no lock. A run lasting minutes wrote a stale count over requests made meanwhile, and two sites on a network could both take the last request. Meter::claim() now checks and counts in one step under a MySQL named lock, just before each send.
  • The end-of-run state write saved a stale copy of the whole pulls option, reverting any pause, edit or delete an editor made during the run. It now re-reads the option and writes only the run’s own fields.
  • The request-args filter could raise redirection, and a redirect carries the key header to any host. Redirects are now fixed at 0 after the filter.
  • Saving the pull editor with a slug that belonged to a different pull replaced that pull silently. It is now a field error.

Limits and what is next

  • The plugin cannot see your NewsAPI account or verify your plan. The meter is local, and the API’s own quota error is the final word. The API does not document when its counter resets, so the meter uses UTC midnight.
  • The content field the API returns is truncated and the plugin never fetches the article page for more.
  • Images are stored as a URL in meta and not downloaded unless a site turns on newsapi_pull_sideload_image.
  • On a multisite network with per-site activation, sites sharing a key share a quota that no site can see. Network-activate to keep one meter per key.
  • A draft is a lead, not an article to publish as is. You must hold the rights to republish anything you publish. The README says this is information, not legal advice.
  • There are no open issues at the time of writing.

Test status: this is v0.1.0. The review reports 125 PHPUnit tests, PHPStan level 8 and phpcs clean on a local PHP 8.3. The tests use stubs and no live NewsAPI account, and the plugin has not run on a live site. CI has not yet run on GitHub. It was specified, built and reviewed in separate roles by language-model coding agents under the editor’s direction.

Install

It needs WordPress 6.4 or later and PHP 8.1 or later. The intl extension is optional.

The package is not on Packagist yet, so first add a Composer vcs repository entry for https://github.com/IndianOpinion/newsapi-pull.

composer require indianopinion/newsapi-pull

Without Composer, put the folder in wp-content/plugins/newsapi-pull/ and activate it. Nothing runs until you choose a plan.

All 47 utilities · newsapi-pull on GitHub

Search Trending
Loading

Signing-in 3 seconds...

Signing-up 3 seconds...

All fields are required.