wiki-uap research crawler

If you are reading this, you probably saw this user-agent in your logs:

wiki-uap/0.1 (+https://uap-wiki.ai.ten3.it/bot.html; research crawler)

This page explains what it is, how it behaves, and how to make it stop.

What it is

A small, non-commercial research crawler that indexes publicly released declassified government records relating to unidentified anomalous phenomena — Project Blue Book, NARA's Record Group 615, FOIA reading-room releases and similar — to build a citation-backed research index. It fetches documents that agencies have already published for public access. It does not attempt to access anything that is not publicly available, and it does not bypass access controls of any kind.

How it behaves

• Honors robots.txt on every host, every request.
• At most one request every two seconds per host, one connection at a time. Often slower.
• Uses conditional requests (If-None-Match / If-Modified-Since) and an on-disk cache so it does not re-download what it already has.
• Backs off exponentially on 429 and 5xx.
• Identifies itself with this user-agent on every request. It never impersonates a browser.
• Does not download bulk archive files (multi-gigabyte zips). It fetches catalog metadata and individual documents only.

Sites it may visit

archives.gov and catalog.archives.gov (NARA), aaro.mil, nsa.gov, vault.fbi.gov, cia.gov (CREST reading room), dtic.mil, dia.mil, archive.org, theblackvault.com, governmentattic.org.

How to make it stop

Add this to your robots.txt and it will not fetch from your site again:

User-agent: wiki-uap
Disallow: /

That is honored immediately on the next run, without contacting anyone.

Contact

This is a low-volume, non-commercial research crawler operated by a single small team. The robots.txt rule above is the intended contact path: it is honored automatically on the next run, with no need to reach anyone. If it ever fails to respect that rule, that is a bug on our side — a 403 or a rate-limit response from your server will also be honored and backed off from.