AESOP · research collector
The AESOP research collector
If aesop-arena-tactical-research appears in your server's logs, this page is about it. The collector reads
public web pages and public APIs — encyclopaedia revision histories and news feeds, for example — to prepare
open-source data for wargaming exercises.
Why the collector visited your site, how it behaves there, and how to ask it to stop or to slow down — in your robots.txt, by your server's answers, or with the form at the end, which reaches the people who run it.
What it reads
- Public pages and public APIs What anyone can read without an account: encyclopaedia revision histories, news feeds and the articles they point to.
- Nothing behind a wall It never signs in, never fills in a form, and reads nothing behind a login or a paywall.
- For wargaming exercises What it reads becomes open-source data — dated facts about places and events — for exercises that train people to plan and to decide.
How it behaves
- A modest pace It asks for a few pages at a time at most — never a flood of parallel requests — and reads each feed at set intervals.
- Your robots.txt and your terms It reads your robots.txt and keeps to it, and to each site's terms of use. Where a site publishes an API and asks identified bots to use it under the API's own rules — as the encyclopaedias do — it follows those rules there, rather than a robots.txt that keeps crawlers off the same path.
- It takes no for an answer On a feed or a page it reads, a 401, 403, 429 or 451 is taken as a no, never worked around. For a feed, the no lasts until the time a Retry-After gives in seconds or, without one, until a person has looked. A 503 counts as an error: a feed that fails three polls in a row is set aside.
- It says who it is Every request carries the same name, with the address of this page. It never presents itself as a browser or as another robot, and never tries again under another name.
The full name it sends: aesop-arena-tactical-research/1.0 (+https://aesop.methodart.fr/robot/). The same name also marks
the occasional download of a published data file — a map layer or an elevation tile — fetched one file at a time
when an exercise area is prepared.
How to ask it to stop or slow down
-
To stop it, add this group to your robots.txt — it reads robots.txt again at least once a day:
To keep it out of part of your site only, disallow those paths (User-agent: aesop-arena-tactical-research Disallow: /Disallow: /archive/). -
To pause it, answer its requests for your feed with a 429 and a
Retry-Afterheader in seconds: it leaves the feed alone until then. A 503 is counted as an error, not as a request to wait. To set a pace, tell us with the form below. - Or write to us with the form below and say which site. Messages are read by the people who run the collector.
Write to us
What we keep, and for how long
We keep what you send — the topic, the site, your message and, only if you give one, your reply address — with the date it arrived, to act on it and to answer you. Only the people who run this site read it: it is never published, and no e-mail is sent automatically. It is deleted after 180 days.
Your network address is not kept with the message: it is used only to count how many messages one address, or one block of addresses, sends in an hour, and is then forgotten. Opening this page sets one cookie, a security token against forged submissions, which your browser keeps for up to a year. To have a message deleted sooner, ask in a new message.