LodestarBot

LodestarBot is the web crawler of Lodestar GmbH, Buochs, Switzerland (UID CHE-187.728.340).

Purpose

LodestarBot monitors the prices and the availability of sealed One Piece Card Game products in online shops. Lodestar uses the data for its own buying decisions. Our weekly newsletter may cite a few public offers (price and a link to the shop). We do not republish merchant catalogues.

We read product data only: titles, prices, availability and images. We do not read customer, account, cart or checkout pages and we collect no personal data.

Identification

A request that carries this User-Agent with a signature that does not verify against this key directory is not from us.

What we fetch

Only public storefront pages and JSON, with HTTP GET. On Shopify shops:

On about 70 WooCommerce shops: the public Store API product search (/wp-json/wc/store/v1/products, /wp-json/wc/store/products or /?rest_route=/wc/store/v1/products). We do not check robots.txt on these WooCommerce shops; we read only their public Store API.

Our discovery agent reads candidate product pages and checks new shops (/meta.json, /products.json?limit=1) with its own client, which guards against unknown sites. It sends the same User-Agent and the same signature and obeys robots.txt for the same product token. Its few requests are outside the shared pace below.

We do not log in, add to a cart, check out, submit forms or store cookies.

Scale (as of 2026-11-10)

robots.txt

We follow RFC 9309. Before we send a request to a Shopify shop we read its /robots.txt and keep it for up to 24 hours. We obey the group for LodestarBot; when there is none, the * group. The longest matching rule wins and * and $ are supported. A 4xx answer (other than 429) means no restrictions. When robots.txt cannot be read (429, 5xx or a network error) we use the last copy we read, if it is less than 30 days old; otherwise we send no requests to that shop, and we ask again after one hour or its Retry-After, whichever is later. When the group sets a Crawl-delay, it becomes the gap of the shared pace for that shop (up to one hour), so it holds over all our processes; a request that would come sooner waits or is deferred.

Rate limits

We never retry a request that was answered with 429. The 429 answers feed a pause that all our processes share: when many requests of one kind (catalogue feeds or product checks) are answered with 429 within 5 minutes, that kind stops until the longest Retry-After plus a cool-down has passed. Our hourly catalogue check also holds a shop for its Retry-After after a single 429. Our once-a-day catalogue read is not stopped by the shared pause; it stops reading a shop at that shop's first 429.

Opt out or contact

To stop LodestarBot on your shop, add this to your robots.txt:

User-agent: LodestarBot
Disallow: /

Or write to contact@lodestarsystems.ch.

Page built 2026-11-15. Key directory signature valid until 2026-12-15.