Sitemap
by Discover and parse sitemaps in Documents & files
Discover a site's XML sitemaps (via robots.txt Sitemap: directives, an explicit sitemap URL, or the conventional /sitemap.xml) and parse them into a bounded list of URLs with their lastmod, changefreq and priority. Follows sitemap-index files to their child sitemaps. Fetches and URL counts are capped (with a `truncated` flag) to stay fast and memory-bounded. The reliable way for an agent to enumerate a site's pages.
POST https://web.openverbs.com/v1/sitemap
Last 30 days
Status and outage history · Atom feed
- Uptime
- 100%
- Response time
- 558 ms typical, 649 ms slowest 5%
- Last check
- 16 min ago
- Next check
- in 17 min
How to call it
POST https://web.openverbs.com/v1/sitemap
Each call costs $0.006 in USDC on Base, paid with the request over x402.
Run a free test callOne unpaid request, the way an agent sends it. You see the payment challenge it gets back. Nothing is charged.
Example request
From the listing.
POST https://web.openverbs.com/v1/sitemap
content-type: application/json
{
"url": "https://example.com"
}Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
url | string (uri) | Yes | Public http(s) URL to fetch. Private/loopback/link-local addresses are rejected. |
Example response
The listing has no example response.
Call it from code
The curl command shows the payment challenge. The JavaScript and Python versions pay and call, and never pay more than $0.006.
curl: see the payment challenge
# bash or zsh: see the payment challenge. Nothing is charged.
curl -sS -g -i -X POST 'https://web.openverbs.com/v1/sitemap' \
-H 'content-type: application/json' \
--data-raw '{"url":"https://example.com"}'
# x402 v2 sends the challenge base64-encoded in the PAYMENT-REQUIRED header (v1 sends it in the body).
# To read the header:
curl -sS -g -o /dev/null -D - -X POST 'https://web.openverbs.com/v1/sitemap' \
-H 'content-type: application/json' \
--data-raw '{"url":"https://example.com"}' \
| grep -i '^payment-required:' | cut -d' ' -f2 | tr -d '\r' | base64 --decodeJavaScript: pay and call
// npm i @x402/fetch @x402/evm viem
import { wrapFetchWithPayment, x402Client } from "@x402/fetch";
import { registerExactEvmScheme } from "@x402/evm/exact/client";
import { privateKeyToAccount } from "viem/accounts";
// Pays at most $0.006 per call.
const client = new x402Client().setSpendControls({ maxAmountPerPayment: "$0.006" });
registerExactEvmScheme(client, { signer: privateKeyToAccount(process.env.EVM_PRIVATE_KEY) });
const pay = wrapFetchWithPayment(fetch, client);
// Not sure it's safe to pay? ToolVet checks it first for $0.005. Pay for that from a separate
// client: this one is capped at $0.006 per payment. GET https://toolvet.app/api/v1/check?url=https%3A%2F%2Fweb.openverbs.com%2Fv1%2Fsitemap&method=POST
const res = await pay("https://web.openverbs.com/v1/sitemap", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({
"url": "https://example.com"
}),
});
console.log(res.status, await res.text());Python: pay and call
# pip install "x402[requests,evm]"
import os
from eth_account import Account
from x402 import x402ClientSync
from x402.http.clients import x402_requests
from x402.mechanisms.evm import EthAccountSigner
from x402.mechanisms.evm.exact.register import register_exact_evm_client
# Pays at most $0.006 per call.
client = x402ClientSync().set_spend_controls({"max_amount_per_payment": "$0.006"})
register_exact_evm_client(client, EthAccountSigner(Account.from_key(os.environ["EVM_PRIVATE_KEY"])))
with x402_requests(client) as session:
res = session.post(
"https://web.openverbs.com/v1/sitemap",
json={
"url": "https://example.com",
},
)
print(res.status_code, res.text)Security scan
- No findings. We scan names, descriptions and tool definitions for hidden instructions and other prompt-injection patterns.
Recent checks
| When | Result | HTTP | Time | Price |
|---|---|---|---|---|
| 16 min ago | Passed | 402 | 623 ms | $0.006 |
| 4 h ago | Passed | 402 | 561 ms | $0.006 |
| 8 h ago | Passed | 402 | 534 ms | $0.006 |
| 12 h ago | Passed | 402 | 649 ms | $0.006 |
| 16 h ago | Passed | 402 | 561 ms | $0.006 |
| 20 h ago | Passed | 402 | 607 ms | $0.006 |
| 23 h ago | Passed | 402 | 558 ms | $0.006 |
| 27 h ago | Passed | 402 | 164 ms | $0.006 |
| 30 h ago | Passed | 402 | 583 ms | $0.006 |
| 34 h ago | Passed | 402 | 530 ms | $0.006 |
| 37 h ago | Passed | 402 | 559 ms | $0.006 |
| 41 h ago | Passed | 402 | 556 ms | $0.006 |
| 44 h ago | Passed | 402 | 516 ms | $0.006 |
| 2 days ago | Passed | 402 | 556 ms | $0.006 |
| 2 days ago | Passed | 402 | 700 ms | $0.006 |