Web Crawl

Last 30 days

Uptime

100%

Response time

38 ms typical, 40 ms slowest 5%

Last check

2 h ago

Next check

any minute now

How to call it

# See the payment challenge (nothing is charged)
curl -i -X POST "https://agentdata-api.sander-van-aard.workers.dev/web/crawl" \
  -H "content-type: application/json" \
  -d '{"max_pages":3,"url":"https://example.com/"}'

import { wrapFetchWithPayment } from "@x402/fetch";
import { x402Client } from "@x402/core/client";
import { ExactEvmScheme } from "@x402/evm/exact/client";
import { privateKeyToAccount } from "viem/accounts";

const client = new x402Client().register(
  "eip155:8453",
  new ExactEvmScheme(privateKeyToAccount(process.env.AGENT_KEY)),
);
const pay = wrapFetchWithPayment(fetch, client);

// Not sure it's safe to pay? Preflight it first for $0.005:
// GET https://toolvet.app/api/v1/check?url=https%3A%2F%2Fagentdata-api.sander-van-aard.workers.dev%2Fweb%2Fcrawl
const res = await pay("https://agentdata-api.sander-van-aard.workers.dev/web/crawl", {
  method: "POST",
  headers: { "content-type": "application/json" },
  body: JSON.stringify({"max_pages":3,"url":"https://example.com/"}),
});
console.log(await res.json());

Example input

{
  "max_pages": 3,
  "url": "https://example.com/"
}

Example output

{
  "crawled_at": "2026-07-30T08:00:00.000Z",
  "pages": [
    {
      "content": "# Example Domain\n\nThis domain is for use in illustrative examples, with a [More information](https://example.com/more) link.",
      "content_chars": 120,
      "final_url": "https://example.com/",
      "title": "Example Domain",
      "truncated": false,
      "url": "https://example.com/"
    }
  ],
  "pages_charged": 3,
  "pages_crawled": 1,
  "skipped": [],
  "start_url": "https://example.com/"
}

Security scan

• No findings. We scan names, descriptions and tool definitions for hidden instructions and other prompt-injection patterns.

Recent checks