81trust / 100

Sitemap Parser Fetch

by NetIntel in Weather & location

x402 APIPassing, checked 3 h ago

Fetch and parse any XML sitemap or sitemap index file — returns URLs with their priority, change frequency, and last modified date, follows sitemap indexes one level deep (first 3 child sitemaps), and auto-discovers the sitemap from robots.txt or common paths when given a bare domain — so agents can enumerate site content for crawling, indexing, and SEO analysis.

GET https://netintel.dev/sitemap-parser/fetch

Last 30 days

All checks passedSome failedAll failedNot checked
Uptime
100%
Response time
192 ms typical, 200 ms slowest 5%
Last check
3 h ago
Next check
any minute now

How to call it

# See the payment challenge (nothing is charged)
curl -i -X GET "https://netintel.dev/sitemap-parser/fetch?url=https%3A%2F%2Fwww.sitemaps.org%2Fsitemap.xml"
import { wrapFetchWithPayment } from "@x402/fetch";
import { x402Client } from "@x402/core/client";
import { ExactEvmScheme } from "@x402/evm/exact/client";
import { privateKeyToAccount } from "viem/accounts";

const client = new x402Client().register(
  "eip155:8453",
  new ExactEvmScheme(privateKeyToAccount(process.env.AGENT_KEY)),
);
const pay = wrapFetchWithPayment(fetch, client);

// Not sure it's safe to pay? Preflight it first for $0.005:
// GET https://toolvet.app/api/v1/check?url=https%3A%2F%2Fnetintel.dev%2Fsitemap-parser%2Ffetch
const res = await pay("https://netintel.dev/sitemap-parser/fetch?url=https%3A%2F%2Fwww.sitemaps.org%2Fsitemap.xml");
console.log(await res.json());

Example input

{
  "url": "https://www.sitemaps.org/sitemap.xml"
}

Example output

{
  "child_sitemaps_fetched": 0,
  "findings": [],
  "grade": "B",
  "score": 88,
  "sitemap_type": "urlset",
  "source_url": "https://www.sitemaps.org/sitemap.xml",
  "stats": {
    "avg_priority": 0.6,
    "has_changefreq": true,
    "has_lastmod": true,
    "has_priority": true,
    "newest_url_date": "2024-03-10T00:00:00Z",
    "oldest_url_date": "2020-01-01T00:00:00Z"
  },
  "total_urls": 847,
  "urls": [
    {
      "change_frequency": "weekly",
      "last_modified": "2024-03-01T00:00:00Z",
      "priority": 0.8,
      "url": "https://www.sitemaps.org/protocol.html"
    }
  ],
  "urls_returned": 100
}

Security scan

  • No findings. We scan names, descriptions and tool definitions for hidden instructions and other prompt-injection patterns.

Recent checks

WhenResultHTTPTimePrice
3 h agoPassed402180 ms$0.01
6 h agoPassed402322 ms$0.01
9 h agoPassed402200 ms$0.01
14 h agoPassed402192 ms$0.01