ToolVet
78trust / 100

Scrape Metadata

by web-scraper-api-production-bf20.up.railway.app in Search & web

x402 APIPassing, checked 3 h ago

Extract page metadata: title, description, keywords, author, published/modified dates, canonical, language, favicon, and OpenGraph/Twitter card tags.

POST
https://web-scraper-api-production-bf20.up.railway.app/scrape/metadata
Price
$0.002
per call in USDC
Uptime
100%
last 30 days
Response time
362 ms
568 ms slowest 5%
Last check
Passed
3 h ago, next any minute now

Run a free test callHow to call it

Reliability

All checks passedSome failedAll failedNot checked

Status and outage history · Atom feed · Watch with other listings

How to call it

POST https://web-scraper-api-production-bf20.up.railway.app/scrape/metadata

Each call costs $0.002 in USDC on Base, paid with the request over x402.

Run a free test callOne unpaid request, the way an agent sends it. You see the payment challenge it gets back. Nothing is charged.

Example request

From the listing.

POST https://web-scraper-api-production-bf20.up.railway.app/scrape/metadata
content-type: application/json

{
  "url": "https://example.com"
}

Parameters

ParameterTypeRequiredDescription
urlstringYesAbsolute http(s) URL of the page to scrape, e.g. 'https://example.com'.

Example response

The provider's example. We can't check it without paying.

Example response, 29 lines
{
  "author": null,
  "canonical": "https://example.com/",
  "description": "Illustrative example domain reserved for use in documentation.",
  "favicon": "https://example.com/favicon.ico",
  "final_url": "https://example.com/",
  "keywords": null,
  "language": "en",
  "modified": "2024-02-01T12:30:00Z",
  "open_graph": {
    "description": "Illustrative example domain.",
    "image": "https://example.com/og.png",
    "site_name": "Example",
    "title": "Example Domain",
    "type": "website",
    "url": "https://example.com/"
  },
  "published": "2024-01-15T09:00:00Z",
  "title": "Example Domain",
  "twitter": {
    "card": "summary_large_image",
    "creator": "@example",
    "description": "Illustrative example domain.",
    "image": "https://example.com/tw.png",
    "site": "@example",
    "title": "Example Domain"
  },
  "url": "https://example.com"
}

Call it from code

The curl command shows the payment challenge. The JavaScript and Python versions pay and call, and never pay more than $0.002.

curl: see the payment challenge
# bash or zsh: see the payment challenge. Nothing is charged.
curl -sS -g -i -X POST 'https://web-scraper-api-production-bf20.up.railway.app/scrape/metadata' \
  -H 'content-type: application/json' \
  --data-raw '{"url":"https://example.com"}'

# x402 v2 sends the challenge base64-encoded in the PAYMENT-REQUIRED header (v1 sends it in the body).
# To read the header:
curl -sS -g -o /dev/null -D - -X POST 'https://web-scraper-api-production-bf20.up.railway.app/scrape/metadata' \
  -H 'content-type: application/json' \
  --data-raw '{"url":"https://example.com"}' \
  | grep -i '^payment-required:' | cut -d' ' -f2 | tr -d '\r' | base64 --decode
JavaScript: pay and call
// npm i @x402/fetch @x402/evm viem
import { wrapFetchWithPayment, x402Client } from "@x402/fetch";
import { registerExactEvmScheme } from "@x402/evm/exact/client";
import { privateKeyToAccount } from "viem/accounts";

// Pays at most $0.002 per call.
const client = new x402Client().setSpendControls({ maxAmountPerPayment: "$0.002" });
registerExactEvmScheme(client, { signer: privateKeyToAccount(process.env.EVM_PRIVATE_KEY) });
const pay = wrapFetchWithPayment(fetch, client);

// Not sure it's safe to pay? ToolVet checks it first for $0.005. Pay for that from a separate
// client: this one is capped at $0.002 per payment. GET https://toolvet.app/api/v1/check?url=https%3A%2F%2Fweb-scraper-api-production-bf20.up.railway.app%2Fscrape%2Fmetadata&method=POST
const res = await pay("https://web-scraper-api-production-bf20.up.railway.app/scrape/metadata", {
  method: "POST",
  headers: { "content-type": "application/json" },
  body: JSON.stringify({
    "url": "https://example.com"
  }),
});
console.log(res.status, await res.text());
Python: pay and call
# pip install "x402[requests,evm]"
import os
from eth_account import Account
from x402 import x402ClientSync
from x402.http.clients import x402_requests
from x402.mechanisms.evm import EthAccountSigner
from x402.mechanisms.evm.exact.register import register_exact_evm_client

# Pays at most $0.002 per call.
client = x402ClientSync().set_spend_controls({"max_amount_per_payment": "$0.002"})
register_exact_evm_client(client, EthAccountSigner(Account.from_key(os.environ["EVM_PRIVATE_KEY"])))

with x402_requests(client) as session:
    res = session.post(
        "https://web-scraper-api-production-bf20.up.railway.app/scrape/metadata",
        json={
            "url": "https://example.com",
        },
    )
    print(res.status_code, res.text)

Security scan

No findings

We scan names, descriptions and tool definitions for hidden instructions and other prompt-injection patterns.

Recent checks

WhenResultHTTPTimePrice
3 h agoPassed402454 ms$0.002
8 h agoPassed402899 ms$0.002
13 h agoPassed402370 ms$0.002
18 h agoPassed402454 ms$0.002
22 h agoPassed402393 ms$0.002
Show 10 older checks
27 h agoPassed402390 ms$0.002
32 h agoPassed402341 ms$0.002
36 h agoPassed402530 ms$0.002
40 h agoPassed402365 ms$0.002
44 h agoPassed402358 ms$0.002
2 days agoPassed402362 ms$0.002
2 days agoPassed402340 ms$0.002
2 days agoPassed402344 ms$0.002
3 days agoPassed402336 ms$0.002
3 days agoPassed402340 ms$0.002