81trust / 100

Dev Text Chunker

by AgentTools in Onchain & crypto

x402 APIPassing, checked 2 h ago

Split text into retrieval-friendly chunks with a token budget and sentence-aware boundaries. Overlap between consecutive chunks preserves context across splits — the standard preparation step before embedding. Use this when an agent needs to rAG-ready chunks with token budgets and overlap.

GET https://agenttools-hub.vercel.app/api/v1/dev/text-chunker

Last 30 days

All checks passedSome failedAll failedNot checked
Uptime
100%
Response time
144 ms typical, 144 ms slowest 5%
Last check
2 h ago
Next check
any minute now

How to call it

# See the payment challenge (nothing is charged)
curl -i -X GET "https://agenttools-hub.vercel.app/api/v1/dev/text-chunker?overlapTokens=15&targetTokens=60&text=Retrieval-augmented+generation+improves+factuality.+First%2C+documents+are+split+into+chunks.+Second%2C+chunks+are+embedded+into+vectors.+Third%2C+the+agent+retrieves+the+top+matches+for+a+query.+Overlap+keeps+ideas+that+straddle+boundaries+intact.+Finally%2C+the+model+answers+with+citations."
import { wrapFetchWithPayment } from "@x402/fetch";
import { x402Client } from "@x402/core/client";
import { ExactEvmScheme } from "@x402/evm/exact/client";
import { privateKeyToAccount } from "viem/accounts";

const client = new x402Client().register(
  "eip155:8453",
  new ExactEvmScheme(privateKeyToAccount(process.env.AGENT_KEY)),
);
const pay = wrapFetchWithPayment(fetch, client);

// Not sure it's safe to pay? Preflight it first for $0.005:
// GET https://toolvet.app/api/v1/check?url=https%3A%2F%2Fagenttools-hub.vercel.app%2Fapi%2Fv1%2Fdev%2Ftext-chunker
const res = await pay("https://agenttools-hub.vercel.app/api/v1/dev/text-chunker?overlapTokens=15&targetTokens=60&text=Retrieval-augmented+generation+improves+factuality.+First%2C+documents+are+split+into+chunks.+Second%2C+chunks+are+embedded+into+vectors.+Third%2C+the+agent+retrieves+the+top+matches+for+a+query.+Overlap+keeps+ideas+that+straddle+boundaries+intact.+Finally%2C+the+model+answers+with+citations.");
console.log(await res.json());

Example input

{
  "overlapTokens": 15,
  "targetTokens": 60,
  "text": "Retrieval-augmented generation improves factuality. First, documents are split into chunks. Second, chunks are embedded into vectors. Third, the agent retrieves the top matches for a query. Overlap keeps ideas that straddle boundaries intact. Finally, the model answers with citations."
}

Example output

{
  "result": {
    "answer": "2 chunks",
    "answerLabel": "≈65 tokens → chunks of ≤60 tokens, overlap 15",
    "artifacts": {
      "chunks": "[{\"index\":0,\"estTokens\":55,\"text\":\"Retrieval-augmented generation improves factuality. First, documents are split into chunks. Second, chunks are embedded into vectors. Third, the agent retrieves the top matches for a query. Overlap keeps ideas that straddle boundaries intact.\"},{\"index\":1,\"estTokens\":22,\"text\":\"Overlap keeps ideas that straddle boundaries intact. Finally, the model answers with citations.\"}]"
    },
    "details": [
      {
        "label": "Input (chars / est. tokens)",
        "value": "285 / ≈65"
      },
      {
        "label": "Avg chunk size",
        "value": "≈39 tokens"
      },
      {
        "label": "Boundary strategy",
        "value": "paragraph → sentence → word"
      }
    ],
    "steps": [
      "Split into 6 sentence units",
      "Pack units up to 60 estimated tokens",
      "Carry ≈15 tokens of overlap between consecutive chunks"
    ],
    "table": {
      "columns": [
        "Chunk",
        "Est. tokens",
        "Preview"
      ],
      "rows": [
        [
          0,
          55,
          "Retrieval-augmented generation improves factuality. First, d…"
        ],
        [
          1,
          22,
          "Overlap keeps ideas that straddle boundaries intact. Finally…"
        ]
      ]
    },
    "value": 2
  },
  "tool": "text-chunker"
}

Security scan

  • No findings. We scan names, descriptions and tool definitions for hidden instructions and other prompt-injection patterns.

Recent checks

WhenResultHTTPTimePrice
2 h agoPassed402144 ms$0.005
7 h agoPassed402161 ms$0.005