Skip to main content

​
Web Scraper

Extract text content and links from any web page. Uses native fetch and lightweight HTML stripping — no browser or heavy dependencies needed.

​
Quick Start

import { Agent, openai, ScraperToolkit } from "@radaros/core";

const scraper = new ScraperToolkit({ maxLength: 10_000 });

const agent = new Agent({
  name: "reader",
  model: openai("gpt-4o"),
  instructions: "Read web pages and summarize their content.",
  tools: [...scraper.getTools()],
});

const result = await agent.run("Summarize the content of https://radaros.dev");

​
Config

​
maxLength
number
default:"15000"
Max characters of extracted text to return.
​
userAgent
string
Custom User-Agent header for requests.
​
timeout
number
default:"15000"
Request timeout in milliseconds.

​
Tools

ToolDescription
scrape_urlFetch a URL and extract text content. Scripts, styles, nav, and footer are stripped.
scrape_linksExtract all links from a page. Returns link text and absolute URLs.