.svg?dpl=dpl_7W3cBcau2siV1WKfAAFJwKQExk6C){kind=small}
.svg?dpl=dpl_7W3cBcau2siV1WKfAAFJwKQExk6C){kind=small}
Index a documentation site
Crawl the site's documentation paths and collect the pages as Markdown. Set a page limit, exclude unrelated sections, and send the results to your search index or retrieval system.
Configure a crawlWeb API
Fetch, crawl, and search the web
Fetch pages as Markdown, HTML, screenshots, or JSON. Crawl sites with page and path limits, or search for URLs by query and location. Hyperbrowser manages the browser rendering for each request.
await client.web.fetch({
url: "https://example.com",
outputs: { formats: ["markdown", "links"] }
});fetching…
Reading the page…Collect pages for search and analysis
Find pages for an agent to read
Search by query, site, file type, or location. Results include titles, URLs, and snippets. Fetch the selected URLs to get the page content your agent will read.
Search the webExtract named fields
Specify fields such as product name, price, or publication date in a JSON Schema. Add a prompt describing what to extract, and use the same schema for each page.
Extract structured dataCompare product listings across sites
Search for the product, select the relevant URLs, and fetch each listing with a schema for its name, price, and availability. Store the source URL with each record so you can check it against the original page. Request Markdown as well if your agent needs to read the surrounding description.
For a site's full catalog, use a crawl with a page limit and path patterns that select product pages. Set the cache age to zero when you need a fresh fetch. Check response status and the extracted fields before adding records to your database.
Page extraction and web search
Fetch
Render a page in a browser and extract its content in the formats you request.
Crawl
Follow links from a starting URL and extract content from each page. Set a page limit and include or exclude paths by pattern.
Search
Get result titles, URLs, and snippets. Filter by site or file type, specify a country, state, or city, and set the maximum cache age.
Structured JSON
Extract fields using a prompt, a JSON Schema, or both. Apply the same schema to each page in a crawl.
Branding profiles
Extract a site's colors, fonts, logo, and button styles as structured data.
Page readiness
Configure navigation waits and timeouts for pages that render content after the initial load.
Cache controls
Set the maximum age of a cached response. Use zero to bypass cache reads.
Pay with x402
Pay for fetch and search requests with a wallet through x402. No Hyperbrowser API key is required for these requests.
Make a Web API request
Choose an endpoint
Use fetch for one URL, crawl for pages across a site, or search for a query. Set page and path limits before starting a crawl.
Configure the request
For fetch or crawl, choose the output formats and add a schema or prompt for JSON extraction. Set loading waits and cache age as needed. Search has its own location and filter options.
Read the result
Check the response status and read the returned data. Crawls run as background jobs. Use startAndWait in the SDK, or save the job ID and poll for completion.
Set HYPERBROWSER_API_KEY in your environment. This example fetches a page as Markdown and JSON. The schema defines a title field, and the prompt asks for the main heading.
1// yarn add @hyperbrowser/sdk2import { Hyperbrowser } from "@hyperbrowser/sdk";34const client = new Hyperbrowser({5 apiKey: process.env.HYPERBROWSER_API_KEY,6});78const page = await client.web.fetch({9 url: "https://example.com",10 outputs: {11 formats: [12 "markdown",13 {14 type: "json",15 prompt: "Extract the main heading as the title.",16 schema: {17 type: "object",18 properties: { title: { type: "string" } },19 required: ["title"],20 additionalProperties: false,21 },22 },23 ],24 },25});26console.log(page.status);27console.log(page.data?.markdown);28console.log(page.data?.json);Output formats
Set outputs.formats on fetch or crawl. You can request several formats in one call.
| Format | Returns |
|---|---|
| markdown | Page content converted to Markdown |
| html | Raw HTML of the rendered page |
| links | Links found on the page |
| screenshot | A screenshot of the page |
| json | Structured data extracted with a JSON Schema, a prompt, or both |
| branding | Colors, fonts, logo, and button styles |
Web API pricing
1 credit = $0.001.
| Usage | Credits | Price |
|---|---|---|
| Fetch | 1 credit / page | $0.001 / page |
| Fetch with JSON output | +5 credits / page | +$0.005 / page |
| Fetch with ultra stealth or location | +9 credits / page | +$0.009 / page |
| Search | 5 credits / query | $0.005 / query |
| Search with location | 10 credits / query | $0.01 / query |
| AI Extract | 0.03 credits / output token | $30 / million output tokens |
Common questions
Which endpoint should I start with?
If you have a URL, use fetch. If you need pages across a site, use crawl. If you need to find URLs first, use search, then fetch the results you want to read.
Read moreCan I extract fields from a JavaScript-heavy page?
Yes. Fetch renders the page in a browser before extraction. Set navigation waits and timeouts for content that loads later. To extract specific fields, supply a JSON Schema, a prompt, or both.
Read moreHow do I keep a crawl within scope?
Set maxPages and include or exclude path patterns before starting the job. For example, include documentation paths and exclude account pages. Use the job ID to check progress, or let the SDK poll with startAndWait.
Read moreDo I need to create and stop a browser?
No. The Web API manages rendering for each request. Use a browser session when you need to retain a login across operations, perform custom clicks, or control tabs directly.
Read moreOther products
Start with a free account.
Includes 5,000 credits for browsers, sandboxes, and web data.