Quickstart

In this guide, you’ll learn how to scrape Hacker News using scripts and powerful AI agents to automate with Hyperbrowser’s cloud based browsers.

Prerequisites

Get Your API Key

export HYPERBROWSER_API_KEY=<your-api-key>

Install Dependencies

Playwright
Puppeteer

npm install @hyperbrowser/sdk playwright-core dotenv

yarn add @hyperbrowser/sdk playwright-core dotenv

pip install hyperbrowser playwright python-dotenv

uv add hyperbrowser playwright python-dotenv

npm install @hyperbrowser/sdk puppeteer-core dotenv

yarn add @hyperbrowser/sdk puppeteer-core dotenv

pip install hyperbrowser pyppeteer python-dotenv

uv add hyperbrowser pyppeteer python-dotenv

Part 1: Automate with Playwright or Puppeteer Scripts

Let’s start by scraping the top story from Hacker News using Playwright or Puppeteer. This shows how you can launch Hyperbrowser sessions and connect to them with automation libraries like Playwright or Puppeteer.

Playwright
Puppeteer

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { chromium } from "playwright-core";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

async function getTopStory() {
  // Create a cloud browser session
  const session = await client.sessions.create();
  console.log("Session created:", session.id);

  try {
    // Connect Playwright to the cloud browser
    const browser = await chromium.connectOverCDP(session.wsEndpoint);
    const context = browser.contexts()[0];
    const page = context.pages()[0];

    // Navigate to Hacker News
    await page.goto("https://news.ycombinator.com/");

    // Extract the first story title
    const topStory = await page.evaluate(() => {
      const titleElement = document.querySelector(".titleline > a");
      return titleElement ? titleElement.textContent : null;
    });

    console.log("Top Story:", topStory);
    return topStory;
  } catch (err) {
    console.error(`Encountered error: ${err}`);
  } finally {
    // Clean up the session
    await client.sessions.stop(session.id);
  }
}

getTopStory().catch(console.error);

import asyncio
import os
from playwright.async_api import async_playwright
from hyperbrowser import AsyncHyperbrowser
from dotenv import load_dotenv

load_dotenv()

client = AsyncHyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


async def get_top_story():
    # Create a cloud browser session
    session = await client.sessions.create()
    print(f"Session created: {session.id}")

    try:
        # Connect Playwright to the cloud browser
        async with async_playwright() as p:
            browser = await p.chromium.connect_over_cdp(session.ws_endpoint)
            context = browser.contexts[0]
            page = context.pages[0]

            # Navigate to Hacker News
            await page.goto("https://news.ycombinator.com/")

            # Extract the first story title
            top_story = await page.evaluate("""
                () => {
                    const titleElement = document.querySelector('.titleline > a');
                    return titleElement ? titleElement.textContent : null;
                }
            """)

            print(f"Top Story: {top_story}")
            return top_story
    except Exception as e:
        print(f"Error: {e}")
    finally:
        # Clean up the session
        await client.sessions.stop(session.id)


if __name__ == "__main__":
    asyncio.run(get_top_story())

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { connect } from "puppeteer-core";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

async function getTopStory() {
  // Create a cloud browser session
  const session = await client.sessions.create();
  console.log("Session created:", session.id);

  try {
    // Connect Playwright to the cloud browser
    const browser = await connect({
      browserWSEndpoint: session.wsEndpoint,
      defaultViewport: null,
    });
    const page = (await browser.pages())[0];

    // Navigate to Hacker News
    await page.goto("https://news.ycombinator.com/");

    // Extract the first story title
    const topStory = await page.evaluate(() => {
      const titleElement = document.querySelector(".titleline > a");
      return titleElement ? titleElement.textContent : null;
    });

    console.log("Top Story:", topStory);
    return topStory;
  } catch (err) {
    console.error(`Encountered error: ${err}`);
  } finally {
    // Clean up the session
    await client.sessions.stop(session.id);
  }
}

getTopStory().catch(console.error);

import asyncio
import os
from pyppeteer import connect
from hyperbrowser import AsyncHyperbrowser
from dotenv import load_dotenv

load_dotenv()

client = AsyncHyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


async def get_top_story():
    # Create a cloud browser session
    session = await client.sessions.create()
    print(f"Session created: {session.id}")

    try:
        # Connect Puppeteer to the cloud browser
        browser = await connect(
            browserWSEndpoint=session.ws_endpoint, defaultViewport=None
        )
        context = browser._defaultContext
        page = await context.newPage()

        # Navigate to Hacker News
        await page.goto("https://news.ycombinator.com/")

        # Extract the first story title
        top_story = await page.evaluate("""
            () => {
                const titleElement = document.querySelector('.titleline > a');
                return titleElement ? titleElement.textContent : null;
            }
        """)

        print(f"Top Story: {top_story}")
        return top_story
    except Exception as e:
        print(f"Error: {e}")
    finally:
        # Clean up the session
        await client.sessions.stop(session.id)


if __name__ == "__main__":
    asyncio.run(get_top_story())

Part 2: AI-Powered Automation

Now let’s solve the same problem using HyperAgent, our AI-powered browser automation tool. Instead of writing selectors and navigation logic, you simply describe what you want in natural language.

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

async function getTopStoryWithAgent() {
  const result = await client.agents.hyperAgent.startAndWait({
    task: "Go to Hacker News and get the title of the first post",
  });

  console.log("Top Story:", result.data?.finalResult);
  return result.data?.finalResult;
}

getTopStoryWithAgent().catch(console.error);

import os
from hyperbrowser import Hyperbrowser
from hyperbrowser.models import StartHyperAgentTaskParams
from dotenv import load_dotenv

load_dotenv()

client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


def get_top_story_with_agent():
    result = client.agents.hyper_agent.start_and_wait(
        StartHyperAgentTaskParams(
            task="Go to Hacker News and get the title of the first post"
        )
    )

    print(f"Top Story: {result.data.final_result}")
    return result.data.final_result


if __name__ == "__main__":
    get_top_story_with_agent()

curl -X POST https://api.hyperbrowser.ai/api/task/hyper-agent \
  -H 'Content-Type: application/json' \
  -H 'x-api-key: <YOUR_API_KEY>' \
  -d '{
    "task": "Go to Hacker News and get the title of the first post"
  }'

The Difference

With HyperAgent, you don’t need to:

Write CSS selectors
Handle page navigation
Manage browser state
Debug DOM changes

Just describe what you want, and the AI handles the rest. It’s perfect for complex workflows, dynamic sites, and rapid prototyping.

Hyperbrowser also integrates with other popular AI agent frameworks including BrowserUse, Claude Computer Use, Gemini Computer Use, Grok Computer Use, and OpenAI CUA. All agents benefit from Hyperbrowser’s cloud infrastructure, stealth capabilities, and proxy support.

Next Steps

Learn more about managing browsers sessions and other capabilities:

Browser Sessions

Learn about managing cloud browser sessions at scale

Web Scraping APIs

Extract data from websites with our scraping APIs

AI Agents

Dive deeper into HyperAgent and other AI automation tools

API Reference

Explore the complete API documentation

Additional Resources

Browser Integrations

Puppeteer Integration - Connect with Puppeteer for automation
Playwright Integration - Connect with Playwright for automation

SDKs

Node.js SDK - TypeScript/JavaScript SDK reference
Python SDK - Python SDK reference

AI Agent Integrations

BrowserUse - Open-source agent for fast browser automation
Claude Computer Use - Anthropic’s Claude with computer capabilities
Gemini Computer Use - Google’s Gemini computer use agent
Grok Computer Use - xAI’s Grok 4.5 computer use agent
OpenAI CUA - OpenAI’s computer use agent

Get Started

Browser Sessions

Session Configuration

Web API

Agents

HyperAgent Usage

Scraping

Integrations

Prerequisites

Get Your API Key

Install Dependencies

Part 1: Automate with Playwright or Puppeteer Scripts

Part 2: AI-Powered Automation

The Difference

Next Steps

Browser Sessions

Web Scraping APIs

AI Agents

API Reference

Additional Resources

Browser Integrations

SDKs

AI Agent Integrations

​Prerequisites

​Get Your API Key

​Install Dependencies

​Part 1: Automate with Playwright or Puppeteer Scripts

​Part 2: AI-Powered Automation

​The Difference

​Next Steps

Browser Sessions

Web Scraping APIs

AI Agents

API Reference

​Additional Resources

​Browser Integrations

​SDKs

​AI Agent Integrations

Prerequisites

Get Your API Key

Install Dependencies

Part 1: Automate with Playwright or Puppeteer Scripts

Part 2: AI-Powered Automation

The Difference

Next Steps

Additional Resources

Browser Integrations

SDKs

AI Agent Integrations