> ## Documentation Index
> Fetch the complete documentation index at: https://hyperbrowser.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LlamaIndex

> Integrate Hyperbrowser with LlamaIndex for web scraping and document loading

## Installation and Setup

To get started with LlamaIndex and Hyperbrowser, you can install the necessary packages using pip:

```bash theme={null}
pip install llama-index-core llama-index-readers-web hyperbrowser
```

And you should configure credentials by setting the following environment variables:

`HYPERBROWSER_API_KEY=<your-api-key>`

You can get an API Key easily from the [dashboard](https://app.hyperbrowser.ai). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY` or you can pass it via the `api_key` argument in the `HyperbrowserWebReader` constructor.

## Usage

Once you have your API Key and have installed the packages you can load webpages into LlamaIndex using `HyperbrowserWebReader`.

```python theme={null}
from llama_index.readers.web import HyperbrowserWebReader

reader = HyperbrowserWebReader(api_key="your_api_key_here")
```

To load data, you can specify the operation to be performed by the loader. The default operation is `scrape`. For `scrape`, you can provide a single URL or a list of URLs to be scraped. For `crawl`, you can only provide a single URL. The `crawl` operation will crawl the provided page and subpages and return a document for each page. `HyperbrowserWebReader` supports loading and lazy loading data in both sync and async modes.

```python theme={null}
documents = reader.load_data(
    urls=["https://example.com"],
    operation="scrape",
)
```

Optional params for the loader can also be provided in the `params` argument. For more information on the supported params, you can see the params on the [scraping guide](/web-scraping/scrape).

<Info>
  The params will be Snake case for python code, so here for example, it is `max_pages` instead of `maxPages`.
</Info>

```python theme={null}
# Scrape
documents = reader.load_data(
    urls=["https://example.com"],
    operation="scrape",
    params={"scrape_options": {"include_tags": ["h1", "h2", "p"]}},
)

# Crawl
documents = reader.load_data(
    urls=["https://example.com"],
    operation="crawl",
    params={
        "max_pages": 10,
        "scrape_options": {
            "formats": ["markdown"],
        },
        "session_options": {
            "use_stealth": True,
        }
    }
)
```