Building a YouTube Data Extraction Tool with Model Context Protocol and Hyperbrowser

In this cookbook, we'll build a powerful YouTube data extraction server using the Model Context Protocol (MCP) and Hyperbrowser. This combination allows AI models to access YouTube data directly from your local machine through a standardized protocol.

With this setup, you'll be able to:

Extract video descriptions from any YouTube video
Retrieve recent videos from any YouTube channel
Access community posts from YouTube creators

The Model Context Protocol acts as a bridge between AI models and your local tools, enabling AI assistants to interact with web content they couldn't otherwise access. By the end, you'll have a robust server that can be extended for various YouTube data extraction needs!

Prerequisites

Before starting, you'll need:

A Hyperbrowser API key (sign up at hyperbrowser.ai if you don't have one)
The MCP Python package (pip install mcp)
Playwright for Python (pip install playwright)
Pydantic for adding models when doing structured extraction (pip install pydantic)
Python 3.9+ installed

Store your API key in a .env file or set it as an environment variable as needed for the MCP client.

So, for example for Claude, we'd modify the claude_desktop_config.json like so:

{
  "mcpServers": {
    "hyperbrowser-ytb": {
      "command": "<PATH TO PYTHON>",
      "args": [
        "<PATH TO MAIN.PY>/main.py"
      ],
      "env": {
        "HYPERBROWSER_API_KEY": "<HYPERBROWSER_API_KEY>"
      }
    }
  }
}

Step 1: Initialize the MCP Server

We start by initializing a Fast MCP server. The Model Context Protocol (MCP) creates a standardized way for AI models to discover and interact with tools running on your local machine.

Unlike traditional API integrations where the AI needs to be specifically trained on each endpoint, MCP allows AIs to dynamically discover available tools and their capabilities, making integration much more flexible and powerful.

import asyncio
import json
import os

from hyperbrowser import Hyperbrowser
from hyperbrowser.models.scrape import StartScrapeJobParams, ScrapeOptions
from hyperbrowser.models.extract import StartExtractJobParams
from hyperbrowser.models.session import CreateSessionParams
from playwright.async_api import async_playwright
from typing import List

from pydantic import BaseModel
from mcp.server.fastmcp import FastMCP

mcp = FastMCP("hyperbrowser-ytb")

Step 2: Create a Video Description Extraction Tool

Our first tool enables AI models to extract the description from any YouTube video. YouTube descriptions often contain valuable information including timestamps, links to related content, and detailed explanations.

We use Hyperbrowser's targeted scraping capabilities to extract only the description element from the page, making the extraction both efficient and precise. The include_tags parameter focuses our scraping on just the description div.

@mcp.tool()
def get_youtube_video_description(url: str) -> str:
    """Get the description of a youtube video from it's url"""
    hb = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))
    resp = hb.scrape.start_and_wait(
        StartScrapeJobParams(
            url=url,
            scrape_options=ScrapeOptions(include_tags=["div#description"]),
            session_options=CreateSessionParams(use_proxy=True),
        )
    )
    if resp.data:
        return resp.data.model_dump_json()
    else:
        raise ValueError(
            "Could not get video description: Youtube scrape result was None"
        )

Step 3: Build a Channel Video Listing Tool

Next, we create a more complex tool that extracts recent videos from any YouTube channel. This tool goes beyond simple scraping by:

Establishing a live browser session through Hyperbrowser
Using Playwright to interact with the page's dynamic content
Extracting structured data including video URLs, titles, and thumbnails

This approach handles YouTube's JavaScript-heavy interface that would be difficult to scrape with traditional methods. The async implementation ensures efficient execution with proper waiting for content to load.

@mcp.tool()
async def get_youtube_channel_recent_videos(channel_url: str):
    """Get the url,title,and thumbnail for the recent videos of a youtube channel"""
    channel_url = (
        channel_url if channel_url.endswith("/videos") else f"{channel_url}/videos"
    )
    hb = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))
    session = hb.sessions.create(CreateSessionParams(use_proxy=True))
    if session.ws_endpoint is None:
        raise Exception("Could not make a hyperbrowser session")
    
    async with async_playwright() as p:
        browser = await p.chromium.connect_over_cdp(session.ws_endpoint)
        page = await browser.new_page()
        await page.goto(channel_url)
        await asyncio.sleep(2.5)
        grid_elements = await page.query_selector_all("ytd-rich-grid-media")
        video_infos = []
        for grid_element in grid_elements:
            video_info = {}
            link_element = await grid_element.query_selector(
                "a.ytd-rich-grid-media.focus-on-expand"
            )
            if link_element:
                href = await link_element.get_attribute("href")
                title = await link_element.inner_text()
                if href:
                    video_info["url"] = href
                if title:
                    video_info["title"] = title
            img_element = await grid_element.query_selector("img")
            if img_element:
                img_src = await img_element.get_attribute("src")
                if img_src:
                    video_info["thumbnail"] = img_src
            video_infos.append(json.dumps(video_info))

        return video_infos

Step 4: Create a Community Posts Extraction Tool

Our final tool enables access to a channel's community posts - content that's typically not available through any official API. Community posts often contain announcements, polls, and updates that provide valuable context about a creator's work.

We use Hyperbrowser's extract feature with Pydantic models to create a structured representation of the community posts. This combines the power of web automation with strong typing for reliable data extraction.

class CommunityPost(BaseModel):
    text: str
    comments: str
    link: str


class CommunityPostList(BaseModel):
    posts: List[CommunityPost]


@mcp.tool()
def get_youtube_channel_recent_posts(channel_url: str) -> str:
    """Get the recent community posts of a youtube channel from it's url"""
    channel_url = (
        channel_url
        if channel_url.endswith("/community")
        else f"{channel_url}/community"
    )
    hb = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))
    session = hb.extract.start_and_wait(
        StartExtractJobParams(
            urls=[channel_url],
            session_options=CreateSessionParams(use_proxy=True),
            schema=CommunityPostList,
        )
    )

    return session.model_dump_json()

Step 5: Running the MCP Server

Now we'll launch our MCP server to make our YouTube tools available to AI models. The MCP server uses stdio (standard input/output) as its transport mechanism, making it compatible with a wide range of clients, including Claude Desktop, Cline, Cursor, Windsurf among many more.

When an AI model connects to this server, it will discover all three of our YouTube tools along with their documentation, parameter types, and return types - all through the standardized MCP protocol.

if __name__ == "__main__":
    # Initialize and run the server
    mcp.run(transport="stdio")

Conclusion

In this cookbook, we've built a powerful YouTube data extraction server using the Model Context Protocol and Hyperbrowser. This combination enables AI models to access YouTube content that would otherwise be unavailable to them.

By leveraging MCP, we've created a standardized interface that allows any compatible AI to:

Extract video descriptions
List recent videos from channels
Access community posts

All without requiring custom training or hardcoded integrations for each tool.

Next Steps

To take this further, you might consider:

Adding tools for video comments extraction
Implementing search functionality across YouTube
Creating tools for channel statistics and metrics
Building a user interface that works alongside the MCP server
Adding authentication to access private or members-only content

The MCP protocol opens up possibilities far beyond YouTube - any web-based or local data source can be made available to AI models using this same pattern.