Firecrawl
Firecrawl is an open-source API that lets developers search, scrape, and interact with web pages at scale, converting content into clean markdown or structured data for AI applications. It handles proxies, rate limits, and JavaScript-heavy pages, making it reliable for real-time agents and data extraction. It is designed for developers building AI agents, data pipelines, or applications that need web content.
🎬 Firecrawl AI clearly explained (and how to make $$) · Greg Isenberg
✨ Key features
- Search the web and get full page content from results.
- Scrape any URL to markdown, HTML, screenshots, or structured JSON.
- Interact with pages using AI prompts or code after scraping.
- Crawl entire websites and map all URLs with a single request.
- Batch scrape thousands of URLs asynchronously.
- Agent endpoint for autonomous data gathering with structured output.
🎯 Use cases
- Extract clean markdown from websites for LLM training or RAG.
- Search the web and retrieve content for AI agents.
- Automate data collection from multiple pages or entire sites.
- Interact with dynamic pages like Amazon to search and click.
- Gather structured data like pricing or founder info via agent.
📦 Installation
🧰 Requirements: Requires an API key from firecrawl.dev; supports Python, Node.js, Go, and CLI; no OS specified.
pip install firecrawl-py
npm install firecrawl
go get github.com/firecrawl/firecrawl-go
🚀 Usage
from firecrawl import Firecrawl
app = Firecrawl(api_key="fc-YOUR_API_KEY")
# Scrape a single URL
doc = app.scrape("https://firecrawl.dev", formats=["markdown"])
print(doc.markdown)
# Search the web
results = app.search("firecrawl", limit=5)
❓ FAQ
How do I get an API key?
Sign up at firecrawl.dev to get your API key, which you use in the SDKs or API calls.
What output formats are supported?
Scrape supports markdown, HTML, screenshots, and structured JSON, among others.
Can I use Firecrawl with AI agents?
Yes, you can connect it to any AI agent or MCP client using the Skill or MCP setup commands.
Does the Agent endpoint require URLs?
No, the Agent endpoint can work without URLs; you can describe what you need and it will search and retrieve data.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.