|
1
|
# Firecrawl |
|
2
|
|
|
3
|
Proxy requests to Firecrawl via Replit-managed billing. |
|
4
|
|
|
5
|
## Callback |
|
6
|
|
|
7
|
Use `externalApi__firecrawl` in `code_execution`. |
|
8
|
|
|
9
|
## Allowed operations |
|
10
|
|
|
11
|
- `POST` `/scrape` - Scrape any URL and get its content in markdown, HTML, or other formats. |
|
12
|
- `POST` `/batch/scrape` - Scrape multiple URLs and get their content in markdown, HTML, or other formats. |
|
13
|
- `POST` `/map` - Discover all URLs on a website without scraping their content. |
|
14
|
- `POST` `/crawl` - Scrape an entire website by following links from a starting URL. The same per-page scrape option costs apply to each page crawled. |
|
15
|
- `POST` `/agent` - Autonomous web research agent. 5 daily runs free; usage-based pricing beyond that. |
|
16
|
- `POST` `/search` - Search the web and optionally scrape the results. Rounded up per 10 results (e.g., 11 results = 4 credits). Additional per-page scrape costs apply to each result that is scraped. Enterprise ZDR search costs 10 credits / 10 results. |
|
17
|
- `POST` | `GET` | `DELETE` `/*` - All other paths are no-charge. |
|
18
|
|
|
19
|
Authorization is handled automatically by Replit. Do not pass an `Authorization` header. |
|
20
|
|
|
21
|
## Skill |
|
22
|
|
|
23
|
## Firecrawl quickstart |
|
24
|
|
|
25
|
Scrape, crawl, and search the web through Firecrawl passthrough |
|
26
|
billing. Send the required fields as an object in `body` (it is |
|
27
|
serialized for you — do not pre-stringify) — an empty body is |
|
28
|
rejected. |
|
29
|
|
|
30
|
```javascript |
|
31
|
const result = await externalApi__firecrawl({ |
|
32
|
path: '/scrape', |
|
33
|
method: 'POST', |
|
34
|
headers: {'Content-Type': 'application/json'}, |
|
35
|
body: {url: 'https://docs.replit.com', formats: ['markdown']}, |
|
36
|
}) |
|
37
|
|
|
38
|
console.log(result.body.data.markdown) |
|
39
|
``` |
|
40
|
|
|
41
|
Authorization is managed by passthrough billing. Do not set an |
|
42
|
`Authorization` header manually. |
|
43
|
|
|
44
|
Be conservative: scraping is billed per request and adds latency. |
|
45
|
Don't scrape every result from a search — pick the few most |
|
46
|
relevant URLs. Keep no more than ~5 scrapes in flight at once and |
|
47
|
prefer `/batch/scrape` over many parallel `/scrape` calls. |
|
48
|
|
|
49
|
## Example |
|
50
|
|
|
51
|
```javascript |
|
52
|
const result = await externalApi__firecrawl({ |
|
53
|
path: '/scrape', |
|
54
|
method: 'POST', |
|
55
|
query: {}, |
|
56
|
}) |
|
57
|
|
|
58
|
console.log(result.status) |
|
59
|
console.log(result.body) |
|
60
|
``` |