Base URL

Documentation

API reference

Prairie Scrape exposes a clean REST API over managed web scraping infrastructure. Authenticate with your API key, send JSON, get clean data back.

Getting started

Create a free account and grab your API key from the dashboard. Then send your first scrape request:

# Set your key once per shell session
export PRAIRIE_API_KEY="fc_live_YOUR_KEY"

# -w prints the HTTP status so we can branch on it
HTTP_STATUS=$(curl -sS -o /tmp/prairie.json -w "%{http_code}" \
  -X POST https://prairieai.dev/api/public/v1/scrape \
  -H "Authorization: Bearer $PRAIRIE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "url": "https://example.com", "formats": ["markdown", "html"] }')

case "$HTTP_STATUS" in
  2*) cat /tmp/prairie.json ;;
  401) echo "Invalid API key — check Authorization header." >&2; exit 1 ;;
  402) echo "Out of credits — top up at https://prairieai.dev/dashboard/credits" >&2; exit 2 ;;
  429) echo "Rate limited — retry after a few seconds." >&2; exit 3 ;;
  *)   echo "Request failed ($HTTP_STATUS):" >&2; cat /tmp/prairie.json >&2; exit 1 ;;
esac

Try it now

POST /api/public/v1/scrape

Sends a real request through the Prairie gateway to the upstream scraping instance. Pick one of your keys (or create one on the Keys page) and a URL to scrape.

Loading…
Response will appear here.

Authentication

All requests authenticate via the Authorization header. Keys start with fc_live_ and can be rotated anytime from the dashboard.

Authorization: Bearer fc_live_82x1prairie_k7v9z2qm4rt8nfa2pdlx

Errors

The API returns standard HTTP status codes. Errors include a JSON body with a code and message.

{ "success": false, "error": "Insufficient credits", "code": "insufficient_credits" }
400bad_request

Malformed JSON or missing required field.

401unauthorized

Missing or invalid API key.

402insufficient_credits

Top up on the Credits page.

429rate_limited

Too many requests — slow down and retry.

502bad_gateway

Upstream scraping engine unreachable.

504timeout

Upstream took too long to respond.

Troubleshooting

Cloudflare error 1003 — Direct IP access not allowed

If the gateway returns a 502 with a Cloudflare HTML body that says "Error 1003 — Direct IP access not allowed", the upstream scraping engine is behind Cloudflare and rejects requests that arrive via a raw IP address instead of its hostname.

The gateway proxies requests to SCRAPE_BASE_URL. When that value is an IP, Cloudflare blocks the call. Use the hostname the upstream is configured with in Cloudflare.

# Correct — use the hostname, not the IP
SCRAPE_BASE_URL=https://scrape.yourdomain.com

# Wrong — IP will trigger Cloudflare 1003
SCRAPE_BASE_URL=http://203.0.113.10:3002

How to set SCRAPE_BASE_URL

Set the environment variable in your hosting dashboard or secrets manager. The gateway reads it at runtime and falls back to the default only when the variable is absent.

Lovable Cloud

Add SCRAPE_BASE_URL in Project Settings → Secrets.

Vercel

Add SCRAPE_BASE_URL in Settings → Environment Variables.

Railway / Render

Add SCRAPE_BASE_URL in the service's Environment tab.

Self-hosted (Docker)

Pass -e SCRAPE_BASE_URL=https://… when starting the container.

Check the current value

Send a request to the gateway and inspect the x-prairie-duration-ms header in the response — the gateway adds it to every proxied response. If you get a 502, read the response body: a plain JSON error means the upstream is down, while an HTML body usually means Cloudflare (or another reverse proxy) is intercepting the request.

POST/api/public/v1/scrape1 credit

Fetches a single URL and returns clean markdown, HTML, and metadata. Add `screenshot` to `formats` to capture a screenshot in the same call.

Full URLhttps://prairieai.dev/api/public/v1/scrape

Sample request

# Set your key once per shell session
export PRAIRIE_API_KEY="fc_live_YOUR_KEY"

# -w prints the HTTP status so we can branch on it
HTTP_STATUS=$(curl -sS -o /tmp/prairie.json -w "%{http_code}" \
  -X POST https://prairieai.dev/api/public/v1/scrape \
  -H "Authorization: Bearer $PRAIRIE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "url": "https://example.com", "formats": ["markdown", "html"] }')

case "$HTTP_STATUS" in
  2*) cat /tmp/prairie.json ;;
  401) echo "Invalid API key — check Authorization header." >&2; exit 1 ;;
  402) echo "Out of credits — top up at https://prairieai.dev/dashboard/credits" >&2; exit 2 ;;
  429) echo "Rate limited — retry after a few seconds." >&2; exit 3 ;;
  *)   echo "Request failed ($HTTP_STATUS):" >&2; cat /tmp/prairie.json >&2; exit 1 ;;
esac

Request body

{
  "url": "https://example.com",
  "formats": ["markdown", "html"]
}

Response

{
  "success": true,
  "data": {
    "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
    "html": "<h1>Example Domain</h1>...",
    "metadata": {
      "title": "Example Domain",
      "statusCode": 200,
      "sourceURL": "https://example.com"
    }
  }
}