# Cancel a task
Source: https://docs.parse.bot/api-reference/dispatch/cancel-a-task
/openapi.json post /dispatch/tasks/{task_id}/cancel
Cancel an in-flight build or revision task. Only tasks in a non-terminal state (queued, running, needs_input) can be cancelled.
# Check for available updates
Source: https://docs.parse.bot/api-reference/dispatch/check-for-available-updates
/openapi.json get /dispatch/tasks/{task_id}/updates
Compare your API against the canonical version to find new or updated endpoints. Returns a list of changes you can preview and merge.
# Create a new API from a URL
Source: https://docs.parse.bot/api-reference/dispatch/create-a-new-api-from-a-url
/openapi.json post /dispatch
Submit a website URL to generate an API. If a matching API already exists, returns it instantly. Otherwise, queues a job to build one. Optionally describe the data you need in the `task` field.
# Get SDK usage example
Source: https://docs.parse.bot/api-reference/dispatch/get-sdk-usage-example
/openapi.json get /dispatch/tasks/{task_id}/sdk-example
Get a code example showing how to call this API using the Parse SDK.
# Get task detail
Source: https://docs.parse.bot/api-reference/dispatch/get-task-detail
/openapi.json get /dispatch/tasks/{task_id}
Get full details for a dispatch task, including the generated API spec with endpoint definitions and execution URLs when completed.
# Get the in-flight task for a scraper
Source: https://docs.parse.bot/api-reference/dispatch/get-the-in-flight-task-for-a-scraper
/openapi.json get /dispatch/scrapers/{scraper_id}/active-task
Return the active (queued/running/needs_input) revision or extension task for a given scraper, if any. Useful for showing live progress on an API you already own. Returns `{ "task": null }` when nothing is in flight.
# List revision history
Source: https://docs.parse.bot/api-reference/dispatch/list-revision-history
/openapi.json get /dispatch/tasks/{task_id}/revisions
List all revisions (child tasks) for a given task.
# List your APIs
Source: https://docs.parse.bot/api-reference/dispatch/list-your-apis
/openapi.json get /dispatch/tasks
List all dispatch tasks for the current user, with optional status filter and pagination.
# Merge updates from canonical
Source: https://docs.parse.bot/api-reference/dispatch/merge-updates-from-canonical
/openapi.json post /dispatch/tasks/{task_id}/merge-updates
Apply selected endpoint updates from the canonical version into your API. New endpoints also merge code; spec-only updates preserve your code.
# Preview an endpoint update
Source: https://docs.parse.bot/api-reference/dispatch/preview-an-endpoint-update
/openapi.json post /dispatch/tasks/{task_id}/preview-update/{endpoint_name}
Test an endpoint against the canonical version to see the updated behavior before merging. Returns the execution result plus sample inputs.
# Respond to input prompts
Source: https://docs.parse.bot/api-reference/dispatch/respond-to-input-prompts
/openapi.json post /dispatch/{task_id}
Send a response when the agent requests user input during API creation. Task status must be 'needs_input'.
# Revise a completed API
Source: https://docs.parse.bot/api-reference/dispatch/revise-a-completed-api
/openapi.json post /dispatch/tasks/{task_id}/revise
Submit a revision request for a completed API. Describe what to change — the system will classify and queue the revision.
# Rollback last merge
Source: https://docs.parse.bot/api-reference/dispatch/rollback-last-merge
/openapi.json post /dispatch/tasks/{task_id}/rollback-merge
Restore your API to its pre-merge state. Only the most recent merge can be rolled back.
# Execute an API endpoint (GET)
Source: https://docs.parse.bot/api-reference/execute/execute-an-api-endpoint-get
/openapi.json get /scraper/{scraper_id}/{endpoint_name}
Call a generated API endpoint using query parameters. Use GET when the endpoint spec defines GET as the method. Parameters are passed as query strings and auto-coerced to the correct types.
# Execute an API endpoint (POST)
Source: https://docs.parse.bot/api-reference/execute/execute-an-api-endpoint-post
/openapi.json post /scraper/{scraper_id}/{endpoint_name}
Call a generated API endpoint by scraper ID and endpoint name. Pass endpoint parameters in the JSON body.
# Export as MCP tools
Source: https://docs.parse.bot/api-reference/export/export-as-mcp-tools
/openapi.json get /dispatch/tasks/{task_id}/export/mcp
Export a completed API as MCP (Model Context Protocol) tool definitions. Returns tool names, input schemas, and endpoint metadata.
# Get a callable API's OpenAPI document
Source: https://docs.parse.bot/api-reference/export/get-a-callable-apis-openapi-document
/openapi.json get /v1/apis/{scraper_id}/openapi.json
Return the complete OpenAPI 3.1 document for an API you own or an active marketplace canonical. Optionally pin it to an available version snapshot.
# Fork a marketplace API privately
Source: https://docs.parse.bot/api-reference/marketplace/fork-a-marketplace-api-privately
/openapi.json post /marketplace/apis/{id}/fork
Create a private, desynced copy of the canonical: no upstream merges, and your revisions don't propagate back. Returns a `scraper_id` you own and call at POST /scraper/{scraper_id}/{endpoint_name}.
# Get a marketplace API's detail
Source: https://docs.parse.bot/api-reference/marketplace/get-a-marketplace-apis-detail
/openapi.json get /marketplace/apis/{id}
Full detail for one marketplace listing, including its `canonical_scraper_id` and endpoint list. No authentication required. Use `canonical_scraper_id` to call the shared canonical directly: POST /scraper/{canonical_scraper_id}/{endpoint_name}.
# Search and browse the marketplace
Source: https://docs.parse.bot/api-reference/marketplace/search-and-browse-the-marketplace
/openapi.json get /marketplace/apis
Search the public marketplace of pre-built APIs. No authentication required. Returns hybrid-ranked results when `q` is provided, or a browseable list ordered by `sort` otherwise. Use this to find an existing API for a site before building your own with POST /dispatch.
# Search marketplace endpoints
Source: https://docs.parse.bot/api-reference/marketplace/search-marketplace-endpoints
/openapi.json get /marketplace/search-endpoints
Search across individual endpoints in the marketplace (not just APIs). No authentication required. Returns two groups: matching APIs and matching endpoints. Useful when you know the data you want but not which API provides it.
# Subscribe to a marketplace API
Source: https://docs.parse.bot/api-reference/marketplace/subscribe-to-a-marketplace-api
/openapi.json post /marketplace/apis/{id}/subscribe
Create your own copy of the canonical, pinned to its current release and added to your account (My APIs). Returns a `scraper_id` you call at POST /scraper/{scraper_id}/{endpoint_name}. Pinned, so upstream changes don't alter your contract until you merge updates. Stays sync-eligible.
# Authenticated APIs
Source: https://docs.parse.bot/authenticated-apis
Build and call APIs for sites that require login credentials
Some Parse APIs are built from sites that require login. These APIs have **login endpoints** that create sessions and **protected endpoints** that reuse the authenticated state. This page covers building one against a real account, then calling it.
Authenticated APIs are started from the [Parse dashboard](https://parse.bot) (check **"This site needs a login"** on the New API form) or programmatically over the API and MCP by setting `allow_auth: true` — after a one-time disclaimer acceptance in the dashboard. Free accounts cannot create authenticated APIs; paid plans include 5 or 10 logged-in accounts per site, and the limit can be [raised on request](#how-many-accounts-you-can-connect).
## Security overview
Parse signs in the same way you would, so the endpoints it builds see exactly the data and permissions of the account you connect.
We recommend using a test account for initial builds where the site offers one. Parse is designed to ask for your permission before taking destructive actions, but building is an agentic process — an agent operates a real browser session on your account while it constructs endpoints — so we recommend still exercising caution, especially with accounts that can move money, place orders, or message other people.
Authenticated builds sign in to the target site as you. We assume you have reviewed that site's terms of service and are authorized to access the account and its data this way. You are responsible for how your account is used; Parse accepts no liability for violations of a third-party site's terms.
Credentials you provide are treated as secrets end to end:
* **A fresh keypair per build.** Every build generates its own ephemeral RSA-4096 keypair. The public key is handed to your browser; the private key exists only for that build.
* **Encrypted before they leave your browser.** Credentials are encrypted client-side with that build's public key, so only that one build can read them.
* **Decrypted only in memory.** The worker running your build decrypts them in memory to drive the login. They are never written to disk and never written to logs, and the private key is wiped from memory when the build finishes.
* **Network logs suppressed.** Parse normally records the network requests behind a build for debugging. For authenticated builds that capture is turned off, so credentials in flight are never recorded.
* **Generated code is scanned before publish.** Every authenticated build's final code goes through an automated check — an LLM review plus pattern matching — that rejects code with hardcoded credentials. Your login is supplied at call time, not baked into the scraper.
* **Never shared publicly.** Authenticated APIs are never contributed to the public marketplace. They stay private to your account.
* **Login attempts are rate-limited.** Login endpoints are capped at 3 attempts per hour, counted per site per account.
Parse is currently undergoing a SOC 2 audit. The audit is in progress — we're happy to share status with customers who need it; reach out from the [dashboard](https://parse.bot).
Once the API is built, calls are protected separately: session state is encrypted with a key only you hold. See [Security](#security) below.
## How many accounts you can connect
A **session** is one account you're logged into for a given site. The cap is counted per site, so being logged into two accounts on the same site uses two of them, while one account each on two different sites uses one apiece.
| Your plan | Sessions per site |
| ------------------- | ------------------------------------------ |
| Free | None — authenticated APIs aren't available |
| Paid plans | 5 |
| Higher-volume plans | 10 |
Need more accounts on a site than your plan includes? Reach out to support with your use case, the website, and your expected needs — limits can be raised per account. We'll need proof that you are authorized to control the accounts you use with Parse.
## During the build: interactive prompts
Building against a login is a conversation. Rather than guessing, the agent pauses and asks you — in the dashboard, while the build is running:
| Prompt | What it asks for | Expires after |
| ----------------------- | --------------------------------------------------------------------------- | ------------- |
| **Credential request** | The username and password for the site | \~5 minutes |
| **2FA / one-time code** | A code from your authenticator app, SMS, or email | \~5 minutes |
| **Clarifying question** | A choice between options — e.g. "which employee should I use for the test?" | \~15 minutes |
| **Confirmation** | Your go-ahead before a test action that could change or delete data | \~15 minutes |
If a prompt goes unanswered past its window it expires and the build **parks** — it stops where it is instead of guessing. Everything verified up to that point is kept (see below).
Credential and 2FA prompts are short-lived by design — a one-time code is usually only valid for a minute or two anyway. Stay on the build page while an authenticated build is running so you can answer promptly.
The dashboard is the easiest place to answer build-time prompts. A build started over the API pauses at `status=needs_input`; the simplest path is to open the build page and answer there, but you can also [answer over the API](#answering-prompts-over-the-api). Never send build-time credentials through MCP chat — an MCP client hands you the build page URL instead. (Calling a finished API's login endpoint with credentials over HTTP is a separate thing, and is covered below.)
### Answering prompts over the API
While a build waits at `status=needs_input`, `GET /dispatch/tasks/{task_id}` returns the prompt on `user_input_prompt`: a `request_id`, the list of fields to fill, and a `public_key_pem` — a fresh RSA-4096 public key that exists only for this build. Responses are encrypted client-side against that key, exactly like the dashboard does:
1. Build the plaintext JSON object:
```json theme={null}
{
"request_id": "",
"outcome": "submitted",
"values": { "": "" },
"user_note": ""
}
```
2. Generate a random 256-bit AES key and 12-byte IV, and encrypt the plaintext with **AES-256-GCM** (no additional authenticated data).
3. Wrap the AES key with the prompt's public key using **RSA-OAEP** (SHA-256 hash, MGF1-SHA256, no label).
4. Submit the envelope — exactly these three keys, each base64-encoded:
```bash theme={null}
curl -X POST https://api.parse.bot/dispatch/{task_id} \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"request_id": "",
"user_response": {
"encrypted_key": "",
"iv": "",
"ciphertext": ""
}
}'
```
The private key lives only in the worker running your build and is wiped when the build ends, so an envelope can never be decrypted later or by anyone else.
Keep credentials out of model context. If an agent is driving your integration, fetch secrets from a secret store inside your own code and encrypt them there — never paste credentials into an agent conversation or send them through MCP.
## Endpoints are saved as they go
Endpoints are saved incrementally, as each one is verified — not in one batch at the end. Practically:
* A build that fails partway through **keeps the endpoints that already worked**. You don't lose the successful half of a build to a failure in the last step.
* The same applies to a parked build waiting on a prompt that expired.
* From there you continue or [revise](/dashboard#4-revise-in-plain-english) the API rather than starting over from scratch.
Incremental saving applies to new builds. Revisions to an existing API still apply their changes at completion.
## Endpoint session tags
Each endpoint in an authenticated API has a session tag:
| Tag | Meaning |
| ----------------------------- | ---------------------------------------------------------------------------------------------- |
| `initiates_session: true` | Login endpoint. Returns `session_id` + `encryption_key` in the response. |
| `depends_on_session: "login"` | Protected endpoint. Requires `session_id` + `encryption_key` from a prior login call. |
| *(neither)* | Public endpoint. Works without any session, even on APIs that have other auth-gated endpoints. |
## Step 1: Call the login endpoint
```bash theme={null}
curl -X POST https://api.parse.bot/scraper/{scraper_id}/login \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"username": "user@example.com", "password": "s3cret"}'
```
The response includes your data plus two session values:
```json theme={null}
{
"data": { "logged_in": true, "display_name": "Jane" },
"session_id": "550e8400-e29b-41d4-a716-446655440000",
"encryption_key": "gAAAAABk..."
}
```
Store both `session_id` and `encryption_key` — you need them for every protected endpoint call. Parse does not store the encryption key.
## Step 2: Call protected endpoints
Pass the session credentials via headers (preferred) or request body:
```bash theme={null}
# Option A: Headers (recommended)
curl -X POST https://api.parse.bot/scraper/{scraper_id}/get_inbox \
-H "X-API-Key: YOUR_API_KEY" \
-H "X-Session-ID: 550e8400-e29b-41d4-a716-446655440000" \
-H "X-Encryption-Key: gAAAAABk..." \
-H "Content-Type: application/json" \
-d '{"folder": "inbox", "limit": 10}'
# Option B: Body params
curl -X POST https://api.parse.bot/scraper/{scraper_id}/get_inbox \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"session_id": "550e8400-e29b-...",
"encryption_key": "gAAAAABk...",
"folder": "inbox",
"limit": 10
}'
```
The session state is automatically updated after each call — you don't need to track cookies or tokens yourself.
## Full example
```python theme={null}
import httpx
API_KEY = "your-api-key"
BASE = "https://api.parse.bot"
HEADERS = {"X-API-Key": API_KEY, "Content-Type": "application/json"}
scraper_id = "scraper-456"
with httpx.Client(headers=HEADERS, timeout=60) as client:
# 1. Login
resp = client.post(f"{BASE}/scraper/{scraper_id}/login", json={
"username": "user@example.com",
"password": "s3cret",
})
login = resp.json()
session_id = login["session_id"]
encryption_key = login["encryption_key"]
# 2. Call a protected endpoint
resp = client.post(
f"{BASE}/scraper/{scraper_id}/get_inbox",
headers={
**HEADERS,
"X-Session-ID": session_id,
"X-Encryption-Key": encryption_key,
},
json={"folder": "inbox", "limit": 10},
)
print(resp.json())
```
## Session details
* **One session per user per scraper**: Logging in again with the same user upserts the existing session.
* **Proxy binding**: The proxy IP used during login is reused for all protected endpoints to prevent session invalidation from IP changes.
* **Session updates are transparent**: After each protected call, the session state is re-encrypted and saved automatically.
* **Rate limiting**: Login endpoints enforce a rate limit of 3 login attempts per hour, counted per site per account — not per scraper. Several APIs built from the same site share one budget.
## Error responses
| Code | Detail | When |
| ---- | -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- |
| 400 | `This endpoint requires session_id and encryption_key. Call the login endpoint first.` | Protected endpoint called without session |
| 400 | `Invalid session or encryption key` | Wrong encryption key or corrupted session |
| 404 | `Session not found` | Invalid session\_id or session belongs to a different scraper |
| 403 | Auth session limit reached | You've hit your [per-site session cap](#how-many-accounts-you-can-connect) |
## Security
**At call time:**
* **Fernet session encryption**: Session state is encrypted with your `encryption_key` before storage. Parse stores only ciphertext — without your key, the data is unreadable.
* **HMAC replay protection**: Serialized session blobs are signed with a 30-minute TTL.
* **No plaintext credential storage**: Credentials provided during API creation are encrypted end-to-end and never stored in plaintext.
**At build time:** credentials are encrypted in your browser against a per-build RSA-4096 key, decrypted only in the worker's memory, kept out of disk and logs, and scanned out of the generated code before publish. Authenticated APIs are never published to the marketplace. See the [security overview](#security-overview) for the full picture.
# Authentication
Source: https://docs.parse.bot/authentication
Get an API key and authenticate your requests
Parse has two ways to authenticate, depending on how you're connecting:
* **API key** — for REST calls, scripts, and backends. (This page.)
* **OAuth** — for MCP clients like Claude Code and Cursor, which handle login for you. (See [MCP Server](/mcp).)
## Get an API key
API keys are created in the dashboard — you can't mint one over the API (creating a key requires a logged-in session, not another key).
1. Log in to [parse.bot](https://parse.bot)
2. Go to **Settings → API Keys**
3. Click **Create key**, give it a name (e.g. `production`, `laptop`, `ci`), and optionally set an expiry
4. **Copy the key immediately** — the full value is shown only once
You can create as many keys as you like — one per app or environment is good practice, so you can revoke a single key without disrupting the others. Every key starts with the prefix `pmx_`.
Treat keys like passwords. Store them in a secret manager or environment variable, never in source control. If a key leaks, revoke it in the dashboard and create a new one — revoking is instant.
## Use your key
Send the key in the `X-API-Key` header on every request:
```bash theme={null}
curl https://api.parse.bot/dispatch/tasks \
-H "X-API-Key: pmx_your_key_here"
```
As an alternative, you can pass it as a bearer token — useful with HTTP clients that have first-class `Authorization` support:
```bash theme={null}
curl https://api.parse.bot/dispatch/tasks \
-H "Authorization: Bearer pmx_your_key_here"
```
Both are equivalent. If you send both headers, `X-API-Key` wins.
### What needs a key
| Surface | Auth |
| ---------------------------------------------------------------- | ------------------------------- |
| `GET /marketplace/*` | **None** — public search/browse |
| `POST /dispatch`, `GET /dispatch/tasks`, revise, export, updates | API key |
| `POST /scraper/{id}/{endpoint}` (execution) | API key |
| `POST /mcp` | OAuth **or** API key |
## MCP clients
If your MCP client supports OAuth (Claude Code, Cursor, Claude Desktop), you don't need a key at all — just point it at the server and it opens a browser login:
```json theme={null}
{
"mcpServers": {
"parse": { "url": "https://api.parse.bot/mcp" }
}
}
```
For clients without OAuth, pass your key in headers instead:
```json theme={null}
{
"mcpServers": {
"parse": {
"url": "https://api.parse.bot/mcp",
"headers": { "X-API-Key": "pmx_your_key_here" }
}
}
}
```
Full setup and the OAuth flow details are in the [MCP Server](/mcp) guide.
## Troubleshooting auth
* **`401 Unauthorized`** — the key is missing, malformed, expired, or revoked. The header *name* is case-insensitive (`X-API-Key`, `x-api-key`, `X-Api-Key` all work), but the key *value* is case-sensitive and must start with `pmx_`. Watch for stray quotes or whitespace around the value.
* **`404 Not Found`** — make sure you're hitting `https://api.parse.bot` and the exact path (e.g. `/dispatch`, not `/api/dispatch`).
See [Errors & Troubleshooting](/errors) for the full reference.
# Dashboard guide
Source: https://docs.parse.bot/dashboard
Build, test, and revise APIs in the Parse dashboard — then call them from your code
The fastest way to build an API is in the [dashboard](https://parse.bot). You paste a URL, Parse builds the API, and you test and revise it visually before wiring it into your app. This guide walks the full UI flow.
## 1. Create a new API
Click **+ New** in the top-left of the nav. Paste the URL you want data from, describe what you need, and Parse either matches an existing API or builds a new one.
![New API submission modal]()
The **Contribute to marketplace** toggle controls whether your API stays linked to the shared canonical for its domain:
* **On** (default) — the build is **free**, and your API stays in sync with the canonical so you can [pull upstream improvements](/updates).
* **Off** — a fully private, desynced API. This build is charged, and the choice is permanent.
Builds typically take \~60 seconds. You'll watch progress live, and the API lands in **My APIs**. On a new build, endpoints are saved as they're verified, so a build that fails partway still keeps the endpoints that already worked. (Revisions to an existing API still apply their changes at completion — see [step 4](#4-revise-in-plain-english).)
If the site requires a login, the build is interactive: it will pause to ask for credentials, a 2FA code, or a clarifying question. Answer promptly — prompts expire (roughly 5 minutes for credentials and 2FA codes, 15 for questions) and an unanswered build parks. We recommend a test account for initial builds where the site offers one: see the [security overview](/authenticated-apis#security-overview) for how credentials are handled and [interactive prompts](/authenticated-apis#during-the-build-interactive-prompts) for the full list.
## 2. Or start from an existing one
Thousands of sites are already built. Open **Discover** to search the [marketplace](/marketplace) and fork or subscribe to an existing API instead of building from scratch.
![Marketplace discover page]()
## 3. Test endpoints in the browser
Open any API to see its endpoints. Pick one, fill in the parameters, and hit **Send request** to run it live and see the real response — no code required. This is the quickest way to confirm an endpoint returns what you expect.
![API detail with endpoint tester and revision bar]()
## 4. Revise in plain English
Not quite right? Use the **revision bar** at the bottom of any API page. Describe the change — "add a search-by-category endpoint", "include the publish date in get\_paper" — and Parse revises the API for you. Mention a specific endpoint with `@`. Revisions run the same way a build does; the `scraper_id` stays the same.
## 5. Grab a code snippet
Once it works, scroll to **Now plug it in**. Copy a ready-to-run snippet — cURL, Python, TypeScript, an MCP config, or the OpenAPI spec — with your API's real URL and IDs already filled in.
![Code snippets: cURL, Python, TypeScript, MCP, OpenAPI]()
The snippet calls the same execution endpoint documented throughout these docs:
```
POST https://api.parse.bot/scraper/{scraper_id}/{endpoint_name}
```
## Next steps
Call your endpoints from code, end to end
Full Python & JavaScript clients
Pull upstream improvements into your API
Connect an AI agent to all your APIs
# Errors & Troubleshooting
Source: https://docs.parse.bot/errors
Decode every status code the execution API can return and act on it correctly
Every request goes to `https://api.parse.bot`. This page is the definitive reference for what each status code means — especially for **executing endpoints** (`POST /scraper/{scraper_id}/{endpoint_name}`), where Parse wraps the outcome in a structured envelope so you can tell *whose* fault a failure is: yours, the target site's, or the scraper's.
## The execution envelope
A successful call returns the endpoint's own JSON (the shape described by its `return_schema`) with HTTP `200`:
```json theme={null}
{ "title": "A Light in the Attic", "price": 51.77, "rating": 3 }
```
A failed call returns a non-2xx status with an `error` envelope. The HTTP code tells you the **category** of failure, and the body carries a machine-readable `status` plus detail:
```json theme={null}
{
"error": {
"status": "upstream_error",
"upstream_status_code": 404,
"message": "The target site returned HTTP 404. See `snippet` for the upstream response body.",
"snippet": "Product not found",
"url": "https://example.com/products/invalid-sku"
},
"status_code": 502
}
```
The `status` field is the most useful signal — it maps 1:1 to the HTTP code:
| HTTP | `status` | What it means | What to do |
| ----- | ---------------- | -------------------------------------------------------- | -------------------------------------------------------------------- |
| `200` | `success` | The endpoint ran and returned data | Use the body |
| `422` | `stale_input` | Your input was invalid, or the resource is gone upstream | Fix the input and retry |
| `429` | — | Rate limit hit (burst or daily) | Honor `Retry-After` and back off |
| `502` | `upstream_error` | The target site returned a non-2xx response | Inspect `upstream_status_code` + `snippet`; **do not blindly retry** |
| `503` | `blocked` | Anti-bot blocked every proxy we tried | Retry later **only if** `retry_after` is present |
| `500` | `error` | The scraper crashed or hit a bug | Request a revision of the API |
| `400` | — | Malformed request (bad header, missing session creds) | Fix the request |
| `401` | — | Missing/invalid/expired API key | Check your key |
| `404` | — | Scraper (or pinned version) doesn't exist or isn't yours | Check the `scraper_id` |
**`502` is not a Parse outage and should not be auto-retried.** It means *the target site* answered the scraper with a non-2xx (e.g. a 404 for an ID that doesn't exist, a 401 for bad upstream credentials, a transient site 500). Read `upstream_status_code` and `snippet` to see what the site actually said before deciding whether to retry.
## 422 — `stale_input` (your input is the problem)
The scraper rejected your input before or during the upstream call — wrong type, missing required field, or a resource the site says is gone.
```json theme={null}
{
"error": {
"status": "stale_input",
"kind": "input_format_invalid",
"message": "trip_type must be 'one_way' or 'round_trip'",
"upstream_status_code": 400,
"url": "https://example.com/api/search"
},
"status_code": 422
}
```
* `kind: "input_format_invalid"` — a parameter is malformed. Fix it against the endpoint's `input_params`.
* `kind: "input_not_found"` — the specific thing you named (a job ID, a product SKU) was confirmed gone by the site (often a 410). Not retryable with the same input.
## 502 — `upstream_error` (the target site failed)
The scraper reached the site, but the site returned a non-2xx. The body tells you exactly what came back:
```json theme={null}
{
"error": {
"status": "upstream_error",
"upstream_status_code": 503,
"message": "The target site returned HTTP 503. See `snippet` for the upstream response body.",
"snippet": "Service Unavailable"
},
"status_code": 502
}
```
Decide based on `upstream_status_code`: a `404`/`410` usually means your input pointed at nothing (don't retry); a `5xx` from the site may be transient (retry with backoff); a `401`/`403` may mean the API's stored credentials need attention (revise the API).
## 503 — `blocked` (anti-bot stopped us)
Parse rotated through proxies and every attempt hit the site's bot protection.
```json theme={null}
{
"error": {
"status": "blocked",
"block_type": "datadome",
"vendor": "datadome",
"kind": "antibot_solvable",
"attempts": 3,
"retry_after": 60,
"message": "Service temporarily unavailable - site protection blocking all proxies"
},
"status_code": 503
}
```
* `retry_after` present (and a `Retry-After` header) → transient; retry after that many seconds.
* `retry_after: null` (and **no** `Retry-After` header) → `kind` is `antibot_unsolvable` or `ip_burned`; retrying soon won't help. If it persists, the API may need a revision.
## 500 — `error` (the scraper bugged out)
The scraper crashed or detected an internal fault. This is the API's problem, not yours.
```json theme={null}
{
"error": {
"status": "error",
"kind": "scraper_bug",
"message": "The scraper crashed during execution. This could be due to bad input, broken scraper code, or a website update."
},
"status_code": 500
}
```
**Fix:** revise the API — `POST /dispatch/tasks/{task_id}/revise` with a description (e.g. "the get\_paper endpoint 500s on valid IDs"). If it's a marketplace/canonical API, the revision improves it for everyone.
## 429 — rate limited
```json theme={null}
{
"error": {
"error": "Rate limit exceeded",
"message": "Too many requests in a short burst. Retry in 5s.",
"limit_type": "burst",
"retry_after": 5
},
"status_code": 429
}
```
* `limit_type: "burst"` — you exceeded the short-term token bucket. `retry_after` is seconds until the next token.
* `limit_type: "daily"` — you hit your daily cap. `retry_after` is seconds until UTC midnight.
Every execution response carries rate-limit headers so you can pace yourself *before* getting a 429:
| Header | Meaning |
| ------------------------------------------- | ------------------------------------------------------- |
| `X-RateLimit-Remaining` | Tokens left in the burst bucket |
| `X-RateLimit-Reset` | Unix time when the next token is available |
| `X-RateLimit-Daily-Remaining` | Requests left today (tiers with a daily cap) |
| `Retry-After` | Seconds to wait (sent on `429`, and on retryable `503`) |
| `X-Credits-Charged` / `X-Credits-Remaining` | Credits this call cost / your balance |
Authenticated login endpoints have a stricter cap (3 attempts/hour per API) — see [Authenticated APIs](/authenticated-apis).
## 400 / 401 / 404 — request problems
* **`400`** — a malformed request: an invalid `API-Snapshot-Version` header, or `session_id`/`encryption_key` sent only one of the pair. Body: `{"error": "", "status_code": 400}`.
* **`401`** — `{"error": "Missing X-API-Key header" | "Invalid API key" | "API key expired", "status_code": 401}`. Send a valid `pmx_…` key in `X-API-Key` (header name is case-insensitive; the value is not). See [Authentication](/authentication).
* **`404`** — `{"error": "Scraper with ID '…' not found", "status_code": 404}`. The scraper or pinned version doesn't exist, or isn't yours — Parse returns `404` rather than reveal another account's resource. Use the `scraper_id` from *your* completed task's `generated_api`.
A bare JSON `{"detail":"Not Found"}` (note: `detail`, not `error`) means the **path** doesn't exist, not the scraper. Check you're calling `POST /scraper/{scraper_id}/{endpoint_name}` exactly, on `https://api.parse.bot`.
## Build failures (task `status: "failed"`)
A dispatch/revision task can finish `failed` rather than `completed`; the task's `error` field says why. Retry with a clearer `task` description, `revise` it, or — if `may_require_auth` was `true` — build it as an [Authenticated API](/authenticated-apis) from the dashboard.
## Still stuck?
* Confirm the base URL: **`https://api.parse.bot`**.
* Verify your key with a cheap call: `GET /dispatch/tasks`.
* **Reach out to support** from the [dashboard](https://parse.bot) — click the chat icon in the **bottom-right corner** to start a conversation with us. Grab the `X-Railway-Request-Id` response header from the failing call and include it; it lets us trace your exact request.
# Code Examples
Source: https://docs.parse.bot/examples
Call an API from your code — plus a separate example for building one programmatically
Most of the time you already have an API — you built or subscribed to it in the [dashboard](/dashboard) — and you just want to call it. That's the first example. The second shows how to *create* an API from code, for when you want the whole flow automated.
Set your key first:
```bash theme={null}
export PARSE_API_KEY="pmx_your_key_here"
```
## Using an API
Say you have an **arxiv.org** API with three endpoints:
| Method | Endpoint | Parameters |
| ------ | ----------------------- | ---------------------------------------------------------- |
| `GET` | `search_papers` | `query`, `author`, `title`, `category`, `sort_by`, `start` |
| `GET` | `get_paper` | `arxiv_id` |
| `GET` | `get_category_taxonomy` | — |
You call each one at `https://api.parse.bot/scraper/{scraper_id}/{endpoint_name}`. `GET` endpoints take query-string params; the response is the endpoint's own JSON. Grab your `scraper_id` from the API's page in the dashboard (or the "Now plug it in" snippet).
```python Python theme={null}
import os
import httpx
SCRAPER_ID = "9380e1b0-fae2-4340-9056-3d416f86c775"
client = httpx.Client(
base_url=f"https://api.parse.bot/scraper/{SCRAPER_ID}",
headers={"X-API-Key": os.environ["PARSE_API_KEY"]},
timeout=60,
)
def call(endpoint: str, **params):
r = client.get(f"/{endpoint}", params=params)
r.raise_for_status() # raises on 4xx/5xx — see the Errors guide
return r.json()
# Search for papers
results = call("search_papers", query="diffusion models", sort_by="relevance")
print(results)
# Fetch one paper's metadata
paper = call("get_paper", arxiv_id="2301.00001")
print(paper)
# Browse the category taxonomy (no params)
taxonomy = call("get_category_taxonomy")
print(taxonomy)
```
```javascript JavaScript theme={null}
const SCRAPER_ID = "9380e1b0-fae2-4340-9056-3d416f86c775";
const BASE = `https://api.parse.bot/scraper/${SCRAPER_ID}`;
const headers = { "X-API-Key": process.env.PARSE_API_KEY };
async function call(endpoint, params = {}) {
const qs = new URLSearchParams(params).toString();
const res = await fetch(`${BASE}/${endpoint}?${qs}`, { headers });
if (!res.ok) throw new Error(`${endpoint} ${res.status}: ${await res.text()}`);
return res.json();
}
const results = await call("search_papers", { query: "diffusion models", sort_by: "relevance" });
console.log(results);
const paper = await call("get_paper", { arxiv_id: "2301.00001" });
console.log(paper);
const taxonomy = await call("get_category_taxonomy");
console.log(taxonomy);
```
```bash curl theme={null}
# GET endpoint with query params
curl "https://api.parse.bot/scraper/9380e1b0-fae2-4340-9056-3d416f86c775/get_paper?arxiv_id=2301.00001" \
-H "X-API-Key: $PARSE_API_KEY"
```
Some endpoints are `POST` instead of `GET` — send those params in a JSON body (`-d '{"page": 1}'` / `json={...}` / `body: JSON.stringify(...)`) instead of the query string. The endpoint's page in the dashboard tells you which method it uses.
## Creating an API from code
This is the automated build flow: submit a URL, poll until it's ready, then call it. Use it when you want to spin up APIs programmatically rather than in the dashboard.
```python theme={null}
import os
import time
import httpx
BASE = "https://api.parse.bot"
client = httpx.Client(
base_url=BASE,
headers={"X-API-Key": os.environ["PARSE_API_KEY"]},
timeout=60,
)
def create_api(url: str, task: str | None = None) -> str:
"""Kick off a build and return the task_id."""
r = client.post("/dispatch", json={"url": url, "task": task})
r.raise_for_status()
body = r.json()
print(f"task_id={body['task_id']} matched={body['matched']}")
return body["task_id"]
def wait_for_completion(task_id: str, interval: float = 4.0, timeout: float = 300):
"""Poll until the task reaches a terminal state. Returns the generated_api."""
deadline = time.time() + timeout
while time.time() < deadline:
task = client.get(f"/dispatch/tasks/{task_id}").json()
status = task["status"]
print(f"status={status}")
if status == "completed":
return task["generated_api"]
if status == "failed":
raise RuntimeError(f"Build failed: {task.get('error')}")
if status == "cancelled":
raise RuntimeError("Build was cancelled")
if status == "needs_input":
# Answer the agent's question. Inspect task["user_input_prompt"]
# to see what it's asking; here we send a generic example.
print(f"agent needs input: {task.get('user_input_prompt')}")
client.post(f"/dispatch/{task_id}", json={
"user_response": {"search_term": "example"},
})
time.sleep(interval)
raise TimeoutError(f"Task {task_id} did not finish in {timeout}s")
if __name__ == "__main__":
task_id = create_api("https://books.toscrape.com", "get book titles, prices, and ratings")
api = wait_for_completion(task_id)
print(f"\nBuilt '{api['name']}' — {len(api['endpoints'])} endpoint(s):")
for ep in api["endpoints"]:
print(f" {ep['method']} {ep['endpoint_name']} — {ep['description']}")
# Now call it — same pattern as "Using an API" above
scraper_id = api["scraper_id"]
r = client.post(f"/scraper/{scraper_id}/get_books", json={"page": 1})
print("\nResult:", r.json())
```
The JavaScript equivalent follows the same shape — `POST /dispatch`, poll `GET /dispatch/tasks/{id}`, then call `/scraper/{id}/{endpoint}`.
## Notes
* **Reuse one HTTP client** so connections are pooled across calls.
* **Back off on `429`.** Honor `Retry-After` and the `X-RateLimit-*` headers.
* **Check the HTTP status, not just the body.** A `502` means the *target site* failed, a `500` means the scraper bugged out — they call for different handling. See [Errors](/errors).
* The standard library's `urllib` and `requests` work just as well as `httpx`.
# Introduction
Source: https://docs.parse.bot/introduction
Call any website as a structured API
## What is Parse?
Parse turns any website into a structured API you can call over plain HTTP. You build an API once — in the [dashboard](https://parse.bot) or via the API — and then call its typed endpoints from your code, getting clean JSON back. Parse hosts and runs it for you, behind rotating proxies and anti-bot handling, and keeps it working as the site changes.
Endpoints can do more than read. They can **scrape** — search, list, and fetch structured data — and they can **act**: log in, submit forms, and trigger multi-step flows. Anything you could do in a browser can become a typed endpoint.
Most people land here to **call the APIs they've already built**. So that's where we start.
## Calling an API
Every API you build gets a `scraper_id` and one or more named endpoints. Call an endpoint by POSTing to it with your [API key](/authentication):
```bash theme={null}
curl -X POST https://api.parse.bot/scraper/{scraper_id}/{endpoint_name} \
-H "X-API-Key: pmx_your_key_here" \
-H "Content-Type: application/json" \
-d '{"page": 1}'
```
```json theme={null}
{ "title": "A Light in the Attic", "price": 51.77, "rating": 3 }
```
A successful call returns the endpoint's own JSON. GET endpoints take query-string params instead of a body. You can find an endpoint's exact URL, parameters, and a ready-made code snippet on its page in the dashboard, or by fetching the API's [spec](/quickstart#step-2-poll-for-completion). When a call fails, the HTTP status tells you whose problem it is — see [Errors](/errors).
Call your first endpoint, end to end
Full Python & JavaScript clients
Typed Python clients via the parse CLI
## Building an API
You build APIs two ways:
* **In the dashboard** (most common) — paste a URL, describe what you want, and Parse matches an existing API or builds a new one. You can test endpoints, revise them in plain English, and grab a code snippet, all in the UI. See the [Dashboard guide](/dashboard).
* **Via the API** — `POST /dispatch` with a URL, poll until it's built, then call it. See the [Quickstart](/quickstart).
Either way, the result is the same callable API. You can also start from the **[marketplace](/marketplace)** — thousands of popular sites are already built and free to use.
## What else you can do
* **Manage your APIs** — list (`GET /dispatch/tasks`), inspect (`GET /dispatch/tasks/{id}`), and revise or extend them (`POST /dispatch/tasks/{id}/revise`).
* **Stay current** — pull upstream improvements into your API with the [updates](/updates) flow.
* **Authenticated sites** — build APIs for sites that require login, using a real account. Credentials are encrypted in your browser and never stored in plaintext. See [Authenticated APIs](/authenticated-apis).
* **Typed Python clients** — generate fully typed clients for your APIs with the `parse` CLI. See the [Python SDK](/sdk).
* **Export & connect agents** — export any API as OpenAPI 3.1 or MCP tools, or point an AI agent at the hosted [MCP server](/mcp) to discover and call your APIs.
## Base URL & auth
Everything is served from a single host:
```text theme={null}
https://api.parse.bot
```
All endpoints except the public marketplace require your API key in the `X-API-Key` header (keys start with `pmx_`). Create one in the dashboard — see [Authentication](/authentication).
## For AI agents
Parse runs a hosted MCP server at `POST /mcp`. Any MCP-compatible client — Claude Code, Cursor, Claude Desktop — connects with OAuth or an API key and immediately gets tools to search, build, inspect, call, revise, and manage APIs. You can also point an agent at `https://parse.bot/llms.txt` for a machine-readable overview.
## Need help?
Reach out any time from the [dashboard](https://parse.bot) — click the chat icon in the bottom-right corner to start a conversation with the team.
# Marketplace
Source: https://docs.parse.bot/marketplace
Search thousands of pre-built APIs before you build your own
The Parse marketplace is a public catalog of pre-built APIs. Before you build an API from scratch, search the marketplace — if a site is already covered, you can use it instantly and for free.
Browse it two ways:
* **In the browser** — [parse.bot/marketplace](https://parse.bot/marketplace) (the "Discover" tab in the dashboard). Search, filter by category, and preview every endpoint.
* **Over the API** — the public REST endpoints below. No API key required.
## Why it exists: the canonical model
Every site in Parse has one shared, community-maintained API called the **canonical**. When you create an API for a site that already has a canonical, you get your own **copy** (a clone) linked back to it:
* The build is **instant and free** — the work is already done.
* You can **pull upstream improvements** as the canonical gets better (new endpoints, fixes). See [API Updates](/updates).
* Your revisions can flow **back** to the canonical, improving it for everyone.
This is controlled by the `contributes_to_marketplace` flag on `POST /dispatch` (default `true`). Set it to `false` only if you want a fully private, desynced API — that build is charged and the choice is permanent.
## Search APIs
```bash theme={null}
curl "https://api.parse.bot/marketplace/apis?q=amazon&limit=10"
```
Query parameters:
| Param | Default | Description |
| ---------- | ------- | ------------------------------------------------------------------------------- |
| `q` | — | Free-text search across name, description, and endpoints. Omit to browse. |
| `category` | — | Filter by category slug. |
| `sort` | `top` | Ordering when browsing: `top` (highest rated), `recent` (newest), `name` (A→Z). |
| `country` | — | ISO-2 code (e.g. `US`) for region-aware ranking. |
| `semantic` | `true` | Include semantic (embedding) ranking when searching. |
| `limit` | `50` | 1–200. |
| `offset` | `0` | For pagination. |
Response:
```json theme={null}
{
"items": [
{
"id": "mp-789",
"slug": "amazon-product-search",
"name": "Amazon Product Search",
"description": "Search products and fetch details, prices, and reviews",
"source_url": "https://www.amazon.com",
"is_authenticated": false,
"endpoint_count": 4,
"primary_category": "ecommerce",
"secondary_categories": ["retail"],
"endpoint_preview": [
{ "endpoint_name": "search_products", "method": "POST", "summary": "Search by keyword" }
]
}
],
"total": 1,
"limit": 10,
"offset": 0
}
```
## Search individual endpoints
When you know the *data* you want but not which API has it, search across endpoints directly:
```bash theme={null}
curl "https://api.parse.bot/marketplace/search-endpoints?q=flight%20prices"
```
This returns two groups — matching APIs and matching endpoints — so you can jump straight to the right endpoint. Params: `q` (required, 1–120 chars), `api_limit` (default 6), `endpoint_limit` (default 12), `country`.
## Using a marketplace API
There are four ways to consume a marketplace API, depending on whether you want your own copy and whether you want to stay in sync with the canonical.
| Path | Endpoint | Your own copy? | Version | Shows in My APIs |
| ------------------------------- | ------------------------------------------------- | -------------------- | ------------------------ | ---------------- |
| **Call the canonical directly** | `POST /scraper/{canonical_scraper_id}/{endpoint}` | No | Always latest | No |
| **Subscribe** | `POST /marketplace/apis/{id}/subscribe` | Yes (pinned clone) | Pinned at subscribe time | Yes |
| **Fork privately** | `POST /marketplace/apis/{id}/fork` | Yes (desynced clone) | Independent | Yes |
| **Dispatch by URL** | `POST /dispatch` with the `source_url` | Yes (clone) | Pinned at build time | Yes |
### 1. Call the canonical directly (simplest)
Every marketplace API has a shared **canonical** scraper that *any* user can call with just their API key — no subscribe or fork needed. Get its `canonical_scraper_id` (and endpoint list) from the detail endpoint, then call it:
```bash theme={null}
# Look up the canonical id + endpoints for a listing
curl "https://api.parse.bot/marketplace/apis/{id}"
# Call it directly — always runs the latest canonical
curl -X POST https://api.parse.bot/scraper/{canonical_scraper_id}/{endpoint_name} \
-H "X-API-Key: $PARSE_API_KEY" \
-H "Content-Type: application/json" \
-d '{ ... }'
```
You always run the latest canonical, it doesn't appear in My APIs, and you can't pin a version. Great for quick, stateless use.
### 2. Subscribe — get your own pinned copy
`POST /marketplace/apis/{id}/subscribe` creates your own copy of the canonical, **pinned to its current release**, and adds it to My APIs. The response gives you a `scraper_id` to call:
```bash theme={null}
curl -X POST https://api.parse.bot/marketplace/apis/{id}/subscribe \
-H "X-API-Key: $PARSE_API_KEY"
# → { "scraper_id": "...", "canonical_scraper_id": "..." }
```
Because it's pinned, upstream changes won't alter your contract until you choose to [merge updates](/updates). Subscriptions stay sync-eligible, so improvements can flow both ways.
### 3. Fork privately — an isolated copy
`POST /marketplace/apis/{id}/fork` creates a **private, desynced** copy: no upstream merges, and your revisions don't propagate back to the canonical. Use this when you want to customize freely without being tied to the shared version:
```bash theme={null}
curl -X POST https://api.parse.bot/marketplace/apis/{id}/fork \
-H "X-API-Key: $PARSE_API_KEY"
# → { "scraper_id": "...", "canonical_scraper_id": "..." }
```
### 4. Dispatch by URL
If you have a URL and don't know whether it's already in the marketplace, just `POST /dispatch` with it. If a canonical exists you get your own clone back instantly (`matched: true`); otherwise Parse builds one. See the [Quickstart](/quickstart).
AI agents can do all of this through the [MCP server](/mcp) with the `marketplace_search`, `marketplace_subscribe`, and `call_endpoint` tools — no manual polling.
# MCP Server
Source: https://docs.parse.bot/mcp
Connect AI agents to Parse via the Model Context Protocol
Parse exposes a hosted MCP server that lets compatible AI agents find, build, manage, and call APIs.
## Connect your agent
### OAuth (recommended)
Add this to your MCP client config (`claude_desktop_config.json`, `.cursor/mcp.json`, `.claude/settings.json`, etc.):
```json theme={null}
{
"mcpServers": {
"parse": {
"url": "https://api.parse.bot/mcp"
}
}
}
```
The client handles authentication automatically — it opens a browser window for you to sign in, then manages tokens in the background. No API key needed.
### API key
For clients that don't support OAuth, pass your key in headers:
```json theme={null}
{
"mcpServers": {
"parse": {
"url": "https://api.parse.bot/mcp",
"headers": {
"X-API-Key": "YOUR_API_KEY"
}
}
}
}
```
## How it works
The MCP server speaks JSON-RPC 2.0 over HTTP. When your agent connects, it gets a compact set of platform tools for finding, building, calling, and managing APIs:
| Tool | Description |
| ------------------------- | ------------------------------------------------------------------ |
| `marketplace_search` | Search the public marketplace for an existing API |
| `marketplace_categories` | List marketplace categories |
| `marketplace_subscribe` | Add a marketplace API to your account (returns a callable scraper) |
| `marketplace_pin_version` | Pin a subscribed API to a specific version |
| `create_api` | Create a new API from a website URL |
| `list_apis` | List all your APIs (with optional status filter) |
| `get_api` | Get full details for an API by task ID |
| `revise_api` | Modify a completed API with natural language |
| `cancel_task` | Cancel an in-flight build or revision |
| `call_endpoint` | Call any endpoint by scraper ID + endpoint name |
| `fork_updates` | Check, preview, merge, or roll back updates for a forked API |
| `list_versions` | List an API's version history |
| `list_revisions` | List the revision history for a task |
APIs for sites that require a login can be created over MCP by passing `allow_auth: true` to `create_api` — after a one-time disclaimer acceptance in the [dashboard](https://parse.bot). The build pauses at `needs_input` for a person to sign in and answer its prompts in the web app; credentials are never supplied to a build through MCP chat. Calling a finished authenticated API with `call_endpoint` also works — see [Authenticated APIs](/authenticated-apis).
Inspect an account API with `get_api`, or a marketplace API with `marketplace_search` detail mode, then use the returned endpoint details with `call_endpoint`. Match `params` to the endpoint's advertised parameter schema.
Completed describe results also include a top-level `spec_url` for the complete OpenAPI 3.1 document. Fetch that URL with your `X-API-Key`; the document is not inlined into the MCP response. Account APIs point to the contract that produced their endpoints, while marketplace detail pins the canonical API's latest release. Add `?version=N` to an unpinned URL when you need a specific available version.
```bash theme={null}
curl -fsS "$SPEC_URL" \
-H "X-API-Key: $PARSE_API_KEY"
```
## OAuth flow details
Parse implements OAuth 2.1 with PKCE. MCP clients handle this automatically, but if you're building a custom integration:
1. **Discovery**: `GET /.well-known/oauth-authorization-server` returns all endpoints
2. **Registration**: `POST /oauth/register` to get a `client_id`
3. **Authorization**: `GET /oauth/authorize` with PKCE challenge — redirects to consent page
4. **Token exchange**: `POST /oauth/token` with the authorization code + PKCE verifier
5. **Refresh**: `POST /oauth/token` with `grant_type=refresh_token` — tokens rotate automatically
Access tokens are JWTs that expire after 1 hour; the client uses the refresh token to get new ones. Refresh tokens are single-use (valid 7 days) and a fresh one is issued on every exchange.
## Protocol details
* **Endpoint**: `POST https://api.parse.bot/mcp`
* **Transport**: Streamable HTTP
* **Supported protocol versions**: `2025-03-26`, `2025-06-18`
* **Supported methods**: `initialize`, `notifications/initialized`, `tools/list`, `tools/call`, `ping`
Prefer code over an agent runtime? The same APIs are available as a typed [Python SDK](/sdk) and as plain [REST endpoints](/quickstart) — the MCP server is one of three surfaces over the same contract.
# Quickstart
Source: https://docs.parse.bot/quickstart
Create and call your first API in minutes
This guide walks the full flow end to end **over the API**: find or create an API from a URL, wait for it to build, then call it. Every request goes to `https://api.parse.bot` with your API key in the `X-API-Key` header.
Prefer to do this in the browser? The [Dashboard guide](/dashboard) walks the same flow through the UI — paste a URL, test endpoints, revise, and grab a snippet. Use this page if you want to drive it programmatically.
## Prerequisites
* A Parse account at [parse.bot](https://parse.bot)
* An API key from the dashboard (**Settings → API Keys**). See [Authentication](/authentication).
Set your key as an environment variable so you can copy-paste the examples:
```bash theme={null}
export PARSE_API_KEY="pmx_your_key_here"
```
## Step 0 (optional): Check the marketplace first
Thousands of sites already have a pre-built API. Searching first is free, instant, and needs no auth:
```bash theme={null}
curl "https://api.parse.bot/marketplace/apis?q=books.toscrape.com"
```
If you find a match, you can use it right away — there are four ways, depending on whether you want your own copy:
| | Route |
| -------------------------------------------- | ------------------------------------------------------ |
| Call the canonical directly (any API key) | `POST /scraper/{canonical_scraper_id}/{endpoint_name}` |
| Subscribe — your own pinned copy, in My APIs | `POST /marketplace/apis/{id}/subscribe` |
| Fork privately — isolated, desynced copy | `POST /marketplace/apis/{id}/fork` |
| Dispatch by URL — clone it instantly | `POST /dispatch` (Step 1 below) |
Get the `canonical_scraper_id` and endpoint list from `GET /marketplace/apis/{id}`. See the [Marketplace guide](/marketplace) for the details and trade-offs. If there's no match, dispatch your URL (below) and Parse builds it from scratch.
## Step 1: Create an API
Submit a URL and, optionally, describe the data you want:
```bash theme={null}
curl -X POST https://api.parse.bot/dispatch \
-H "X-API-Key: $PARSE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://books.toscrape.com",
"task": "get book titles, prices, and ratings"
}'
```
Response:
```json theme={null}
{
"task_id": "abc123-...",
"matched": false,
"may_require_auth": false
}
```
* `matched: true` — an existing API already covered this site; it's ready to call now.
* `matched: false` — a build job was queued. Poll `task_id` until it completes (Step 2).
* `may_require_auth: true` — the site likely needs a login. Start an authenticated build by resubmitting with `allow_auth: true` (paid plans, after a one-time disclaimer acceptance in the [dashboard](https://parse.bot)) — see [Authenticated APIs](/authenticated-apis).
**`contributes_to_marketplace`** (optional, default `true`). When `true`, your API stays linked to the shared canonical for its domain: you can [pull upstream improvements](/updates), and the build is free. Set it to `false` to build a fully private, desynced API (this is a charged build and the choice is permanent). Most users should leave it `true`.
## Step 2: Poll for completion
```bash theme={null}
curl https://api.parse.bot/dispatch/tasks/abc123-... \
-H "X-API-Key: $PARSE_API_KEY"
```
Poll every 3–5 seconds until `status` is `completed`. A typical build takes anywhere from a few seconds (marketplace match) to a couple of minutes (a fresh build on a complex site).
The `status` field moves through this lifecycle:
| Status | Meaning |
| ------------- | ------------------------------------------------------------------------------------------------------------------- |
| `queued` | Accepted, waiting for a worker. |
| `running` | The agent is actively building. `progress` shows live detail. |
| `needs_input` | The agent needs something from you (e.g. a sample search term). Respond via `POST /dispatch/{task_id}` — see below. |
| `completed` | ✅ Done. `generated_api` is populated and the endpoints are callable. |
| `failed` | ❌ Build failed. `error` explains why. |
| `cancelled` | You cancelled it via `POST /dispatch/tasks/{task_id}/cancel`. |
A completed task includes a `generated_api` object with everything needed to call it:
```json theme={null}
{
"id": "abc123-...",
"url": "https://books.toscrape.com",
"status": "completed",
"generated_api": {
"marketplace_id": "mp-789",
"scraper_id": "scraper-456",
"name": "Books to Scrape",
"source_url": "https://books.toscrape.com",
"execution_base_url": "https://api.parse.bot/scraper/scraper-456",
"endpoints": [
{
"method": "POST",
"endpoint_name": "get_books",
"description": "Get book listings with titles, prices, and ratings",
"input_params": {
"page": { "type": "integer", "description": "Page number" }
},
"return_schema": { "...": "..." }
}
]
}
}
```
The agent occasionally needs a hint — most often a realistic input to test an endpoint (e.g. a search term). The `user_input_prompt` field describes what it's asking for. Answer it and the build resumes:
```bash theme={null}
curl -X POST https://api.parse.bot/dispatch/abc123-... \
-H "X-API-Key: $PARSE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"user_response": {"search_term": "python"}}'
```
## Step 3: Call an endpoint
Use the `scraper_id` and an `endpoint_name` from the spec. Send parameters in the JSON body for `POST` endpoints, or as query-string params for `GET` endpoints:
```bash theme={null}
curl -X POST https://api.parse.bot/scraper/scraper-456/get_books \
-H "X-API-Key: $PARSE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"page": 1}'
```
A successful call returns the endpoint's own JSON — the shape defined by its `return_schema`, with no wrapper:
```json theme={null}
[
{ "title": "A Light in the Attic", "price": 51.77, "rating": 3 },
{ "title": "Tipping the Velvet", "price": 53.74, "rating": 1 }
]
```
Each execution also returns rate-limit and credit headers (`X-RateLimit-Remaining`, `X-RateLimit-Reset`, `X-Credits-Remaining`, …) so you can pace your calls. When a call fails, the HTTP status tells you whose fault it is — a `502` means the *target site* failed, a `500` means the scraper bugged out. See [Errors & Troubleshooting](/errors).
## Step 4: Revise your API
Need a change? Describe it in plain English:
```bash theme={null}
curl -X POST https://api.parse.bot/dispatch/tasks/abc123-.../revise \
-H "X-API-Key: $PARSE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"revision": "add an endpoint to search books by genre"}'
```
```json theme={null}
{ "task_id": "revision-task-id", "revision_type": "revision" }
```
Poll the new `task_id` exactly like Step 2. Your `scraper_id` stays the same — new and updated endpoints become callable when the revision completes.
## Step 5: Export or connect an agent
```bash theme={null}
# OpenAPI 3.1
curl https://api.parse.bot/v1/apis/your-scraper-id/openapi.json \
-H "X-API-Key: $PARSE_API_KEY"
# MCP tool definitions
curl https://api.parse.bot/dispatch/tasks/abc123-.../export/mcp \
-H "X-API-Key: $PARSE_API_KEY"
```
Or skip exporting entirely and connect an AI agent straight to the hosted MCP server — it can discover this API (and all your others) and call their endpoints. See [MCP Server](/mcp). To call your APIs from Python with fully typed clients, use the [Python SDK](/sdk).
## What's next
Typed Python clients generated by the parse CLI
Full Python & JavaScript clients with a polling helper
Search pre-built APIs before you build
Pull upstream improvements into your APIs
Read the execution envelope — whose fault a 422 / 502 / 503 is
# Python SDK
Source: https://docs.parse.bot/sdk
Typed Python clients for your Parse APIs, generated by the parse CLI
`parse-sdk` is the runtime and the `parse` CLI. It generates a project-local, fully typed `parse_apis` package — the client for every API your key can call, with autocompleting resources, methods, enums, and typed responses. Clients are generated from the same spec that powers the REST and MCP surfaces.
## Install
The SDK is managed with [uv](https://docs.astral.sh/uv/). Inside a uv project (run `uv init` first if you don't have a `pyproject.toml`):
```bash theme={null}
uv add parse-sdk # runtime + `parse` CLI, into your project's .venv
uv run parse init # one-time: authenticate, scaffold parse_apis, first sync
```
`uv add` installs the `parse` console script into the project's `.venv`, which isn't on your `PATH`. Invoke it as **`uv run parse `**, or `source .venv/bin/activate` once and call `parse` directly. Examples below use the `uv run` form.
`parse init` is the one-time setup: it scaffolds the committed `parse_apis` package, records it as an editable path dependency (`[tool.uv.sources]`), appends a deny-by-default `.gitignore` block, ensures credentials, and runs the first `parse sync`. Run it from your project root, where the `pyproject.toml` lives.
## Authenticate
`parse init` prompts for an API key on first run. To authenticate explicitly:
```bash theme={null}
uv run parse login # browser sign-in (OAuth 2.1 + PKCE) — the default
uv run parse login --no-web # paste an API key (dashboard: Settings → API Keys)
uv run parse login --api-key YOUR_KEY # non-interactive (CI, agents)
```
Credentials are saved to `~/.config/parse/credentials` (the same path on every OS). The CLI and generated clients also read `PARSE_API_KEY` from the environment. `uv run parse whoami` shows the active key and base URL.
## Add an API
A fresh project tracks **all** APIs your key can call. To curate an explicit set, add APIs by slug.
### One of your own APIs
```bash theme={null}
uv run parse list # see your account APIs and their slugs
uv run parse add my_zillow_api # track it, then sync
```
### A marketplace API
Pull a ready-made API from the [marketplace](/marketplace) by slug, listing id, or URL:
```bash theme={null}
uv run parse search walmart # find it
uv run parse add --marketplace walmart-com-api # add a pinned snapshot
```
`parse add --marketplace` downloads a **pinned snapshot** of a shared canonical API — it won't change underneath you. Pass `--version N` to pin a specific release (the default is the latest at add time). To *edit* the API, subscribe and swap to your own copy in the dashboard.
## Use it
Once you've added an API (here, the Walmart marketplace API above):
```python theme={null}
from parse_apis.walmart_com_api import Walmart, Sort, ProductNotFound
from parse_apis import RateLimitError # runtime errors, re-exported
client = Walmart() # picks up your `parse login` credentials
# Typed resources + methods, with autocomplete and typed responses
for product in client.productsummaries.search(query="laptop", sort=Sort.PRICE_LOW, limit=5):
print(product.name, product.price, product.rating)
# Typed errors
try:
client.products.get(product_id="0000000000")
except ProductNotFound as exc:
print(f"not found: {exc.product_id}")
except RateLimitError:
...
```
Run it in the project environment:
```bash theme={null}
uv run python app.py
```
Resources and methods are generated from **your** API's spec. The client class follows the API's root type; the import module follows its slug. `parse add` prints the exact `from parse_apis.… import …` line, which also heads the generated `README.md` and `example.py` in each `parse_apis/src/parse_apis//` directory.
## Project layout & manifest
`parse init` creates an installed, editable package — not a loose folder:
```text theme={null}
parse_apis/
pyproject.toml # committed: name="parse-apis", pins parse-sdk==X
AGENTS.md CLAUDE.md # committed "start here" pointer (survives a fresh clone)
src/parse_apis/
__init__.py # committed scaffold (re-exports runtime errors)
py.typed # committed
walmart_com_api/ # generated client + README.md + example.py (gitignored)
_manifest.json # generated (gitignored)
AGENTS.md CLAUDE.md # generated cross-API index (gitignored)
```
Commit the scaffold files (those not marked *generated*); the generated payload is per-key (it reveals your API inventory) and stays gitignored. Because `parse_apis` is an installed editable package, `import parse_apis` resolves from any directory.
The repo curates which APIs it syncs via a `[tool.parse]` table in your `pyproject.toml`. `parse add` and `parse remove` manage it for you — you rarely edit it by hand:
```toml theme={null}
[tool.parse]
# Account APIs to track. Omit `apis` entirely to track ALL of them;
# an empty list means none.
apis = ["my_zillow_api"]
# Marketplace APIs downloaded into this repo, each pinned to a version.
[[tool.parse.marketplace]]
slug = "walmart-com-api"
version = 3
```
Reconcile the generated package with the manifest at any time:
```bash theme={null}
uv run parse sync # generate/refresh exactly what the manifest declares
uv run parse sync --check # CI dry run: report drift without writing, exit 1 on any finding
uv run parse clean # remove the generated payload, keep the committed scaffold
```
**Commit the `[tool.parse]` table; the generated `parse_apis/` payload is your choice.** The manifest is the source of truth. Commit the payload too, or `.gitignore` it and run `parse sync` on checkout / in CI. After upgrading `parse-sdk`, re-run `parse sync` — generated code subclasses the runtime by version, and `init`/`sync` restamp the pin in `parse_apis/pyproject.toml`. ([Release history](https://pypi.org/project/parse-sdk/#history).)
## Agents & scripting
Most commands take `--json` for stable, machine-readable output — agents should prefer it. (`init`, `login`, and `clean` have no `--json`.) Errors still go to stderr; stdout stays parseable.
```bash theme={null}
uv run parse list --json # [{ "id", "slug", "name", "modeled", "endpoints", "resources" }, …]
uv run parse sync --json # { "written", "skipped", "removed", "quarantined", "failures" }
uv run parse doctor --json # { "ok": true|false, "findings": [ … ] }
```
`parse sync --check` exits non-zero on drift, so it drops straight into CI. The generated `AGENTS.md` and `CLAUDE.md` files give a coding agent a cross-API index of every client, its resources, and example calls.
### Setup prompt for coding agents
Paste this into Claude Code, Codex, or any coding agent to set up and use the SDK end to end:
```text theme={null}
Set up and use the Parse Python SDK in this project.
1. Ensure this is a uv project — if there is no pyproject.toml, run `uv init`.
2. Install the SDK: `uv add parse-sdk`.
3. Authenticate: if PARSE_API_KEY is not set, get a key from the Parse
dashboard (parse.bot → Settings → API Keys) and run
`uv run parse login --api-key `.
4. Initialize: `uv run parse init` (now non-interactive — credentials exist).
5. List the APIs this key can call: `uv run parse list --json`.
6. Add the API(s) I need by slug: `uv run parse add ` (or
`uv run parse add --marketplace ` for a marketplace API).
This regenerates parse_apis.
7. Read `parse_apis/src/parse_apis//README.md` for the exact import line, then write
code that imports the typed client from `parse_apis.` and calls its
resources and methods. Catch typed errors re-exported from `parse_apis`.
8. Run it with `uv run python .py`.
Always invoke the CLI as `uv run parse …`. Prefer `--json` output when parsing
results. If imports or editor autocomplete don't resolve, run
`uv run parse doctor --fix`.
```
## Command reference
Run every command as `uv run parse ` (or bare `parse` in an activated venv).
| Command | What it does |
| ---------------------------------------------------------------- | ---------------------------------------------------------------------------- |
| `parse init [--api-key K] [--base-url U]` | One-time setup: authenticate, scaffold `parse_apis`, record deps, first sync |
| `parse login [--web \| --no-web] [--api-key K] [--base-url U]` | Sign in via browser (default), or save an API key |
| `parse whoami [--json]` | Show the active key + base URL |
| `parse list [--long] [--json]` | List the APIs your key can call (no generation) |
| `parse search [--limit N] [--json]` | Search the marketplace |
| `parse add [--json]` | Track account API(s) by slug, then sync |
| `parse add --marketplace [--version N] [--json]` | Add a pinned marketplace API |
| `parse remove [--json]` | Stop tracking API(s), then reconcile |
| `parse sync [--check] [--clean] [--json]` | Reconcile the generated package with the manifest |
| `parse clean` | Remove the generated payload, keep the committed scaffold |
| `parse doctor [--fix] [--json]` | Diagnose (and optionally fix) install / auth / import issues |
| `parse help [--json]` | Command overview |
`parse doctor` checks uv, the `parse_apis` scaffold, your pyproject deps and editable source, `parse-sdk` version skew, whether the project interpreter can import `parse_sdk` + `parse_apis` (the usual cause of dead autocomplete), and credentials. `parse doctor --fix` auto-remediates what it safely can.
Prefer no SDK? Every API is also a plain [REST endpoint](/quickstart) and an [MCP server](/mcp).
# API Updates
Source: https://docs.parse.bot/updates
Keep your APIs current with upstream improvements
When you create an API from a URL that already has a shared ("canonical") version, Parse gives you a personal copy. The canonical version may get improvements over time — new endpoints, better schemas, or bug fixes. The updates system lets you pull those changes into your copy when you're ready.
## Check for updates
```bash theme={null}
curl https://api.parse.bot/dispatch/tasks/{task_id}/updates \
-H "X-API-Key: YOUR_API_KEY"
```
Returns a list of endpoints that have changed since you last cloned or merged:
```json theme={null}
{
"updates": [
{
"endpoint_name": "search_products",
"type": "new",
"description": "Search products by keyword",
"method": "POST",
"user_has_called": false
},
{
"endpoint_name": "get_product",
"type": "updated",
"description": "Get product details by ID",
"method": "POST",
"user_has_called": true
}
]
}
```
* `type: "new"` — this endpoint didn't exist when you got your copy
* `type: "updated"` — the canonical version has been modified
* `user_has_called` — whether you've used this endpoint before (helps you gauge impact)
If there are no updates, `updates` will be an empty array.
## Preview before merging
Test an endpoint against the canonical version to see the updated behavior without changing your copy:
```bash theme={null}
curl -X POST https://api.parse.bot/dispatch/tasks/{task_id}/preview-update/search_products \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"params": {"query": "test"}}'
```
Returns the execution result from the canonical version plus sample inputs for the endpoint.
## Merge updates
Pick which endpoints to merge:
```bash theme={null}
curl -X POST https://api.parse.bot/dispatch/tasks/{task_id}/merge-updates \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"endpoints": ["search_products", "get_product"]}'
```
```json theme={null}
{
"merged": 2,
"endpoints": ["search_products", "get_product"],
"code_merged": true
}
```
* `code_merged: true` means the underlying scraper code was also updated (happens when new endpoints are added)
* Spec-only updates preserve your existing code
## Rollback
If a merge doesn't work as expected, undo it:
```bash theme={null}
curl -X POST https://api.parse.bot/dispatch/tasks/{task_id}/rollback-merge \
-H "X-API-Key: YOUR_API_KEY"
```
```json theme={null}
{
"rolled_back": true
}
```
Only the most recent merge can be rolled back. After rolling back, you can re-check updates and try again.