QuickCrawl

Reference

API

Integrate QuickCrawl via HTTP. Try the live extractor on the home page.

Base URL: Same origin (e.g. http://localhost:3000).

GET /health

JSON health check for monitoring and deploy probes.

{
  "status": "ok",
  "service": "quickcrawl",
  "version": "1.0.0"
}

POST /crawl

Fetches a URL and returns Markdown plus optional metadata.

Request

Headers: Content-Type: application/json

{
  "url": "https://example.com"
}

Success (200)

{
  "success": true,
  "data": {
    "markdown": "# Example Domain\n\n...",
    "title": "Example Domain",
    "metadata": { "ogTitle": "...", "ogImage": "..." }
  }
}

Validation error (422)

{
  "error": "Invalid input",
  "issues": [{ "path": ["url"], "message": "..." }]
}

Crawl failure (500)

{
  "success": false,
  "error": { "type": "unknown", "message": "..." }
}

Rate limit (429)

{ "error": "Rate limit exceeded" }

Default: 10 requests per 60s per IP.

Example

curl -X POST http://localhost:3000/crawl \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

Environment

VariableDefaultDescription
RATE_LIMIT_MAX_REQUESTS10Max requests per window
RATE_LIMIT_WINDOW_MS60000Window (ms)
CACHE_TTL_MS300000Cache TTL
CACHE_MAX_SIZE100Max cache entries