indexly
REST API

API Reference

The Indexly REST API gives Pro, Agency, and Enterprise users programmatic access to sites, crawls, and sitemaps.

Authentication

All API requests must include a Bearer token in the Authorization header. Generate tokens from Settings → API Tokens.

http 
Authorization: Bearer <your-api-token>

Plan requirement

API access is available on Pro, Agency, and Enterprise plans only. Requests from Free or Starter accounts receive a 403 response.

Base URL

base url 
https://indexly.dev/api

All responses are JSON. Paginated list endpoints follow Laravel's standard pagination envelope with data, links, and meta keys.

Endpoints

GET /api/sites

Return a paginated list of all sites belonging to the authenticated user.

Example request

bash 
curl https://indexly.dev/api/sites \
  -H "Authorization: Bearer <token>" \
  -H "Accept: application/json"

Example response 200 OK

json 
{
  "data": [
    {
      "id": 1,
      "name": "My Website",
      "url": "https://example.com",
      "crawl_frequency": "weekly",
      "is_active": true,
      "last_crawled_at": "2026-04-10T14:23:00+00:00",
      "next_crawl_at": "2026-04-17T14:23:00+00:00",
      "created_at": "2026-04-01T09:00:00+00:00",
      "latest_crawl": { "id": 42, "status": "completed", ... },
      "sitemap_url": "https://cdn.indexly.dev/sitemaps/1/1/42/sitemap.xml"
    }
  ],
  "links": { ... },
  "meta": { "current_page": 1, "total": 1, ... }
}
POST /api/sites

Create a new site. Subject to your plan's site limit.

Request body

Field Type Required Notes
name string Yes Max 100 characters
url string Yes Must be a valid http/https URL. Private IPs are rejected.
crawl_frequency string Yes manual, weekly, or daily

Example request

bash 
curl -X POST https://indexly.dev/api/sites \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{"name":"My Blog","url":"https://myblog.com","crawl_frequency":"weekly"}'

Example response 201 Created

json 
{
  "data": {
    "id": 2,
    "name": "My Blog",
    "url": "https://myblog.com",
    "crawl_frequency": "weekly",
    "is_active": true,
    "last_crawled_at": null,
    "next_crawl_at": null,
    "created_at": "2026-04-12T10:00:00+00:00",
    "latest_crawl": null,
    "sitemap_url": null
  }
}
GET /api/sites/{site}

Return a single site by ID, including the latest crawl and sitemap URL.

Example request

bash 
curl https://indexly.dev/api/sites/1 \
  -H "Authorization: Bearer <token>" \
  -H "Accept: application/json"

Same shape as a single item in the list response. Returns 404 if not found or not owned by the authenticated user.

DELETE /api/sites/{site}

Soft-delete a site. Crawl history and sitemaps are retained.

Example request

bash 
curl -X DELETE https://indexly.dev/api/sites/2 \
  -H "Authorization: Bearer <token>" \
  -H "Accept: application/json"

Returns 204 No Content on success.

POST /api/sites/{site}/crawl

Trigger a new crawl for the given site. The crawl is queued asynchronously. Rate-limited to 10 requests per minute.

Returns 409 Conflict if a crawl is already queued or running for this site.

Example request

bash 
curl -X POST https://indexly.dev/api/sites/1/crawl \
  -H "Authorization: Bearer <token>" \
  -H "Accept: application/json"

Example response 202 Accepted

json 
{
  "data": {
    "id": 43,
    "site_id": 1,
    "status": "queued",
    "pages_found": null,
    "pages_in_sitemap": null,
    "started_at": null,
    "completed_at": null,
    "sitemap_url": null,
    "created_at": "2026-04-12T10:05:00+00:00"
  }
}
GET /api/sites/{site}/crawls

Return a paginated crawl history for the given site (20 per page, newest first).

Example request

bash 
curl https://indexly.dev/api/sites/1/crawls \
  -H "Authorization: Bearer <token>" \
  -H "Accept: application/json"

Example response 200 OK

json 
{
  "data": [
    {
      "id": 42,
      "site_id": 1,
      "status": "completed",
      "pages_found": 312,
      "pages_in_sitemap": 310,
      "started_at": "2026-04-10T14:20:00+00:00",
      "completed_at": "2026-04-10T14:23:00+00:00",
      "sitemap_url": "https://cdn.indexly.dev/sitemaps/1/1/42/sitemap.xml",
      "created_at": "2026-04-10T14:20:00+00:00"
    }
  ],
  "links": { ... },
  "meta": { "current_page": 1, "total": 5, ... }
}
GET /api/crawls/{crawl}

Return a single crawl by ID. Includes error_message only when status is failed.

Example request

bash 
curl https://indexly.dev/api/crawls/42 \
  -H "Authorization: Bearer <token>" \
  -H "Accept: application/json"

Crawl status values

Value Meaning
queued Waiting in the queue
running Actively crawling
completed Finished successfully — sitemap available
failed Crawl failed — check error_message
GET /api/crawls/{crawl}/sitemap

Redirects (302) to the public sitemap XML URL on Cloudflare R2. Returns 404 if no sitemap is available for this crawl.

Example request

bash 
curl -L https://indexly.dev/api/crawls/42/sitemap \
  -H "Authorization: Bearer <token>" \
  -H "Accept: application/xml"

Use -L with curl to follow the redirect and land on the raw XML file.

Rate Limiting

Rate limit headers are returned on every response. When a limit is exceeded, the API responds with 429 Too Many Requests.

Endpoint group Limit Keyed by
All API endpoints 60 requests / minute User ID
POST /sites/{site}/crawl 10 requests / minute User ID

Error Responses

All error responses follow a consistent shape with a human-readable message field and an optional errors object on validation failures.

Status Meaning
401 Missing or invalid Bearer token
402 Insufficient crawl credits
403 Plan does not include API access, or resource belongs to another user
404 Resource not found
409 Crawl already queued or running for this site
422 Validation error (field details in errors key) or SSRF-blocked URL
429 Rate limit exceeded

Error body shape

json 
{
  "message": "Human-readable description of the error.",
  "errors": {
    "url": ["The url field is required."]
  }
}
Getting Started Generate an API token