Create a job

curl --request POST \
  --url https://go-fetch.io/api/v1/jobs/create/ \
  --header 'Content-Type: application/json' \
  --header 'X-API-Key: <api-key>' \
  --data '{}'

import requests

url = "https://go-fetch.io/api/v1/jobs/create/"

payload = {}
headers = {
    "X-API-Key": "<api-key>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {'X-API-Key': '<api-key>', 'Content-Type': 'application/json'},
  body: JSON.stringify({})
};

fetch('https://go-fetch.io/api/v1/jobs/create/', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "organization": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "organization_name": "<string>",
  "environment": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "environment_name": "<string>",
  "scraper_type": "<string>",
  "status": "pending",
  "is_terminal": true,
  "will_auto_retry": true,
  "items_scraped": 123,
  "output_dataset_url": "<string>",
  "error_message": "<string>",
  "error_code": "CANCELLED",
  "error_description": "<string>",
  "attempt_history": [
    {
      "attempt": 123,
      "status": "<string>",
      "error_message": "<string>",
      "failed_at": "2023-11-07T05:31:56Z"
    }
  ],
  "started_at": "2023-11-07T05:31:56Z",
  "completed_at": "2023-11-07T05:31:56Z",
  "duration_seconds": 123,
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "retry_count": 123,
  "can_retry": true,
  "partial_results_count": 123,
  "last_progress_at": "2023-11-07T05:31:56Z",
  "ad_hoc_webhooks": [
    {
      "url": "<string>",
      "events": [
        "<string>"
      ],
      "signing_secret": "<string>"
    }
  ],
  "scraper_metadata": {},
  "started_from": "2023-12-25",
  "is_initial_crawl": true,
  "price_per_1000_at_creation": "<string>",
  "platform_prices_at_creation": "<unknown>",
  "minimum_charge_at_creation": true,
  "estimated_cost": "<string>",
  "actual_cost": "<string>",
  "credits_deducted": true,
  "config": "<unknown>",
  "max_retries": 123,
  "can_resume": true
}

POST

api

jobs

create

Create a job

curl --request POST \
  --url https://go-fetch.io/api/v1/jobs/create/ \
  --header 'Content-Type: application/json' \
  --header 'X-API-Key: <api-key>' \
  --data '{}'

import requests

url = "https://go-fetch.io/api/v1/jobs/create/"

payload = {}
headers = {
    "X-API-Key": "<api-key>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {'X-API-Key': '<api-key>', 'Content-Type': 'application/json'},
  body: JSON.stringify({})
};

fetch('https://go-fetch.io/api/v1/jobs/create/', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "organization": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "organization_name": "<string>",
  "environment": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "environment_name": "<string>",
  "scraper_type": "<string>",
  "status": "pending",
  "is_terminal": true,
  "will_auto_retry": true,
  "items_scraped": 123,
  "output_dataset_url": "<string>",
  "error_message": "<string>",
  "error_code": "CANCELLED",
  "error_description": "<string>",
  "attempt_history": [
    {
      "attempt": 123,
      "status": "<string>",
      "error_message": "<string>",
      "failed_at": "2023-11-07T05:31:56Z"
    }
  ],
  "started_at": "2023-11-07T05:31:56Z",
  "completed_at": "2023-11-07T05:31:56Z",
  "duration_seconds": 123,
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "retry_count": 123,
  "can_retry": true,
  "partial_results_count": 123,
  "last_progress_at": "2023-11-07T05:31:56Z",
  "ad_hoc_webhooks": [
    {
      "url": "<string>",
      "events": [
        "<string>"
      ],
      "signing_secret": "<string>"
    }
  ],
  "scraper_metadata": {},
  "started_from": "2023-12-25",
  "is_initial_crawl": true,
  "price_per_1000_at_creation": "<string>",
  "platform_prices_at_creation": "<unknown>",
  "minimum_charge_at_creation": true,
  "estimated_cost": "<string>",
  "actual_cost": "<string>",
  "credits_deducted": true,
  "config": "<unknown>",
  "max_retries": 123,
  "can_resume": true
}

Authorizations

X-API-Key

string

header

required

Body

scraper_type

enum<string>

required

instagram - instagram
instagram_profile - instagram_profile
instagram_posts - instagram_posts
instagram_comments - instagram_comments
youtube - youtube
tiktok - tiktok
tiktok_comments - tiktok_comments
facebook - facebook
facebook_posts - facebook_posts
facebook_profile - facebook_profile
google_serp - google_serp
tiktok_search - tiktok_search
tiktok_profile - tiktok_profile

Available options:

instagram,

instagram_profile,

instagram_posts,

instagram_comments,

youtube,

tiktok,

tiktok_comments,

facebook,

facebook_posts,

facebook_profile,

google_serp,

tiktok_search,

tiktok_profile

config

any

input

any

organization

string<uuid>

timeout

integer

default:28800

Job timeout in seconds (min 5 min, max 11 hours). Default 8 hours.

webhooks

object[]

Show child attributes

Response

201 - application/json

string<uuid>

required

read-only

organization

string<uuid>

required

organization_name

string

required

read-only

environment

string<uuid>

required

read-only

environment_name

string

required

read-only

scraper_type

string

required

status

enum<string>

required

pending - Pending
running - Running
completed - Completed
failed - Failed
timed_out - Timed Out
cancelled - Cancelled

Available options:

pending,

running,

completed,

failed,

timed_out,

cancelled

is_terminal

boolean

required

read-only

will_auto_retry

boolean

required

read-only

items_scraped

integer

required

read-only

output_dataset_url

string<uri-reference>

required

read-only

The job's results endpoint, a relative path served with your API key. Empty until the job completes.

error_message

string

required

read-only

error_code

required

Why the job did not complete; null unless it failed, timed out or was cancelled.

CANCELLED - CANCELLED
TIMEOUT - TIMEOUT
PLATFORM_ERROR - PLATFORM_ERROR
PROFILE_EXCLUDED - PROFILE_EXCLUDED
INPUT_ERROR - INPUT_ERROR
INFRA_TERMINATED - INFRA_TERMINATED
RATE_LIMITED - RATE_LIMITED
BLOCKED - BLOCKED
UNKNOWN - UNKNOWN

Available options:

CANCELLED,

TIMEOUT,

PLATFORM_ERROR,

PROFILE_EXCLUDED,

INPUT_ERROR,

INFRA_TERMINATED,

RATE_LIMITED,

BLOCKED,

UNKNOWN

error_description

string

required

read-only

Why the job ended, in words for a person: what happened and what to do next. Empty when error_message is. Show this one; error_message is the scraper's own technical text.

attempt_history

object[]

required

read-only

Show child attributes

started_at

string<date-time> | null

required

read-only

completed_at

string<date-time> | null

required

read-only

duration_seconds

integer | null

required

read-only

created_at

string<date-time>

required

read-only

updated_at

string<date-time>

required

read-only

retry_count

integer

required

read-only

Number of times this job has been retried

can_retry

boolean

required

read-only

partial_results_count

integer

required

read-only

Items stored so far while the job runs.

last_progress_at

string<date-time> | null

required

read-only

Last time progress was reported by the scraper

ad_hoc_webhooks

object[] | null

required

read-only

The per-run webhooks sent with the create request.

Show child attributes

scraper_metadata

object | null

required

read-only

What the scraper reported about how complete the collection is, when it reported anything: for example coverage, restrictions, targets_failed and dataAsOf. The keys vary by scraper type.

Show child attributes

started_from

string<date> | null

required

read-only

Scrape posts newer than this date. NULL = all-time / initial crawl.

is_initial_crawl

boolean

required

read-only

True if this is a deep historical scrape; False for date-bounded regular crawls.

price_per_1000_at_creation

string<decimal> | null

required

read-only

Price per 1,000 items at job creation (USD) - locked for historical reporting

Pattern: ^-?\d{0,6}(?:\.\d{0,4})?$

platform_prices_at_creation

any | null

required

read-only

Rates per 1,000 rows by platform, locked at creation, for scraper types billed per platform; null when the job bills at price_per_1000_at_creation.

minimum_charge_at_creation

boolean | null

required

read-only

Whether this job's scraper type has a minimum charge per job that returned results.

estimated_cost

string<decimal> | null

required

read-only

The estimate the credit check used when the job was created (USD): estimated items × the locked rate. Not a forecast; see actual_cost.

Pattern: ^-?\d{0,8}(?:\.\d{0,4})?$

actual_cost

string<decimal> | null

required

read-only

Actual cost based on items_scraped (USD) - set on completion

Pattern: ^-?\d{0,8}(?:\.\d{0,4})?$

credits_deducted

boolean

required

read-only

Whether credits have been deducted for this job

config

any

Scraper configuration/input parameters

max_retries

integer

Maximum number of retries allowed for this job

can_resume

boolean

Whether the job can resume from partial state

Overview

API keys

Jobs

Organizations

Pricing

Usage

Webhooks

Create a job

Authorizations

Body

Response