Jobs and results

Everything you collect is a job: one scraper, one input. A job runs in the background, so you create it, wait for it to finish, then read its results.

Create a job

POST /api/v1/jobs/create/ with a scraper_type and its input in config:

curl https://go-fetch.io/api/v1/jobs/create/ \
  -H "X-API-Key: $GOFETCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "scraper_type": "youtube",
    "config": {"channelUrls": ["https://www.youtube.com/@NobelPrize"], "limit": 25},
    "timeout": 3600
  }'

Field	Required	What it does
`scraper_type`	yes	Which scraper runs. See the scraper guides.
`config`	yes	The scraper’s input. `input` is accepted as another name.
`timeout`	no	Seconds the job may run, from 300 to 39,600. The default depends on the scraper: 8 hours for most, 2 hours for `tiktok_search`, 1 hour for `tiktok_profile` and `google_serp`.
`webhooks`	no	Webhooks for this job only. See Webhooks.

The answer is 201 with the job. The price is locked at this moment, and the job’s credit is checked. A request is refused with 402 if the balance can’t cover its estimate, or if that estimate plus the quoted estimated_cost of the organization’s pending and running jobs exceeds 10 times the balance (Billing). A 402 creates no job, so retrying the request after backing off is safe. How config is checked:

A field the scraper doesn’t know is ignored, not refused. Check spelling and case: a misspelt limit runs with the default.
An input a scraper accepts but can’t act on with the value you sent is ignored too. Each guide lists them.
Either way the job still runs, and its log (GET /api/v1/jobs/{id}/logs/) gets one WARNING naming the fields, for example Input(s) limitt are not recognised for instagram and had no effect; check the spelling.
An empty list ("usernames": []) is refused with 400.
Numbers, links and dates the scraper guide gives a range for are checked, and a value outside it is refused with 400 naming the field.

Statuses

`status`	Means
`pending`	Created, not yet handed to a scraper.
`running`	Collecting. Results already delivered can be read.
`completed`	Finished. `items_scraped` holds the billed result count and `actual_cost` the charge.
`failed`	Stopped by an error. `error_message` and `error_code` say why.
`timed_out`	Ran past its `timeout`.
`cancelled`	Stopped by you (not charged), or by its spending cap (billed for the rows it delivered, at most the cap).

Wait on is_terminal, not on status. A job that fails while it still has automatic retries left reads "status": "failed" with "is_terminal": false and "will_auto_retry": true: it is about to run again, under the same id. is_terminal is true once the job is completed or cancelled, or failed or timed_out with no retry coming. The SDK’s call() and wait_for_finish() already wait for it. Each earlier attempt is listed in attempt_history with its status, error_message and failed_at.

`error_code`

A job that did not complete has an error_code, so your code can decide what to do without reading error_message:

`error_code`	Means	What to do
`INPUT_ERROR`	The input can’t work, for example an account that does not exist.	Fix the input.
`BLOCKED`	The platform refused the requests.	Try again later.
`RATE_LIMITED`	The platform throttled the requests.	Try again later.
`TIMEOUT`	The job ran past its `timeout`.	Ask for less, or raise `timeout`.
`INFRA_TERMINATED`	Our side stopped the job.	Retry it.
`PLATFORM_ERROR`	The scraper is not available on our side.	Write to support.
`PROFILE_EXCLUDED`	Every target is on our exclusion list.	Remove those targets.
`CANCELLED`	You cancelled the job, or its spending cap did.
`UNKNOWN`	None of the above.	Read `error_message`.

A finished job

A real completed job, from a test run of the quickstart on a local stack. Amounts are shown as "…": rates change, so the docs leave them out.

{
  "id": "d1c519f1-9ae0-44c0-8362-1afe2f01a0f5",
  "organization": "0d17a952-5d4f-457c-95a0-1812bb75aa86",
  "organization_name": "Local Test",
  "environment": "919aed5c-b794-4ca0-a22a-0a6fc0d47a80",
  "environment_name": "Docs samples",
  "scraper_type": "instagram",
  "config": {
    "limit": 10,
    "timeout": 28800,
    "usernames": [
      "nasa"
    ]
  },
  "status": "completed",
  "is_terminal": true,
  "will_auto_retry": false,
  "items_scraped": 2,
  "output_dataset_url": "/api/v1/jobs/d1c519f1-9ae0-44c0-8362-1afe2f01a0f5/results/",
  "error_message": "",
  "error_code": null,
  "attempt_history": [],
  "started_at": "2026-10-15T08:24:49.206473Z",
  "completed_at": "2026-10-15T08:24:52.146760Z",
  "duration_seconds": 2,
  "created_at": "2026-10-15T08:24:49.202036Z",
  "updated_at": "2026-10-15T08:24:52.146802Z",
  "retry_count": 0,
  "max_retries": 3,
  "can_retry": false,
  "can_resume": true,
  "partial_results_count": 0,
  "last_progress_at": null,
  "ad_hoc_webhooks": null,
  "scraper_metadata": null,
  "started_from": null,
  "is_initial_crawl": true,
  "price_per_1000_at_creation": "…",
  "platform_prices_at_creation": null,
  "minimum_charge_at_creation": true,
  "estimated_cost": "…",
  "actual_cost": "…",
  "credits_deducted": true
}

Fields worth knowing:

Field	What it holds
`items_scraped`	Billed result rows, excluding $0 status rows.
`actual_cost`, `credits_deducted`	The final charge in USD, and whether it has been taken from your balance.
`estimated_cost`	The estimate the credit check used when the job was created.
`price_per_1000_at_creation`, `minimum_charge_at_creation`	The rate locked for this job, and whether its scraper type has a minimum charge per job.
`started_from`, `is_initial_crawl`	The start of the job’s date filter, and whether the job counts as a full crawl or a recent one (rates).
`partial_results_count`, `last_progress_at`	Progress while the job runs.
`can_retry`, `retry_count`, `max_retries`	Whether `POST /api/v1/jobs/{id}/retry/` will run it again.
`scraper_metadata`	What the scraper reported about completeness. See below.
`environment`, `environment_name`	The environment of the key that created it.
`ad_hoc_webhooks`	The job’s own webhooks.
`output_dataset_url`	The job’s results endpoint (relative path), served with your API key.

Read the results

GET /api/v1/jobs/{id}/results/?offset=0&limit=100 returns a page of rows:

curl "https://go-fetch.io/api/v1/jobs/$JOB_ID/results/?offset=0&limit=100" \
  -H "X-API-Key: $GOFETCH_API_KEY"

limit is 50 by default and at most 100. Page with offset until you have total rows, or a page comes back with count 0.
total includes $0 status rows, such as Instagram’s profile_error, so it can exceed items_scraped.
A running job already returns the rows delivered so far.
Each row is the scraper’s own record. Its fields are in the scraper’s guide.
Rows don’t always come back in input order. Match them to your input with the field the guide names (input, facebookUrl, search_term, …).

With the SDK, iterate_items() pages for you:

import os

from gofetch import GoFetchClient

client = GoFetchClient(api_key=os.environ["GOFETCH_API_KEY"])

items = client.dataset(os.environ["JOB_ID"]).list_items()
print(items.total, "rows")
for item in client.dataset(os.environ["JOB_ID"]).iterate_items():
    print(item.get("type"), item.get("url"))

Results are deleted 5 days after collection. After that, /results/ returns total: 0. Job records (configuration, status, counts and usage) stay while your account is active. Fetch the results, or take them by webhook, within that window.

Is it complete?

completed means the job ran to its end. It does not always mean every target delivered everything: a private account, a page that shows nothing to signed-out visitors, or a platform that stopped answering partway leaves gaps. scraper_metadata says what happened, with keys that depend on the scraper:

Key	What it says
`completeness`	How much of what you asked for was collected, per target.
`coverage`	For comment scrapers: comments collected against the count each post shows.
`restrictions`	Targets the platform withheld from signed-out visitors, and why.
`targets_failed`	Targets that returned nothing, with a `reason` and whether retrying can help (`retriable`).
`excluded_targets`	Targets removed because they are on our exclusion list: people who asked not to be collected. Their rows are never returned.
`profiles_scraped`, `profiles_failed`	Targets collected and targets that failed.

Each scraper guide lists what it reports.

Logs, cancel and retry

Request	Does
`GET /api/v1/jobs/{id}/logs/`	The progress messages the scraper reported, oldest first, and any warning about the job’s input.
`POST /api/v1/jobs/{id}/cancel/`	Stops a pending or running job, which is then not charged. A finished job is returned as it is.
`POST /api/v1/jobs/{id}/retry/`	Runs a `failed`, `timed_out` or `cancelled` job again, or a `completed` one that delivered nothing, while `can_retry` is `true`. Its credit is checked again. A job its spending cap stopped can’t be retried (`400`): create a new job instead. A job you cancelled yourself can.
`GET /api/v1/jobs/?status=completed&scraper_type=tiktok`	Your organization’s jobs, newest first, filtered by `status`, `scraper_type` or `environment`.

curl https://go-fetch.io/api/v1/jobs/$JOB_ID/logs/ \
  -H "X-API-Key: $GOFETCH_API_KEY"

Lists and paging

Every list answers {"total", "offset", "limit", "count", "results"}. Page with offset and limit: 25 rows by default, at most 100. The audit log is the exception: it pages with page and page_size (100 by default) and answers {"count", "next", "previous", "results"}.

Get started

Concepts

Scrapers

Python SDK

Create a job

Statuses

`error_code`

A finished job

Read the results

Is it complete?

Logs, cancel and retry

Lists and paging

​Create a job

​Statuses

​error_code

​A finished job

​Read the results

​Is it complete?

​Logs, cancel and retry

​Lists and paging

Create a job

Statuses

`error_code`

A finished job

Read the results

Is it complete?

Logs, cancel and retry

Lists and paging