> For the complete documentation index, see [llms.txt](https://docs.amigo.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.amigo.ai/developer-guide/platform-api/data-world-model/intake.md).

# Customer Data Intake

These endpoints manage the **dataset intake catalog**: registered dataset contracts, direct file uploads, files discovered by mapped [intake sources](/developer-guide/platform-api/data-world-model/intake-sources.md), and dataset update runs. They require a workspace-scoped Platform bearer credential.

Dataset intake is separate from [upload-link intake](/developer-guide/platform-api/data-world-model/intake-links.md). A file received through a public upload link is not assigned to a dataset and does not appear in the dataset file catalog.

{% hint style="info" %}
The Developer Console's dataset-intake surface is feature-gated. API availability does not guarantee that the Intake tab is enabled for every production environment.
{% endhint %}

## Files

### List Files

Returns a paginated dataset-file catalog. Results can be sorted by ingestion time, filename, status, dataset, or size; filtered by status; searched by filename; and optionally include withdrawn document versions.

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/files" method="get" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

The status vocabulary is `received`, `scanned`, `processing`, `curated`, `rejected`, `failed`, and `held`. Current upload paths commonly use:

| Status     | Meaning                                                        |
| ---------- | -------------------------------------------------------------- |
| `received` | Stored and waiting for, or eligible for, downstream processing |
| `curated`  | Validation or configured processing completed successfully     |
| `rejected` | The file did not satisfy the tabular contract                  |
| `failed`   | Document extraction or another processing step failed          |

`scanned`, `processing`, and `held` are valid wire values, but they do not by themselves prove that malware scanning ran. Use `error_reason` for failure context and treat unknown future values defensively.

### Upload File

Uploads one file as `multipart/form-data` to a registered dataset. The service enforces a 100 MB request cap.

`max_size_mb` is stored on the dataset contract and returned by the dataset list, but the current upload runtime does not enforce a lower per-dataset value. Enforce that value in the client if your workflow depends on it.

Before persistence, the service computes a SHA-256 hash and returns the existing file row for byte-identical content already present in the same workspace and dataset. Malware scanning runs only when a scanner is configured for the deployment.

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/files" method="post" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

Upload behavior depends on the dataset mode:

* **Snapshot** - The current processing format is CSV. Depending on workspace configuration, validation can complete synchronously (`curated` or `rejected`) or the file can enter asynchronous processing as `received`.
* **Document** - The filename extension must be in the dataset's accepted set. Omit `document_id` to create a logical document, or pass an existing document ID to add a same-type version. Documents enter as `received` and require the extraction processor for a terminal verdict.

{% hint style="warning" %}
Dataset registration accepts `csv`, `xls`, and `xlsx` as snapshot type tokens, but current snapshot validators and processing read CSV bytes. Use CSV for current processing workflows.
{% endhint %}

Common route errors include:

| Status | Condition                                                                                                                                                    |
| ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `413`  | File exceeds the global 100 MB cap                                                                                                                           |
| `422`  | Missing dataset contract, invalid or empty filename, disallowed document type, invalid document version target, or an infected file when scanning is enabled |
| `503`  | The configured malware scanner is unavailable                                                                                                                |

### Download File

Downloads the original catalog file. The response includes `Content-Disposition: attachment` and `Cache-Control: no-store`; a file outside the workspace scope returns `404`.

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/files/{file\_id}/download" method="get" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

## Dataset Contracts

### List Datasets

Returns registered dataset contracts with their mode, representative file type, accepted document types, schema version, field count, and recorded `max_size_mb` value.

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/datasets" method="get" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

### Register a Dataset

The server infers `ingestion_mode` from `file_type`:

* **Snapshot** - `csv`, `xls`, or `xlsx`; requires one or more primary-key columns and fields. The supported field types are `str`, `int`, `float`, `bool`, `date`, and `datetime`.
* **Document** - Any other lowercase alphanumeric type; ignores the tabular primary key and field list. `accepted_file_types` can contain up to 16 non-tabular types.

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/schema/register" method="post" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

Document contracts accept `document_processing.extraction_mode` values of `text_extract`, `OCR`, or `hybrid`, but the current processor implements `text_extract` only. `OCR` and `hybrid` reach a failed processing verdict rather than performing OCR.

With `text_extract`, the current processor handles PDF, DOCX, and UTF-8 text types such as `txt`, `text`, `md`, and `markdown`. An unknown type is treated as text only when its bytes decode as UTF-8; otherwise processing fails. Registering an accepted extension does not guarantee parser support.

| Status | Condition                                                                               |
| ------ | --------------------------------------------------------------------------------------- |
| `409`  | A dataset with this name already exists in the workspace                                |
| `422`  | Invalid fields, types, primary key, extraction mode, or mixed tabular/document type set |

## Update a Dataset

`POST /v1/{workspace_id}/intake/datasets/{dataset}/update` starts the one-click workflow for a dataset backed by at least one active Google Drive source. It checks mapped folders, creates batches for newly landed files, starts preparation, and then starts publication when preparation succeeds.

This route does not refresh a dataset made only from manual uploads. If no active Drive source maps to the dataset, the accepted run transitions to `failed` with an explanatory error.

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/datasets/{dataset}/update" method="post" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

The `202 Accepted` response is a durable run record, not a completion receipt. Its status is one of `checking_drive`, `preparing`, `publishing`, `completed`, `needs_review`, or `failed`. Poll the run until it reaches a terminal status:

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/update-runs/{run\_id}" method="get" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

The run response includes `source_ids`, `batch_ids`, timestamps, and optional `materialize_run_id` and `error`. Starting another update while one is active can return the existing run rather than create parallel work.

## External HMAC Upload

{% hint style="warning" %}
`POST /v1/{workspace_id}/intake/files/external` is a compatibility path, not the dataset upload endpoint. Although the route is in the public schema, the current service is initialized with no customer HMAC secrets. It is therefore dormant and every customer slug returns `404`. Do not integrate until Amigo explicitly enables and provisions the path.
{% endhint %}

When provisioned, this raw-body route requires both a Platform bearer credential and these headers:

| Header                         | Value                                                              |
| ------------------------------ | ------------------------------------------------------------------ |
| `x-amigo-intake-sha256`        | Lowercase SHA-256 of the body, exactly 64 hexadecimal characters   |
| `x-amigo-intake-timestamp`     | Unix time in seconds; accepted within a five-minute window         |
| `x-amigo-intake-signature`     | Hex HMAC-SHA256 over `sha256:content_type:customer_slug:timestamp` |
| `x-amigo-intake-customer-slug` | Provisioned customer identifier                                    |
| `x-amigo-intake-filename`      | Original filename                                                  |
| `x-amigo-intake-content-type`  | Optional MIME type; defaults to `application/octet-stream`         |

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/files/external" method="post" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

The external path verifies the checksum and records a separate integration upload. It does not assign the file to a dataset, apply a dataset contract, create a source batch, or enter a dataset update run. Malware scanning is still deployment-dependent; the response can report `scan_status: skipped` when no scanner is configured.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.amigo.ai/developer-guide/platform-api/data-world-model/intake.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.