> For the complete documentation index, see [llms.txt](https://docs.amigo.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.amigo.ai/developer-guide/platform-api/data-world-model/intake-sources.md).

# Intake Sources

Intake sources map Google Drive folders to registered [dataset intake](/developer-guide/platform-api/data-world-model/intake.md) contracts. Files are discovered when you explicitly trigger a source sync or start a dataset update; registering a source does not create a continuous schedule.

Source-discovered files enter the dataset catalog and carry a batch association. Files uploaded directly to a dataset do not have a batch. Files received through [upload links](/developer-guide/platform-api/data-world-model/intake-links.md) use a separate upload store and never join the dataset catalog automatically.

Current source registration creates the `google_shared_drive` source type. Depending on credential setup, the mapped folder can be in a Shared Drive or be a folder shared with the configured account.

## Register a Source

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/sources" method="post" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

```json
{
  "display_name": "Clinic intake documents",
  "drive_id": "0ABcDeFgHiJkLmNoPq",
  "folders": [
    {
      "folder_id": "1AbCdEfGhIjKlMnOpQrStUv",
      "dataset": "patient-records"
    }
  ]
}
```

{% hint style="info" %}
`credential_ssm_param_path` is an opaque provisioning reference retained for wire compatibility. The API does not accept source credential material in the registration body and never returns the credential. Coordinate the out-of-band credential step with Amigo; do not construct or parse the reference yourself.
{% endhint %}

## List Sources

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/sources" method="get" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

## Trigger a Source Sync

A source sync performs discovery immediately and returns one batch summary per mapped folder.

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/sources/{source\_id}/sync" method="post" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

For each folder, the sync creates a batch, recursively lists files, downloads eligible content, and calls the same catalog upload service used by direct dataset uploads. Byte-identical content is deduplicated. A batch with no newly landed rows returns `completed` with `files_found: 0`; otherwise it returns `ready`.

Processing is separate from source discovery. Use the batch processing endpoint for manual orchestration, or use the [dataset update run](/developer-guide/platform-api/data-world-model/intake.md#update-a-dataset) to coordinate discovery, preparation, and publication.

{% openapi src="<https://api.platform.amigo.ai/v1/openapi.json>" path="/v1/{workspace\_id}/intake/batches/{batch\_id}/process" method="post" %}
<https://api.platform.amigo.ai/v1/openapi.json>
{% endopenapi %}

{% hint style="warning" %}
A source sync can still return `201` when individual files were skipped because of source download errors, contract mismatches, the 100 MB upload limit, or scanning failures. Reconcile the returned `files_found` count and catalog rows instead of treating the HTTP status as proof that every source file landed.
{% endhint %}

Current source syncs consider up to 1,000 files in a mapped folder tree per run. Split larger collections into multiple mappings or coordinate a different ingestion plan with Amigo.

| Status | Condition                                                  |
| ------ | ---------------------------------------------------------- |
| `404`  | Source ID is not present in the workspace                  |
| `422`  | The out-of-band source credential has not been provisioned |
| `502`  | Google Drive authentication failed                         |

## Batch Lifecycle

| Status       | Meaning                                                                             |
| ------------ | ----------------------------------------------------------------------------------- |
| `discovered` | Batch record created while discovery is in progress                                 |
| `ready`      | At least one new file is associated with the batch; processing has not been settled |
| `processing` | Batch processing has started                                                        |
| `completed`  | Processing completed, or the sync found no new content to process                   |
| `failed`     | Batch processing failed                                                             |

Snapshot batches are processed in version order. Document batches can be processed together. A source sync response is therefore a discovery receipt, not a processing or publication receipt.

## Authentication

Source management, sync, and batch routes require a workspace-scoped Platform bearer credential. See [Authentication and API Keys](/developer-guide/platform-api/platform-api/authentication.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.amigo.ai/developer-guide/platform-api/data-world-model/intake-sources.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.