Intake Sources

Register and manage external file sources for the customer data intake pipeline.

Intake sources let you connect external file storage (such as Google Shared Drive) to the intake pipeline. Once registered, a source maps folders in the external storage to intake datasets. The platform discovers files in those folders and lands them into the intake catalog automatically.

Sources are one of three ways files enter the intake catalog, alongside direct uploads and shareable upload links. Files landed by a source sync are grouped into batches; direct uploads are not.

Endpoints

Method	Path	Description
POST	/v1/{workspace_id}/intake/sources	Register a new intake source
GET	/v1/{workspace_id}/intake/sources	List registered intake sources
POST	/v1/{workspace_id}/intake/sources/{source_id}/sync	Trigger a sync for a source

Register a Source

POST /v1/{workspace_id}/intake/sources

Registers a new external file source for the workspace. Currently supports Google Shared Drive sources.

Request Body

Field	Type	Required	Description
display_name	string	Yes	Human-readable name for the source (1-200 characters)
drive_id	string	No	The Shared Drive identifier. If omitted, resolved automatically at sync time
folders	array	Yes	One or more folder-to-dataset mappings (1-100 entries)
folders[].folder_id	string	Yes	The Drive folder identifier
folders[].dataset	string	Yes	The intake dataset slug to land files into (2-63 characters, lowercase alphanumeric with hyphens and underscores)

Response (201 Created)

Field	Type	Description
id	string (UUID)	Unique source identifier
source_type	string	The source type (e.g. google_shared_drive)
display_name	string	Human-readable name
drive_id	string or null	The Shared Drive identifier, if provided
folders	array	The folder-to-dataset mappings
credential_ssm_param_path	string	The secure secret-store location where the service account key must be provisioned. The credential itself is never returned
status	string	Source status (e.g. active)
created_ts	string	ISO-8601 creation timestamp

Example Request

{
  "display_name": "Clinic intake documents",
  "drive_id": "0ABcDeFgHiJkLmNoPq",
  "folders": [
    {
      "folder_id": "1AbCdEfGhIjKlMnOpQrStUv",
      "dataset": "patient-records"
    },
    {
      "folder_id": "2BcDeFgHiJkLmNoPqRsTuVw",
      "dataset": "lab-results"
    }
  ]
}

Example Response

{
  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "source_type": "google_shared_drive",
  "display_name": "Clinic intake documents",
  "drive_id": "0ABcDeFgHiJkLmNoPq",
  "folders": [
    {
      "folder_id": "1AbCdEfGhIjKlMnOpQrStUv",
      "dataset": "patient-records"
    },
    {
      "folder_id": "2BcDeFgHiJkLmNoPqRsTuVw",
      "dataset": "lab-results"
    }
  ],
  "credential_ssm_param_path": "<secret-store-location>",
  "status": "active",
  "created_ts": "2026-07-15T10:30:00+00:00"
}

Credential setup: The response includes credential_ssm_param_path, the secure secret-store location where you must provision the service account key out of band. The platform never stores or returns the credential itself - only a reference to its storage location.

List Sources

GET /v1/{workspace_id}/intake/sources

Returns a paginated list of registered intake sources for the workspace.

Query Parameters

Parameter	Type	Default	Description
limit	integer	50	Maximum number of sources to return (1-200)
continuation_token	integer	0	Pagination offset
sort_by	string	-created_ts	Sort field and direction. Prefix with - for descending. Supported fields: created_ts, display_name

Response (200 OK)

Field	Type	Description
items	array	Array of source objects (same shape as the register response)
has_more	boolean	Whether more results are available
continuation_token	integer or null	Token to pass for the next page, if has_more is true

Trigger a Sync

POST /v1/{workspace_id}/intake/sources/{source_id}/sync

Runs a sync cycle for the source immediately: the platform scans the mapped folders, lands newly discovered files, and returns the resulting batches (201 Created).

Status	Condition
404	Unknown source ID
422	Source credential not yet provisioned to the secret store
502	External storage authentication failed

Authentication

All intake source endpoints require a valid API key with workspace access.

Batches

When a source syncs, files are grouped into batches. A batch represents the set of files discovered and processed together in a single sync cycle. Each batch tracks:

• The source and dataset it belongs to
• The number of files found
• Processing status: discovered, ready, processing, completed, or failed

Files landed by a source sync are linked to their batch. Files uploaded directly through the console or upload API or through an upload link have no batch association.