Skip to main content
POST
This endpoint creates a workflow that persists until it is explicitly deleted (a long-lived workflow). To create a workflow that exists only for the duration of a single job run using local files as input, use the create job endpoint instead.

Body

string
required
Workflow name.
string
required
Workflow type. Use custom to configure processing nodes. The other accepted values, basic, advanced, and platinum, are non-operational and will be removed in a future release.
Workflows with neither source_id nor source_ids use a local file source. A local-source workflow must use custom and cannot have a repeating schedule. Run these workflows through the API or Python SDK because Unstructured Pipelines cannot run them.
string
ID of the source connector.
string
ID of the destination connector.
array
Processing nodes. Each node requires name, type, and subtype, and can include an id and settings object. For a custom workflow with source or destination connectors, this field is required.For more information on workflow nodes, see Workflow nodes.
string
ID of a pre-built workflow template to use as the basis for the workflow.
string
Repeating run schedule. Valid values and their cron equivalents:If omitted, the workflow does not automatically run on a repeating schedule.Workflows with a local source cannot be set to run on a repeating schedule.
boolean
Default: false. If true, reprocesses all documents in the source location on every run. If false, the workflow excludes from future processing any files Unstructured determines are unchanged since the last time the workflow ran.Unstructured determines if a document has changed based on the document version. For each workflow, Unstructured maintains a record of documents (and their versions, if present) processed by that workflow. Each document record consists of:
  • A record_id derived from the document name and path.
  • A record_version derived from either the document Etag (if the source provider generates one) or the source provider’s native version identifier.
When you set reprocess_all to false for a source connector that supports reprocess_all, Unstructured uses this list of records to determine whether or not to process each document:
  • If the record_id does not exist in the workflow records, Unstructured processes the document.
  • If the record_id exists, but the record_version has changed, or there is no record_version, Unstructured processes the document.
The following table lists out the possible record-id and record_version combinations, and the action Unstructured takes in each case:
Renaming a document results in a new record_id; Unstructured will then reprocess the renamed document when the workflow runs.
The following table lists the source connectors that support the reprocess_all setting. The Record version base column specifies the versioning information Unstructured uses to generated the corresponding record version for each processed document.Source connectors that do not support reprocess_all reprocess every document in the source location each time the workflow runs.Additional considerations to take into account when setting reprocess_all to false:
  • Unstructured only adds document records for documents that it successfully processes. Documents that failed to process will be reprocessed the next time the workflow is run.
  • Because S3 ETags are content-based, changing the metadata on an S3 object will not result in it being reprocessed.
  • For source providers that support the S3 protocol, be aware that deleting an object and then reuploading it to the source location will maintain the same record_id, but may result in a different record_version being generated. This is especially true of multipart uploads. This results in Unstructured reprocessing the document.
  • For source providers that offer Key Management Services (KMS), be aware that server-side encryption can change document ETags. This results in the the record_version of a document changing, and Unstructured reprocessing the document.
  • If you clone or recreate a source connector, the resulting connector does not include the document processing history of the previous connector.
  • Changing a workflow’s configuration does not automatically result in Unstructured reprocessing all documents. For example, changing chunker, embedder, enrichment, or partitioner settings may not result in reprocessing all document. To reprocess all documents using new workflow settings, set reprocess_all to true for at least the next workflow run.
array
Source connector IDs for a workflow with multiple sources. Use either source_ids or source_id.
array
Destination connector IDs for a workflow with multiple destinations. Use either destination_ids or destination_id.
boolean
Skip the job preflight check for this workflow. Preflight runs by default.
string
Workspace-scoped workflow identifier. If omitted, the API derives it from name with a six-character suffix. You cannot change the key after creating the workflow.

Response

string
required
Unique identifier for the workflow.
string
required
Workflow name.
string
Workflow type, when available: custom, basic, advanced, or platinum.
string
required
Workflow state: active or inactive.
string
required
ISO 8601 timestamp when the workflow was created.
object
Repeating run schedule, with crontab_entries containing cron_expression values.
string
ISO 8601 timestamp when the workflow was last updated.
array
required
Source connector IDs bound to the workflow.
array
required
Destination connector IDs bound to the workflow.
array
required
Workflow processing pipeline nodes.
string
Workspace-scoped workflow identifier.
boolean
true reprocesses all source documents on every run. With false, supported source connectors can skip unchanged documents. Other connectors still reprocess all documents.
boolean
true skips job preflight checks. false uses the default preflight behavior.
object
Edit lock details while someone edits the workflow in the console. Updates and deletes by other users return HTTP 423 until the editor saves or the lock expires.