Chunk by Similarity strategy

The Chunk by Similarity strategy uses an embedding model to identify topically similar consecutive elements and combines them into chunks.

Type: chunk

Subtype: chunk_by_similarity

Usage guidance

Best used when your source documents have content that is unstructured but topically coherent.

For more information, see Chunk by similarity strategy.

Settings

unstructured_api_url · string

If specified, overrides the default API URL used for chunker calls. Default: none (uses Unstructured’s internal default).

unstructured_api_key · string

If specified, overrides the default API key used for chunker calls. Default: none (uses Unstructured’s internal default).

include_orig_elements · boolean

If true, the elements used to form a chunk appear in .metadata.orig_elements for that chunk. Default: false.

new_after_n_chars · integer

Soft maximum length of a chunk in characters. Closes a section after reaching approximately this length. Default: none.

max_characters · integer

Hard maximum number of characters in a chunk. Default: none.

overlap · integer

Number of trailing characters from the prior text-split chunk to prepend to each subsequent chunk formed by splitting an oversized element. Default: none.

overlap_all · boolean

If true, applies overlap to chunks formed by combining whole elements, not just oversized ones. Use with caution — this can introduce noise into otherwise clean semantic units. Default: false.

contextual_chunking_strategy · string · deprecated

Deprecated. The v1 contextual chunking option is no longer available in the workflow designer and is not recommended. To apply contextual chunking, add a Contextual Chunker node after this chunker. Allowed value: v1. Default: none.

similarity_threshold · number

Minimum similarity score required for consecutive elements to be combined into the same chunk. Must be between 0.0 and 1.0 exclusive (i.e., 0.01 to 0.99). Default: none.

Python SDK

chunk_by_similarity_chunker_workflow_node = WorkflowNode(
    name="Chunker",
    subtype="chunk_by_similarity",
    type="chunk",
    settings={
        "unstructured_api_url": None,
        "unstructured_api_key": None,
        "include_orig_elements": <True|False>,
        "new_after_n_chars": <new-after-n-chars>,
        "max_characters": <max-characters>,
        "overlap": <overlap>,
        "overlap_all": <True|False>,
        "contextual_chunking_strategy": "<contextual-chunking-strategy>",
        "similarity_threshold": <similarity-threshold>
    }
)

{
    "name": "Chunker",
    "type": "chunk",
    "subtype": "chunk_by_similarity",
    "settings": {
        "unstructured_api_url": null,
        "unstructured_api_key": null,
        "include_orig_elements": <true|false>,
        "new_after_n_chars": <new-after-n-chars>,
        "max_characters": <max-characters>,
        "overlap": <overlap>,
        "overlap_all": <true|false>,
        "contextual_chunking_strategy": "<contextual-chunking-strategy>",
        "similarity_threshold": <similarity-threshold>
    }
}