Execution Details COMPLETED
Documentation Search with AI Assistant
Execution Information
Execution ID: 5bb4fd71-ce9b-49e6-ae27-f18c6252a040
Workflow ID:
documentation_search_workflow
Status: COMPLETED
Started: 2025-08-14 11:23:05
Completed: N/A
Duration:
Quick Actions
Execution Inputs
{
"EXECUTION_ID": "5bb4fd71-ce9b-49e6-ae27-f18c6252a040",
"execution_id": "5bb4fd71-ce9b-49e6-ae27-f18c6252a040",
"search_query": "how to create a loop node and use it?"
}
Execution Outputs
Workflow Execution Summary
Mode: Distributed
Tasks: 3/3
Duration: N/A
Dependencies: No
Tasks: 3/3
Duration: N/A
Dependencies: No
Started: 2025-08-14 11:23:05
Ended: 2025-08-14 11:23:36
Status: SUCCESS
Ended: 2025-08-14 11:23:36
Status: SUCCESS
Task Results & Logs
ai_assistant
COMPLETED
AI Response (Live Stream)
Generating response...
Waiting for AI response...
This shows the AI response as it's being generated. The final result appears below after completion.
Duration:
N/A
Dependencies: No
Dependencies: No
Thread: N/A
Started:
Started:
vector_search
COMPLETED
Duration:
Calculating...
Dependencies: No
Dependencies: No
Thread: N/A
Started: 11:23:07
Started: 11:23:07
fetch_documentation
COMPLETED
Duration:
Calculating...
Dependencies: No
Dependencies: No
Thread: N/A
Started: 11:23:06
Started: 11:23:06
Complete Execution Output (JSON):
{
"ai_assistant": {
"agent_type": "assistant",
"execution_details": {
"agent_type": "assistant",
"input_length": 7301,
"model": "moonshotai/kimi-k2",
"output_length": 4529,
"provider": "openrouter",
"system_message": "You are a helpful AI assistant that answers questions about workflow engine documentation.\n\nYour role is to:\n1. Analyze the provided documentation search results\n2. Answer the user\u0027s question based on the relevant documentation content\n3. Provide clear, accurate, and helpful responses\n4. Include specific examples when possible\n5. Reference the documentation sections used in your answer\n\nIf the search results don\u0027t contain enough information to answer the question, \nsay so clearly and suggest what additional information might be needed.\n"
},
"input_format": "text",
"model_client_id": "openrouter_kimi",
"output": "## Creating and Using Loop Nodes in the Workflow Engine\n\nBased on the documentation, here\u0027s a comprehensive guide on how to create and use loop nodes:\n\n### 1. Basic Loop Node Structure\n\nA loop node is defined in YAML with the following essential components:\n\n**Required Configuration:**\n```yaml\n- id: financial_processing_loop\n type: loop\n loop_type: for # REQUIRED: \u0027for\u0027 or \u0027while\u0027\n max_iterations: 3 # REQUIRED for \u0027for\u0027 loops\n iteration_variable: company_index # Optional: defaults to \u0027iteration\u0027\n```\n\n**Supported Loop Types:**\n- `loop_type: for` - Fixed iterations (documentation section: content.chunk_178)\n- `loop_type: while` - Conditional loops\n\n### 2. Complete Loop Node Example\n\nHere\u0027s a comprehensive example showing all the pieces working together:\n\n```yaml\n- id: financial_processing_loop\n type: loop\n loop_type: for\n max_iterations: 3\n iteration_variable: company_index\n depends_on: [list_companies] # External dependency to previous task\n \n # State variables to maintain across iterations\n state_variables:\n processed_companies: []\n total_extractions: 0\n successful_extractions: 0\n \n # Tasks that run in each iteration\n loop_tasks:\n - id: process_current_company\n type: script\n script: |\n # Access array data from external task\n companies_data = inputs.get(\u0027list_companies\u0027, {})\n company_idx = inputs.get(\u0027company_index\u0027, 0)\n \n # Access loop state variables\n processed = loop_state.get(\u0027processed_companies\u0027, [])\n \n # Process current company\n company = companies_data[\u0027companies\u0027][company_idx]\n result = process_company(company)\n \n outputs = {\"company_name\": company[\u0027name\u0027], \"status\": \"success\"}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n*(Documentation reference: content.chunk_62 - Complete Loop Task Example)*\n\n### 3. Data Access Within Loop Tasks\n\nLoop tasks have special data access patterns:\n\n- **Loop state variables** are accessed via injected context files\n- **External task outputs** are automatically flattened and available\n- **Iteration variables** are injected from the loop scope\n\nExample of accessing data within a loop task:\n```yaml\nloop_tasks:\n - id: debug_loop_task\n type: script\n script: |\n # Debug: Print all available data\n print(f\"Available inputs: {list(inputs.keys())}\")\n print(f\"Loop state keys: {list(loop_state.keys())}\")\n \n # Access loop state variables\n processed_items = loop_state.get(\u0027processed_items\u0027, [])\n \n # Access external task data (flattened)\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\n```\n\n*(Documentation reference: content.chunk_68 - Debugging Loop Task Data Access)*\n\n### 4. Context Injection\n\nEach loop task automatically receives context through temporary files:\n\n```python\nimport json\nimport os\n\n# Auto-injected at the start of each loop task script:\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\n inputs = json.load(f) # Contains workflow data + flattened previous outputs\n \nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\n loop_state = json.load(f) # Contains current state variables\n\noutputs = {}\nstate_updates = {}\n```\n\n*(Documentation reference: content.chunk_67 - Loop Task Context Injection and Output Flattening)*\n\n### 5. Practical Loop Patterns\n\n**For loops with arrays from previous tasks:**\n```yaml\n- id: process_items_loop\n type: loop\n loop_type: for\n max_iterations: 5\n state_variables:\n processed_count: 0\n \n loop_tasks:\n - id: process_individual_item\n type: script\n script: |\n items = inputs.get(\u0027fetch_items\u0027, {}).get(\u0027data\u0027, [])\n current_idx = inputs.get(\u0027iteration\u0027, 0)\n \n item = items[current_idx]\n processed_item = {\n \"id\": item[\"id\"],\n \"processed_at\": \"${current_timestamp}\",\n \"status\": \"processed\"\n }\n \n outputs = {\"processed_item\": processed_item}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n*(Documentation reference: content.chunk_81 - Loop through arrays from previous tasks)*\n\n### Key Points to Remember:\n\n1. **State variables** are automatically managed across iterations\n2. **External dependencies** must be specified with `depends_on`\n3. **Iteration variables** are injected into each task automatically\n4. **Outputs are auto-flattened** for downstream tasks\n5. Use **debug printing** to understand available data structure\n__OUTPUTS__ {\"ai_response\": \"## Creating and Using Loop Nodes in the Workflow Engine\\n\\nBased on the documentation, here\u0027s a comprehensive guide on how to create and use loop nodes:\\n\\n### 1. Basic Loop Node Structure\\n\\nA loop node is defined in YAML with the following essential components:\\n\\n**Required Configuration:**\\n```yaml\\n- id: financial_processing_loop\\n type: loop\\n loop_type: for # REQUIRED: \u0027for\u0027 or \u0027while\u0027\\n max_iterations: 3 # REQUIRED for \u0027for\u0027 loops\\n iteration_variable: company_index # Optional: defaults to \u0027iteration\u0027\\n```\\n\\n**Supported Loop Types:**\\n- `loop_type: for` - Fixed iterations (documentation section: content.chunk_178)\\n- `loop_type: while` - Conditional loops\\n\\n### 2. Complete Loop Node Example\\n\\nHere\u0027s a comprehensive example showing all the pieces working together:\\n\\n```yaml\\n- id: financial_processing_loop\\n type: loop\\n loop_type: for\\n max_iterations: 3\\n iteration_variable: company_index\\n depends_on: [list_companies] # External dependency to previous task\\n \\n # State variables to maintain across iterations\\n state_variables:\\n processed_companies: []\\n total_extractions: 0\\n successful_extractions: 0\\n \\n # Tasks that run in each iteration\\n loop_tasks:\\n - id: process_current_company\\n type: script\\n script: |\\n # Access array data from external task\\n companies_data = inputs.get(\u0027list_companies\u0027, {})\\n company_idx = inputs.get(\u0027company_index\u0027, 0)\\n \\n # Access loop state variables\\n processed = loop_state.get(\u0027processed_companies\u0027, [])\\n \\n # Process current company\\n company = companies_data[\u0027companies\u0027][company_idx]\\n result = process_company(company)\\n \\n outputs = {\\\"company_name\\\": company[\u0027name\u0027], \\\"status\\\": \\\"success\\\"}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n*(Documentation reference: content.chunk_62 - Complete Loop Task Example)*\\n\\n### 3. Data Access Within Loop Tasks\\n\\nLoop tasks have special data access patterns:\\n\\n- **Loop state variables** are accessed via injected context files\\n- **External task outputs** are automatically flattened and available\\n- **Iteration variables** are injected from the loop scope\\n\\nExample of accessing data within a loop task:\\n```yaml\\nloop_tasks:\\n - id: debug_loop_task\\n type: script\\n script: |\\n # Debug: Print all available data\\n print(f\\\"Available inputs: {list(inputs.keys())}\\\")\\n print(f\\\"Loop state keys: {list(loop_state.keys())}\\\")\\n \\n # Access loop state variables\\n processed_items = loop_state.get(\u0027processed_items\u0027, [])\\n \\n # Access external task data (flattened)\\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n```\\n\\n*(Documentation reference: content.chunk_68 - Debugging Loop Task Data Access)*\\n\\n### 4. Context Injection\\n\\nEach loop task automatically receives context through temporary files:\\n\\n```python\\nimport json\\nimport os\\n\\n# Auto-injected at the start of each loop task script:\\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\\n inputs = json.load(f) # Contains workflow data + flattened previous outputs\\n \\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\\n loop_state = json.load(f) # Contains current state variables\\n\\noutputs = {}\\nstate_updates = {}\\n```\\n\\n*(Documentation reference: content.chunk_67 - Loop Task Context Injection and Output Flattening)*\\n\\n### 5. Practical Loop Patterns\\n\\n**For loops with arrays from previous tasks:**\\n```yaml\\n- id: process_items_loop\\n type: loop\\n loop_type: for\\n max_iterations: 5\\n state_variables:\\n processed_count: 0\\n \\n loop_tasks:\\n - id: process_individual_item\\n type: script\\n script: |\\n items = inputs.get(\u0027fetch_items\u0027, {}).get(\u0027data\u0027, [])\\n current_idx = inputs.get(\u0027iteration\u0027, 0)\\n \\n item = items[current_idx]\\n processed_item = {\\n \\\"id\\\": item[\\\"id\\\"],\\n \\\"processed_at\\\": \\\"${current_timestamp}\\\",\\n \\\"status\\\": \\\"processed\\\"\\n }\\n \\n outputs = {\\\"processed_item\\\": processed_item}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n*(Documentation reference: content.chunk_81 - Loop through arrays from previous tasks)*\\n\\n### Key Points to Remember:\\n\\n1. **State variables** are automatically managed across iterations\\n2. **External dependencies** must be specified with `depends_on`\\n3. **Iteration variables** are injected into each task automatically\\n4. **Outputs are auto-flattened** for downstream tasks\\n5. Use **debug printing** to understand available data structure\", \"model_client_id\": \"openrouter_kimi\", \"agent_type\": \"assistant\", \"model\": \"moonshotai/kimi-k2\", \"provider\": \"openrouter\", \"status\": \"completed\", \"token_usage\": {\"prompt_tokens\": 2028, \"completion_tokens\": 1002, \"total_tokens\": 3030, \"estimated_cost\": {\"amount\": 0.12096, \"currency\": \"USD\"}, \"model\": \"moonshotai/kimi-k2\", \"provider\": \"openrouter\"}}",
"output_format": "text",
"return_code": 0,
"status": "completed",
"stderr": "",
"task_id": "ai_assistant",
"token_usage": {
"completion_tokens": 1002,
"estimated_cost": {
"amount": 0.12096,
"currency": "USD"
},
"model": "moonshotai/kimi-k2",
"prompt_tokens": 2028,
"provider": "openrouter",
"total_tokens": 3030
}
},
"execution_summary": {
"completed_tasks": 3,
"dependencies_detected": false,
"end_time": "2025-08-14T11:23:36.444342",
"execution_mode": "distributed",
"start_time": "2025-08-14T11:23:05.738262",
"total_tasks": 3
},
"fetch_documentation": {
"documentation": {
"content": "# YAML Workflow Structure Reference\n\nComplete reference for the YAML workflow definition structure supported by the Assistants Workflow Engine.\n\n## Table of Contents\n\n1. [Complete Schema Overview](#complete-schema-overview)\n2. [Required vs Optional Fields](#required-vs-optional-fields)\n3. [Data Types Reference](#data-types-reference)\n4. [Field Validation Rules](#field-validation-rules)\n5. [Parser Behavior](#parser-behavior)\n6. [Migration Guide](#migration-guide)\n\n## Complete Schema Overview\n\n```yaml\n# === WORKFLOW METADATA (Required) ===\nid: string # REQUIRED: Unique workflow identifier\nname: string # REQUIRED: Human-readable workflow name\n\n# === WORKFLOW METADATA (Optional) ===\nversion: string | integer # Default: 1\nnamespace: string # Default: \"default\"\ndescription: string # Default: \"\"\ntenant_id: string # Default: null\n\n# === EXECUTION CONFIGURATION ===\ntimeout_seconds: integer|string # Default: 3600 (1 hour) - Workflow-level timeout (supports duration strings like \"1h\", \"30m\")\nlabels: object # Default: {}\nmetadata: object # Default: {}\n\n# === DATA FLOW DEFINITIONS ===\ninputs: # Default: []\n - name: string # REQUIRED\n type: string # REQUIRED: string|integer|boolean|object|array\n description: string # Optional\n required: boolean # Default: false\n default: any # Optional default value\n\noutputs: object # Default: {}\n output_name:\n type: string # REQUIRED\n source: string # REQUIRED: task_id or task_id.field\n description: string # Optional\n\nvariables: object # Default: {}\n\n# === MODEL CLIENT CONFIGURATION (Required for AI tasks) ===\nmodel_clients: # REQUIRED: For ai_agent, chatbot, continuous_conversation tasks\n - id: string # REQUIRED: Unique identifier to reference in tasks\n provider: string # REQUIRED: openai|anthropic|openrouter|groq\n config: object # REQUIRED: Provider-specific config\n model: string # REQUIRED: Model name (e.g., gpt-4, claude-3-sonnet-20240229)\n api_key: string # REQUIRED: API key for the provider\n temperature: number # Optional: 0.0-2.0, controls randomness\n max_tokens: integer # Optional: Maximum tokens to generate\n timeout: integer # Optional: Request timeout in seconds\n base_url: string # Optional: Custom API base URL\n\n# Model Client Examples:\n# OpenAI Configuration:\n# model_clients:\n# - id: gpt4_model\n# provider: openai\n# config:\n# model: gpt-4-turbo-preview\n# api_key: \"sk-your-openai-api-key\"\n# temperature: 0.7\n# max_tokens: 2000\n#\n# Anthropic Configuration:\n# model_clients:\n# - id: claude_model\n# provider: anthropic\n# config:\n# model: claude-3-sonnet-20240229\n# api_key: \"sk-ant-your-anthropic-api-key\"\n# temperature: 0.7\n# max_tokens: 1500\n#\n# Multiple Models:\n# model_clients:\n# - id: gpt4_model\n# provider: openai\n# config:\n# model: gpt-4-turbo-preview\n# api_key: \"sk-your-openai-api-key\"\n# - id: claude_model\n# provider: anthropic\n# config:\n# model: claude-3-sonnet-20240229\n# api_key: \"sk-ant-your-anthropic-api-key\"\n\n# === TASK DEFINITIONS (Required) ===\ntasks: # REQUIRED: List of tasks\n - id: string # REQUIRED: Unique task identifier\n name: string # Optional: Human-readable name\n type: string # REQUIRED: Task type\n description: string # Optional\n \n # DEPENDENCY CONFIGURATION\n depends_on: array # Optional: Execution dependencies and template resolution\n previous_node: string # REQUIRED: Task connectivity and UI hierarchy (essential for proper workflow execution)\n \n timeout_seconds: integer|string # Optional: Task-specific timeout (supports duration strings like \"5m\", \"30s\")\n worker_group: string # Optional: Target worker group\n conditions: array # Optional: Execution conditions\n execute_on_routes: array # Optional: Routes for conditional execution\n \n # Input/Output mapping\n input_mapping: object # Optional: Map workflow context to task inputs\n output_mapping: object # Optional: Map task outputs to workflow context\n \n # Retry configuration\n retry: object # Optional: Task-specific retry policy\n maxAttempts: integer # Default: 1\n initialDelay: string|integer # Default: \"1s\"\n maxDelay: string|integer # Default: \"300s\"\n backoffMultiplier: number # Default: 2.0\n retryOn: array # Default: [\"TEMPORARY_FAILURE\"]\n \n # === TASK TYPE SPECIFIC CONFIGURATIONS ===\n # See detailed sections below for each task type\n\n# === WORKFLOW-LEVEL RETRY POLICY ===\nretry: object # Optional: Global retry policy\n retryOn: array # Default: [\"TEMPORARY_FAILURE\"]\n maxAttempts: integer # Default: 1\n initialDelay: string|integer # Default: \"1s\"\n maxDelay: string|integer # Default: \"300s\"\n backoffMultiplier: number # Default: 2.0\n\n# === TRIGGERS ===\ntriggers: # Optional: List of triggers\n - name: string # REQUIRED\n type: string # REQUIRED: webhook|schedule|event\n description: string # Optional\n config: object # REQUIRED: Trigger-specific config\n input_mapping: array # Optional: Map trigger data to workflow inputs\n - webhook_field: string # Source field\n workflow_input: string # Target workflow input\n```\n\n## Required vs Optional Fields\n\n### Required Fields (Workflow Level)\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `id` | string | Unique identifier for the workflow |\n| `tasks` | array | List of task definitions (can be empty) |\n\n### Required Fields (Task Level)\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `id` | string | Unique identifier for the task |\n| `type` | string | Task type (script, http, ai_agent, etc.) |\n| `previous_node` | string | Reference to previous task for UI connectivity (required for proper task linking, except for the first task) |\n\n### Required Fields (Input Definition)\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `name` | string | Input parameter name |\n| `type` | string | Data type (string, integer, boolean, object, array) |\n\n### Required Fields (Output Definition)\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `type` | string | Output data type |\n| `source` | string | Source task ID or task_id.field_name |\n\n## Data Types Reference\n\n### Primitive Types\n\n```yaml\n# String type\ninputs:\n - name: customer_name\n type: string\n default: \"John Doe\"\n description: \"Customer full name\"\n\n# Integer type \ninputs:\n - name: batch_size\n type: integer\n default: 100\n description: \"Number of records per batch\"\n\n# Boolean type\ninputs:\n - name: include_archived\n type: boolean\n default: false\n description: \"Include archived records\"\n```\n\n### Complex Types\n\n```yaml\n# Object type\ninputs:\n - name: config_params\n type: object\n default:\n timeout: 30\n retries: 3\n debug: false\n description: \"Configuration parameters\"\n\n# Array type\ninputs:\n - name: customer_ids\n type: array\n default: [\"cust_001\", \"cust_002\"]\n description: \"List of customer identifiers\"\n```\n\n### Duration Types\n\nDuration values can be specified as strings or integers:\n\n```yaml\n# String format (recommended)\ntimeout_seconds: \"1h\" # 1 hour\ntimeout_seconds: \"30m\" # 30 minutes \ntimeout_seconds: \"45s\" # 45 seconds\ntimeout_seconds: \"2d\" # 2 days\n\n# Integer format (seconds)\ntimeout_seconds: 3600 # 1 hour in seconds\n\n# Supported units\n# s, sec, second, seconds\n# m, min, minute, minutes \n# h, hour, hours\n# d, day, days\n```\n\n### Timeout Configuration and Execution Behavior\n\nThe workflow engine supports configurable timeouts at both workflow and task levels with intelligent timeout-free communication for long-running tasks.\n\n#### Timeout Hierarchy\n\nTimeout values are resolved in this order of precedence:\n1. **Task-level `timeout_seconds`**: Specific timeout for individual tasks\n2. **Workflow-level `timeout_seconds`**: Default for all tasks in the workflow\n3. **System default**: 300 seconds (5 minutes) for tasks, 3600 seconds (1 hour) for workflows\n\n```yaml\n# Workflow-level timeout (applies to all tasks by default)\ntimeout_seconds: \"2h\" # 2 hour workflow timeout\n\ntasks:\n - id: quick_task\n type: script\n # Uses workflow-level timeout (2h)\n script: \"print(\u0027Quick task\u0027)\"\n \n - id: custom_timeout_task\n type: script\n timeout_seconds: \"30m\" # Override with 30 minute timeout\n script: \"print(\u0027Custom timeout task\u0027)\"\n \n - id: long_running_task\n type: script\n timeout_seconds: 600 # 10 minutes (integer format)\n script: |\n import time\n time.sleep(300) # 5 minute processing\n print(\"Long task completed\")\n```\n\n#### Timeout-Free Communication\n\nThe workflow engine automatically uses **timeout-free communication** for long-running tasks to prevent timeout-related failures:\n\n- **Loop tasks**: Always use timeout-free communication (event-driven execution)\n- **Script tasks**: Use timeout-free communication when `timeout_seconds \u003e 180` (host timeout threshold)\n- **Other task types**: Use standard timeout-based communication\n\n```yaml\ntasks:\n - id: short_script\n type: script\n timeout_seconds: 60 # Uses standard timeout-based communication\n script: \"print(\u0027Short script\u0027)\"\n \n - id: long_script\n type: script\n timeout_seconds: 600 # Uses timeout-free communication (600 \u003e 180)\n script: |\n import time\n print(\"\ud83d\ude80 This uses timeout-free communication\")\n time.sleep(300) # Won\u0027t cause timeout issues\n print(\"Long script completed\")\n \n - id: data_processing_loop\n type: loop\n timeout_seconds: \"1h\" # Always uses timeout-free communication\n loop_type: for\n max_iterations: 10\n # Loop tasks automatically use timeout-free communication\n```\n\n#### Loop Task Timeout Inheritance\n\nLoop tasks can pass timeout configuration to their inner script tasks:\n\n```yaml\n- id: batch_processing_loop\n type: loop\n timeout_seconds: \"30m\" # Loop-level timeout\n loop_type: for\n max_iterations: 5\n loop_tasks:\n - id: process_batch\n type: script\n # Inherits 30-minute timeout from parent loop\n script: |\n print(\"Processing batch with inherited timeout\")\n # This script will use the 30-minute timeout from the loop\n```\n\n#### Best Practices for Timeout Configuration\n\n1. **Use duration strings** for readability: `\"5m\"` vs `300`\n2. **Set appropriate timeouts** based on expected execution time plus buffer\n3. **Long-running tasks**: Set `timeout_seconds \u003e 180` to enable timeout-free communication\n4. **Loop tasks**: Always get timeout-free communication automatically\n5. **Critical tasks**: Use shorter timeouts for fail-fast behavior\n6. **External API calls**: Use longer timeouts to account for network latency\n\n```yaml\n# Example: Well-configured timeout workflow\ntimeout_seconds: \"1h\" # Reasonable workflow timeout\n\ntasks:\n - id: api_call\n type: http\n timeout_seconds: \"30s\" # Quick timeout for API calls\n url: \"https://api.example.com/data\"\n \n - id: data_processing\n type: script\n timeout_seconds: \"10m\" # Longer timeout for processing (uses timeout-free)\n script: |\n # Process large dataset\n process_data()\n \n - id: batch_processing\n type: loop\n timeout_seconds: \"45m\" # Generous timeout for loop (timeout-free)\n loop_type: for\n max_iterations: 100\n # Inner tasks inherit the 45-minute timeout\n```\n\n## Field Validation Rules\n\n### ID Fields\n\n```yaml\n# Valid IDs\nid: \"customer_processor\" # \u2705 Valid\nid: \"process-data\" # \u2705 Valid\nid: \"task_001\" # \u2705 Valid\n\n# Invalid IDs\nid: \"process data\" # \u274c Spaces not allowed\nid: \"123_start\" # \u274c Cannot start with number\nid: \"\" # \u274c Cannot be empty\n```\n\n### Timeout Values\n\n```yaml\n# Valid timeouts\ntimeout_seconds: 300 # \u2705 Positive integer\ntimeout_seconds: \"5m\" # \u2705 Duration string\ntimeout_seconds: \"1h30m\" # \u2705 Complex duration\n\n# Invalid timeouts\ntimeout_seconds: 0 # \u274c Must be positive\ntimeout_seconds: -60 # \u274c Cannot be negative\ntimeout_seconds: \"invalid\" # \u274c Invalid duration format\n```\n\n### Dependency Types and Validation\n\n#### `depends_on` vs `previous_node`\n\n**`depends_on` - Execution Dependencies:**\n- Controls actual task execution order\n- Enables template resolution and data access\n- Required for accessing outputs from other tasks\n- Validates that referenced tasks exist\n- Multiple tasks can be specified\n\n**`previous_node` - Task Connectivity and UI Hierarchy:**\n- **REQUIRED for proper task connectivity in UI**\n- Essential for workflow visualization and task linking\n- Critical for execution flow representation\n- Single task reference only\n- **Without this field, tasks won\u0027t be properly connected in the UI**\n\n```yaml\ntasks:\n - id: fetch_data\n type: http\n url: \"https://api.example.com/data\"\n \n - id: validate_data\n type: script\n depends_on: [fetch_data] # \u2705 EXECUTION: wait for fetch_data to complete\n previous_node: fetch_data # \u2705 UI: show as child of fetch_data in hierarchy\n script: |\n data = ${fetch_data} # \u2705 Can access because of depends_on\n if not validate(data):\n raise ValueError(\"Invalid data\")\n \n # Required: Use outputs dict and __OUTPUTS__ marker for proper parsing\n outputs[\u0027status\u0027] = \u0027valid\u0027\n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n \n - id: process_data\n type: script\n depends_on: [validate_data] # \u2705 EXECUTION: wait for validation\n previous_node: validate_data # \u2705 UI: show as child of validate_data\n script: |\n validation_result = ${validate_data} # \u2705 Accessible via depends_on\n # Process data...\n\n - id: separate_task\n type: script\n previous_node: process_data # \u2705 UI: appears under process_data in hierarchy\n # NO depends_on means this can run immediately (parallel to other tasks)\n script: |\n # This task runs independently but appears under process_data in UI\n # Cannot access ${process_data} because no depends_on relationship\n print(\"Independent task\")\n\n### Complete Example with Proper Hierarchy\n\n```yaml\ntasks:\n - id: fetch_customer_data\n name: \"Fetch Customer Data\"\n type: http\n url: \"https://api.example.com/customers/${customer_id}\"\n \n - id: validate_customer\n name: \"Validate Customer Data\"\n type: script\n depends_on: [fetch_customer_data] # \u2705 EXECUTION: Wait for data fetch\n previous_node: fetch_customer_data # \u2705 UI: Show under fetch_customer_data\n script: |\n customer = ${fetch_customer_data} # \u2705 Can access because of depends_on\n if not customer.get(\u0027email\u0027):\n raise ValueError(\"Invalid customer data\")\n print(\"__OUTPUTS__ {\u0027status\u0027: \u0027valid\u0027}\")\n \n - id: process_customer\n name: \"Process Customer\"\n type: ai_agent\n depends_on: [validate_customer] # \u2705 EXECUTION: Wait for validation\n previous_node: validate_customer # \u2705 UI: Show under validate_customer\n config:\n model_client_id: gpt4_model\n user_message: \"Process customer: ${fetch_customer_data}\"\n \n - id: send_notification\n name: \"Send Notification\"\n type: script\n depends_on: [process_customer] # \u2705 EXECUTION: Wait for processing\n previous_node: process_customer # \u2705 UI: Show under process_customer\n script: |\n result = ${process_customer}\n print(f\"Notification sent for: ${fetch_customer_data.name}\")\n \n - id: parallel_audit_log\n name: \"Audit Log (Parallel)\"\n type: script\n depends_on: [fetch_customer_data] # \u2705 EXECUTION: Only needs initial data\n previous_node: fetch_customer_data # \u2705 UI: Show under fetch_customer_data\n script: |\n # This runs in parallel with validation/processing\n customer = ${fetch_customer_data}\n print(f\"Audit: Customer {customer.id} accessed at ${current_timestamp}\")\n```\n\n # Invalid examples\n - id: invalid_task\n type: script\n depends_on: [nonexistent_task] # \u274c VALIDATION ERROR: task doesn\u0027t exist\n previous_node: another_missing # \u26a0\ufe0f UI WARNING: task doesn\u0027t exist (but not fatal)\n script: \"print(\u0027Invalid\u0027)\"\n```\n\n#### Key Differences:\n\n| Aspect | `depends_on` | `previous_node` |\n|--------|--------------|-----------------|\n| **Purpose** | Execution control | Task connectivity \u0026 UI organization |\n| **Impact** | Blocks task until dependencies complete | **REQUIRED for UI task linking** |\n| **Template Access** | \u2705 Enables `${task_id}` resolution | \u274c No template access |\n| **Validation** | \u274c Parser error if task missing | \u26a0\ufe0f Warning if task missing |\n| **Multiple References** | \u2705 Array of task IDs | \u274c Single task ID only |\n| **Execution Order** | \u2705 Enforced | \u274c No execution impact |\n| **UI Connectivity** | \u274c No UI impact | \u2705 **ESSENTIAL for proper task connection** |\n\n### Circular Dependency Detection\n\n```yaml\n# This will be rejected by the parser\ntasks:\n - id: task_a\n depends_on: [task_b] # \u274c Circular dependency\n \n - id: task_b \n depends_on: [task_a] # \u274c Circular dependency\n```\n\n## Parser Behavior\n\n### Field Name Compatibility\n\nThe parser supports both camelCase and snake_case for field names:\n\n```yaml\ntasks:\n - id: flexible_task\n # Both formats are accepted\n dependsOn: [other_task] # camelCase\n depends_on: [other_task] # snake_case\n \n inputMapping: # camelCase\n source_field: \"target_field\"\n input_mapping: # snake_case \n source_field: \"target_field\"\n \n outputMapping: {} # camelCase\n output_mapping: {} # snake_case\n \n workerGroup: \"gpu-workers\" # camelCase\n worker_group: \"gpu-workers\" # snake_case\n```\n\n### Default Value Assignment\n\n```yaml\n# Fields with defaults are automatically populated\nid: my_workflow\n# name defaults to id value: \"my_workflow\"\n# version defaults to: 1\n# namespace defaults to: \"default\" \n# timeout_seconds defaults to: 3600\n# description defaults to: \"\"\n\ntasks:\n - id: simple_task\n type: script\n # name defaults to id: \"simple_task\"\n # timeout_seconds defaults to: 300\n # retry.maxAttempts defaults to: 1\n```\n\n### Template Resolution Context\n\nThe parser provides these built-in variables for template resolution:\n\n```yaml\ntasks:\n - id: context_aware_task\n type: script\n script: |\n # Workflow context\n workflow_id = \"${workflow_id}\" # Current workflow ID\n execution_id = \"${execution_id}\" # Unique execution ID\n namespace = \"${namespace}\" # Workflow namespace\n \n # Timing context\n current_timestamp = \"${current_timestamp}\" # ISO timestamp\n started_at = \"${started_at}\" # Workflow start time\n \n # Input context\n user_input = \"${input_parameter}\" # Workflow inputs\n \n # Task context\n previous_result = ${previous_task} # Previous task output\n specific_field = ${task_id.field_name} # Specific output field\n \n # Route context (for conditional workflows)\n route_info = ${_route_context} # Current route information\n```\n\n### Error Handling During Parsing\n\n```yaml\n# The parser will validate and report these errors:\n\n# 1. Missing required fields\n# ERROR: Missing required field: \u0027id\u0027\n\n# 2. Invalid YAML syntax \n# ERROR: Invalid YAML: mapping values are not allowed here\n\n# 3. Circular dependencies\n# ERROR: Circular dependency detected involving task \u0027task_a\u0027\n\n# 4. Invalid task references\n# ERROR: Task \u0027task_b\u0027 depends on non-existent task \u0027invalid_task\u0027\n\n# 5. Invalid duration formats\n# ERROR: Invalid duration: \u0027not_a_duration\u0027\n\n# 6. Invalid field values\n# ERROR: Timeout must be positive, got -10\n```\n\n## Migration Guide\n\n### From Version 1.0 to 2.0\n\n```yaml\n# OLD FORMAT (v1.0)\nworkflow:\n id: my_workflow\n tasks:\n - task_id: process_data\n task_type: python_script\n \n# NEW FORMAT (v2.0) \nid: my_workflow\ntasks:\n - id: process_data\n type: script\n language: python\n```\n\n### Deprecated Fields\n\n| Old Field | New Field | Status |\n|-----------|-----------|--------|\n| `task_type` | `type` | Deprecated in v2.0 |\n| `task_id` | `id` | Deprecated in v2.0 |\n| `python_script` | `script` (type) | Deprecated in v2.0 |\n| `plugin` | `mcp` (type) | Deprecated in v2.1 |\n\n### Backward Compatibility\n\nThe parser maintains backward compatibility for deprecated fields but will show warnings:\n\n```yaml\n# This will work but show a warning\ntasks:\n - task_id: old_style_task # \u26a0\ufe0f Warning: Use \u0027id\u0027 instead\n task_type: python_script # \u26a0\ufe0f Warning: Use \u0027type: script\u0027 instead\n```\n\n## Advanced Schema Features\n\n### Conditional Field Requirements\n\nSome fields become required based on task type:\n\n```yaml\n# HTTP tasks require URL\n- id: api_call\n type: http\n url: \"https://api.example.com\" # \u2705 Required for HTTP tasks\n \n# AI agent tasks require model_client_id\n- id: ai_task\n type: ai_agent\n config:\n model_client_id: gpt4 # \u2705 Required for AI agent tasks\n```\n\n### Dynamic Field Validation\n\nField validation can depend on other field values:\n\n```yaml\n# Loop tasks require loop-specific fields\n- id: data_loop\n type: loop\n loop_type: \"for\" # Determines required fields\n max_iterations: 10 # \u2705 Required for \u0027for\u0027 loops\n # exit_condition: not needed for \u0027for\u0027 loops\n \n- id: condition_loop \n type: loop\n loop_type: \"while\" # Determines required fields\n exit_condition: \"${count} \u003e= 10\" # \u2705 Required for \u0027while\u0027 loops\n # max_iterations: optional for \u0027while\u0027 loops\n```\n\n## Template Resolution and Data Flow\n\nTemplate resolution is a core feature that allows tasks to access data from previous tasks, workflow inputs, and built-in variables using the `${variable_name}` syntax. This is essential for creating dynamic workflows where tasks pass data to each other.\n\n### Basic Template Syntax\n\n```yaml\n# Access workflow inputs\nscript: |\n customer_name = \"${customer_name}\" # From workflow inputs\n \n# Access previous task outputs\nscript: |\n previous_result = ${data_fetch_task} # Entire output from data_fetch_task\n specific_field = ${data_fetch_task.customer_id} # Specific field\n \n# Access built-in variables\nscript: |\n workflow_id = \"${workflow_id}\"\n execution_id = \"${execution_id}\"\n timestamp = \"${current_timestamp}\"\n```\n\n### Available Built-in Variables\n\n| Variable | Description | Example Value |\n|----------|-------------|---------------|\n| `${workflow_id}` | Current workflow identifier | `customer_processor` |\n| `${execution_id}` | Unique execution identifier | `exec_12345_abcde` |\n| `${namespace}` | Workflow namespace | `production` |\n| `${current_timestamp}` | ISO timestamp | `2024-01-15T10:30:00Z` |\n| `${started_at}` | Workflow start time | `2024-01-15T10:29:45Z` |\n\n### Task Output Resolution\n\nTasks can access outputs from other tasks they depend on:\n\n```yaml\ntasks:\n - id: fetch_customer\n type: http\n url: \"https://api.example.com/customers/${customer_id}\"\n \n - id: process_customer\n type: script\n depends_on: [fetch_customer] # Required for template access\n previous_node: fetch_customer\n script: |\n # Access entire response\n customer_data = ${fetch_customer}\n \n # Access specific fields\n customer_name = \"${fetch_customer.name}\"\n customer_email = \"${fetch_customer.email}\"\n \n # Process the data\n result = {\n \"processed_name\": customer_name.upper(),\n \"domain\": customer_email.split(\"@\")[1]\n }\n print(f\"__OUTPUTS__ {json.dumps(result)}\")\n```\n\n### AI Agent Output Resolution\n\nAI agent tasks return structured output that requires specific access patterns:\n\n```yaml\n# AI Agent task\n- id: ai_analysis_task\n type: ai_agent\n config:\n model_client_id: gpt4_model\n system_message: \"Analyze the customer data and provide insights.\"\n user_message: \"Customer data: ${customer_data}\"\n\n# Accessing AI agent output in subsequent tasks\n- id: process_ai_response\n type: script\n depends_on: [ai_analysis_task]\n previous_node: ai_analysis_task\n script: |\n # CORRECT: AI agents return structured output\n agent_output = ${ai_analysis_task}\n ai_response = agent_output[\u0027ai_response\u0027].strip()\n \n # Additional metadata available\n token_usage = agent_output.get(\u0027token_usage\u0027, {})\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\n \n # Process the AI response\n processed_analysis = {\n \"analysis\": ai_response,\n \"tokens_used\": token_usage.get(\u0027total_tokens\u0027, 0),\n \"model\": model_used,\n \"processed_at\": \"${current_timestamp}\"\n }\n \n print(f\"__OUTPUTS__ {json.dumps(processed_analysis)}\")\n\n# INCORRECT: Direct access won\u0027t work\n- id: incorrect_ai_access\n type: script\n depends_on: [ai_analysis_task]\n script: |\n # \u274c This will not work correctly\n ai_response = ${ai_analysis_task} # Returns full agent object\n # ai_response is now a dict, not the actual response text\n```\n\n### AI Agent Response Structure\n\nAI agent tasks return a structured object with the following format:\n\n```json\n{\n \"ai_response\": \"The actual AI-generated text response\",\n \"model\": \"gpt-4-turbo-preview\",\n \"token_usage\": {\n \"prompt_tokens\": 150,\n \"completion_tokens\": 75,\n \"total_tokens\": 225\n },\n \"estimated_cost\": {\n \"amount\": 0.00337,\n \"currency\": \"USD\"\n },\n \"provider\": \"openai\",\n \"timestamp\": \"2024-01-15T10:30:00Z\"\n}\n```\n\n### Loop Task Data Access and Dependencies\n\nLoop tasks have a special execution context that differs from regular workflow tasks. Understanding how to access data within loops is crucial for building functional workflows.\n\n#### Loop Task Internal Data Access\n\nWithin loop tasks, data access works through injected context files rather than template resolution:\n\n```yaml\n# Loop task with state variables and internal task dependencies\n- id: data_processing_loop\n type: loop\n loop_type: for\n max_iterations: 3\n state_variables:\n processed_items: []\n total_processed: 0\n error_count: 0\n loop_tasks:\n # First task in loop\n - id: fetch_data\n type: script\n script: |\n # Access loop state variables through injected context\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\n processed_items = loop_state.get(\u0027processed_items\u0027, [])\n \n # Access workflow inputs and external task outputs\n external_data = inputs.get(\u0027previous_workflow_task\u0027, {})\n iteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable\n \n # Process data\n batch_data = fetch_batch_data(iteration_num)\n \n outputs = {\"batch_data\": batch_data, \"batch_size\": len(batch_data)}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n \n # Second task depends on first task in same loop\n - id: process_data\n type: script\n depends_on: [fetch_data] # \u2705 CORRECT: Dependencies within loop tasks work normally\n script: |\n # \u2705 CORRECT: Access previous loop task through inputs context\n fetch_result = inputs.get(\u0027fetch_data\u0027, {})\n batch_data = fetch_result.get(\u0027batch_data\u0027, [])\n \n # \u274c INCORRECT: This won\u0027t work in loop context\n # batch_data = ${fetch_data.batch_data} # Template resolution fails in loops\n \n # Access and update state variables\n current_processed = loop_state.get(\u0027processed_items\u0027, [])\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\n \n # Process the batch\n processed_batch = process_batch(batch_data)\n current_processed.extend(processed_batch)\n \n # Update state variables using state_updates\n state_updates[\u0027processed_items\u0027] = current_processed\n state_updates[\u0027total_processed\u0027] = current_total + len(processed_batch)\n \n outputs = {\"processed_count\": len(processed_batch)}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n print(f\"__STATE_UPDATES__ {json.dumps(state_updates)}\") # \u2705 REQUIRED for state updates\n```\n\n#### Key Differences in Loop Task Data Access\n\n| Access Pattern | Regular Tasks | Loop Tasks |\n|---------------|---------------|------------|\n| **Previous Task Output** | `${previous_task}` | `inputs.get(\u0027field_name\u0027, default)` (flattened) |\n| **State Variables** | N/A | `loop_state.get(\u0027variable_name\u0027, default)` |\n| **State Updates** | N/A | `state_updates[\u0027variable_name\u0027] = value` |\n| **Workflow Inputs** | `\"${input_name}\"` | `inputs.get(\u0027input_name\u0027, \u0027default\u0027)` |\n| **Dependencies** | `depends_on: [task_id]` | Same: `depends_on: [task_id]` |\n\n#### Critical: Loop Task Output Flattening\n\n**Important:** The loop executor flattens task outputs into individual fields in the `inputs` dictionary, rather than storing them as nested objects under the task name.\n\n```yaml\n# First loop task outputs:\noutputs = {\"company_name\": \"ACME Corp\", \"company_id\": \"123\", \"status\": \"success\"}\nprint(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n\n# Second loop task accessing the data:\nscript: |\n # \u274c INCORRECT: This won\u0027t work - no nested object\n # company_data = inputs.get(\u0027first_task\u0027, {})\n # company_name = company_data.get(\u0027company_name\u0027)\n \n # \u2705 CORRECT: Access flattened fields directly\n company_name = inputs.get(\u0027company_name\u0027, \u0027\u0027)\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\n \n print(f\"Company: {company_name}, ID: {company_id}, Status: {task_status}\")\n```\n\n#### Complete Loop Task Example\n\n```yaml\n- id: financial_processing_loop\n type: loop\n loop_type: for\n max_iterations: 3\n iteration_variable: company_index\n depends_on: [list_companies] # External dependency\n \n state_variables:\n processed_companies: []\n total_extractions: 0\n successful_extractions: 0\n \n loop_tasks:\n - id: process_current_company\n type: script\n script: |\n # Access external workflow task data\n companies_data = inputs.get(\u0027list_companies\u0027, {})\n company_idx = inputs.get(\u0027company_index\u0027, 0)\n \n # Access state variables\n processed = loop_state.get(\u0027processed_companies\u0027, [])\n \n # Process company\n company = companies_data[\u0027companies\u0027][company_idx]\n result = process_company(company)\n \n outputs = {\"company_name\": company[\u0027name\u0027], \"status\": \"success\"}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n \n - id: extract_financial_data\n type: script\n depends_on: [process_current_company] # Depend on previous loop task\n requirements:\n - requests\u003e=2.28.0\n - some-ai-library\u003e=1.0.0\n script: |\n # \u2705 CORRECT: Access flattened outputs from previous loop task\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\n \n # \u274c INCORRECT: Loop executor flattens outputs, no nested access\n # company_data = inputs.get(\u0027process_current_company\u0027, {})\n # company_name = company_data.get(\u0027company_name\u0027, \u0027Unknown\u0027)\n \n # \u274c INCORRECT: Template resolution doesn\u0027t work in loops\n # company_data = ${process_current_company}\n \n # Access external data\n companies_list = inputs.get(\u0027list_companies\u0027, {})\n \n # Perform AI extraction\n extraction_result = extract_with_ai({\"company_name\": company_name, \"company_id\": company_id})\n \n # Update state variables\n current_processed = loop_state.get(\u0027processed_companies\u0027, [])\n current_total = loop_state.get(\u0027total_extractions\u0027, 0)\n current_successful = loop_state.get(\u0027successful_extractions\u0027, 0)\n \n current_processed.append({\n \"company_name\": company_name,\n \"extraction_result\": extraction_result\n })\n \n # State updates are required for persistence\n state_updates[\u0027processed_companies\u0027] = current_processed\n state_updates[\u0027total_extractions\u0027] = current_total + 1\n state_updates[\u0027successful_extractions\u0027] = current_successful + (1 if extraction_result else 0)\n \n outputs = {\"extracted\": bool(extraction_result)}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n print(f\"__STATE_UPDATES__ {json.dumps(state_updates)}\") # \u2705 REQUIRED\n\n# Accessing loop task outputs in subsequent workflow tasks\n- id: process_loop_results\n type: script\n depends_on: [financial_processing_loop]\n previous_node: financial_processing_loop\n script: |\n # \u2705 CORRECT: Access final state of loop\n loop_result = ${financial_processing_loop}\n loop_final_state = loop_result.get(\u0027final_state\u0027, {})\n \n # Access specific state variables from final state\n processed_companies = loop_final_state.get(\u0027processed_companies\u0027, [])\n total_processed = loop_final_state.get(\u0027total_extractions\u0027, 0)\n successful_count = loop_final_state.get(\u0027successful_extractions\u0027, 0)\n \n # Access loop execution metadata\n iterations_completed = loop_result.get(\u0027iterations_completed\u0027, 0)\n \n print(f\"Loop completed {iterations_completed} iterations\")\n print(f\"Processed {len(processed_companies)} companies\")\n print(f\"Success rate: {successful_count}/{total_processed}\")\n \n outputs = {\n \"companies_processed\": len(processed_companies),\n \"success_rate\": f\"{successful_count}/{total_processed}\",\n \"iterations\": iterations_completed\n }\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n#### Loop Task Context Injection and Output Flattening\n\nThe loop executor injects context through temporary files and flattens task outputs for easier access:\n\n```python\n# Auto-injected at the start of each loop task script:\nimport json\nimport os\n\n# Load context from temporary files (auto-generated)\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\n inputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\n\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\n loop_state = json.load(f) # Contains current state variables\n\noutputs = {}\nstate_updates = {}\n\n# Your script code follows...\n```\n\n#### Debugging Loop Task Data Access\n\nWhen debugging loop tasks, always print available keys to understand what data is accessible:\n\n```yaml\n- id: debug_loop_task\n type: script\n depends_on: [previous_loop_task]\n script: |\n # Debug: Print all available data\n print(f\"\ud83d\udd0d Available inputs: {list(inputs.keys())}\")\n print(f\"\ud83d\udd0d Loop state keys: {list(loop_state.keys())}\")\n \n # Debug: Print specific values\n for key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\n value = inputs.get(key, \u0027NOT_FOUND\u0027)\n print(f\"\ud83d\udd0d {key}: {value}\")\n \n # Now access the flattened data\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\n \n # Process your data...\n outputs = {\"processed\": f\"Company {company_name}\"}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n#### Real Example: Task Output Flattening\n\n```yaml\n# Task 1 outputs this structure:\n__OUTPUTS__ {\"company_name\": \"LTTS\", \"company_id\": \"123\", \"fy_folders\": [{\"id\": \"456\", \"name\": \"FY24\"}], \"status\": \"success\"}\n\n# Task 2 receives flattened inputs:\ninputs = {\n \"folder_id\": \"...\", # Workflow input\n \"gemini_api_key\": \"...\", # Workflow input \n \"list_company_folders\": {...}, # External task output\n \"company_index\": 0, # Loop iteration variable\n \"company_name\": \"LTTS\", # \u2705 Flattened from Task 1\n \"company_id\": \"123\", # \u2705 Flattened from Task 1\n \"fy_folders\": [{\"id\": \"456\", \"name\": \"FY24\"}], # \u2705 Flattened from Task 1\n \"status\": \"success\", # \u2705 Flattened from Task 1\n \"total_extractions\": 0, # Loop state variable\n \"processed_companies\": [] # Loop state variable\n}\n```\n\n#### Loop Task Special Output Fields\n\n| Field | Description | Example Access |\n|-------|-------------|----------------|\n| `final_state` | Final values of all state variables | `${loop_task.final_state}` |\n| `final_state[\"variable_name\"]` | Specific state variable from final state | `${loop_task.final_state[\"total_processed\"]}` |\n| `iterations_completed` | Number of iterations executed | `${loop_task.iterations_completed}` |\n\n#### Safe Template Resolution Pattern for Loop States\n\nWhen accessing loop final state fields, use this safe parsing pattern:\n\n```yaml\nscript: |\n try:\n # Template resolution with safety checks\n final_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\n processed_count_str = \u0027\u0027\u0027${loop_task.final_state[\"processed_count\"]}\u0027\u0027\u0027\n items_str = \u0027\u0027\u0027${loop_task.final_state[\"processed_items\"]}\u0027\u0027\u0027\n iterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027\n \n # Safe parsing with fallbacks\n final_state = json.loads(final_state_str) if not final_state_str.startswith(\u0027UNRESOLVED_\u0027) else {}\n processed_count = int(processed_count_str) if processed_count_str.isdigit() else 0\n processed_items = json.loads(items_str) if not items_str.startswith(\u0027UNRESOLVED_\u0027) else []\n iterations = int(iterations_str) if iterations_str.isdigit() else 0\n \n print(f\"Successfully processed {processed_count} items in {iterations} iterations\")\n \n except Exception as e:\n print(f\"Error accessing loop results: {str(e)}\")\n # Handle error case\n```\n```\n\n### Real-World AI Agent Example\n\nBased on actual WhatsApp workflow patterns:\n\n```yaml\n# AI greeting agent\n- id: greeting_agent\n type: ai_agent\n config:\n model_client_id: assistant_model\n system_message: \"You are a helpful assistant. Generate a friendly greeting.\"\n user_message: \"User said: ${user_message}\"\n\n# Send the AI response via API\n- id: send_greeting\n type: script\n depends_on: [greeting_agent]\n previous_node: greeting_agent\n script: |\n import requests\n import json\n \n # CORRECT: Access AI response from structured output\n agent_output = ${greeting_agent}\n ai_greeting = agent_output[\u0027ai_response\u0027].strip()\n \n phone_number = \"${phone_number}\"\n api_token = \"${API_TOKEN}\"\n \n print(f\"\ud83e\udd16 AI generated greeting: {ai_greeting}\")\n \n # Send via API\n url = f\"https://api.example.com/messages/{phone_number}\"\n headers = {\"Authorization\": f\"Bearer {api_token}\"}\n \n response = requests.post(url, headers=headers, json={\n \"message\": ai_greeting\n })\n \n result = {\n \"status\": \"sent\",\n \"ai_response\": ai_greeting,\n \"api_response\": response.json(),\n \"tokens_used\": agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\n }\n \n print(f\"__OUTPUTS__ {json.dumps(result)}\")\n```\n\n### Complex Template Patterns\n\n```yaml\n# Conditional templates with router outputs\n- id: router_task\n type: conditional_router\n conditions:\n - name: \"premium_customer\"\n condition: \"${customer_data.tier} == \u0027premium\u0027\"\n route: \"premium_processing\"\n\n- id: premium_handler\n type: script\n depends_on: [router_task]\n previous_node: router_task\n execute_on_routes: [\"premium_processing\"]\n script: |\n # Access router decision\n selected_route = \"${router_task.selected_route}\"\n route_data = ${router_task.route_data}\n \n # Process premium customer\n customer = ${customer_data}\n premium_benefits = calculate_benefits(customer)\n\n# Loop through arrays from previous tasks\n- id: process_items\n type: script\n depends_on: [fetch_items]\n previous_node: fetch_items\n script: |\n items = ${fetch_items.data}\n processed_items = []\n \n for item in items:\n processed_item = {\n \"id\": item[\"id\"],\n \"processed_at\": \"${current_timestamp}\",\n \"status\": \"processed\"\n }\n processed_items.append(processed_item)\n \n print(f\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\")\n\n# Nested object access\n- id: user_preferences\n type: script\n depends_on: [fetch_user]\n previous_node: fetch_user\n script: |\n # Access nested user preferences\n email_notifications = ${fetch_user.preferences.notifications.email}\n theme_setting = \"${fetch_user.preferences.ui.theme}\"\n language = \"${fetch_user.profile.language}\"\n \n settings = {\n \"notifications\": email_notifications,\n \"theme\": theme_setting,\n \"locale\": language\n }\n print(f\"__OUTPUTS__ {json.dumps(settings)}\")\n```\n\n### Template Resolution Rules\n\n1. **Dependency Requirement**: Tasks must have `depends_on` relationship to access other task outputs\n2. **String vs Object**: Use quotes for string values, no quotes for objects/arrays\n3. **Field Navigation**: Use dot notation for nested field access\n4. **Error Handling**: Unresolved templates become `UNRESOLVED_variable_name`\n\n```yaml\n# \u2705 Correct template usage\n- id: correct_task\n depends_on: [previous_task]\n previous_node: previous_task\n script: |\n string_value = \"${previous_task.name}\" # String with quotes\n object_value = ${previous_task.data} # Object without quotes\n nested_value = \"${previous_task.user.email}\" # Nested field\n\n# \u274c Incorrect template usage \n- id: incorrect_task\n # Missing depends_on - template won\u0027t resolve\n script: |\n value = ${previous_task.name} # Will become UNRESOLVED_previous_task.name\n```\n\n## Script Task Output Handling\n\nScript tasks must use the `__OUTPUTS__` marker pattern for proper output parsing by the workflow engine.\n\n### Required Output Pattern\n\n```yaml\ntasks:\n - id: example_script\n type: script\n script: |\n # Your script logic here\n data = process_data()\n \n # Required: Store outputs in outputs dict\n outputs = {}\n outputs[\u0027result\u0027] = data\n outputs[\u0027status\u0027] = \u0027success\u0027\n outputs[\u0027count\u0027] = len(data)\n \n # Required: Print with __OUTPUTS__ marker for engine parsing\n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n### Output Access in Subsequent Tasks\n\n```yaml\ntasks:\n - id: data_processor\n type: script\n script: |\n # Process and store outputs\n results = {\"processed_count\": 100, \"errors\": []}\n \n outputs = {}\n outputs[\u0027results\u0027] = results\n outputs[\u0027status\u0027] = \u0027completed\u0027\n \n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n \n - id: result_handler\n type: script\n depends_on: [data_processor]\n script: |\n # Access previous task outputs\n processor_output = ${data_processor}\n processed_count = processor_output[\u0027results\u0027][\u0027processed_count\u0027]\n status = processor_output[\u0027status\u0027]\n \n print(f\"Previous task processed {processed_count} items with status: {status}\")\n \n # Store new outputs\n outputs = {\u0027message\u0027: f\u0027Handled {processed_count} results\u0027}\n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n### Error Handling in Scripts\n\n```yaml\ntasks:\n - id: safe_script\n type: script\n script: |\n try:\n result = risky_operation()\n outputs = {\u0027result\u0027: result, \u0027status\u0027: \u0027success\u0027}\n except Exception as e:\n outputs = {\u0027error\u0027: str(e), \u0027status\u0027: \u0027failed\u0027}\n \n # Always print outputs, even for errors\n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n## Triggers and Workflow Activation\n\nTriggers define how workflows are started automatically based on external events. They enable event-driven workflow execution.\n\n### Webhook Triggers\n\nWebhook triggers allow external systems to start workflows by sending HTTP requests.\n\n```yaml\ntriggers:\n - name: customer_webhook\n type: webhook\n description: \"Triggered when customer data is updated\"\n config:\n path: \"/webhooks/customer-updated\" # Webhook endpoint path\n method: POST # HTTP method (GET, POST, PUT, etc.)\n authentication:\n type: token # none|token|signature|basic\n token: \"${WEBHOOK_SECRET}\" # Required if type is token\n header_name: \"X-Webhook-Token\" # Optional: custom header name\n validation:\n required_fields: [\"customer.id\", \"customer.email\"] # Required payload fields\n content_type: \"application/json\" # Expected content type\n input_mapping:\n - webhook_field: customer.id # Field from webhook payload\n workflow_input: customer_id # Map to workflow input\n - webhook_field: customer.email\n workflow_input: customer_email\n - webhook_field: customer.updated_at\n workflow_input: update_timestamp\n```\n\n### Scheduled Triggers\n\nSchedule workflows to run at specific times or intervals.\n\n```yaml\ntriggers:\n - name: daily_report\n type: schedule\n description: \"Generate daily customer report\"\n config:\n cron: \"0 9 * * *\" # Daily at 9 AM (cron format)\n timezone: \"UTC\" # Optional: timezone\n enabled: true # Optional: enable/disable\n max_instances: 1 # Optional: max concurrent instances\n input_mapping:\n - workflow_input: report_date\n default_value: \"${current_date}\" # Use built-in variable\n - workflow_input: report_type\n default_value: \"daily\"\n\n - name: weekly_cleanup\n type: schedule\n description: \"Weekly database cleanup\"\n config:\n cron: \"0 2 * * 0\" # Weekly on Sunday at 2 AM\n timezone: \"America/New_York\"\n input_mapping:\n - workflow_input: cleanup_before_date\n default_value: \"${date_minus_7_days}\"\n```\n\n### Event Triggers\n\nEvent triggers respond to system events from various sources.\n\n```yaml\ntriggers:\n - name: file_uploaded\n type: event\n description: \"Process uploaded files\"\n config:\n event_type: \"file.uploaded\" # Event type to listen for\n source: \"storage_service\" # Event source system\n filter: # Optional: event filtering\n file_type: [\"csv\", \"json\", \"xlsx\"]\n file_size_max: \"100MB\"\n path_pattern: \"/uploads/customers/*\"\n retry: # Optional: retry configuration\n max_attempts: 3\n initial_delay: \"5s\"\n input_mapping:\n - webhook_field: file.path\n workflow_input: file_path\n - webhook_field: file.size\n workflow_input: file_size\n - webhook_field: file.type\n workflow_input: file_type\n\n - name: order_status_changed\n type: event\n description: \"Handle order status changes\"\n config:\n event_type: \"order.status_changed\"\n source: \"order_management\"\n filter:\n status: [\"completed\", \"cancelled\", \"refunded\"]\n priority: [\"high\", \"urgent\"]\n input_mapping:\n - webhook_field: order.id\n workflow_input: order_id\n - webhook_field: order.status\n workflow_input: new_status\n - webhook_field: order.customer_id\n workflow_input: customer_id\n```\n\n### Dependency Triggers\n\nDependency triggers start workflows when other workflows complete.\n\n```yaml\ntriggers:\n - name: post_processing\n type: dependency\n description: \"Run after main processing completes\"\n config:\n depends_on_workflow: \"main_data_processing\" # Workflow to depend on\n depends_on_status: \"SUCCESS\" # SUCCESS|FAILED|COMPLETED\n namespace: \"production\" # Optional: workflow namespace\n timeout_minutes: 60 # Optional: wait timeout\n input_mapping:\n - webhook_field: execution_id # From completed workflow\n workflow_input: source_execution_id\n - webhook_field: workflow_outputs.processed_count\n workflow_input: items_processed\n```\n\n## Model Client Configuration\n\nBefore defining AI-related tasks (ai_agent, chatbot, continuous_conversation), you must configure model clients. Model clients define which AI models to use and how to authenticate with them.\n\n### Basic Structure\n\n```yaml\nmodel_clients:\n - id: unique_identifier # Used to reference this model in tasks\n provider: provider_name # AI provider (openai, anthropic, etc.)\n config:\n model: model_name # Specific model to use\n api_key: your_api_key # Authentication key\n # Additional provider-specific options...\n```\n\n### Supported Providers\n\n#### OpenAI Provider\n\n```yaml\nmodel_clients:\n - id: gpt4_model\n provider: openai\n config:\n model: gpt-4-turbo-preview # Available: gpt-4, gpt-4-turbo, gpt-3.5-turbo\n api_key: \"sk-your-openai-api-key\"\n temperature: 0.7 # Optional: 0.0-2.0\n max_tokens: 2000 # Optional: Maximum response tokens\n timeout: 30 # Optional: Request timeout (seconds)\n```\n\n#### Anthropic Provider\n\n```yaml\nmodel_clients:\n - id: claude_model\n provider: anthropic\n config:\n model: claude-3-sonnet-20240229 # Available: claude-3-opus, claude-3-sonnet, claude-3-haiku\n api_key: \"sk-ant-your-anthropic-api-key\"\n temperature: 0.7 # Optional: 0.0-1.0\n max_tokens: 1500 # Optional: Maximum response tokens\n timeout: 30 # Optional: Request timeout (seconds)\n```\n\n#### OpenRouter Provider\n\n```yaml\nmodel_clients:\n - id: openrouter_model\n provider: openrouter\n config:\n model: anthropic/claude-3-sonnet # Available: Various models via OpenRouter\n api_key: \"sk-or-your-openrouter-api-key\"\n base_url: \"https://openrouter.ai/api/v1\" # Optional: Custom endpoint\n temperature: 0.7\n max_tokens: 2000\n```\n\n#### Multiple Model Clients\n\nYou can define multiple model clients and reference them by their `id` in tasks:\n\n```yaml\nmodel_clients:\n - id: fast_model\n provider: openai\n config:\n model: gpt-3.5-turbo\n api_key: \"sk-your-openai-api-key\"\n temperature: 0.3\n \n - id: smart_model\n provider: anthropic\n config:\n model: claude-3-opus-20240229\n api_key: \"sk-ant-your-anthropic-api-key\"\n temperature: 0.7\n \n - id: creative_model\n provider: openai\n config:\n model: gpt-4-turbo-preview\n api_key: \"sk-your-openai-api-key\"\n temperature: 1.2\n```\n\n### Using Model Clients in Tasks\n\nReference model clients in AI tasks using the `model_client_id` field:\n\n```yaml\ntasks:\n - id: analysis_task\n type: ai_agent\n config:\n model_client_id: smart_model # References the claude model above\n system_message: \"You are a data analyst...\"\n user_message: \"Analyze this data: ${input_data}\"\n \n - id: creative_task\n type: ai_agent\n config:\n model_client_id: creative_model # References the creative GPT-4 model\n system_message: \"You are a creative writer...\"\n user_message: \"Write a story about ${topic}\"\n```\n\n### Environment Variables for API Keys\n\nFor security, use environment variables instead of hardcoding API keys:\n\n```yaml\nmodel_clients:\n - id: secure_model\n provider: openai\n config:\n model: gpt-4-turbo-preview\n api_key: \"${OPENAI_API_KEY}\" # References environment variable\n temperature: 0.7\n```\n\n### Common Configuration Options\n\n| Field | Description | Required | Default |\n|-------|-------------|----------|---------|\n| `id` | Unique identifier for referencing in tasks | Yes | - |\n| `provider` | AI provider name | Yes | - |\n| `config.model` | Specific model name | Yes | - |\n| `config.api_key` | Authentication API key | Yes | - |\n| `config.temperature` | Response randomness (0.0-2.0) | No | 0.7 |\n| `config.max_tokens` | Maximum response tokens | No | Provider default |\n| `config.timeout` | Request timeout in seconds | No | 30 |\n| `config.base_url` | Custom API endpoint | No | Provider default |\n\n## Secrets Management and Environment Variables\n\nProper secrets management is crucial for security. Never hardcode sensitive information like API keys, passwords, or tokens directly in workflow YAML files.\n\n### Using Environment Variables\n\nThe recommended approach is to use environment variables for all sensitive data:\n\n```yaml\n# \u2705 RECOMMENDED: Use environment variables\nmodel_clients:\n - id: secure_openai\n provider: openai\n config:\n model: gpt-4-turbo-preview\n api_key: \"${OPENAI_API_KEY}\" # References environment variable\n\n# HTTP tasks with environment variables\ntasks:\n - id: secure_api_call\n type: http\n url: \"${API_BASE_URL}/customers\" # Environment variable for URL\n headers:\n Authorization: \"Bearer ${API_TOKEN}\" # Environment variable for token\n X-API-Key: \"${EXTERNAL_API_KEY}\" # Environment variable for API key\n```\n\n### Environment Variable Patterns\n\n```yaml\n# Database connections\n- id: database_task\n type: storage\n config:\n storage_type: database\n connection_string: \"${DATABASE_URL}\" # Full connection string\n # or individual components:\n host: \"${DB_HOST}\"\n port: \"${DB_PORT}\"\n database: \"${DB_NAME}\"\n username: \"${DB_USER}\"\n password: \"${DB_PASSWORD}\"\n\n# Third-party service credentials\n- id: external_service\n type: http\n url: \"${SERVICE_BASE_URL}/api/v1/data\"\n headers:\n Authorization: \"Bearer ${SERVICE_TOKEN}\"\n X-Client-ID: \"${SERVICE_CLIENT_ID}\"\n X-Client-Secret: \"${SERVICE_CLIENT_SECRET}\"\n\n# MCP server credentials\n- id: mcp_task\n type: mcp\n mcp_server: \"@agentic/mcp-notion-server\"\n environment_vars:\n NOTION_TOKEN: \"${NOTION_API_TOKEN}\"\n WORKSPACE_ID: \"${NOTION_WORKSPACE_ID}\"\n```\n\n### Security Best Practices\n\n#### \u274c What NOT to Do\n\n```yaml\n# DON\u0027T: Hardcode secrets\nmodel_clients:\n - id: insecure_example\n provider: openai\n config:\n api_key: \"sk-actual-api-key-here\" # SECURITY RISK!\n\n# DON\u0027T: Put secrets in comments\ntasks:\n - id: bad_example\n type: http\n url: \"https://api.example.com\"\n # API key is: sk-12345... DON\u0027T DO THIS!\n headers:\n Authorization: \"Bearer sk-real-token\" # SECURITY RISK!\n```\n\n#### \u2705 What TO Do\n\n```yaml\n# DO: Use environment variables\nmodel_clients:\n - id: secure_example\n provider: openai\n config:\n api_key: \"${OPENAI_API_KEY}\" # Secure approach\n\n# DO: Use different variables for different environments\n- id: environment_aware_task\n type: http\n url: \"${API_BASE_URL}\" # Different URLs per environment\n headers:\n Authorization: \"Bearer ${API_TOKEN}\" # Different tokens per environment\n X-Environment: \"${ENVIRONMENT_NAME}\" # Track environment\n\n# DO: Use descriptive variable names\nenvironment_vars:\n CUSTOMER_API_TOKEN: \"${CUSTOMER_SERVICE_TOKEN}\"\n BILLING_API_KEY: \"${BILLING_SERVICE_API_KEY}\"\n NOTIFICATION_WEBHOOK_SECRET: \"${NOTIFICATION_SECRET}\"\n```\n\n### Environment Variable Naming Conventions\n\n| Pattern | Example | Use Case |\n|---------|---------|----------|\n| `SERVICE_API_KEY` | `OPENAI_API_KEY` | Service API keys |\n| `SERVICE_BASE_URL` | `CUSTOMER_API_BASE_URL` | Service endpoints |\n| `DB_*` | `DB_HOST`, `DB_PASSWORD` | Database credentials |\n| `WEBHOOK_*` | `WEBHOOK_SECRET_TOKEN` | Webhook authentication |\n| `*_TIMEOUT` | `HTTP_REQUEST_TIMEOUT` | Timeout configurations |\n\n### Dynamic Environment Variables\n\nUse template resolution to create dynamic environment variable names:\n\n```yaml\n# Different API keys per customer\n- id: customer_specific_api\n type: http\n url: \"https://api.partner.com/data\"\n headers:\n Authorization: \"Bearer ${PARTNER_API_KEY_${customer_tier}}\" # Resolves to PARTNER_API_KEY_PREMIUM\n\n# Environment-specific configurations\n- id: env_specific_task\n type: storage\n config:\n connection_string: \"${DATABASE_URL_${environment}}\" # DATABASE_URL_PRODUCTION\n```\n\n### Secrets in Different Task Types\n\n```yaml\n# AI Agent with multiple secure credentials\n- id: secure_ai_task\n type: ai_agent\n config:\n model_client_id: secure_model\n system_message: \"You are a secure assistant with access to customer data.\"\n user_message: \"Process customer ${customer_id}\"\n environment_vars:\n CUSTOMER_DB_TOKEN: \"${CUSTOMER_DATABASE_TOKEN}\"\n AUDIT_LOG_KEY: \"${AUDIT_LOGGING_API_KEY}\"\n\n# Script task with secure file access\n- id: secure_script_task\n type: script\n environment_vars:\n AWS_ACCESS_KEY_ID: \"${AWS_ACCESS_KEY}\"\n AWS_SECRET_ACCESS_KEY: \"${AWS_SECRET_KEY}\"\n S3_BUCKET_NAME: \"${SECURE_BUCKET_NAME}\"\n script: |\n import boto3\n import os\n \n # Credentials automatically available from environment\n s3_client = boto3.client(\u0027s3\u0027)\n bucket = os.environ[\u0027S3_BUCKET_NAME\u0027]\n \n # Process secure files...\n\n# Storage task with encrypted connections\n- id: secure_storage_task\n type: storage\n config:\n storage_type: database\n connection_string: \"${ENCRYPTED_DB_CONNECTION_STRING}\"\n ssl_mode: \"require\"\n ssl_cert: \"${SSL_CERT_PATH}\"\n ssl_key: \"${SSL_KEY_PATH}\"\n```\n\n### Environment Setup Examples\n\n#### Development Environment (.env file)\n```bash\n# Development API keys (non-production)\nOPENAI_API_KEY=sk-dev-api-key-here\nCUSTOMER_API_BASE_URL=https://api-dev.example.com\nDATABASE_URL=postgresql://dev:pass@localhost:5432/workflow_dev\nENVIRONMENT_NAME=development\n```\n\n#### Production Environment\n```bash\n# Production API keys (secure)\nOPENAI_API_KEY=sk-prod-api-key-here\nCUSTOMER_API_BASE_URL=https://api.example.com\nDATABASE_URL=postgresql://prod_user:secure_pass@prod-db:5432/workflow_prod\nENVIRONMENT_NAME=production\n```\n\n## Workflow Execution and Orchestration\n\nUnderstanding how workflows execute and how tasks are orchestrated is essential for building reliable, efficient workflows.\n\n### Execution Flow\n\nWorkflows follow a directed acyclic graph (DAG) execution model:\n\n1. **Initialization**: Workflow starts with inputs and model clients\n2. **Task Scheduling**: Tasks are scheduled based on dependencies\n3. **Parallel Execution**: Independent tasks run simultaneously\n4. **Data Flow**: Outputs flow between tasks via template resolution\n5. **Completion**: Workflow completes when all tasks finish\n\n```yaml\n# Example execution flow\ntasks:\n # Stage 1: Independent tasks (run in parallel)\n - id: fetch_customer # Runs immediately\n type: http\n url: \"https://api.example.com/customers/${customer_id}\"\n \n - id: fetch_products # Runs in parallel with fetch_customer\n type: http\n url: \"https://api.example.com/products\"\n \n # Stage 2: Dependent tasks (wait for Stage 1)\n - id: validate_customer # Waits for fetch_customer\n type: script\n depends_on: [fetch_customer]\n previous_node: fetch_customer\n script: # validation logic...\n \n - id: filter_products # Waits for fetch_products\n type: script\n depends_on: [fetch_products]\n previous_node: fetch_products\n script: # filtering logic...\n \n # Stage 3: Final processing (waits for Stage 2)\n - id: generate_recommendations # Waits for both validation and filtering\n type: ai_agent\n depends_on: [validate_customer, filter_products]\n previous_node: validate_customer # UI hierarchy under customer validation\n config:\n model_client_id: gpt4_model\n system_message: \"Generate product recommendations for customer.\"\n user_message: |\n Customer: ${validate_customer}\n Available products: ${filter_products}\n```\n\n### Task States and Lifecycle\n\nEach task goes through these states during execution:\n\n| State | Description | Next States |\n|-------|-------------|-------------|\n| `PENDING` | Task created, waiting for dependencies | `RUNNING`, `SKIPPED` |\n| `RUNNING` | Task is actively executing | `COMPLETED`, `FAILED` |\n| `COMPLETED` | Task finished successfully | - |\n| `FAILED` | Task encountered an error | `RUNNING` (if retries) |\n| `SKIPPED` | Task skipped due to conditions | - |\n| `TIMEOUT` | Task exceeded timeout limit | `RUNNING` (if retries) |\n\n### Conditional Execution\n\nTasks can be executed conditionally based on routes or conditions:\n\n```yaml\n# Router-based conditional execution\n- id: data_router\n type: conditional_router\n depends_on: [fetch_data]\n previous_node: fetch_data\n conditions:\n - name: \"large_dataset\"\n condition: \"${fetch_data.record_count} \u003e 10000\"\n route: \"batch_processing\"\n - name: \"small_dataset\"\n condition: \"${fetch_data.record_count} \u003c= 1000\"\n route: \"direct_processing\"\n default_route: \"standard_processing\"\n\n# Tasks that execute on specific routes\n- id: batch_processor\n type: script\n depends_on: [data_router]\n previous_node: data_router\n execute_on_routes: [\"batch_processing\"] # Only runs for large datasets\n script: |\n # Handle large dataset in batches\n data = ${fetch_data}\n process_in_batches(data, batch_size=1000)\n\n- id: direct_processor\n type: script\n depends_on: [data_router]\n previous_node: data_router\n execute_on_routes: [\"direct_processing\"] # Only runs for small datasets\n script: |\n # Process small dataset directly\n data = ${fetch_data}\n process_directly(data)\n```\n\n### Error Handling and Retries\n\nConfigure retry policies for resilient workflow execution:\n\n```yaml\n# Workflow-level retry policy\nretry:\n retryOn: [\"TEMPORARY_FAILURE\", \"TIMEOUT\", \"NETWORK_ERROR\"]\n maxAttempts: 3\n initialDelay: \"1s\"\n maxDelay: \"60s\"\n backoffMultiplier: 2.0\n\ntasks:\n # Task-specific retry policy (overrides workflow-level)\n - id: external_api_call\n type: http\n url: \"https://unreliable-api.example.com/data\"\n retry:\n maxAttempts: 5 # More retries for unreliable API\n initialDelay: \"2s\"\n maxDelay: \"120s\"\n backoffMultiplier: 2.0\n retryOn: [\"TIMEOUT\", \"HTTP_5XX\", \"NETWORK_ERROR\"]\n \n # Task with no retries\n - id: critical_operation\n type: script\n retry:\n maxAttempts: 1 # No retries for critical operations\n script: |\n # This operation should not be retried if it fails\n perform_critical_database_operation()\n```\n\n### Parallel Execution Patterns\n\n```yaml\n# Fan-out pattern: One task triggers multiple parallel tasks\n- id: data_source\n type: http\n url: \"https://api.example.com/data\"\n\n- id: process_customers # Parallel processing\n type: script\n depends_on: [data_source]\n previous_node: data_source\n script: # process customer data from data_source\n\n- id: process_orders # Parallel processing\n type: script\n depends_on: [data_source]\n previous_node: data_source\n script: # process order data from data_source\n\n- id: process_products # Parallel processing\n type: script\n depends_on: [data_source]\n previous_node: data_source\n script: # process product data from data_source\n\n# Fan-in pattern: Multiple tasks feed into one task\n- id: generate_report # Waits for all parallel tasks\n type: ai_agent\n depends_on: [process_customers, process_orders, process_products]\n previous_node: process_customers # UI hierarchy\n config:\n model_client_id: gpt4_model\n system_message: \"Generate comprehensive report from processed data.\"\n user_message: |\n Customer data: ${process_customers}\n Order data: ${process_orders}\n Product data: ${process_products}\n```\n\n### Resource Management\n\nControl resource usage and task distribution:\n\n```yaml\n# Worker group targeting\n- id: cpu_intensive_task\n type: script\n worker_group: \"high-cpu-workers\" # Target specific worker group\n script: |\n # CPU-intensive processing\n perform_heavy_computation()\n\n- id: gpu_ml_task\n type: ai_agent\n worker_group: \"gpu-workers\" # Target GPU workers\n config:\n model_client_id: large_model\n # Large model processing...\n\n# Timeout configuration\n- id: long_running_task\n type: script\n timeout_seconds: 3600 # 1 hour timeout\n script: |\n # Long-running data processing\n process_large_dataset()\n\n# Workflow-level timeout\ntimeout_seconds: 7200 # 2 hour workflow timeout\n```\n\n## Task Type Configurations\n\nThis section provides the complete YAML configuration for each supported task type.\n\n**Supported Task Types:**\n- `script` - Execute Python code with full workflow context\n- `http` - Make HTTP requests to external APIs\n- `ai_agent` - Integrate with AI models for intelligent processing \n- `loop` - Execute tasks repeatedly (for/while loops)\n- `conditional_router` - Route execution based on conditions\n- `mcp` - Execute tools via Model Context Protocol servers\n- `storage` - Data persistence and retrieval operations\n- `continuous_conversation` - Interactive AI conversations\n- `human_input` - Tasks requiring human input or approval\n- `chatbot` - Session-based chatbot conversations\n\n### Script Tasks\n\nExecute Python code with full workflow context access.\n\n```yaml\n- id: script_task\n name: \"Python Script Task\"\n type: script\n description: \"Execute Python code with dependencies and environment\"\n \n # Script configuration - script field is directly under task, NOT under config\n script: | # REQUIRED: Python code to execute\n import json\n import requests\n \n # Access workflow inputs\n input_value = \"${workflow_input}\"\n \n # Access previous task outputs\n previous_result = ${previous_task}\n \n # Your processing logic\n result = {\"processed\": True, \"value\": input_value}\n \n # Output results (must be JSON serializable)\n print(f\"__OUTPUTS__ {json.dumps(result)}\")\n \n # Alternative: use \u0027code\u0027 field instead of \u0027script\u0027\n code: | # Alternative to \u0027script\u0027 field\n print(\"Hello World\")\n \n # Package management\n requirements: # Optional: Python packages\n - requests==2.31.0\n - pandas==1.5.3\n - numpy==1.24.3\n \n # Environment configuration\n environment: # Optional: Environment variables\n API_KEY: \"${secrets.api_key}\"\n DEBUG: \"true\"\n CUSTOM_VAR: \"value\"\n \n environment_vars: # Alternative to \u0027environment\u0027\n DATABASE_URL: \"${secrets.db_url}\"\n \n # Secrets configuration\n secrets_config: # Optional: Secrets access\n api_key: \"secret_name\"\n db_password: \"db_secret\"\n```\n\n### HTTP Tasks\n\nMake HTTP requests to external APIs and services.\n\n```yaml\n- id: http_task\n name: \"HTTP API Call\"\n type: http\n description: \"Call external REST API\"\n \n # HTTP configuration\n method: POST # Default: GET\n url: \"https://api.example.com/data\" # REQUIRED: Target URL\n \n # Headers configuration\n headers: # Optional: HTTP headers\n Content-Type: \"application/json\"\n Authorization: \"Bearer ${secrets.api_token}\"\n X-Custom-Header: \"workflow-engine\"\n User-Agent: \"WorkflowEngine/1.0\"\n \n # Request body\n body: | # Optional: Request body (for POST/PUT)\n {\n \"customer_id\": \"${customer_id}\",\n \"timestamp\": \"${current_timestamp}\",\n \"data\": ${previous_task}\n }\n \n # Response validation\n expected_status: [200, 201, 202] # Default: [200]\n expectedStatus: [200, 201] # Alternative field name\n \n # Additional HTTP options\n timeout_seconds: 30 # HTTP request timeout\n follow_redirects: true # Follow HTTP redirects\n verify_ssl: true # Verify SSL certificates\n```\n\n### AI Agent Tasks\n\nIntegrate with AI models for intelligent processing.\n\n```yaml\n- id: ai_agent_task\n name: \"AI Processing Agent\"\n type: ai_agent\n description: \"Process data using AI model\"\n \n # AI configuration\n config:\n model_client_id: gpt4_model # REQUIRED: Reference to model client\n \n system_message: | # Optional: System prompt\n You are a data analyst. Analyze the provided data and extract insights.\n Return your analysis in JSON format with the following structure:\n {\n \"insights\": [\"insight1\", \"insight2\"],\n \"recommendations\": [\"rec1\", \"rec2\"],\n \"confidence_score\": 0.95\n }\n \n user_message: | # REQUIRED: User prompt with template resolution\n Please analyze this customer data:\n ${previous_task}\n \n Customer ID: ${customer_id}\n Date Range: ${date_range}\n \n # Model parameters\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\n max_tokens: 1500 # Optional: Maximum response tokens\n top_p: 1.0 # Optional: Nucleus sampling\n frequency_penalty: 0.0 # Optional: Frequency penalty\n presence_penalty: 0.0 # Optional: Presence penalty\n \n # Tools configuration for function calling\n tools: # Optional: List of MCP tool names\n - current_time # Time-related tools\n - relative_time\n - days_in_month\n - weather_api # Weather tools\n - database_query # Database tools\n - notification_service # Communication tools\n \n # Required for tool resolution\n tenant_id: \"your-tenant-uuid\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\n \n # Input/output format\n input_format: \"text\" # Optional: text|json|structured\n output_format: \"structured\" # Optional: text|json|structured\n\n### Important: AI Agent Output Access\n\nAI agent tasks return structured objects, not direct text responses. Always access the AI response using this pattern:\n\n```yaml\n# In subsequent script tasks\nscript: |\n # CORRECT: Access AI response from structured output\n agent_output = ${ai_agent_task}\n ai_response = agent_output[\u0027ai_response\u0027].strip()\n \n # Additional metadata available\n tokens_used = agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\n cost_estimate = agent_output.get(\u0027estimated_cost\u0027, {}).get(\u0027amount\u0027, 0.0)\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\n \n print(f\"AI Response: {ai_response}\")\n print(f\"Tokens used: {tokens_used}\")\n print(f\"Estimated cost: ${cost_estimate:.6f}\")\n```\n\n```yaml\n# In workflow outputs - specify the ai_response field\noutputs:\n ai_analysis:\n type: string\n source: ai_agent_task.ai_response # Access the ai_response field\n description: \"AI-generated analysis text\"\n \n total_tokens:\n type: integer \n source: ai_agent_task.token_usage.total_tokens\n description: \"Total tokens consumed\"\n```\n```\n\n### Loop Tasks\n\nExecute tasks repeatedly with for-loop and while-loop patterns.\n\n```yaml\n# For-Loop (Fixed Iterations)\n- id: for_loop_task\n name: \"For Loop Processor\"\n type: loop\n description: \"Process data in fixed batches\"\n \n # Loop configuration\n loop_type: for # REQUIRED: for|while\n max_iterations: 10 # REQUIRED for \u0027for\u0027 loops\n iteration_variable: batch_index # Default: iteration\n \n # State variables (persistent across iterations)\n state_variables:\n total_processed: 0\n error_count: 0\n batch_results: []\n \n # Tasks to execute in each iteration\n loop_tasks: # REQUIRED: Tasks to run in loop\n - id: process_batch\n name: \"Process Current Batch\"\n type: script\n script: |\n batch_idx = ${batch_index} # Access iteration variable\n total = ${total_processed} # Access state variable\n \n # Process current batch\n result = process_batch_data(batch_idx)\n \n print(f\"__OUTPUTS__ {json.dumps(result)}\")\n \n - id: update_state\n name: \"Update Loop State\"\n type: script\n depends_on: [process_batch]\n script: |\n # Update state variables\n new_total = ${total_processed} + ${process_batch.items_processed}\n \n result = {\"total_processed\": new_total}\n print(f\"__OUTPUTS__ {json.dumps(result)}\")\n\n# While-Loop (Conditional)\n- id: while_loop_task\n name: \"While Loop Quality Processor\"\n type: loop\n description: \"Process until quality threshold met\"\n \n # Loop configuration\n loop_type: while # REQUIRED: for|while\n exit_condition: \"${average_quality} \u003e= 85 || ${iteration_count} \u003e= 20\" # REQUIRED for \u0027while\u0027\n max_iterations: 50 # Optional: Safety limit\n iteration_variable: iteration_count # Default: iteration\n \n # State variables\n state_variables:\n average_quality: 0\n items_processed: 0\n quality_scores: []\n \n # Loop tasks\n loop_tasks:\n - id: assess_quality\n type: script\n script: |\n current_quality = calculate_quality()\n print(f\"__OUTPUTS__ {json.dumps({\u0027quality\u0027: current_quality})}\")\n```\n\n\n### Conditional Router Tasks\n\nRoute execution to different paths based on conditions.\n\n```yaml\n- id: conditional_router_task\n name: \"Data Type Router\"\n type: conditional_router\n description: \"Route processing based on data characteristics\"\n \n # Router configuration\n router_type: condition # Default: condition\n \n # Routing conditions\n conditions: # REQUIRED: List of conditions\n - name: \"large_dataset\"\n condition: \"${data_size} \u003e 10000\"\n route: \"batch_processing\"\n - name: \"small_dataset\" \n condition: \"${data_size} \u003c= 1000\"\n route: \"direct_processing\"\n - name: \"medium_dataset\"\n condition: \"${data_size} \u003e 1000 \u0026\u0026 ${data_size} \u003c= 10000\"\n route: \"standard_processing\"\n \n default_route: \"standard_processing\" # REQUIRED: Default route if no conditions match\n \n # Alternative: evaluation script for complex routing\n evaluation_script: | # Optional: Custom routing logic\n if data_type == \"json\":\n return \"json_processing\"\n elif data_type == \"xml\":\n return \"xml_processing\"\n else:\n return \"default_processing\"\n \n # Route mapping for complex scenarios\n route_mapping: # Optional: Route name mappings\n json_processing: \"specialized_json\"\n xml_processing: \"specialized_xml\"\n\n# Tasks that execute on specific routes\n- id: batch_handler\n name: \"Handle Large Dataset\"\n type: script\n execute_on_routes: [\"batch_processing\"] # REQUIRED: Routes this task runs on\n depends_on: [conditional_router_task]\n script: |\n print(\"Processing large dataset in batches...\")\n\n- id: direct_handler\n name: \"Handle Small Dataset\" \n type: script\n execute_on_routes: [\"direct_processing\"]\n depends_on: [conditional_router_task]\n script: |\n print(\"Processing small dataset directly...\")\n```\n\n### MCP Tasks\n\nExecute tools via Model Context Protocol servers.\n\n```yaml\n- id: mcp_task\n name: \"MCP Tool Execution\"\n type: mcp\n description: \"Execute external tool via MCP server\"\n \n # MCP server configuration\n mcp_server: \"@agentic/mcp-notion-server\" # REQUIRED: MCP server package\n tool_name: \"update_database\" # REQUIRED: Tool to execute\n runtime: \"npx\" # REQUIRED: npx|uvx\n \n # Server startup\n server_args: [] # Optional: Server startup arguments\n server_startup_timeout: 30 # Optional: Server startup timeout (seconds)\n \n # Tool execution\n tool_arguments: # REQUIRED: Arguments for the tool\n database_id: \"${secrets.notion_database_id}\"\n properties:\n title: \"Workflow Results\"\n status: \"Completed\"\n processed_count: \"${previous_task.count}\"\n timestamp: \"${current_timestamp}\"\n \n # Environment configuration\n environment_vars: # Optional: Environment variables\n NOTION_TOKEN: \"${secrets.notion_token}\"\n DEBUG: \"true\"\n \n # Timeout configuration\n timeout_seconds: 120 # Optional: Tool execution timeout\n```\n\n### Storage Tasks\n\nManage data persistence and retrieval operations.\n\n```yaml\n- id: storage_task\n name: \"Data Storage Operation\"\n type: storage\n description: \"Save/load data to/from storage\"\n \n # Storage operation\n operation: save # REQUIRED: save|load|delete|list\n \n # Storage configuration\n storage_type: \"file\" # REQUIRED: file|database|cache\n path: \"/data/results/${execution_id}.json\" # REQUIRED: Storage path\n \n # Data configuration\n data: ${previous_task} # REQUIRED for \u0027save\u0027: Data to store\n format: \"json\" # Optional: json|yaml|csv|parquet\n compression: \"gzip\" # Optional: gzip|bz2|none\n \n # Metadata\n metadata: # Optional: Additional metadata\n workflow_id: \"${workflow_id}\"\n created_at: \"${current_timestamp}\"\n source_task: \"previous_task\"\n \n # File options\n create_directories: true # Optional: Create parent directories\n overwrite: true # Optional: Overwrite existing files\n backup: false # Optional: Create backup before overwrite\n \n # Database options (when storage_type: \"database\")\n table_name: \"workflow_results\" # Required for database storage\n connection_string: \"${secrets.db_url}\" # Required for database storage\n \n # Cache options (when storage_type: \"cache\")\n cache_key: \"result_${execution_id}\" # Required for cache storage\n ttl: 3600 # Optional: Time to live (seconds)\n```\n\n\n### Continuous Conversation Tasks\n\nCreate interactive AI-powered conversations.\n\n```yaml\n- id: conversation_task\n name: \"AI Conversation Interface\"\n type: continuous_conversation\n description: \"Interactive conversation with AI assistant\"\n \n # Conversation configuration\n config:\n # AI configuration\n ai_config:\n model_client_id: support_assistant # REQUIRED: AI model to use\n system_prompt: | # REQUIRED: System instructions\n You are a helpful customer support representative.\n Be friendly, professional, and solution-oriented.\n \n Guidelines:\n - Always be polite and respectful\n - Ask clarifying questions when needed\n - Provide clear and actionable solutions\n \n temperature: 0.7 # Optional: AI creativity\n max_tokens: 1000 # Optional: Response length limit\n \n # UI configuration\n ui_config:\n title: \"Customer Support Chat\" # Optional: Interface title\n placeholder: \"How can I help you today?\" # Optional: Input placeholder\n theme: \"light\" # Optional: light|dark\n show_timestamp: true # Optional: Show message timestamps\n \n # Termination conditions\n termination:\n keywords: [\"goodbye\", \"bye\", \"thanks\", \"done\"] # Optional: End conversation keywords\n max_turns: 50 # Optional: Maximum conversation turns\n timeout_minutes: 30 # Optional: Conversation timeout\n inactivity_timeout: 10 # Optional: Inactivity timeout (minutes)\n \n # Context configuration\n context:\n system_context: \"Customer support conversation\" # Optional: Context description\n user_id: \"${customer_id}\" # Optional: User identifier\n session_data: ${previous_task} # Optional: Additional context data\n```\n\n### Human Input Tasks\n\nTasks requiring human input or approval.\n\n```yaml\n- id: human_input_task\n name: \"Human Input Required\"\n type: human_input\n description: \"Wait for human input or approval\"\n \n # Human input configuration\n config:\n prompt: \"Please review and approve the processed data\" # REQUIRED: Prompt for human\n input_type: \"approval\" # REQUIRED: approval|text|selection\n \n # Input options (for selection type)\n options: [\"approve\", \"reject\", \"request_changes\"] # Optional: For selection input\n \n # Timeout configuration\n timeout_minutes: 60 # Optional: How long to wait for input (minutes)\n \n # Conversation settings\n conversation_mode: false # Optional: Add to conversation history\n next_task: \"process_approval\" # Optional: Task to trigger after input\n end_condition: \"bye|exit|quit\" # Optional: Keywords to end conversation\n \n # Input validation\n required: true # Optional: Whether input is required\n \n # Additional context\n context_data: ${previous_task} # Optional: Data to show to human\n```\n\n### ChatBot Tasks\n\nSession-based chatbot conversations with streaming support.\n\n```yaml\n- id: chatbot_task\n name: \"AI Chatbot\"\n type: chatbot\n description: \"Interactive chatbot with session management\"\n \n # Chatbot configuration\n config:\n # AI model configuration\n model_client_id: gpt4_model # REQUIRED: Reference to model client\n \n system_message: | # REQUIRED: System prompt for chatbot\n You are a helpful AI assistant. Be friendly, helpful, and concise.\n Answer questions clearly and ask for clarification when needed.\n \n # Session management\n session_id: \"${session_id}\" # Optional: Session identifier (auto-generated if not provided)\n max_history_messages: 20 # Optional: Maximum conversation history to maintain\n \n # Model parameters\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\n max_tokens: 1000 # Optional: Maximum response tokens\n stream: true # Optional: Enable streaming responses\n \n # Tools configuration for function calling\n tools: # Optional: List of MCP tool names\n - current_time # Time-related tools\n - relative_time\n - days_in_month\n - weather_forecast # Weather tools\n - send_notification # Communication tools\n \n # Required for tool resolution\n tenant_id: \"your-tenant-uuid\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\n \n # UI configuration\n ui_config:\n title: \"AI Assistant\" # Optional: Chat interface title\n placeholder: \"Type your message...\" # Optional: Input placeholder\n theme: \"light\" # Optional: light|dark\n show_typing_indicator: true # Optional: Show typing animation\n \n # Response configuration\n response_config:\n max_retries: 3 # Optional: Retry failed requests\n timeout_seconds: 30 # Optional: Response timeout\n \n # Input message (for single-turn mode)\n user_message: \"${user_input}\" # Optional: Direct message input\n \n # Context data\n context_data: ${previous_task} # Optional: Additional context\n```\n\n---\n\n## Common Errors and Corrections\n\nThis section documents frequent mistakes encountered when creating workflows and their proper corrections.\n\n### 1. Missing Required `id` Field\n\n**\u274c Error:**\n```yaml\nname: \"My Workflow\"\ndescription: \"A workflow without id\"\ntasks:\n - name: \"Task 1\"\n type: script\n```\n\n**\u2705 Correct:**\n```yaml\nid: \"my_workflow\" # REQUIRED: Unique workflow identifier\nname: \"My Workflow\"\ndescription: \"A workflow with proper id\"\ntasks:\n - id: task_1 # REQUIRED: Unique task identifier\n name: \"Task 1\"\n type: script\n```\n\n### 2. Incorrect Task Structure (Object vs Array)\n\n**\u274c Error:**\n```yaml\ntasks:\n task_1: # Wrong: tasks as object mapping\n type: script\n script: \"print(\u0027hello\u0027)\"\n task_2:\n type: ai_agent\n```\n\n**\u2705 Correct:**\n```yaml\ntasks:\n - id: task_1 # Correct: tasks as array with id field\n type: script\n script: \"print(\u0027hello\u0027)\"\n - id: task_2\n type: ai_agent\n```\n\n### 3. Incorrect Inputs Format (Object vs Array)\n\n**\u274c Error:**\n```yaml\ninputs:\n user_name: # Wrong: inputs as object mapping\n type: string\n required: true\n age:\n type: integer\n```\n\n**\u2705 Correct:**\n```yaml\ninputs:\n - name: user_name # Correct: inputs as array with name field\n type: string\n required: true\n - name: age\n type: integer\n```\n\n### 4. Script Configuration Location Error\n\n**\u274c Error:**\n```yaml\n- id: script_task\n type: script\n config: # Wrong: script under config\n script: |\n print(\"Hello\")\n requirements:\n - requests\n```\n\n**\u2705 Correct:**\n```yaml\n- id: script_task\n type: script\n script: | # Correct: script directly under task\n print(\"Hello\")\n # requirements can be at task level if needed\n```\n\n### 5. Individual Task Inputs (Not Supported)\n\n**\u274c Error:**\n```yaml\n- id: task_1\n type: script\n script: \"print(\u0027hello\u0027)\"\n inputs: # Wrong: tasks don\u0027t have individual inputs\n search_query: ${search_query}\n prev_result: ${previous_task}\n```\n\n**\u2705 Correct:**\n```yaml\n- id: task_1\n type: script\n depends_on: [previous_task] # Use depends_on for data access\n script: |\n # Access workflow inputs directly\n search_query = \"${search_query}\"\n # Access previous task outputs via template resolution\n prev_result = ${previous_task}\n```\n\n### 6. Incorrect Script Output Handling\n\n**\u274c Error:**\n```yaml\nscript: |\n result = process_data()\n outputs[\u0027result\u0027] = result # Wrong: missing __OUTPUTS__ pattern\n```\n\n**\u2705 Correct:**\n```yaml\nscript: |\n result = process_data()\n \n # Required: Use outputs dict and __OUTPUTS__ marker\n outputs = {}\n outputs[\u0027result\u0027] = result\n outputs[\u0027status\u0027] = \u0027success\u0027\n \n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n### 7. AI Agent Configuration Errors\n\n**\u274c Error:**\n```yaml\n- id: ai_task\n type: ai_agent\n config:\n model_client_id: my_model\n prompt: \"Process this data: ${data}\" # Wrong: use user_message\n```\n\n**\u2705 Correct:**\n```yaml\n- id: ai_task\n type: ai_agent\n config:\n model_client_id: my_model\n system_message: \"You are a helpful assistant.\"\n user_message: \"Process this data: ${data}\" # Correct field name\n```\n\n### 8. Missing Dependencies for Template Resolution\n\n**\u274c Error:**\n```yaml\n- id: task_2\n type: script\n # Missing depends_on\n script: |\n data = ${task_1} # Will become UNRESOLVED_task_1\n```\n\n**\u2705 Correct:**\n```yaml\n- id: task_2\n type: script\n depends_on: [task_1] # Required for template access\n script: |\n data = ${task_1} # Now resolves correctly\n```\n\n### 9. Incorrect Model Client Configuration\n\n**\u274c Error:**\n```yaml\nmodel_clients:\n my_model: # Wrong: object mapping format\n type: openai\n api_key: \"sk-...\"\n```\n\n**\u2705 Correct:**\n```yaml\nmodel_clients:\n - id: my_model # Correct: array format with id field\n provider: openai\n config:\n model: \"gpt-4\"\n api_key: \"sk-...\"\n```\n\n### 10. Empty String Literals Causing Python Syntax Errors\n\n**\u274c Error:**\n```yaml\nscript: |\n def process_data(path=\"\"): # Wrong: empty string can cause syntax issues\n current_path = f\"{path}.{key}\" if path else key\n \n # This can lead to: SyntaxError: unterminated string literal\n```\n\n**\u2705 Correct:**\n```yaml\nscript: |\n def process_data(path=None): # Correct: use None instead of empty string\n current_path = f\"{path}.{key}\" if path else key\n # Or use path = path or \"default_value\"\n```\n\n### 11. YAML Indentation and Structure Issues\n\n**\u274c Error:**\n```yaml\ntasks:\n- id: task_1\n type: script\nscript: | # Wrong: inconsistent indentation\n print(\"hello\")\n```\n\n**\u2705 Correct:**\n```yaml\ntasks:\n- id: task_1\n type: script\n script: | # Correct: consistent indentation\n print(\"hello\")\n```\n\n### Best Practices Summary\n\n1. **Always include required `id` fields** for workflows and tasks\n2. **Use array format** for `tasks`, `inputs`, and `model_clients`\n3. **Script field goes directly under task**, not under `config`\n4. **No individual task inputs** - use template resolution instead\n5. **Use `depends_on`** to access previous task outputs\n6. **Print `__OUTPUTS__`** in scripts for proper output parsing\n7. **Use `user_message`** (not `prompt`) for AI agent tasks\n8. **Avoid empty string literals** - use `None` or meaningful defaults\n9. **Follow consistent YAML indentation** throughout the file\n\n## MCP Tools and Function Calling\n\nAI agents and chatbots can use MCP (Model Context Protocol) tools for function calling, enabling them to execute external tools and services.\n\n### Tools Configuration\n\nBoth `ai_agent` and `chatbot` tasks support tools configuration:\n\n```yaml\n# AI Agent with MCP tools\n- id: ai_with_tools\n type: ai_agent\n config:\n model_client_id: gpt4_model\n system_message: |\n You are a helpful assistant with access to various tools.\n Use the available tools when needed to help users.\n user_message: \"What time is it and what\u0027s the weather like?\"\n \n # Tools configuration\n tools: # List of MCP tool names\n - current_time # Time tools\n - relative_time\n - days_in_month\n - weather_forecast # Weather tools\n - database_query # Database tools\n - send_notification # Communication tools\n \n tenant_id: \"550e8400-e29b-41d4-a716-446655440000\" # Required for tool resolution\n\n# Chatbot with MCP tools\n- id: chatbot_with_tools\n type: chatbot\n config:\n model_client_id: gpt4_model\n system_message: |\n You are a helpful chatbot with access to tools.\n Use tools when users ask for specific information or actions.\n \n # Tools configuration (same format as AI agent)\n tools:\n - current_time\n - weather_api\n - user_preferences\n \n tenant_id: \"550e8400-e29b-41d4-a716-446655440000\" # Required for tool resolution\n```\n\n### How Tools Work\n\n1. **Tool Resolution**: The workflow engine resolves tool names to actual MCP server deployments using the `tenant_id`\n2. **Function Calling**: When the AI model decides to use a tool, it calls the `tool_executor` function\n3. **Tool Execution**: The engine executes the specified tool via the appropriate MCP server\n4. **Response Integration**: Tool results are fed back to the AI model for response generation\n\n### Tool Configuration Requirements\n\n| Field | Required | Description |\n|-------|----------|-------------|\n| `tools` | No | List of MCP tool names available to the AI |\n| `tenant_id` | **Yes** (when using tools) | Tenant UUID for tool resolution |\n\n### Example: Complete AI Agent with Tools\n\n```yaml\nid: intelligent_assistant\nname: \"Intelligent Assistant with Tools\"\n\nmodel_clients:\n - id: gpt4_model\n provider: openai\n config:\n model: gpt-4o-mini\n api_key: \"${OPENAI_API_KEY}\"\n temperature: 0.3\n\n# Tenant ID can be set at workflow level for all tasks\ntenant_id: \"0572fa8d-d7c3-47db-9219-ef40b80d42b7\"\n\ntasks:\n - id: assistant_with_tools\n type: ai_agent\n config:\n model_client_id: gpt4_model\n system_message: |\n You are a helpful assistant with access to various tools:\n - Time-related tools: current_time, relative_time, days_in_month\n - Weather tools: weather_forecast\n - Communication: send_notification\n \n Use these tools when users ask for:\n - Current time or date information\n - Weather forecasts\n - Sending notifications\n \n Always explain what tool you\u0027re using and why.\n \n user_message: \"${user_query}\"\n \n # MCP tools configuration\n tools:\n - current_time\n - relative_time\n - days_in_month\n - weather_forecast\n - send_notification\n \n # Tool resolution requires tenant_id\n tenant_id: \"0572fa8d-d7c3-47db-9219-ef40b80d42b7\"\n \n temperature: 0.7\n max_tokens: 1000\n\ninputs:\n - name: user_query\n type: string\n required: true\n default: \"What time is it and can you send a notification to user123?\"\n\noutputs:\n ai_response:\n type: string\n source: assistant_with_tools.ai_response\n description: \"AI response using tools when needed\"\n```\n\n### Tool Execution Flow\n\n1. **User Input**: User provides a query that might require tool usage\n2. **AI Analysis**: AI model analyzes the query and decides if tools are needed\n3. **Tool Calling**: AI calls `tool_executor(deployment_id, tool_name, arguments)`\n4. **Tool Resolution**: Engine resolves tool name to MCP server using tenant_id\n5. **Tool Execution**: MCP server executes the requested tool\n6. **Result Processing**: Tool results are returned to the AI model\n7. **Response Generation**: AI generates final response incorporating tool results\n\n### Best Practices for Tools\n\n1. **Clear System Messages**: Explain available tools and when to use them\n2. **Tool Categories**: Group related tools (time, weather, database, etc.)\n3. **Error Handling**: AI models handle tool execution failures gracefully\n4. **Tool Documentation**: Include tool descriptions in system messages\n5. **Tenant Configuration**: Ensure tenant_id matches your MCP server setup\n\n---\n\n*This comprehensive reference covers all supported task types and their YAML configurations. Use this as a quick reference when building workflows with different task types.*",
"format": "markdown"
},
"execution_details": {
"actual_result": {
"output": "Trying to fetch documentation from: http://host.docker.internal:5000/api/docs/yaml-reference\n\u274c Connection failed: http://host.docker.internal:5000/api/docs/yaml-reference - HTTPConnectionPool(host=\u0027host.docker.internal\u0027, port=5000): Max retries exceeded with url: /api/docs/yaml-reference (Caused by NameResolutionError(\"\u003curllib3.connection.HTTPConnection object at 0x7d0629aa9010\u003e: Failed to resolve \u0027host.docker.internal\u0027 ([Errno -2] Name or service not known)\"))\nTrying to fetch documentation from: http://172.17.0.1:5000/api/docs/yaml-reference\n\u2705 Successfully connected to: http://172.17.0.1:5000/api/docs/yaml-reference\nSuccessfully fetched documentation from http://172.17.0.1:5000/api/docs/yaml-reference: 97047 characters\n__OUTPUTS__ {\"documentation\": {\"content\": \"# YAML Workflow Structure Reference\\n\\nComplete reference for the YAML workflow definition structure supported by the Assistants Workflow Engine.\\n\\n## Table of Contents\\n\\n1. [Complete Schema Overview](#complete-schema-overview)\\n2. [Required vs Optional Fields](#required-vs-optional-fields)\\n3. [Data Types Reference](#data-types-reference)\\n4. [Field Validation Rules](#field-validation-rules)\\n5. [Parser Behavior](#parser-behavior)\\n6. [Migration Guide](#migration-guide)\\n\\n## Complete Schema Overview\\n\\n```yaml\\n# === WORKFLOW METADATA (Required) ===\\nid: string # REQUIRED: Unique workflow identifier\\nname: string # REQUIRED: Human-readable workflow name\\n\\n# === WORKFLOW METADATA (Optional) ===\\nversion: string | integer # Default: 1\\nnamespace: string # Default: \\\"default\\\"\\ndescription: string # Default: \\\"\\\"\\ntenant_id: string # Default: null\\n\\n# === EXECUTION CONFIGURATION ===\\ntimeout_seconds: integer|string # Default: 3600 (1 hour) - Workflow-level timeout (supports duration strings like \\\"1h\\\", \\\"30m\\\")\\nlabels: object # Default: {}\\nmetadata: object # Default: {}\\n\\n# === DATA FLOW DEFINITIONS ===\\ninputs: # Default: []\\n - name: string # REQUIRED\\n type: string # REQUIRED: string|integer|boolean|object|array\\n description: string # Optional\\n required: boolean # Default: false\\n default: any # Optional default value\\n\\noutputs: object # Default: {}\\n output_name:\\n type: string # REQUIRED\\n source: string # REQUIRED: task_id or task_id.field\\n description: string # Optional\\n\\nvariables: object # Default: {}\\n\\n# === MODEL CLIENT CONFIGURATION (Required for AI tasks) ===\\nmodel_clients: # REQUIRED: For ai_agent, chatbot, continuous_conversation tasks\\n - id: string # REQUIRED: Unique identifier to reference in tasks\\n provider: string # REQUIRED: openai|anthropic|openrouter|groq\\n config: object # REQUIRED: Provider-specific config\\n model: string # REQUIRED: Model name (e.g., gpt-4, claude-3-sonnet-20240229)\\n api_key: string # REQUIRED: API key for the provider\\n temperature: number # Optional: 0.0-2.0, controls randomness\\n max_tokens: integer # Optional: Maximum tokens to generate\\n timeout: integer # Optional: Request timeout in seconds\\n base_url: string # Optional: Custom API base URL\\n\\n# Model Client Examples:\\n# OpenAI Configuration:\\n# model_clients:\\n# - id: gpt4_model\\n# provider: openai\\n# config:\\n# model: gpt-4-turbo-preview\\n# api_key: \\\"sk-your-openai-api-key\\\"\\n# temperature: 0.7\\n# max_tokens: 2000\\n#\\n# Anthropic Configuration:\\n# model_clients:\\n# - id: claude_model\\n# provider: anthropic\\n# config:\\n# model: claude-3-sonnet-20240229\\n# api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n# temperature: 0.7\\n# max_tokens: 1500\\n#\\n# Multiple Models:\\n# model_clients:\\n# - id: gpt4_model\\n# provider: openai\\n# config:\\n# model: gpt-4-turbo-preview\\n# api_key: \\\"sk-your-openai-api-key\\\"\\n# - id: claude_model\\n# provider: anthropic\\n# config:\\n# model: claude-3-sonnet-20240229\\n# api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n\\n# === TASK DEFINITIONS (Required) ===\\ntasks: # REQUIRED: List of tasks\\n - id: string # REQUIRED: Unique task identifier\\n name: string # Optional: Human-readable name\\n type: string # REQUIRED: Task type\\n description: string # Optional\\n \\n # DEPENDENCY CONFIGURATION\\n depends_on: array # Optional: Execution dependencies and template resolution\\n previous_node: string # REQUIRED: Task connectivity and UI hierarchy (essential for proper workflow execution)\\n \\n timeout_seconds: integer|string # Optional: Task-specific timeout (supports duration strings like \\\"5m\\\", \\\"30s\\\")\\n worker_group: string # Optional: Target worker group\\n conditions: array # Optional: Execution conditions\\n execute_on_routes: array # Optional: Routes for conditional execution\\n \\n # Input/Output mapping\\n input_mapping: object # Optional: Map workflow context to task inputs\\n output_mapping: object # Optional: Map task outputs to workflow context\\n \\n # Retry configuration\\n retry: object # Optional: Task-specific retry policy\\n maxAttempts: integer # Default: 1\\n initialDelay: string|integer # Default: \\\"1s\\\"\\n maxDelay: string|integer # Default: \\\"300s\\\"\\n backoffMultiplier: number # Default: 2.0\\n retryOn: array # Default: [\\\"TEMPORARY_FAILURE\\\"]\\n \\n # === TASK TYPE SPECIFIC CONFIGURATIONS ===\\n # See detailed sections below for each task type\\n\\n# === WORKFLOW-LEVEL RETRY POLICY ===\\nretry: object # Optional: Global retry policy\\n retryOn: array # Default: [\\\"TEMPORARY_FAILURE\\\"]\\n maxAttempts: integer # Default: 1\\n initialDelay: string|integer # Default: \\\"1s\\\"\\n maxDelay: string|integer # Default: \\\"300s\\\"\\n backoffMultiplier: number # Default: 2.0\\n\\n# === TRIGGERS ===\\ntriggers: # Optional: List of triggers\\n - name: string # REQUIRED\\n type: string # REQUIRED: webhook|schedule|event\\n description: string # Optional\\n config: object # REQUIRED: Trigger-specific config\\n input_mapping: array # Optional: Map trigger data to workflow inputs\\n - webhook_field: string # Source field\\n workflow_input: string # Target workflow input\\n```\\n\\n## Required vs Optional Fields\\n\\n### Required Fields (Workflow Level)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `id` | string | Unique identifier for the workflow |\\n| `tasks` | array | List of task definitions (can be empty) |\\n\\n### Required Fields (Task Level)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `id` | string | Unique identifier for the task |\\n| `type` | string | Task type (script, http, ai_agent, etc.) |\\n| `previous_node` | string | Reference to previous task for UI connectivity (required for proper task linking, except for the first task) |\\n\\n### Required Fields (Input Definition)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `name` | string | Input parameter name |\\n| `type` | string | Data type (string, integer, boolean, object, array) |\\n\\n### Required Fields (Output Definition)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `type` | string | Output data type |\\n| `source` | string | Source task ID or task_id.field_name |\\n\\n## Data Types Reference\\n\\n### Primitive Types\\n\\n```yaml\\n# String type\\ninputs:\\n - name: customer_name\\n type: string\\n default: \\\"John Doe\\\"\\n description: \\\"Customer full name\\\"\\n\\n# Integer type \\ninputs:\\n - name: batch_size\\n type: integer\\n default: 100\\n description: \\\"Number of records per batch\\\"\\n\\n# Boolean type\\ninputs:\\n - name: include_archived\\n type: boolean\\n default: false\\n description: \\\"Include archived records\\\"\\n```\\n\\n### Complex Types\\n\\n```yaml\\n# Object type\\ninputs:\\n - name: config_params\\n type: object\\n default:\\n timeout: 30\\n retries: 3\\n debug: false\\n description: \\\"Configuration parameters\\\"\\n\\n# Array type\\ninputs:\\n - name: customer_ids\\n type: array\\n default: [\\\"cust_001\\\", \\\"cust_002\\\"]\\n description: \\\"List of customer identifiers\\\"\\n```\\n\\n### Duration Types\\n\\nDuration values can be specified as strings or integers:\\n\\n```yaml\\n# String format (recommended)\\ntimeout_seconds: \\\"1h\\\" # 1 hour\\ntimeout_seconds: \\\"30m\\\" # 30 minutes \\ntimeout_seconds: \\\"45s\\\" # 45 seconds\\ntimeout_seconds: \\\"2d\\\" # 2 days\\n\\n# Integer format (seconds)\\ntimeout_seconds: 3600 # 1 hour in seconds\\n\\n# Supported units\\n# s, sec, second, seconds\\n# m, min, minute, minutes \\n# h, hour, hours\\n# d, day, days\\n```\\n\\n### Timeout Configuration and Execution Behavior\\n\\nThe workflow engine supports configurable timeouts at both workflow and task levels with intelligent timeout-free communication for long-running tasks.\\n\\n#### Timeout Hierarchy\\n\\nTimeout values are resolved in this order of precedence:\\n1. **Task-level `timeout_seconds`**: Specific timeout for individual tasks\\n2. **Workflow-level `timeout_seconds`**: Default for all tasks in the workflow\\n3. **System default**: 300 seconds (5 minutes) for tasks, 3600 seconds (1 hour) for workflows\\n\\n```yaml\\n# Workflow-level timeout (applies to all tasks by default)\\ntimeout_seconds: \\\"2h\\\" # 2 hour workflow timeout\\n\\ntasks:\\n - id: quick_task\\n type: script\\n # Uses workflow-level timeout (2h)\\n script: \\\"print(\u0027Quick task\u0027)\\\"\\n \\n - id: custom_timeout_task\\n type: script\\n timeout_seconds: \\\"30m\\\" # Override with 30 minute timeout\\n script: \\\"print(\u0027Custom timeout task\u0027)\\\"\\n \\n - id: long_running_task\\n type: script\\n timeout_seconds: 600 # 10 minutes (integer format)\\n script: |\\n import time\\n time.sleep(300) # 5 minute processing\\n print(\\\"Long task completed\\\")\\n```\\n\\n#### Timeout-Free Communication\\n\\nThe workflow engine automatically uses **timeout-free communication** for long-running tasks to prevent timeout-related failures:\\n\\n- **Loop tasks**: Always use timeout-free communication (event-driven execution)\\n- **Script tasks**: Use timeout-free communication when `timeout_seconds \u003e 180` (host timeout threshold)\\n- **Other task types**: Use standard timeout-based communication\\n\\n```yaml\\ntasks:\\n - id: short_script\\n type: script\\n timeout_seconds: 60 # Uses standard timeout-based communication\\n script: \\\"print(\u0027Short script\u0027)\\\"\\n \\n - id: long_script\\n type: script\\n timeout_seconds: 600 # Uses timeout-free communication (600 \u003e 180)\\n script: |\\n import time\\n print(\\\"\\ud83d\\ude80 This uses timeout-free communication\\\")\\n time.sleep(300) # Won\u0027t cause timeout issues\\n print(\\\"Long script completed\\\")\\n \\n - id: data_processing_loop\\n type: loop\\n timeout_seconds: \\\"1h\\\" # Always uses timeout-free communication\\n loop_type: for\\n max_iterations: 10\\n # Loop tasks automatically use timeout-free communication\\n```\\n\\n#### Loop Task Timeout Inheritance\\n\\nLoop tasks can pass timeout configuration to their inner script tasks:\\n\\n```yaml\\n- id: batch_processing_loop\\n type: loop\\n timeout_seconds: \\\"30m\\\" # Loop-level timeout\\n loop_type: for\\n max_iterations: 5\\n loop_tasks:\\n - id: process_batch\\n type: script\\n # Inherits 30-minute timeout from parent loop\\n script: |\\n print(\\\"Processing batch with inherited timeout\\\")\\n # This script will use the 30-minute timeout from the loop\\n```\\n\\n#### Best Practices for Timeout Configuration\\n\\n1. **Use duration strings** for readability: `\\\"5m\\\"` vs `300`\\n2. **Set appropriate timeouts** based on expected execution time plus buffer\\n3. **Long-running tasks**: Set `timeout_seconds \u003e 180` to enable timeout-free communication\\n4. **Loop tasks**: Always get timeout-free communication automatically\\n5. **Critical tasks**: Use shorter timeouts for fail-fast behavior\\n6. **External API calls**: Use longer timeouts to account for network latency\\n\\n```yaml\\n# Example: Well-configured timeout workflow\\ntimeout_seconds: \\\"1h\\\" # Reasonable workflow timeout\\n\\ntasks:\\n - id: api_call\\n type: http\\n timeout_seconds: \\\"30s\\\" # Quick timeout for API calls\\n url: \\\"https://api.example.com/data\\\"\\n \\n - id: data_processing\\n type: script\\n timeout_seconds: \\\"10m\\\" # Longer timeout for processing (uses timeout-free)\\n script: |\\n # Process large dataset\\n process_data()\\n \\n - id: batch_processing\\n type: loop\\n timeout_seconds: \\\"45m\\\" # Generous timeout for loop (timeout-free)\\n loop_type: for\\n max_iterations: 100\\n # Inner tasks inherit the 45-minute timeout\\n```\\n\\n## Field Validation Rules\\n\\n### ID Fields\\n\\n```yaml\\n# Valid IDs\\nid: \\\"customer_processor\\\" # \\u2705 Valid\\nid: \\\"process-data\\\" # \\u2705 Valid\\nid: \\\"task_001\\\" # \\u2705 Valid\\n\\n# Invalid IDs\\nid: \\\"process data\\\" # \\u274c Spaces not allowed\\nid: \\\"123_start\\\" # \\u274c Cannot start with number\\nid: \\\"\\\" # \\u274c Cannot be empty\\n```\\n\\n### Timeout Values\\n\\n```yaml\\n# Valid timeouts\\ntimeout_seconds: 300 # \\u2705 Positive integer\\ntimeout_seconds: \\\"5m\\\" # \\u2705 Duration string\\ntimeout_seconds: \\\"1h30m\\\" # \\u2705 Complex duration\\n\\n# Invalid timeouts\\ntimeout_seconds: 0 # \\u274c Must be positive\\ntimeout_seconds: -60 # \\u274c Cannot be negative\\ntimeout_seconds: \\\"invalid\\\" # \\u274c Invalid duration format\\n```\\n\\n### Dependency Types and Validation\\n\\n#### `depends_on` vs `previous_node`\\n\\n**`depends_on` - Execution Dependencies:**\\n- Controls actual task execution order\\n- Enables template resolution and data access\\n- Required for accessing outputs from other tasks\\n- Validates that referenced tasks exist\\n- Multiple tasks can be specified\\n\\n**`previous_node` - Task Connectivity and UI Hierarchy:**\\n- **REQUIRED for proper task connectivity in UI**\\n- Essential for workflow visualization and task linking\\n- Critical for execution flow representation\\n- Single task reference only\\n- **Without this field, tasks won\u0027t be properly connected in the UI**\\n\\n```yaml\\ntasks:\\n - id: fetch_data\\n type: http\\n url: \\\"https://api.example.com/data\\\"\\n \\n - id: validate_data\\n type: script\\n depends_on: [fetch_data] # \\u2705 EXECUTION: wait for fetch_data to complete\\n previous_node: fetch_data # \\u2705 UI: show as child of fetch_data in hierarchy\\n script: |\\n data = ${fetch_data} # \\u2705 Can access because of depends_on\\n if not validate(data):\\n raise ValueError(\\\"Invalid data\\\")\\n \\n # Required: Use outputs dict and __OUTPUTS__ marker for proper parsing\\n outputs[\u0027status\u0027] = \u0027valid\u0027\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: process_data\\n type: script\\n depends_on: [validate_data] # \\u2705 EXECUTION: wait for validation\\n previous_node: validate_data # \\u2705 UI: show as child of validate_data\\n script: |\\n validation_result = ${validate_data} # \\u2705 Accessible via depends_on\\n # Process data...\\n\\n - id: separate_task\\n type: script\\n previous_node: process_data # \\u2705 UI: appears under process_data in hierarchy\\n # NO depends_on means this can run immediately (parallel to other tasks)\\n script: |\\n # This task runs independently but appears under process_data in UI\\n # Cannot access ${process_data} because no depends_on relationship\\n print(\\\"Independent task\\\")\\n\\n### Complete Example with Proper Hierarchy\\n\\n```yaml\\ntasks:\\n - id: fetch_customer_data\\n name: \\\"Fetch Customer Data\\\"\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: validate_customer\\n name: \\\"Validate Customer Data\\\"\\n type: script\\n depends_on: [fetch_customer_data] # \\u2705 EXECUTION: Wait for data fetch\\n previous_node: fetch_customer_data # \\u2705 UI: Show under fetch_customer_data\\n script: |\\n customer = ${fetch_customer_data} # \\u2705 Can access because of depends_on\\n if not customer.get(\u0027email\u0027):\\n raise ValueError(\\\"Invalid customer data\\\")\\n print(\\\"__OUTPUTS__ {\u0027status\u0027: \u0027valid\u0027}\\\")\\n \\n - id: process_customer\\n name: \\\"Process Customer\\\"\\n type: ai_agent\\n depends_on: [validate_customer] # \\u2705 EXECUTION: Wait for validation\\n previous_node: validate_customer # \\u2705 UI: Show under validate_customer\\n config:\\n model_client_id: gpt4_model\\n user_message: \\\"Process customer: ${fetch_customer_data}\\\"\\n \\n - id: send_notification\\n name: \\\"Send Notification\\\"\\n type: script\\n depends_on: [process_customer] # \\u2705 EXECUTION: Wait for processing\\n previous_node: process_customer # \\u2705 UI: Show under process_customer\\n script: |\\n result = ${process_customer}\\n print(f\\\"Notification sent for: ${fetch_customer_data.name}\\\")\\n \\n - id: parallel_audit_log\\n name: \\\"Audit Log (Parallel)\\\"\\n type: script\\n depends_on: [fetch_customer_data] # \\u2705 EXECUTION: Only needs initial data\\n previous_node: fetch_customer_data # \\u2705 UI: Show under fetch_customer_data\\n script: |\\n # This runs in parallel with validation/processing\\n customer = ${fetch_customer_data}\\n print(f\\\"Audit: Customer {customer.id} accessed at ${current_timestamp}\\\")\\n```\\n\\n # Invalid examples\\n - id: invalid_task\\n type: script\\n depends_on: [nonexistent_task] # \\u274c VALIDATION ERROR: task doesn\u0027t exist\\n previous_node: another_missing # \\u26a0\\ufe0f UI WARNING: task doesn\u0027t exist (but not fatal)\\n script: \\\"print(\u0027Invalid\u0027)\\\"\\n```\\n\\n#### Key Differences:\\n\\n| Aspect | `depends_on` | `previous_node` |\\n|--------|--------------|-----------------|\\n| **Purpose** | Execution control | Task connectivity \u0026 UI organization |\\n| **Impact** | Blocks task until dependencies complete | **REQUIRED for UI task linking** |\\n| **Template Access** | \\u2705 Enables `${task_id}` resolution | \\u274c No template access |\\n| **Validation** | \\u274c Parser error if task missing | \\u26a0\\ufe0f Warning if task missing |\\n| **Multiple References** | \\u2705 Array of task IDs | \\u274c Single task ID only |\\n| **Execution Order** | \\u2705 Enforced | \\u274c No execution impact |\\n| **UI Connectivity** | \\u274c No UI impact | \\u2705 **ESSENTIAL for proper task connection** |\\n\\n### Circular Dependency Detection\\n\\n```yaml\\n# This will be rejected by the parser\\ntasks:\\n - id: task_a\\n depends_on: [task_b] # \\u274c Circular dependency\\n \\n - id: task_b \\n depends_on: [task_a] # \\u274c Circular dependency\\n```\\n\\n## Parser Behavior\\n\\n### Field Name Compatibility\\n\\nThe parser supports both camelCase and snake_case for field names:\\n\\n```yaml\\ntasks:\\n - id: flexible_task\\n # Both formats are accepted\\n dependsOn: [other_task] # camelCase\\n depends_on: [other_task] # snake_case\\n \\n inputMapping: # camelCase\\n source_field: \\\"target_field\\\"\\n input_mapping: # snake_case \\n source_field: \\\"target_field\\\"\\n \\n outputMapping: {} # camelCase\\n output_mapping: {} # snake_case\\n \\n workerGroup: \\\"gpu-workers\\\" # camelCase\\n worker_group: \\\"gpu-workers\\\" # snake_case\\n```\\n\\n### Default Value Assignment\\n\\n```yaml\\n# Fields with defaults are automatically populated\\nid: my_workflow\\n# name defaults to id value: \\\"my_workflow\\\"\\n# version defaults to: 1\\n# namespace defaults to: \\\"default\\\" \\n# timeout_seconds defaults to: 3600\\n# description defaults to: \\\"\\\"\\n\\ntasks:\\n - id: simple_task\\n type: script\\n # name defaults to id: \\\"simple_task\\\"\\n # timeout_seconds defaults to: 300\\n # retry.maxAttempts defaults to: 1\\n```\\n\\n### Template Resolution Context\\n\\nThe parser provides these built-in variables for template resolution:\\n\\n```yaml\\ntasks:\\n - id: context_aware_task\\n type: script\\n script: |\\n # Workflow context\\n workflow_id = \\\"${workflow_id}\\\" # Current workflow ID\\n execution_id = \\\"${execution_id}\\\" # Unique execution ID\\n namespace = \\\"${namespace}\\\" # Workflow namespace\\n \\n # Timing context\\n current_timestamp = \\\"${current_timestamp}\\\" # ISO timestamp\\n started_at = \\\"${started_at}\\\" # Workflow start time\\n \\n # Input context\\n user_input = \\\"${input_parameter}\\\" # Workflow inputs\\n \\n # Task context\\n previous_result = ${previous_task} # Previous task output\\n specific_field = ${task_id.field_name} # Specific output field\\n \\n # Route context (for conditional workflows)\\n route_info = ${_route_context} # Current route information\\n```\\n\\n### Error Handling During Parsing\\n\\n```yaml\\n# The parser will validate and report these errors:\\n\\n# 1. Missing required fields\\n# ERROR: Missing required field: \u0027id\u0027\\n\\n# 2. Invalid YAML syntax \\n# ERROR: Invalid YAML: mapping values are not allowed here\\n\\n# 3. Circular dependencies\\n# ERROR: Circular dependency detected involving task \u0027task_a\u0027\\n\\n# 4. Invalid task references\\n# ERROR: Task \u0027task_b\u0027 depends on non-existent task \u0027invalid_task\u0027\\n\\n# 5. Invalid duration formats\\n# ERROR: Invalid duration: \u0027not_a_duration\u0027\\n\\n# 6. Invalid field values\\n# ERROR: Timeout must be positive, got -10\\n```\\n\\n## Migration Guide\\n\\n### From Version 1.0 to 2.0\\n\\n```yaml\\n# OLD FORMAT (v1.0)\\nworkflow:\\n id: my_workflow\\n tasks:\\n - task_id: process_data\\n task_type: python_script\\n \\n# NEW FORMAT (v2.0) \\nid: my_workflow\\ntasks:\\n - id: process_data\\n type: script\\n language: python\\n```\\n\\n### Deprecated Fields\\n\\n| Old Field | New Field | Status |\\n|-----------|-----------|--------|\\n| `task_type` | `type` | Deprecated in v2.0 |\\n| `task_id` | `id` | Deprecated in v2.0 |\\n| `python_script` | `script` (type) | Deprecated in v2.0 |\\n| `plugin` | `mcp` (type) | Deprecated in v2.1 |\\n\\n### Backward Compatibility\\n\\nThe parser maintains backward compatibility for deprecated fields but will show warnings:\\n\\n```yaml\\n# This will work but show a warning\\ntasks:\\n - task_id: old_style_task # \\u26a0\\ufe0f Warning: Use \u0027id\u0027 instead\\n task_type: python_script # \\u26a0\\ufe0f Warning: Use \u0027type: script\u0027 instead\\n```\\n\\n## Advanced Schema Features\\n\\n### Conditional Field Requirements\\n\\nSome fields become required based on task type:\\n\\n```yaml\\n# HTTP tasks require URL\\n- id: api_call\\n type: http\\n url: \\\"https://api.example.com\\\" # \\u2705 Required for HTTP tasks\\n \\n# AI agent tasks require model_client_id\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: gpt4 # \\u2705 Required for AI agent tasks\\n```\\n\\n### Dynamic Field Validation\\n\\nField validation can depend on other field values:\\n\\n```yaml\\n# Loop tasks require loop-specific fields\\n- id: data_loop\\n type: loop\\n loop_type: \\\"for\\\" # Determines required fields\\n max_iterations: 10 # \\u2705 Required for \u0027for\u0027 loops\\n # exit_condition: not needed for \u0027for\u0027 loops\\n \\n- id: condition_loop \\n type: loop\\n loop_type: \\\"while\\\" # Determines required fields\\n exit_condition: \\\"${count} \u003e= 10\\\" # \\u2705 Required for \u0027while\u0027 loops\\n # max_iterations: optional for \u0027while\u0027 loops\\n```\\n\\n## Template Resolution and Data Flow\\n\\nTemplate resolution is a core feature that allows tasks to access data from previous tasks, workflow inputs, and built-in variables using the `${variable_name}` syntax. This is essential for creating dynamic workflows where tasks pass data to each other.\\n\\n### Basic Template Syntax\\n\\n```yaml\\n# Access workflow inputs\\nscript: |\\n customer_name = \\\"${customer_name}\\\" # From workflow inputs\\n \\n# Access previous task outputs\\nscript: |\\n previous_result = ${data_fetch_task} # Entire output from data_fetch_task\\n specific_field = ${data_fetch_task.customer_id} # Specific field\\n \\n# Access built-in variables\\nscript: |\\n workflow_id = \\\"${workflow_id}\\\"\\n execution_id = \\\"${execution_id}\\\"\\n timestamp = \\\"${current_timestamp}\\\"\\n```\\n\\n### Available Built-in Variables\\n\\n| Variable | Description | Example Value |\\n|----------|-------------|---------------|\\n| `${workflow_id}` | Current workflow identifier | `customer_processor` |\\n| `${execution_id}` | Unique execution identifier | `exec_12345_abcde` |\\n| `${namespace}` | Workflow namespace | `production` |\\n| `${current_timestamp}` | ISO timestamp | `2024-01-15T10:30:00Z` |\\n| `${started_at}` | Workflow start time | `2024-01-15T10:29:45Z` |\\n\\n### Task Output Resolution\\n\\nTasks can access outputs from other tasks they depend on:\\n\\n```yaml\\ntasks:\\n - id: fetch_customer\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: process_customer\\n type: script\\n depends_on: [fetch_customer] # Required for template access\\n previous_node: fetch_customer\\n script: |\\n # Access entire response\\n customer_data = ${fetch_customer}\\n \\n # Access specific fields\\n customer_name = \\\"${fetch_customer.name}\\\"\\n customer_email = \\\"${fetch_customer.email}\\\"\\n \\n # Process the data\\n result = {\\n \\\"processed_name\\\": customer_name.upper(),\\n \\\"domain\\\": customer_email.split(\\\"@\\\")[1]\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n```\\n\\n### AI Agent Output Resolution\\n\\nAI agent tasks return structured output that requires specific access patterns:\\n\\n```yaml\\n# AI Agent task\\n- id: ai_analysis_task\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Analyze the customer data and provide insights.\\\"\\n user_message: \\\"Customer data: ${customer_data}\\\"\\n\\n# Accessing AI agent output in subsequent tasks\\n- id: process_ai_response\\n type: script\\n depends_on: [ai_analysis_task]\\n previous_node: ai_analysis_task\\n script: |\\n # CORRECT: AI agents return structured output\\n agent_output = ${ai_analysis_task}\\n ai_response = agent_output[\u0027ai_response\u0027].strip()\\n \\n # Additional metadata available\\n token_usage = agent_output.get(\u0027token_usage\u0027, {})\\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\\n \\n # Process the AI response\\n processed_analysis = {\\n \\\"analysis\\\": ai_response,\\n \\\"tokens_used\\\": token_usage.get(\u0027total_tokens\u0027, 0),\\n \\\"model\\\": model_used,\\n \\\"processed_at\\\": \\\"${current_timestamp}\\\"\\n }\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(processed_analysis)}\\\")\\n\\n# INCORRECT: Direct access won\u0027t work\\n- id: incorrect_ai_access\\n type: script\\n depends_on: [ai_analysis_task]\\n script: |\\n # \\u274c This will not work correctly\\n ai_response = ${ai_analysis_task} # Returns full agent object\\n # ai_response is now a dict, not the actual response text\\n```\\n\\n### AI Agent Response Structure\\n\\nAI agent tasks return a structured object with the following format:\\n\\n```json\\n{\\n \\\"ai_response\\\": \\\"The actual AI-generated text response\\\",\\n \\\"model\\\": \\\"gpt-4-turbo-preview\\\",\\n \\\"token_usage\\\": {\\n \\\"prompt_tokens\\\": 150,\\n \\\"completion_tokens\\\": 75,\\n \\\"total_tokens\\\": 225\\n },\\n \\\"estimated_cost\\\": {\\n \\\"amount\\\": 0.00337,\\n \\\"currency\\\": \\\"USD\\\"\\n },\\n \\\"provider\\\": \\\"openai\\\",\\n \\\"timestamp\\\": \\\"2024-01-15T10:30:00Z\\\"\\n}\\n```\\n\\n### Loop Task Data Access and Dependencies\\n\\nLoop tasks have a special execution context that differs from regular workflow tasks. Understanding how to access data within loops is crucial for building functional workflows.\\n\\n#### Loop Task Internal Data Access\\n\\nWithin loop tasks, data access works through injected context files rather than template resolution:\\n\\n```yaml\\n# Loop task with state variables and internal task dependencies\\n- id: data_processing_loop\\n type: loop\\n loop_type: for\\n max_iterations: 3\\n state_variables:\\n processed_items: []\\n total_processed: 0\\n error_count: 0\\n loop_tasks:\\n # First task in loop\\n - id: fetch_data\\n type: script\\n script: |\\n # Access loop state variables through injected context\\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\\n processed_items = loop_state.get(\u0027processed_items\u0027, [])\\n \\n # Access workflow inputs and external task outputs\\n external_data = inputs.get(\u0027previous_workflow_task\u0027, {})\\n iteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable\\n \\n # Process data\\n batch_data = fetch_batch_data(iteration_num)\\n \\n outputs = {\\\"batch_data\\\": batch_data, \\\"batch_size\\\": len(batch_data)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n # Second task depends on first task in same loop\\n - id: process_data\\n type: script\\n depends_on: [fetch_data] # \\u2705 CORRECT: Dependencies within loop tasks work normally\\n script: |\\n # \\u2705 CORRECT: Access previous loop task through inputs context\\n fetch_result = inputs.get(\u0027fetch_data\u0027, {})\\n batch_data = fetch_result.get(\u0027batch_data\u0027, [])\\n \\n # \\u274c INCORRECT: This won\u0027t work in loop context\\n # batch_data = ${fetch_data.batch_data} # Template resolution fails in loops\\n \\n # Access and update state variables\\n current_processed = loop_state.get(\u0027processed_items\u0027, [])\\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\\n \\n # Process the batch\\n processed_batch = process_batch(batch_data)\\n current_processed.extend(processed_batch)\\n \\n # Update state variables using state_updates\\n state_updates[\u0027processed_items\u0027] = current_processed\\n state_updates[\u0027total_processed\u0027] = current_total + len(processed_batch)\\n \\n outputs = {\\\"processed_count\\\": len(processed_batch)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n print(f\\\"__STATE_UPDATES__ {json.dumps(state_updates)}\\\") # \\u2705 REQUIRED for state updates\\n```\\n\\n#### Key Differences in Loop Task Data Access\\n\\n| Access Pattern | Regular Tasks | Loop Tasks |\\n|---------------|---------------|------------|\\n| **Previous Task Output** | `${previous_task}` | `inputs.get(\u0027field_name\u0027, default)` (flattened) |\\n| **State Variables** | N/A | `loop_state.get(\u0027variable_name\u0027, default)` |\\n| **State Updates** | N/A | `state_updates[\u0027variable_name\u0027] = value` |\\n| **Workflow Inputs** | `\\\"${input_name}\\\"` | `inputs.get(\u0027input_name\u0027, \u0027default\u0027)` |\\n| **Dependencies** | `depends_on: [task_id]` | Same: `depends_on: [task_id]` |\\n\\n#### Critical: Loop Task Output Flattening\\n\\n**Important:** The loop executor flattens task outputs into individual fields in the `inputs` dictionary, rather than storing them as nested objects under the task name.\\n\\n```yaml\\n# First loop task outputs:\\noutputs = {\\\"company_name\\\": \\\"ACME Corp\\\", \\\"company_id\\\": \\\"123\\\", \\\"status\\\": \\\"success\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n\\n# Second loop task accessing the data:\\nscript: |\\n # \\u274c INCORRECT: This won\u0027t work - no nested object\\n # company_data = inputs.get(\u0027first_task\u0027, {})\\n # company_name = company_data.get(\u0027company_name\u0027)\\n \\n # \\u2705 CORRECT: Access flattened fields directly\\n company_name = inputs.get(\u0027company_name\u0027, \u0027\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\\n \\n print(f\\\"Company: {company_name}, ID: {company_id}, Status: {task_status}\\\")\\n```\\n\\n#### Complete Loop Task Example\\n\\n```yaml\\n- id: financial_processing_loop\\n type: loop\\n loop_type: for\\n max_iterations: 3\\n iteration_variable: company_index\\n depends_on: [list_companies] # External dependency\\n \\n state_variables:\\n processed_companies: []\\n total_extractions: 0\\n successful_extractions: 0\\n \\n loop_tasks:\\n - id: process_current_company\\n type: script\\n script: |\\n # Access external workflow task data\\n companies_data = inputs.get(\u0027list_companies\u0027, {})\\n company_idx = inputs.get(\u0027company_index\u0027, 0)\\n \\n # Access state variables\\n processed = loop_state.get(\u0027processed_companies\u0027, [])\\n \\n # Process company\\n company = companies_data[\u0027companies\u0027][company_idx]\\n result = process_company(company)\\n \\n outputs = {\\\"company_name\\\": company[\u0027name\u0027], \\\"status\\\": \\\"success\\\"}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: extract_financial_data\\n type: script\\n depends_on: [process_current_company] # Depend on previous loop task\\n requirements:\\n - requests\u003e=2.28.0\\n - some-ai-library\u003e=1.0.0\\n script: |\\n # \\u2705 CORRECT: Access flattened outputs from previous loop task\\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\\n \\n # \\u274c INCORRECT: Loop executor flattens outputs, no nested access\\n # company_data = inputs.get(\u0027process_current_company\u0027, {})\\n # company_name = company_data.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n \\n # \\u274c INCORRECT: Template resolution doesn\u0027t work in loops\\n # company_data = ${process_current_company}\\n \\n # Access external data\\n companies_list = inputs.get(\u0027list_companies\u0027, {})\\n \\n # Perform AI extraction\\n extraction_result = extract_with_ai({\\\"company_name\\\": company_name, \\\"company_id\\\": company_id})\\n \\n # Update state variables\\n current_processed = loop_state.get(\u0027processed_companies\u0027, [])\\n current_total = loop_state.get(\u0027total_extractions\u0027, 0)\\n current_successful = loop_state.get(\u0027successful_extractions\u0027, 0)\\n \\n current_processed.append({\\n \\\"company_name\\\": company_name,\\n \\\"extraction_result\\\": extraction_result\\n })\\n \\n # State updates are required for persistence\\n state_updates[\u0027processed_companies\u0027] = current_processed\\n state_updates[\u0027total_extractions\u0027] = current_total + 1\\n state_updates[\u0027successful_extractions\u0027] = current_successful + (1 if extraction_result else 0)\\n \\n outputs = {\\\"extracted\\\": bool(extraction_result)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n print(f\\\"__STATE_UPDATES__ {json.dumps(state_updates)}\\\") # \\u2705 REQUIRED\\n\\n# Accessing loop task outputs in subsequent workflow tasks\\n- id: process_loop_results\\n type: script\\n depends_on: [financial_processing_loop]\\n previous_node: financial_processing_loop\\n script: |\\n # \\u2705 CORRECT: Access final state of loop\\n loop_result = ${financial_processing_loop}\\n loop_final_state = loop_result.get(\u0027final_state\u0027, {})\\n \\n # Access specific state variables from final state\\n processed_companies = loop_final_state.get(\u0027processed_companies\u0027, [])\\n total_processed = loop_final_state.get(\u0027total_extractions\u0027, 0)\\n successful_count = loop_final_state.get(\u0027successful_extractions\u0027, 0)\\n \\n # Access loop execution metadata\\n iterations_completed = loop_result.get(\u0027iterations_completed\u0027, 0)\\n \\n print(f\\\"Loop completed {iterations_completed} iterations\\\")\\n print(f\\\"Processed {len(processed_companies)} companies\\\")\\n print(f\\\"Success rate: {successful_count}/{total_processed}\\\")\\n \\n outputs = {\\n \\\"companies_processed\\\": len(processed_companies),\\n \\\"success_rate\\\": f\\\"{successful_count}/{total_processed}\\\",\\n \\\"iterations\\\": iterations_completed\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n#### Loop Task Context Injection and Output Flattening\\n\\nThe loop executor injects context through temporary files and flattens task outputs for easier access:\\n\\n```python\\n# Auto-injected at the start of each loop task script:\\nimport json\\nimport os\\n\\n# Load context from temporary files (auto-generated)\\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\\n inputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\\n\\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\\n loop_state = json.load(f) # Contains current state variables\\n\\noutputs = {}\\nstate_updates = {}\\n\\n# Your script code follows...\\n```\\n\\n#### Debugging Loop Task Data Access\\n\\nWhen debugging loop tasks, always print available keys to understand what data is accessible:\\n\\n```yaml\\n- id: debug_loop_task\\n type: script\\n depends_on: [previous_loop_task]\\n script: |\\n # Debug: Print all available data\\n print(f\\\"\\ud83d\\udd0d Available inputs: {list(inputs.keys())}\\\")\\n print(f\\\"\\ud83d\\udd0d Loop state keys: {list(loop_state.keys())}\\\")\\n \\n # Debug: Print specific values\\n for key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\\n value = inputs.get(key, \u0027NOT_FOUND\u0027)\\n print(f\\\"\\ud83d\\udd0d {key}: {value}\\\")\\n \\n # Now access the flattened data\\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n \\n # Process your data...\\n outputs = {\\\"processed\\\": f\\\"Company {company_name}\\\"}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n#### Real Example: Task Output Flattening\\n\\n```yaml\\n# Task 1 outputs this structure:\\n__OUTPUTS__ {\\\"company_name\\\": \\\"LTTS\\\", \\\"company_id\\\": \\\"123\\\", \\\"fy_folders\\\": [{\\\"id\\\": \\\"456\\\", \\\"name\\\": \\\"FY24\\\"}], \\\"status\\\": \\\"success\\\"}\\n\\n# Task 2 receives flattened inputs:\\ninputs = {\\n \\\"folder_id\\\": \\\"...\\\", # Workflow input\\n \\\"gemini_api_key\\\": \\\"...\\\", # Workflow input \\n \\\"list_company_folders\\\": {...}, # External task output\\n \\\"company_index\\\": 0, # Loop iteration variable\\n \\\"company_name\\\": \\\"LTTS\\\", # \\u2705 Flattened from Task 1\\n \\\"company_id\\\": \\\"123\\\", # \\u2705 Flattened from Task 1\\n \\\"fy_folders\\\": [{\\\"id\\\": \\\"456\\\", \\\"name\\\": \\\"FY24\\\"}], # \\u2705 Flattened from Task 1\\n \\\"status\\\": \\\"success\\\", # \\u2705 Flattened from Task 1\\n \\\"total_extractions\\\": 0, # Loop state variable\\n \\\"processed_companies\\\": [] # Loop state variable\\n}\\n```\\n\\n#### Loop Task Special Output Fields\\n\\n| Field | Description | Example Access |\\n|-------|-------------|----------------|\\n| `final_state` | Final values of all state variables | `${loop_task.final_state}` |\\n| `final_state[\\\"variable_name\\\"]` | Specific state variable from final state | `${loop_task.final_state[\\\"total_processed\\\"]}` |\\n| `iterations_completed` | Number of iterations executed | `${loop_task.iterations_completed}` |\\n\\n#### Safe Template Resolution Pattern for Loop States\\n\\nWhen accessing loop final state fields, use this safe parsing pattern:\\n\\n```yaml\\nscript: |\\n try:\\n # Template resolution with safety checks\\n final_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\\n processed_count_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_count\\\"]}\u0027\u0027\u0027\\n items_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_items\\\"]}\u0027\u0027\u0027\\n iterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027\\n \\n # Safe parsing with fallbacks\\n final_state = json.loads(final_state_str) if not final_state_str.startswith(\u0027UNRESOLVED_\u0027) else {}\\n processed_count = int(processed_count_str) if processed_count_str.isdigit() else 0\\n processed_items = json.loads(items_str) if not items_str.startswith(\u0027UNRESOLVED_\u0027) else []\\n iterations = int(iterations_str) if iterations_str.isdigit() else 0\\n \\n print(f\\\"Successfully processed {processed_count} items in {iterations} iterations\\\")\\n \\n except Exception as e:\\n print(f\\\"Error accessing loop results: {str(e)}\\\")\\n # Handle error case\\n```\\n```\\n\\n### Real-World AI Agent Example\\n\\nBased on actual WhatsApp workflow patterns:\\n\\n```yaml\\n# AI greeting agent\\n- id: greeting_agent\\n type: ai_agent\\n config:\\n model_client_id: assistant_model\\n system_message: \\\"You are a helpful assistant. Generate a friendly greeting.\\\"\\n user_message: \\\"User said: ${user_message}\\\"\\n\\n# Send the AI response via API\\n- id: send_greeting\\n type: script\\n depends_on: [greeting_agent]\\n previous_node: greeting_agent\\n script: |\\n import requests\\n import json\\n \\n # CORRECT: Access AI response from structured output\\n agent_output = ${greeting_agent}\\n ai_greeting = agent_output[\u0027ai_response\u0027].strip()\\n \\n phone_number = \\\"${phone_number}\\\"\\n api_token = \\\"${API_TOKEN}\\\"\\n \\n print(f\\\"\\ud83e\\udd16 AI generated greeting: {ai_greeting}\\\")\\n \\n # Send via API\\n url = f\\\"https://api.example.com/messages/{phone_number}\\\"\\n headers = {\\\"Authorization\\\": f\\\"Bearer {api_token}\\\"}\\n \\n response = requests.post(url, headers=headers, json={\\n \\\"message\\\": ai_greeting\\n })\\n \\n result = {\\n \\\"status\\\": \\\"sent\\\",\\n \\\"ai_response\\\": ai_greeting,\\n \\\"api_response\\\": response.json(),\\n \\\"tokens_used\\\": agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\\n }\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n```\\n\\n### Complex Template Patterns\\n\\n```yaml\\n# Conditional templates with router outputs\\n- id: router_task\\n type: conditional_router\\n conditions:\\n - name: \\\"premium_customer\\\"\\n condition: \\\"${customer_data.tier} == \u0027premium\u0027\\\"\\n route: \\\"premium_processing\\\"\\n\\n- id: premium_handler\\n type: script\\n depends_on: [router_task]\\n previous_node: router_task\\n execute_on_routes: [\\\"premium_processing\\\"]\\n script: |\\n # Access router decision\\n selected_route = \\\"${router_task.selected_route}\\\"\\n route_data = ${router_task.route_data}\\n \\n # Process premium customer\\n customer = ${customer_data}\\n premium_benefits = calculate_benefits(customer)\\n\\n# Loop through arrays from previous tasks\\n- id: process_items\\n type: script\\n depends_on: [fetch_items]\\n previous_node: fetch_items\\n script: |\\n items = ${fetch_items.data}\\n processed_items = []\\n \\n for item in items:\\n processed_item = {\\n \\\"id\\\": item[\\\"id\\\"],\\n \\\"processed_at\\\": \\\"${current_timestamp}\\\",\\n \\\"status\\\": \\\"processed\\\"\\n }\\n processed_items.append(processed_item)\\n \\n print(f\\\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\\\")\\n\\n# Nested object access\\n- id: user_preferences\\n type: script\\n depends_on: [fetch_user]\\n previous_node: fetch_user\\n script: |\\n # Access nested user preferences\\n email_notifications = ${fetch_user.preferences.notifications.email}\\n theme_setting = \\\"${fetch_user.preferences.ui.theme}\\\"\\n language = \\\"${fetch_user.profile.language}\\\"\\n \\n settings = {\\n \\\"notifications\\\": email_notifications,\\n \\\"theme\\\": theme_setting,\\n \\\"locale\\\": language\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(settings)}\\\")\\n```\\n\\n### Template Resolution Rules\\n\\n1. **Dependency Requirement**: Tasks must have `depends_on` relationship to access other task outputs\\n2. **String vs Object**: Use quotes for string values, no quotes for objects/arrays\\n3. **Field Navigation**: Use dot notation for nested field access\\n4. **Error Handling**: Unresolved templates become `UNRESOLVED_variable_name`\\n\\n```yaml\\n# \\u2705 Correct template usage\\n- id: correct_task\\n depends_on: [previous_task]\\n previous_node: previous_task\\n script: |\\n string_value = \\\"${previous_task.name}\\\" # String with quotes\\n object_value = ${previous_task.data} # Object without quotes\\n nested_value = \\\"${previous_task.user.email}\\\" # Nested field\\n\\n# \\u274c Incorrect template usage \\n- id: incorrect_task\\n # Missing depends_on - template won\u0027t resolve\\n script: |\\n value = ${previous_task.name} # Will become UNRESOLVED_previous_task.name\\n```\\n\\n## Script Task Output Handling\\n\\nScript tasks must use the `__OUTPUTS__` marker pattern for proper output parsing by the workflow engine.\\n\\n### Required Output Pattern\\n\\n```yaml\\ntasks:\\n - id: example_script\\n type: script\\n script: |\\n # Your script logic here\\n data = process_data()\\n \\n # Required: Store outputs in outputs dict\\n outputs = {}\\n outputs[\u0027result\u0027] = data\\n outputs[\u0027status\u0027] = \u0027success\u0027\\n outputs[\u0027count\u0027] = len(data)\\n \\n # Required: Print with __OUTPUTS__ marker for engine parsing\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### Output Access in Subsequent Tasks\\n\\n```yaml\\ntasks:\\n - id: data_processor\\n type: script\\n script: |\\n # Process and store outputs\\n results = {\\\"processed_count\\\": 100, \\\"errors\\\": []}\\n \\n outputs = {}\\n outputs[\u0027results\u0027] = results\\n outputs[\u0027status\u0027] = \u0027completed\u0027\\n \\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: result_handler\\n type: script\\n depends_on: [data_processor]\\n script: |\\n # Access previous task outputs\\n processor_output = ${data_processor}\\n processed_count = processor_output[\u0027results\u0027][\u0027processed_count\u0027]\\n status = processor_output[\u0027status\u0027]\\n \\n print(f\\\"Previous task processed {processed_count} items with status: {status}\\\")\\n \\n # Store new outputs\\n outputs = {\u0027message\u0027: f\u0027Handled {processed_count} results\u0027}\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### Error Handling in Scripts\\n\\n```yaml\\ntasks:\\n - id: safe_script\\n type: script\\n script: |\\n try:\\n result = risky_operation()\\n outputs = {\u0027result\u0027: result, \u0027status\u0027: \u0027success\u0027}\\n except Exception as e:\\n outputs = {\u0027error\u0027: str(e), \u0027status\u0027: \u0027failed\u0027}\\n \\n # Always print outputs, even for errors\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n## Triggers and Workflow Activation\\n\\nTriggers define how workflows are started automatically based on external events. They enable event-driven workflow execution.\\n\\n### Webhook Triggers\\n\\nWebhook triggers allow external systems to start workflows by sending HTTP requests.\\n\\n```yaml\\ntriggers:\\n - name: customer_webhook\\n type: webhook\\n description: \\\"Triggered when customer data is updated\\\"\\n config:\\n path: \\\"/webhooks/customer-updated\\\" # Webhook endpoint path\\n method: POST # HTTP method (GET, POST, PUT, etc.)\\n authentication:\\n type: token # none|token|signature|basic\\n token: \\\"${WEBHOOK_SECRET}\\\" # Required if type is token\\n header_name: \\\"X-Webhook-Token\\\" # Optional: custom header name\\n validation:\\n required_fields: [\\\"customer.id\\\", \\\"customer.email\\\"] # Required payload fields\\n content_type: \\\"application/json\\\" # Expected content type\\n input_mapping:\\n - webhook_field: customer.id # Field from webhook payload\\n workflow_input: customer_id # Map to workflow input\\n - webhook_field: customer.email\\n workflow_input: customer_email\\n - webhook_field: customer.updated_at\\n workflow_input: update_timestamp\\n```\\n\\n### Scheduled Triggers\\n\\nSchedule workflows to run at specific times or intervals.\\n\\n```yaml\\ntriggers:\\n - name: daily_report\\n type: schedule\\n description: \\\"Generate daily customer report\\\"\\n config:\\n cron: \\\"0 9 * * *\\\" # Daily at 9 AM (cron format)\\n timezone: \\\"UTC\\\" # Optional: timezone\\n enabled: true # Optional: enable/disable\\n max_instances: 1 # Optional: max concurrent instances\\n input_mapping:\\n - workflow_input: report_date\\n default_value: \\\"${current_date}\\\" # Use built-in variable\\n - workflow_input: report_type\\n default_value: \\\"daily\\\"\\n\\n - name: weekly_cleanup\\n type: schedule\\n description: \\\"Weekly database cleanup\\\"\\n config:\\n cron: \\\"0 2 * * 0\\\" # Weekly on Sunday at 2 AM\\n timezone: \\\"America/New_York\\\"\\n input_mapping:\\n - workflow_input: cleanup_before_date\\n default_value: \\\"${date_minus_7_days}\\\"\\n```\\n\\n### Event Triggers\\n\\nEvent triggers respond to system events from various sources.\\n\\n```yaml\\ntriggers:\\n - name: file_uploaded\\n type: event\\n description: \\\"Process uploaded files\\\"\\n config:\\n event_type: \\\"file.uploaded\\\" # Event type to listen for\\n source: \\\"storage_service\\\" # Event source system\\n filter: # Optional: event filtering\\n file_type: [\\\"csv\\\", \\\"json\\\", \\\"xlsx\\\"]\\n file_size_max: \\\"100MB\\\"\\n path_pattern: \\\"/uploads/customers/*\\\"\\n retry: # Optional: retry configuration\\n max_attempts: 3\\n initial_delay: \\\"5s\\\"\\n input_mapping:\\n - webhook_field: file.path\\n workflow_input: file_path\\n - webhook_field: file.size\\n workflow_input: file_size\\n - webhook_field: file.type\\n workflow_input: file_type\\n\\n - name: order_status_changed\\n type: event\\n description: \\\"Handle order status changes\\\"\\n config:\\n event_type: \\\"order.status_changed\\\"\\n source: \\\"order_management\\\"\\n filter:\\n status: [\\\"completed\\\", \\\"cancelled\\\", \\\"refunded\\\"]\\n priority: [\\\"high\\\", \\\"urgent\\\"]\\n input_mapping:\\n - webhook_field: order.id\\n workflow_input: order_id\\n - webhook_field: order.status\\n workflow_input: new_status\\n - webhook_field: order.customer_id\\n workflow_input: customer_id\\n```\\n\\n### Dependency Triggers\\n\\nDependency triggers start workflows when other workflows complete.\\n\\n```yaml\\ntriggers:\\n - name: post_processing\\n type: dependency\\n description: \\\"Run after main processing completes\\\"\\n config:\\n depends_on_workflow: \\\"main_data_processing\\\" # Workflow to depend on\\n depends_on_status: \\\"SUCCESS\\\" # SUCCESS|FAILED|COMPLETED\\n namespace: \\\"production\\\" # Optional: workflow namespace\\n timeout_minutes: 60 # Optional: wait timeout\\n input_mapping:\\n - webhook_field: execution_id # From completed workflow\\n workflow_input: source_execution_id\\n - webhook_field: workflow_outputs.processed_count\\n workflow_input: items_processed\\n```\\n\\n## Model Client Configuration\\n\\nBefore defining AI-related tasks (ai_agent, chatbot, continuous_conversation), you must configure model clients. Model clients define which AI models to use and how to authenticate with them.\\n\\n### Basic Structure\\n\\n```yaml\\nmodel_clients:\\n - id: unique_identifier # Used to reference this model in tasks\\n provider: provider_name # AI provider (openai, anthropic, etc.)\\n config:\\n model: model_name # Specific model to use\\n api_key: your_api_key # Authentication key\\n # Additional provider-specific options...\\n```\\n\\n### Supported Providers\\n\\n#### OpenAI Provider\\n\\n```yaml\\nmodel_clients:\\n - id: gpt4_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview # Available: gpt-4, gpt-4-turbo, gpt-3.5-turbo\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 0.7 # Optional: 0.0-2.0\\n max_tokens: 2000 # Optional: Maximum response tokens\\n timeout: 30 # Optional: Request timeout (seconds)\\n```\\n\\n#### Anthropic Provider\\n\\n```yaml\\nmodel_clients:\\n - id: claude_model\\n provider: anthropic\\n config:\\n model: claude-3-sonnet-20240229 # Available: claude-3-opus, claude-3-sonnet, claude-3-haiku\\n api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n temperature: 0.7 # Optional: 0.0-1.0\\n max_tokens: 1500 # Optional: Maximum response tokens\\n timeout: 30 # Optional: Request timeout (seconds)\\n```\\n\\n#### OpenRouter Provider\\n\\n```yaml\\nmodel_clients:\\n - id: openrouter_model\\n provider: openrouter\\n config:\\n model: anthropic/claude-3-sonnet # Available: Various models via OpenRouter\\n api_key: \\\"sk-or-your-openrouter-api-key\\\"\\n base_url: \\\"https://openrouter.ai/api/v1\\\" # Optional: Custom endpoint\\n temperature: 0.7\\n max_tokens: 2000\\n```\\n\\n#### Multiple Model Clients\\n\\nYou can define multiple model clients and reference them by their `id` in tasks:\\n\\n```yaml\\nmodel_clients:\\n - id: fast_model\\n provider: openai\\n config:\\n model: gpt-3.5-turbo\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 0.3\\n \\n - id: smart_model\\n provider: anthropic\\n config:\\n model: claude-3-opus-20240229\\n api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n temperature: 0.7\\n \\n - id: creative_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 1.2\\n```\\n\\n### Using Model Clients in Tasks\\n\\nReference model clients in AI tasks using the `model_client_id` field:\\n\\n```yaml\\ntasks:\\n - id: analysis_task\\n type: ai_agent\\n config:\\n model_client_id: smart_model # References the claude model above\\n system_message: \\\"You are a data analyst...\\\"\\n user_message: \\\"Analyze this data: ${input_data}\\\"\\n \\n - id: creative_task\\n type: ai_agent\\n config:\\n model_client_id: creative_model # References the creative GPT-4 model\\n system_message: \\\"You are a creative writer...\\\"\\n user_message: \\\"Write a story about ${topic}\\\"\\n```\\n\\n### Environment Variables for API Keys\\n\\nFor security, use environment variables instead of hardcoding API keys:\\n\\n```yaml\\nmodel_clients:\\n - id: secure_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"${OPENAI_API_KEY}\\\" # References environment variable\\n temperature: 0.7\\n```\\n\\n### Common Configuration Options\\n\\n| Field | Description | Required | Default |\\n|-------|-------------|----------|---------|\\n| `id` | Unique identifier for referencing in tasks | Yes | - |\\n| `provider` | AI provider name | Yes | - |\\n| `config.model` | Specific model name | Yes | - |\\n| `config.api_key` | Authentication API key | Yes | - |\\n| `config.temperature` | Response randomness (0.0-2.0) | No | 0.7 |\\n| `config.max_tokens` | Maximum response tokens | No | Provider default |\\n| `config.timeout` | Request timeout in seconds | No | 30 |\\n| `config.base_url` | Custom API endpoint | No | Provider default |\\n\\n## Secrets Management and Environment Variables\\n\\nProper secrets management is crucial for security. Never hardcode sensitive information like API keys, passwords, or tokens directly in workflow YAML files.\\n\\n### Using Environment Variables\\n\\nThe recommended approach is to use environment variables for all sensitive data:\\n\\n```yaml\\n# \\u2705 RECOMMENDED: Use environment variables\\nmodel_clients:\\n - id: secure_openai\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"${OPENAI_API_KEY}\\\" # References environment variable\\n\\n# HTTP tasks with environment variables\\ntasks:\\n - id: secure_api_call\\n type: http\\n url: \\\"${API_BASE_URL}/customers\\\" # Environment variable for URL\\n headers:\\n Authorization: \\\"Bearer ${API_TOKEN}\\\" # Environment variable for token\\n X-API-Key: \\\"${EXTERNAL_API_KEY}\\\" # Environment variable for API key\\n```\\n\\n### Environment Variable Patterns\\n\\n```yaml\\n# Database connections\\n- id: database_task\\n type: storage\\n config:\\n storage_type: database\\n connection_string: \\\"${DATABASE_URL}\\\" # Full connection string\\n # or individual components:\\n host: \\\"${DB_HOST}\\\"\\n port: \\\"${DB_PORT}\\\"\\n database: \\\"${DB_NAME}\\\"\\n username: \\\"${DB_USER}\\\"\\n password: \\\"${DB_PASSWORD}\\\"\\n\\n# Third-party service credentials\\n- id: external_service\\n type: http\\n url: \\\"${SERVICE_BASE_URL}/api/v1/data\\\"\\n headers:\\n Authorization: \\\"Bearer ${SERVICE_TOKEN}\\\"\\n X-Client-ID: \\\"${SERVICE_CLIENT_ID}\\\"\\n X-Client-Secret: \\\"${SERVICE_CLIENT_SECRET}\\\"\\n\\n# MCP server credentials\\n- id: mcp_task\\n type: mcp\\n mcp_server: \\\"@agentic/mcp-notion-server\\\"\\n environment_vars:\\n NOTION_TOKEN: \\\"${NOTION_API_TOKEN}\\\"\\n WORKSPACE_ID: \\\"${NOTION_WORKSPACE_ID}\\\"\\n```\\n\\n### Security Best Practices\\n\\n#### \\u274c What NOT to Do\\n\\n```yaml\\n# DON\u0027T: Hardcode secrets\\nmodel_clients:\\n - id: insecure_example\\n provider: openai\\n config:\\n api_key: \\\"sk-actual-api-key-here\\\" # SECURITY RISK!\\n\\n# DON\u0027T: Put secrets in comments\\ntasks:\\n - id: bad_example\\n type: http\\n url: \\\"https://api.example.com\\\"\\n # API key is: sk-12345... DON\u0027T DO THIS!\\n headers:\\n Authorization: \\\"Bearer sk-real-token\\\" # SECURITY RISK!\\n```\\n\\n#### \\u2705 What TO Do\\n\\n```yaml\\n# DO: Use environment variables\\nmodel_clients:\\n - id: secure_example\\n provider: openai\\n config:\\n api_key: \\\"${OPENAI_API_KEY}\\\" # Secure approach\\n\\n# DO: Use different variables for different environments\\n- id: environment_aware_task\\n type: http\\n url: \\\"${API_BASE_URL}\\\" # Different URLs per environment\\n headers:\\n Authorization: \\\"Bearer ${API_TOKEN}\\\" # Different tokens per environment\\n X-Environment: \\\"${ENVIRONMENT_NAME}\\\" # Track environment\\n\\n# DO: Use descriptive variable names\\nenvironment_vars:\\n CUSTOMER_API_TOKEN: \\\"${CUSTOMER_SERVICE_TOKEN}\\\"\\n BILLING_API_KEY: \\\"${BILLING_SERVICE_API_KEY}\\\"\\n NOTIFICATION_WEBHOOK_SECRET: \\\"${NOTIFICATION_SECRET}\\\"\\n```\\n\\n### Environment Variable Naming Conventions\\n\\n| Pattern | Example | Use Case |\\n|---------|---------|----------|\\n| `SERVICE_API_KEY` | `OPENAI_API_KEY` | Service API keys |\\n| `SERVICE_BASE_URL` | `CUSTOMER_API_BASE_URL` | Service endpoints |\\n| `DB_*` | `DB_HOST`, `DB_PASSWORD` | Database credentials |\\n| `WEBHOOK_*` | `WEBHOOK_SECRET_TOKEN` | Webhook authentication |\\n| `*_TIMEOUT` | `HTTP_REQUEST_TIMEOUT` | Timeout configurations |\\n\\n### Dynamic Environment Variables\\n\\nUse template resolution to create dynamic environment variable names:\\n\\n```yaml\\n# Different API keys per customer\\n- id: customer_specific_api\\n type: http\\n url: \\\"https://api.partner.com/data\\\"\\n headers:\\n Authorization: \\\"Bearer ${PARTNER_API_KEY_${customer_tier}}\\\" # Resolves to PARTNER_API_KEY_PREMIUM\\n\\n# Environment-specific configurations\\n- id: env_specific_task\\n type: storage\\n config:\\n connection_string: \\\"${DATABASE_URL_${environment}}\\\" # DATABASE_URL_PRODUCTION\\n```\\n\\n### Secrets in Different Task Types\\n\\n```yaml\\n# AI Agent with multiple secure credentials\\n- id: secure_ai_task\\n type: ai_agent\\n config:\\n model_client_id: secure_model\\n system_message: \\\"You are a secure assistant with access to customer data.\\\"\\n user_message: \\\"Process customer ${customer_id}\\\"\\n environment_vars:\\n CUSTOMER_DB_TOKEN: \\\"${CUSTOMER_DATABASE_TOKEN}\\\"\\n AUDIT_LOG_KEY: \\\"${AUDIT_LOGGING_API_KEY}\\\"\\n\\n# Script task with secure file access\\n- id: secure_script_task\\n type: script\\n environment_vars:\\n AWS_ACCESS_KEY_ID: \\\"${AWS_ACCESS_KEY}\\\"\\n AWS_SECRET_ACCESS_KEY: \\\"${AWS_SECRET_KEY}\\\"\\n S3_BUCKET_NAME: \\\"${SECURE_BUCKET_NAME}\\\"\\n script: |\\n import boto3\\n import os\\n \\n # Credentials automatically available from environment\\n s3_client = boto3.client(\u0027s3\u0027)\\n bucket = os.environ[\u0027S3_BUCKET_NAME\u0027]\\n \\n # Process secure files...\\n\\n# Storage task with encrypted connections\\n- id: secure_storage_task\\n type: storage\\n config:\\n storage_type: database\\n connection_string: \\\"${ENCRYPTED_DB_CONNECTION_STRING}\\\"\\n ssl_mode: \\\"require\\\"\\n ssl_cert: \\\"${SSL_CERT_PATH}\\\"\\n ssl_key: \\\"${SSL_KEY_PATH}\\\"\\n```\\n\\n### Environment Setup Examples\\n\\n#### Development Environment (.env file)\\n```bash\\n# Development API keys (non-production)\\nOPENAI_API_KEY=sk-dev-api-key-here\\nCUSTOMER_API_BASE_URL=https://api-dev.example.com\\nDATABASE_URL=postgresql://dev:pass@localhost:5432/workflow_dev\\nENVIRONMENT_NAME=development\\n```\\n\\n#### Production Environment\\n```bash\\n# Production API keys (secure)\\nOPENAI_API_KEY=sk-prod-api-key-here\\nCUSTOMER_API_BASE_URL=https://api.example.com\\nDATABASE_URL=postgresql://prod_user:secure_pass@prod-db:5432/workflow_prod\\nENVIRONMENT_NAME=production\\n```\\n\\n## Workflow Execution and Orchestration\\n\\nUnderstanding how workflows execute and how tasks are orchestrated is essential for building reliable, efficient workflows.\\n\\n### Execution Flow\\n\\nWorkflows follow a directed acyclic graph (DAG) execution model:\\n\\n1. **Initialization**: Workflow starts with inputs and model clients\\n2. **Task Scheduling**: Tasks are scheduled based on dependencies\\n3. **Parallel Execution**: Independent tasks run simultaneously\\n4. **Data Flow**: Outputs flow between tasks via template resolution\\n5. **Completion**: Workflow completes when all tasks finish\\n\\n```yaml\\n# Example execution flow\\ntasks:\\n # Stage 1: Independent tasks (run in parallel)\\n - id: fetch_customer # Runs immediately\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: fetch_products # Runs in parallel with fetch_customer\\n type: http\\n url: \\\"https://api.example.com/products\\\"\\n \\n # Stage 2: Dependent tasks (wait for Stage 1)\\n - id: validate_customer # Waits for fetch_customer\\n type: script\\n depends_on: [fetch_customer]\\n previous_node: fetch_customer\\n script: # validation logic...\\n \\n - id: filter_products # Waits for fetch_products\\n type: script\\n depends_on: [fetch_products]\\n previous_node: fetch_products\\n script: # filtering logic...\\n \\n # Stage 3: Final processing (waits for Stage 2)\\n - id: generate_recommendations # Waits for both validation and filtering\\n type: ai_agent\\n depends_on: [validate_customer, filter_products]\\n previous_node: validate_customer # UI hierarchy under customer validation\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Generate product recommendations for customer.\\\"\\n user_message: |\\n Customer: ${validate_customer}\\n Available products: ${filter_products}\\n```\\n\\n### Task States and Lifecycle\\n\\nEach task goes through these states during execution:\\n\\n| State | Description | Next States |\\n|-------|-------------|-------------|\\n| `PENDING` | Task created, waiting for dependencies | `RUNNING`, `SKIPPED` |\\n| `RUNNING` | Task is actively executing | `COMPLETED`, `FAILED` |\\n| `COMPLETED` | Task finished successfully | - |\\n| `FAILED` | Task encountered an error | `RUNNING` (if retries) |\\n| `SKIPPED` | Task skipped due to conditions | - |\\n| `TIMEOUT` | Task exceeded timeout limit | `RUNNING` (if retries) |\\n\\n### Conditional Execution\\n\\nTasks can be executed conditionally based on routes or conditions:\\n\\n```yaml\\n# Router-based conditional execution\\n- id: data_router\\n type: conditional_router\\n depends_on: [fetch_data]\\n previous_node: fetch_data\\n conditions:\\n - name: \\\"large_dataset\\\"\\n condition: \\\"${fetch_data.record_count} \u003e 10000\\\"\\n route: \\\"batch_processing\\\"\\n - name: \\\"small_dataset\\\"\\n condition: \\\"${fetch_data.record_count} \u003c= 1000\\\"\\n route: \\\"direct_processing\\\"\\n default_route: \\\"standard_processing\\\"\\n\\n# Tasks that execute on specific routes\\n- id: batch_processor\\n type: script\\n depends_on: [data_router]\\n previous_node: data_router\\n execute_on_routes: [\\\"batch_processing\\\"] # Only runs for large datasets\\n script: |\\n # Handle large dataset in batches\\n data = ${fetch_data}\\n process_in_batches(data, batch_size=1000)\\n\\n- id: direct_processor\\n type: script\\n depends_on: [data_router]\\n previous_node: data_router\\n execute_on_routes: [\\\"direct_processing\\\"] # Only runs for small datasets\\n script: |\\n # Process small dataset directly\\n data = ${fetch_data}\\n process_directly(data)\\n```\\n\\n### Error Handling and Retries\\n\\nConfigure retry policies for resilient workflow execution:\\n\\n```yaml\\n# Workflow-level retry policy\\nretry:\\n retryOn: [\\\"TEMPORARY_FAILURE\\\", \\\"TIMEOUT\\\", \\\"NETWORK_ERROR\\\"]\\n maxAttempts: 3\\n initialDelay: \\\"1s\\\"\\n maxDelay: \\\"60s\\\"\\n backoffMultiplier: 2.0\\n\\ntasks:\\n # Task-specific retry policy (overrides workflow-level)\\n - id: external_api_call\\n type: http\\n url: \\\"https://unreliable-api.example.com/data\\\"\\n retry:\\n maxAttempts: 5 # More retries for unreliable API\\n initialDelay: \\\"2s\\\"\\n maxDelay: \\\"120s\\\"\\n backoffMultiplier: 2.0\\n retryOn: [\\\"TIMEOUT\\\", \\\"HTTP_5XX\\\", \\\"NETWORK_ERROR\\\"]\\n \\n # Task with no retries\\n - id: critical_operation\\n type: script\\n retry:\\n maxAttempts: 1 # No retries for critical operations\\n script: |\\n # This operation should not be retried if it fails\\n perform_critical_database_operation()\\n```\\n\\n### Parallel Execution Patterns\\n\\n```yaml\\n# Fan-out pattern: One task triggers multiple parallel tasks\\n- id: data_source\\n type: http\\n url: \\\"https://api.example.com/data\\\"\\n\\n- id: process_customers # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process customer data from data_source\\n\\n- id: process_orders # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process order data from data_source\\n\\n- id: process_products # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process product data from data_source\\n\\n# Fan-in pattern: Multiple tasks feed into one task\\n- id: generate_report # Waits for all parallel tasks\\n type: ai_agent\\n depends_on: [process_customers, process_orders, process_products]\\n previous_node: process_customers # UI hierarchy\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Generate comprehensive report from processed data.\\\"\\n user_message: |\\n Customer data: ${process_customers}\\n Order data: ${process_orders}\\n Product data: ${process_products}\\n```\\n\\n### Resource Management\\n\\nControl resource usage and task distribution:\\n\\n```yaml\\n# Worker group targeting\\n- id: cpu_intensive_task\\n type: script\\n worker_group: \\\"high-cpu-workers\\\" # Target specific worker group\\n script: |\\n # CPU-intensive processing\\n perform_heavy_computation()\\n\\n- id: gpu_ml_task\\n type: ai_agent\\n worker_group: \\\"gpu-workers\\\" # Target GPU workers\\n config:\\n model_client_id: large_model\\n # Large model processing...\\n\\n# Timeout configuration\\n- id: long_running_task\\n type: script\\n timeout_seconds: 3600 # 1 hour timeout\\n script: |\\n # Long-running data processing\\n process_large_dataset()\\n\\n# Workflow-level timeout\\ntimeout_seconds: 7200 # 2 hour workflow timeout\\n```\\n\\n## Task Type Configurations\\n\\nThis section provides the complete YAML configuration for each supported task type.\\n\\n**Supported Task Types:**\\n- `script` - Execute Python code with full workflow context\\n- `http` - Make HTTP requests to external APIs\\n- `ai_agent` - Integrate with AI models for intelligent processing \\n- `loop` - Execute tasks repeatedly (for/while loops)\\n- `conditional_router` - Route execution based on conditions\\n- `mcp` - Execute tools via Model Context Protocol servers\\n- `storage` - Data persistence and retrieval operations\\n- `continuous_conversation` - Interactive AI conversations\\n- `human_input` - Tasks requiring human input or approval\\n- `chatbot` - Session-based chatbot conversations\\n\\n### Script Tasks\\n\\nExecute Python code with full workflow context access.\\n\\n```yaml\\n- id: script_task\\n name: \\\"Python Script Task\\\"\\n type: script\\n description: \\\"Execute Python code with dependencies and environment\\\"\\n \\n # Script configuration - script field is directly under task, NOT under config\\n script: | # REQUIRED: Python code to execute\\n import json\\n import requests\\n \\n # Access workflow inputs\\n input_value = \\\"${workflow_input}\\\"\\n \\n # Access previous task outputs\\n previous_result = ${previous_task}\\n \\n # Your processing logic\\n result = {\\\"processed\\\": True, \\\"value\\\": input_value}\\n \\n # Output results (must be JSON serializable)\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n \\n # Alternative: use \u0027code\u0027 field instead of \u0027script\u0027\\n code: | # Alternative to \u0027script\u0027 field\\n print(\\\"Hello World\\\")\\n \\n # Package management\\n requirements: # Optional: Python packages\\n - requests==2.31.0\\n - pandas==1.5.3\\n - numpy==1.24.3\\n \\n # Environment configuration\\n environment: # Optional: Environment variables\\n API_KEY: \\\"${secrets.api_key}\\\"\\n DEBUG: \\\"true\\\"\\n CUSTOM_VAR: \\\"value\\\"\\n \\n environment_vars: # Alternative to \u0027environment\u0027\\n DATABASE_URL: \\\"${secrets.db_url}\\\"\\n \\n # Secrets configuration\\n secrets_config: # Optional: Secrets access\\n api_key: \\\"secret_name\\\"\\n db_password: \\\"db_secret\\\"\\n```\\n\\n### HTTP Tasks\\n\\nMake HTTP requests to external APIs and services.\\n\\n```yaml\\n- id: http_task\\n name: \\\"HTTP API Call\\\"\\n type: http\\n description: \\\"Call external REST API\\\"\\n \\n # HTTP configuration\\n method: POST # Default: GET\\n url: \\\"https://api.example.com/data\\\" # REQUIRED: Target URL\\n \\n # Headers configuration\\n headers: # Optional: HTTP headers\\n Content-Type: \\\"application/json\\\"\\n Authorization: \\\"Bearer ${secrets.api_token}\\\"\\n X-Custom-Header: \\\"workflow-engine\\\"\\n User-Agent: \\\"WorkflowEngine/1.0\\\"\\n \\n # Request body\\n body: | # Optional: Request body (for POST/PUT)\\n {\\n \\\"customer_id\\\": \\\"${customer_id}\\\",\\n \\\"timestamp\\\": \\\"${current_timestamp}\\\",\\n \\\"data\\\": ${previous_task}\\n }\\n \\n # Response validation\\n expected_status: [200, 201, 202] # Default: [200]\\n expectedStatus: [200, 201] # Alternative field name\\n \\n # Additional HTTP options\\n timeout_seconds: 30 # HTTP request timeout\\n follow_redirects: true # Follow HTTP redirects\\n verify_ssl: true # Verify SSL certificates\\n```\\n\\n### AI Agent Tasks\\n\\nIntegrate with AI models for intelligent processing.\\n\\n```yaml\\n- id: ai_agent_task\\n name: \\\"AI Processing Agent\\\"\\n type: ai_agent\\n description: \\\"Process data using AI model\\\"\\n \\n # AI configuration\\n config:\\n model_client_id: gpt4_model # REQUIRED: Reference to model client\\n \\n system_message: | # Optional: System prompt\\n You are a data analyst. Analyze the provided data and extract insights.\\n Return your analysis in JSON format with the following structure:\\n {\\n \\\"insights\\\": [\\\"insight1\\\", \\\"insight2\\\"],\\n \\\"recommendations\\\": [\\\"rec1\\\", \\\"rec2\\\"],\\n \\\"confidence_score\\\": 0.95\\n }\\n \\n user_message: | # REQUIRED: User prompt with template resolution\\n Please analyze this customer data:\\n ${previous_task}\\n \\n Customer ID: ${customer_id}\\n Date Range: ${date_range}\\n \\n # Model parameters\\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\\n max_tokens: 1500 # Optional: Maximum response tokens\\n top_p: 1.0 # Optional: Nucleus sampling\\n frequency_penalty: 0.0 # Optional: Frequency penalty\\n presence_penalty: 0.0 # Optional: Presence penalty\\n \\n # Tools configuration for function calling\\n tools: # Optional: List of MCP tool names\\n - current_time # Time-related tools\\n - relative_time\\n - days_in_month\\n - weather_api # Weather tools\\n - database_query # Database tools\\n - notification_service # Communication tools\\n \\n # Required for tool resolution\\n tenant_id: \\\"your-tenant-uuid\\\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\\n \\n # Input/output format\\n input_format: \\\"text\\\" # Optional: text|json|structured\\n output_format: \\\"structured\\\" # Optional: text|json|structured\\n\\n### Important: AI Agent Output Access\\n\\nAI agent tasks return structured objects, not direct text responses. Always access the AI response using this pattern:\\n\\n```yaml\\n# In subsequent script tasks\\nscript: |\\n # CORRECT: Access AI response from structured output\\n agent_output = ${ai_agent_task}\\n ai_response = agent_output[\u0027ai_response\u0027].strip()\\n \\n # Additional metadata available\\n tokens_used = agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\\n cost_estimate = agent_output.get(\u0027estimated_cost\u0027, {}).get(\u0027amount\u0027, 0.0)\\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\\n \\n print(f\\\"AI Response: {ai_response}\\\")\\n print(f\\\"Tokens used: {tokens_used}\\\")\\n print(f\\\"Estimated cost: ${cost_estimate:.6f}\\\")\\n```\\n\\n```yaml\\n# In workflow outputs - specify the ai_response field\\noutputs:\\n ai_analysis:\\n type: string\\n source: ai_agent_task.ai_response # Access the ai_response field\\n description: \\\"AI-generated analysis text\\\"\\n \\n total_tokens:\\n type: integer \\n source: ai_agent_task.token_usage.total_tokens\\n description: \\\"Total tokens consumed\\\"\\n```\\n```\\n\\n### Loop Tasks\\n\\nExecute tasks repeatedly with for-loop and while-loop patterns.\\n\\n```yaml\\n# For-Loop (Fixed Iterations)\\n- id: for_loop_task\\n name: \\\"For Loop Processor\\\"\\n type: loop\\n description: \\\"Process data in fixed batches\\\"\\n \\n # Loop configuration\\n loop_type: for # REQUIRED: for|while\\n max_iterations: 10 # REQUIRED for \u0027for\u0027 loops\\n iteration_variable: batch_index # Default: iteration\\n \\n # State variables (persistent across iterations)\\n state_variables:\\n total_processed: 0\\n error_count: 0\\n batch_results: []\\n \\n # Tasks to execute in each iteration\\n loop_tasks: # REQUIRED: Tasks to run in loop\\n - id: process_batch\\n name: \\\"Process Current Batch\\\"\\n type: script\\n script: |\\n batch_idx = ${batch_index} # Access iteration variable\\n total = ${total_processed} # Access state variable\\n \\n # Process current batch\\n result = process_batch_data(batch_idx)\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n \\n - id: update_state\\n name: \\\"Update Loop State\\\"\\n type: script\\n depends_on: [process_batch]\\n script: |\\n # Update state variables\\n new_total = ${total_processed} + ${process_batch.items_processed}\\n \\n result = {\\\"total_processed\\\": new_total}\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n\\n# While-Loop (Conditional)\\n- id: while_loop_task\\n name: \\\"While Loop Quality Processor\\\"\\n type: loop\\n description: \\\"Process until quality threshold met\\\"\\n \\n # Loop configuration\\n loop_type: while # REQUIRED: for|while\\n exit_condition: \\\"${average_quality} \u003e= 85 || ${iteration_count} \u003e= 20\\\" # REQUIRED for \u0027while\u0027\\n max_iterations: 50 # Optional: Safety limit\\n iteration_variable: iteration_count # Default: iteration\\n \\n # State variables\\n state_variables:\\n average_quality: 0\\n items_processed: 0\\n quality_scores: []\\n \\n # Loop tasks\\n loop_tasks:\\n - id: assess_quality\\n type: script\\n script: |\\n current_quality = calculate_quality()\\n print(f\\\"__OUTPUTS__ {json.dumps({\u0027quality\u0027: current_quality})}\\\")\\n```\\n\\n\\n### Conditional Router Tasks\\n\\nRoute execution to different paths based on conditions.\\n\\n```yaml\\n- id: conditional_router_task\\n name: \\\"Data Type Router\\\"\\n type: conditional_router\\n description: \\\"Route processing based on data characteristics\\\"\\n \\n # Router configuration\\n router_type: condition # Default: condition\\n \\n # Routing conditions\\n conditions: # REQUIRED: List of conditions\\n - name: \\\"large_dataset\\\"\\n condition: \\\"${data_size} \u003e 10000\\\"\\n route: \\\"batch_processing\\\"\\n - name: \\\"small_dataset\\\" \\n condition: \\\"${data_size} \u003c= 1000\\\"\\n route: \\\"direct_processing\\\"\\n - name: \\\"medium_dataset\\\"\\n condition: \\\"${data_size} \u003e 1000 \u0026\u0026 ${data_size} \u003c= 10000\\\"\\n route: \\\"standard_processing\\\"\\n \\n default_route: \\\"standard_processing\\\" # REQUIRED: Default route if no conditions match\\n \\n # Alternative: evaluation script for complex routing\\n evaluation_script: | # Optional: Custom routing logic\\n if data_type == \\\"json\\\":\\n return \\\"json_processing\\\"\\n elif data_type == \\\"xml\\\":\\n return \\\"xml_processing\\\"\\n else:\\n return \\\"default_processing\\\"\\n \\n # Route mapping for complex scenarios\\n route_mapping: # Optional: Route name mappings\\n json_processing: \\\"specialized_json\\\"\\n xml_processing: \\\"specialized_xml\\\"\\n\\n# Tasks that execute on specific routes\\n- id: batch_handler\\n name: \\\"Handle Large Dataset\\\"\\n type: script\\n execute_on_routes: [\\\"batch_processing\\\"] # REQUIRED: Routes this task runs on\\n depends_on: [conditional_router_task]\\n script: |\\n print(\\\"Processing large dataset in batches...\\\")\\n\\n- id: direct_handler\\n name: \\\"Handle Small Dataset\\\" \\n type: script\\n execute_on_routes: [\\\"direct_processing\\\"]\\n depends_on: [conditional_router_task]\\n script: |\\n print(\\\"Processing small dataset directly...\\\")\\n```\\n\\n### MCP Tasks\\n\\nExecute tools via Model Context Protocol servers.\\n\\n```yaml\\n- id: mcp_task\\n name: \\\"MCP Tool Execution\\\"\\n type: mcp\\n description: \\\"Execute external tool via MCP server\\\"\\n \\n # MCP server configuration\\n mcp_server: \\\"@agentic/mcp-notion-server\\\" # REQUIRED: MCP server package\\n tool_name: \\\"update_database\\\" # REQUIRED: Tool to execute\\n runtime: \\\"npx\\\" # REQUIRED: npx|uvx\\n \\n # Server startup\\n server_args: [] # Optional: Server startup arguments\\n server_startup_timeout: 30 # Optional: Server startup timeout (seconds)\\n \\n # Tool execution\\n tool_arguments: # REQUIRED: Arguments for the tool\\n database_id: \\\"${secrets.notion_database_id}\\\"\\n properties:\\n title: \\\"Workflow Results\\\"\\n status: \\\"Completed\\\"\\n processed_count: \\\"${previous_task.count}\\\"\\n timestamp: \\\"${current_timestamp}\\\"\\n \\n # Environment configuration\\n environment_vars: # Optional: Environment variables\\n NOTION_TOKEN: \\\"${secrets.notion_token}\\\"\\n DEBUG: \\\"true\\\"\\n \\n # Timeout configuration\\n timeout_seconds: 120 # Optional: Tool execution timeout\\n```\\n\\n### Storage Tasks\\n\\nManage data persistence and retrieval operations.\\n\\n```yaml\\n- id: storage_task\\n name: \\\"Data Storage Operation\\\"\\n type: storage\\n description: \\\"Save/load data to/from storage\\\"\\n \\n # Storage operation\\n operation: save # REQUIRED: save|load|delete|list\\n \\n # Storage configuration\\n storage_type: \\\"file\\\" # REQUIRED: file|database|cache\\n path: \\\"/data/results/${execution_id}.json\\\" # REQUIRED: Storage path\\n \\n # Data configuration\\n data: ${previous_task} # REQUIRED for \u0027save\u0027: Data to store\\n format: \\\"json\\\" # Optional: json|yaml|csv|parquet\\n compression: \\\"gzip\\\" # Optional: gzip|bz2|none\\n \\n # Metadata\\n metadata: # Optional: Additional metadata\\n workflow_id: \\\"${workflow_id}\\\"\\n created_at: \\\"${current_timestamp}\\\"\\n source_task: \\\"previous_task\\\"\\n \\n # File options\\n create_directories: true # Optional: Create parent directories\\n overwrite: true # Optional: Overwrite existing files\\n backup: false # Optional: Create backup before overwrite\\n \\n # Database options (when storage_type: \\\"database\\\")\\n table_name: \\\"workflow_results\\\" # Required for database storage\\n connection_string: \\\"${secrets.db_url}\\\" # Required for database storage\\n \\n # Cache options (when storage_type: \\\"cache\\\")\\n cache_key: \\\"result_${execution_id}\\\" # Required for cache storage\\n ttl: 3600 # Optional: Time to live (seconds)\\n```\\n\\n\\n### Continuous Conversation Tasks\\n\\nCreate interactive AI-powered conversations.\\n\\n```yaml\\n- id: conversation_task\\n name: \\\"AI Conversation Interface\\\"\\n type: continuous_conversation\\n description: \\\"Interactive conversation with AI assistant\\\"\\n \\n # Conversation configuration\\n config:\\n # AI configuration\\n ai_config:\\n model_client_id: support_assistant # REQUIRED: AI model to use\\n system_prompt: | # REQUIRED: System instructions\\n You are a helpful customer support representative.\\n Be friendly, professional, and solution-oriented.\\n \\n Guidelines:\\n - Always be polite and respectful\\n - Ask clarifying questions when needed\\n - Provide clear and actionable solutions\\n \\n temperature: 0.7 # Optional: AI creativity\\n max_tokens: 1000 # Optional: Response length limit\\n \\n # UI configuration\\n ui_config:\\n title: \\\"Customer Support Chat\\\" # Optional: Interface title\\n placeholder: \\\"How can I help you today?\\\" # Optional: Input placeholder\\n theme: \\\"light\\\" # Optional: light|dark\\n show_timestamp: true # Optional: Show message timestamps\\n \\n # Termination conditions\\n termination:\\n keywords: [\\\"goodbye\\\", \\\"bye\\\", \\\"thanks\\\", \\\"done\\\"] # Optional: End conversation keywords\\n max_turns: 50 # Optional: Maximum conversation turns\\n timeout_minutes: 30 # Optional: Conversation timeout\\n inactivity_timeout: 10 # Optional: Inactivity timeout (minutes)\\n \\n # Context configuration\\n context:\\n system_context: \\\"Customer support conversation\\\" # Optional: Context description\\n user_id: \\\"${customer_id}\\\" # Optional: User identifier\\n session_data: ${previous_task} # Optional: Additional context data\\n```\\n\\n### Human Input Tasks\\n\\nTasks requiring human input or approval.\\n\\n```yaml\\n- id: human_input_task\\n name: \\\"Human Input Required\\\"\\n type: human_input\\n description: \\\"Wait for human input or approval\\\"\\n \\n # Human input configuration\\n config:\\n prompt: \\\"Please review and approve the processed data\\\" # REQUIRED: Prompt for human\\n input_type: \\\"approval\\\" # REQUIRED: approval|text|selection\\n \\n # Input options (for selection type)\\n options: [\\\"approve\\\", \\\"reject\\\", \\\"request_changes\\\"] # Optional: For selection input\\n \\n # Timeout configuration\\n timeout_minutes: 60 # Optional: How long to wait for input (minutes)\\n \\n # Conversation settings\\n conversation_mode: false # Optional: Add to conversation history\\n next_task: \\\"process_approval\\\" # Optional: Task to trigger after input\\n end_condition: \\\"bye|exit|quit\\\" # Optional: Keywords to end conversation\\n \\n # Input validation\\n required: true # Optional: Whether input is required\\n \\n # Additional context\\n context_data: ${previous_task} # Optional: Data to show to human\\n```\\n\\n### ChatBot Tasks\\n\\nSession-based chatbot conversations with streaming support.\\n\\n```yaml\\n- id: chatbot_task\\n name: \\\"AI Chatbot\\\"\\n type: chatbot\\n description: \\\"Interactive chatbot with session management\\\"\\n \\n # Chatbot configuration\\n config:\\n # AI model configuration\\n model_client_id: gpt4_model # REQUIRED: Reference to model client\\n \\n system_message: | # REQUIRED: System prompt for chatbot\\n You are a helpful AI assistant. Be friendly, helpful, and concise.\\n Answer questions clearly and ask for clarification when needed.\\n \\n # Session management\\n session_id: \\\"${session_id}\\\" # Optional: Session identifier (auto-generated if not provided)\\n max_history_messages: 20 # Optional: Maximum conversation history to maintain\\n \\n # Model parameters\\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\\n max_tokens: 1000 # Optional: Maximum response tokens\\n stream: true # Optional: Enable streaming responses\\n \\n # Tools configuration for function calling\\n tools: # Optional: List of MCP tool names\\n - current_time # Time-related tools\\n - relative_time\\n - days_in_month\\n - weather_forecast # Weather tools\\n - send_notification # Communication tools\\n \\n # Required for tool resolution\\n tenant_id: \\\"your-tenant-uuid\\\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\\n \\n # UI configuration\\n ui_config:\\n title: \\\"AI Assistant\\\" # Optional: Chat interface title\\n placeholder: \\\"Type your message...\\\" # Optional: Input placeholder\\n theme: \\\"light\\\" # Optional: light|dark\\n show_typing_indicator: true # Optional: Show typing animation\\n \\n # Response configuration\\n response_config:\\n max_retries: 3 # Optional: Retry failed requests\\n timeout_seconds: 30 # Optional: Response timeout\\n \\n # Input message (for single-turn mode)\\n user_message: \\\"${user_input}\\\" # Optional: Direct message input\\n \\n # Context data\\n context_data: ${previous_task} # Optional: Additional context\\n```\\n\\n---\\n\\n## Common Errors and Corrections\\n\\nThis section documents frequent mistakes encountered when creating workflows and their proper corrections.\\n\\n### 1. Missing Required `id` Field\\n\\n**\\u274c Error:**\\n```yaml\\nname: \\\"My Workflow\\\"\\ndescription: \\\"A workflow without id\\\"\\ntasks:\\n - name: \\\"Task 1\\\"\\n type: script\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nid: \\\"my_workflow\\\" # REQUIRED: Unique workflow identifier\\nname: \\\"My Workflow\\\"\\ndescription: \\\"A workflow with proper id\\\"\\ntasks:\\n - id: task_1 # REQUIRED: Unique task identifier\\n name: \\\"Task 1\\\"\\n type: script\\n```\\n\\n### 2. Incorrect Task Structure (Object vs Array)\\n\\n**\\u274c Error:**\\n```yaml\\ntasks:\\n task_1: # Wrong: tasks as object mapping\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n task_2:\\n type: ai_agent\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ntasks:\\n - id: task_1 # Correct: tasks as array with id field\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n - id: task_2\\n type: ai_agent\\n```\\n\\n### 3. Incorrect Inputs Format (Object vs Array)\\n\\n**\\u274c Error:**\\n```yaml\\ninputs:\\n user_name: # Wrong: inputs as object mapping\\n type: string\\n required: true\\n age:\\n type: integer\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ninputs:\\n - name: user_name # Correct: inputs as array with name field\\n type: string\\n required: true\\n - name: age\\n type: integer\\n```\\n\\n### 4. Script Configuration Location Error\\n\\n**\\u274c Error:**\\n```yaml\\n- id: script_task\\n type: script\\n config: # Wrong: script under config\\n script: |\\n print(\\\"Hello\\\")\\n requirements:\\n - requests\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: script_task\\n type: script\\n script: | # Correct: script directly under task\\n print(\\\"Hello\\\")\\n # requirements can be at task level if needed\\n```\\n\\n### 5. Individual Task Inputs (Not Supported)\\n\\n**\\u274c Error:**\\n```yaml\\n- id: task_1\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n inputs: # Wrong: tasks don\u0027t have individual inputs\\n search_query: ${search_query}\\n prev_result: ${previous_task}\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: task_1\\n type: script\\n depends_on: [previous_task] # Use depends_on for data access\\n script: |\\n # Access workflow inputs directly\\n search_query = \\\"${search_query}\\\"\\n # Access previous task outputs via template resolution\\n prev_result = ${previous_task}\\n```\\n\\n### 6. Incorrect Script Output Handling\\n\\n**\\u274c Error:**\\n```yaml\\nscript: |\\n result = process_data()\\n outputs[\u0027result\u0027] = result # Wrong: missing __OUTPUTS__ pattern\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nscript: |\\n result = process_data()\\n \\n # Required: Use outputs dict and __OUTPUTS__ marker\\n outputs = {}\\n outputs[\u0027result\u0027] = result\\n outputs[\u0027status\u0027] = \u0027success\u0027\\n \\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### 7. AI Agent Configuration Errors\\n\\n**\\u274c Error:**\\n```yaml\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: my_model\\n prompt: \\\"Process this data: ${data}\\\" # Wrong: use user_message\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: my_model\\n system_message: \\\"You are a helpful assistant.\\\"\\n user_message: \\\"Process this data: ${data}\\\" # Correct field name\\n```\\n\\n### 8. Missing Dependencies for Template Resolution\\n\\n**\\u274c Error:**\\n```yaml\\n- id: task_2\\n type: script\\n # Missing depends_on\\n script: |\\n data = ${task_1} # Will become UNRESOLVED_task_1\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: task_2\\n type: script\\n depends_on: [task_1] # Required for template access\\n script: |\\n data = ${task_1} # Now resolves correctly\\n```\\n\\n### 9. Incorrect Model Client Configuration\\n\\n**\\u274c Error:**\\n```yaml\\nmodel_clients:\\n my_model: # Wrong: object mapping format\\n type: openai\\n api_key: \\\"sk-...\\\"\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nmodel_clients:\\n - id: my_model # Correct: array format with id field\\n provider: openai\\n config:\\n model: \\\"gpt-4\\\"\\n api_key: \\\"sk-...\\\"\\n```\\n\\n### 10. Empty String Literals Causing Python Syntax Errors\\n\\n**\\u274c Error:**\\n```yaml\\nscript: |\\n def process_data(path=\\\"\\\"): # Wrong: empty string can cause syntax issues\\n current_path = f\\\"{path}.{key}\\\" if path else key\\n \\n # This can lead to: SyntaxError: unterminated string literal\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nscript: |\\n def process_data(path=None): # Correct: use None instead of empty string\\n current_path = f\\\"{path}.{key}\\\" if path else key\\n # Or use path = path or \\\"default_value\\\"\\n```\\n\\n### 11. YAML Indentation and Structure Issues\\n\\n**\\u274c Error:**\\n```yaml\\ntasks:\\n- id: task_1\\n type: script\\nscript: | # Wrong: inconsistent indentation\\n print(\\\"hello\\\")\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ntasks:\\n- id: task_1\\n type: script\\n script: | # Correct: consistent indentation\\n print(\\\"hello\\\")\\n```\\n\\n### Best Practices Summary\\n\\n1. **Always include required `id` fields** for workflows and tasks\\n2. **Use array format** for `tasks`, `inputs`, and `model_clients`\\n3. **Script field goes directly under task**, not under `config`\\n4. **No individual task inputs** - use template resolution instead\\n5. **Use `depends_on`** to access previous task outputs\\n6. **Print `__OUTPUTS__`** in scripts for proper output parsing\\n7. **Use `user_message`** (not `prompt`) for AI agent tasks\\n8. **Avoid empty string literals** - use `None` or meaningful defaults\\n9. **Follow consistent YAML indentation** throughout the file\\n\\n## MCP Tools and Function Calling\\n\\nAI agents and chatbots can use MCP (Model Context Protocol) tools for function calling, enabling them to execute external tools and services.\\n\\n### Tools Configuration\\n\\nBoth `ai_agent` and `chatbot` tasks support tools configuration:\\n\\n```yaml\\n# AI Agent with MCP tools\\n- id: ai_with_tools\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful assistant with access to various tools.\\n Use the available tools when needed to help users.\\n user_message: \\\"What time is it and what\u0027s the weather like?\\\"\\n \\n # Tools configuration\\n tools: # List of MCP tool names\\n - current_time # Time tools\\n - relative_time\\n - days_in_month\\n - weather_forecast # Weather tools\\n - database_query # Database tools\\n - send_notification # Communication tools\\n \\n tenant_id: \\\"550e8400-e29b-41d4-a716-446655440000\\\" # Required for tool resolution\\n\\n# Chatbot with MCP tools\\n- id: chatbot_with_tools\\n type: chatbot\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful chatbot with access to tools.\\n Use tools when users ask for specific information or actions.\\n \\n # Tools configuration (same format as AI agent)\\n tools:\\n - current_time\\n - weather_api\\n - user_preferences\\n \\n tenant_id: \\\"550e8400-e29b-41d4-a716-446655440000\\\" # Required for tool resolution\\n```\\n\\n### How Tools Work\\n\\n1. **Tool Resolution**: The workflow engine resolves tool names to actual MCP server deployments using the `tenant_id`\\n2. **Function Calling**: When the AI model decides to use a tool, it calls the `tool_executor` function\\n3. **Tool Execution**: The engine executes the specified tool via the appropriate MCP server\\n4. **Response Integration**: Tool results are fed back to the AI model for response generation\\n\\n### Tool Configuration Requirements\\n\\n| Field | Required | Description |\\n|-------|----------|-------------|\\n| `tools` | No | List of MCP tool names available to the AI |\\n| `tenant_id` | **Yes** (when using tools) | Tenant UUID for tool resolution |\\n\\n### Example: Complete AI Agent with Tools\\n\\n```yaml\\nid: intelligent_assistant\\nname: \\\"Intelligent Assistant with Tools\\\"\\n\\nmodel_clients:\\n - id: gpt4_model\\n provider: openai\\n config:\\n model: gpt-4o-mini\\n api_key: \\\"${OPENAI_API_KEY}\\\"\\n temperature: 0.3\\n\\n# Tenant ID can be set at workflow level for all tasks\\ntenant_id: \\\"0572fa8d-d7c3-47db-9219-ef40b80d42b7\\\"\\n\\ntasks:\\n - id: assistant_with_tools\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful assistant with access to various tools:\\n - Time-related tools: current_time, relative_time, days_in_month\\n - Weather tools: weather_forecast\\n - Communication: send_notification\\n \\n Use these tools when users ask for:\\n - Current time or date information\\n - Weather forecasts\\n - Sending notifications\\n \\n Always explain what tool you\u0027re using and why.\\n \\n user_message: \\\"${user_query}\\\"\\n \\n # MCP tools configuration\\n tools:\\n - current_time\\n - relative_time\\n - days_in_month\\n - weather_forecast\\n - send_notification\\n \\n # Tool resolution requires tenant_id\\n tenant_id: \\\"0572fa8d-d7c3-47db-9219-ef40b80d42b7\\\"\\n \\n temperature: 0.7\\n max_tokens: 1000\\n\\ninputs:\\n - name: user_query\\n type: string\\n required: true\\n default: \\\"What time is it and can you send a notification to user123?\\\"\\n\\noutputs:\\n ai_response:\\n type: string\\n source: assistant_with_tools.ai_response\\n description: \\\"AI response using tools when needed\\\"\\n```\\n\\n### Tool Execution Flow\\n\\n1. **User Input**: User provides a query that might require tool usage\\n2. **AI Analysis**: AI model analyzes the query and decides if tools are needed\\n3. **Tool Calling**: AI calls `tool_executor(deployment_id, tool_name, arguments)`\\n4. **Tool Resolution**: Engine resolves tool name to MCP server using tenant_id\\n5. **Tool Execution**: MCP server executes the requested tool\\n6. **Result Processing**: Tool results are returned to the AI model\\n7. **Response Generation**: AI generates final response incorporating tool results\\n\\n### Best Practices for Tools\\n\\n1. **Clear System Messages**: Explain available tools and when to use them\\n2. **Tool Categories**: Group related tools (time, weather, database, etc.)\\n3. **Error Handling**: AI models handle tool execution failures gracefully\\n4. **Tool Documentation**: Include tool descriptions in system messages\\n5. **Tenant Configuration**: Ensure tenant_id matches your MCP server setup\\n\\n---\\n\\n*This comprehensive reference covers all supported task types and their YAML configurations. Use this as a quick reference when building workflows with different task types.*\", \"format\": \"markdown\"}, \"status\": \"success\", \"successful_url\": \"http://172.17.0.1:5000/api/docs/yaml-reference\"}\n",
"return_code": 0,
"status": "completed",
"stderr": ""
},
"duration_seconds": 0.820081,
"end_time": "2025-08-14T11:23:07.670722",
"message_sent": true,
"start_time": "2025-08-14T11:23:06.850641",
"timestamp": "2025-08-14T11:23:07.670722",
"worker_executed": true,
"workers_notified": true
},
"output": "Trying to fetch documentation from: http://host.docker.internal:5000/api/docs/yaml-reference\n\u274c Connection failed: http://host.docker.internal:5000/api/docs/yaml-reference - HTTPConnectionPool(host=\u0027host.docker.internal\u0027, port=5000): Max retries exceeded with url: /api/docs/yaml-reference (Caused by NameResolutionError(\"\u003curllib3.connection.HTTPConnection object at 0x7d0629aa9010\u003e: Failed to resolve \u0027host.docker.internal\u0027 ([Errno -2] Name or service not known)\"))\nTrying to fetch documentation from: http://172.17.0.1:5000/api/docs/yaml-reference\n\u2705 Successfully connected to: http://172.17.0.1:5000/api/docs/yaml-reference\nSuccessfully fetched documentation from http://172.17.0.1:5000/api/docs/yaml-reference: 97047 characters\n__OUTPUTS__ {\"documentation\": {\"content\": \"# YAML Workflow Structure Reference\\n\\nComplete reference for the YAML workflow definition structure supported by the Assistants Workflow Engine.\\n\\n## Table of Contents\\n\\n1. [Complete Schema Overview](#complete-schema-overview)\\n2. [Required vs Optional Fields](#required-vs-optional-fields)\\n3. [Data Types Reference](#data-types-reference)\\n4. [Field Validation Rules](#field-validation-rules)\\n5. [Parser Behavior](#parser-behavior)\\n6. [Migration Guide](#migration-guide)\\n\\n## Complete Schema Overview\\n\\n```yaml\\n# === WORKFLOW METADATA (Required) ===\\nid: string # REQUIRED: Unique workflow identifier\\nname: string # REQUIRED: Human-readable workflow name\\n\\n# === WORKFLOW METADATA (Optional) ===\\nversion: string | integer # Default: 1\\nnamespace: string # Default: \\\"default\\\"\\ndescription: string # Default: \\\"\\\"\\ntenant_id: string # Default: null\\n\\n# === EXECUTION CONFIGURATION ===\\ntimeout_seconds: integer|string # Default: 3600 (1 hour) - Workflow-level timeout (supports duration strings like \\\"1h\\\", \\\"30m\\\")\\nlabels: object # Default: {}\\nmetadata: object # Default: {}\\n\\n# === DATA FLOW DEFINITIONS ===\\ninputs: # Default: []\\n - name: string # REQUIRED\\n type: string # REQUIRED: string|integer|boolean|object|array\\n description: string # Optional\\n required: boolean # Default: false\\n default: any # Optional default value\\n\\noutputs: object # Default: {}\\n output_name:\\n type: string # REQUIRED\\n source: string # REQUIRED: task_id or task_id.field\\n description: string # Optional\\n\\nvariables: object # Default: {}\\n\\n# === MODEL CLIENT CONFIGURATION (Required for AI tasks) ===\\nmodel_clients: # REQUIRED: For ai_agent, chatbot, continuous_conversation tasks\\n - id: string # REQUIRED: Unique identifier to reference in tasks\\n provider: string # REQUIRED: openai|anthropic|openrouter|groq\\n config: object # REQUIRED: Provider-specific config\\n model: string # REQUIRED: Model name (e.g., gpt-4, claude-3-sonnet-20240229)\\n api_key: string # REQUIRED: API key for the provider\\n temperature: number # Optional: 0.0-2.0, controls randomness\\n max_tokens: integer # Optional: Maximum tokens to generate\\n timeout: integer # Optional: Request timeout in seconds\\n base_url: string # Optional: Custom API base URL\\n\\n# Model Client Examples:\\n# OpenAI Configuration:\\n# model_clients:\\n# - id: gpt4_model\\n# provider: openai\\n# config:\\n# model: gpt-4-turbo-preview\\n# api_key: \\\"sk-your-openai-api-key\\\"\\n# temperature: 0.7\\n# max_tokens: 2000\\n#\\n# Anthropic Configuration:\\n# model_clients:\\n# - id: claude_model\\n# provider: anthropic\\n# config:\\n# model: claude-3-sonnet-20240229\\n# api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n# temperature: 0.7\\n# max_tokens: 1500\\n#\\n# Multiple Models:\\n# model_clients:\\n# - id: gpt4_model\\n# provider: openai\\n# config:\\n# model: gpt-4-turbo-preview\\n# api_key: \\\"sk-your-openai-api-key\\\"\\n# - id: claude_model\\n# provider: anthropic\\n# config:\\n# model: claude-3-sonnet-20240229\\n# api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n\\n# === TASK DEFINITIONS (Required) ===\\ntasks: # REQUIRED: List of tasks\\n - id: string # REQUIRED: Unique task identifier\\n name: string # Optional: Human-readable name\\n type: string # REQUIRED: Task type\\n description: string # Optional\\n \\n # DEPENDENCY CONFIGURATION\\n depends_on: array # Optional: Execution dependencies and template resolution\\n previous_node: string # REQUIRED: Task connectivity and UI hierarchy (essential for proper workflow execution)\\n \\n timeout_seconds: integer|string # Optional: Task-specific timeout (supports duration strings like \\\"5m\\\", \\\"30s\\\")\\n worker_group: string # Optional: Target worker group\\n conditions: array # Optional: Execution conditions\\n execute_on_routes: array # Optional: Routes for conditional execution\\n \\n # Input/Output mapping\\n input_mapping: object # Optional: Map workflow context to task inputs\\n output_mapping: object # Optional: Map task outputs to workflow context\\n \\n # Retry configuration\\n retry: object # Optional: Task-specific retry policy\\n maxAttempts: integer # Default: 1\\n initialDelay: string|integer # Default: \\\"1s\\\"\\n maxDelay: string|integer # Default: \\\"300s\\\"\\n backoffMultiplier: number # Default: 2.0\\n retryOn: array # Default: [\\\"TEMPORARY_FAILURE\\\"]\\n \\n # === TASK TYPE SPECIFIC CONFIGURATIONS ===\\n # See detailed sections below for each task type\\n\\n# === WORKFLOW-LEVEL RETRY POLICY ===\\nretry: object # Optional: Global retry policy\\n retryOn: array # Default: [\\\"TEMPORARY_FAILURE\\\"]\\n maxAttempts: integer # Default: 1\\n initialDelay: string|integer # Default: \\\"1s\\\"\\n maxDelay: string|integer # Default: \\\"300s\\\"\\n backoffMultiplier: number # Default: 2.0\\n\\n# === TRIGGERS ===\\ntriggers: # Optional: List of triggers\\n - name: string # REQUIRED\\n type: string # REQUIRED: webhook|schedule|event\\n description: string # Optional\\n config: object # REQUIRED: Trigger-specific config\\n input_mapping: array # Optional: Map trigger data to workflow inputs\\n - webhook_field: string # Source field\\n workflow_input: string # Target workflow input\\n```\\n\\n## Required vs Optional Fields\\n\\n### Required Fields (Workflow Level)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `id` | string | Unique identifier for the workflow |\\n| `tasks` | array | List of task definitions (can be empty) |\\n\\n### Required Fields (Task Level)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `id` | string | Unique identifier for the task |\\n| `type` | string | Task type (script, http, ai_agent, etc.) |\\n| `previous_node` | string | Reference to previous task for UI connectivity (required for proper task linking, except for the first task) |\\n\\n### Required Fields (Input Definition)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `name` | string | Input parameter name |\\n| `type` | string | Data type (string, integer, boolean, object, array) |\\n\\n### Required Fields (Output Definition)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `type` | string | Output data type |\\n| `source` | string | Source task ID or task_id.field_name |\\n\\n## Data Types Reference\\n\\n### Primitive Types\\n\\n```yaml\\n# String type\\ninputs:\\n - name: customer_name\\n type: string\\n default: \\\"John Doe\\\"\\n description: \\\"Customer full name\\\"\\n\\n# Integer type \\ninputs:\\n - name: batch_size\\n type: integer\\n default: 100\\n description: \\\"Number of records per batch\\\"\\n\\n# Boolean type\\ninputs:\\n - name: include_archived\\n type: boolean\\n default: false\\n description: \\\"Include archived records\\\"\\n```\\n\\n### Complex Types\\n\\n```yaml\\n# Object type\\ninputs:\\n - name: config_params\\n type: object\\n default:\\n timeout: 30\\n retries: 3\\n debug: false\\n description: \\\"Configuration parameters\\\"\\n\\n# Array type\\ninputs:\\n - name: customer_ids\\n type: array\\n default: [\\\"cust_001\\\", \\\"cust_002\\\"]\\n description: \\\"List of customer identifiers\\\"\\n```\\n\\n### Duration Types\\n\\nDuration values can be specified as strings or integers:\\n\\n```yaml\\n# String format (recommended)\\ntimeout_seconds: \\\"1h\\\" # 1 hour\\ntimeout_seconds: \\\"30m\\\" # 30 minutes \\ntimeout_seconds: \\\"45s\\\" # 45 seconds\\ntimeout_seconds: \\\"2d\\\" # 2 days\\n\\n# Integer format (seconds)\\ntimeout_seconds: 3600 # 1 hour in seconds\\n\\n# Supported units\\n# s, sec, second, seconds\\n# m, min, minute, minutes \\n# h, hour, hours\\n# d, day, days\\n```\\n\\n### Timeout Configuration and Execution Behavior\\n\\nThe workflow engine supports configurable timeouts at both workflow and task levels with intelligent timeout-free communication for long-running tasks.\\n\\n#### Timeout Hierarchy\\n\\nTimeout values are resolved in this order of precedence:\\n1. **Task-level `timeout_seconds`**: Specific timeout for individual tasks\\n2. **Workflow-level `timeout_seconds`**: Default for all tasks in the workflow\\n3. **System default**: 300 seconds (5 minutes) for tasks, 3600 seconds (1 hour) for workflows\\n\\n```yaml\\n# Workflow-level timeout (applies to all tasks by default)\\ntimeout_seconds: \\\"2h\\\" # 2 hour workflow timeout\\n\\ntasks:\\n - id: quick_task\\n type: script\\n # Uses workflow-level timeout (2h)\\n script: \\\"print(\u0027Quick task\u0027)\\\"\\n \\n - id: custom_timeout_task\\n type: script\\n timeout_seconds: \\\"30m\\\" # Override with 30 minute timeout\\n script: \\\"print(\u0027Custom timeout task\u0027)\\\"\\n \\n - id: long_running_task\\n type: script\\n timeout_seconds: 600 # 10 minutes (integer format)\\n script: |\\n import time\\n time.sleep(300) # 5 minute processing\\n print(\\\"Long task completed\\\")\\n```\\n\\n#### Timeout-Free Communication\\n\\nThe workflow engine automatically uses **timeout-free communication** for long-running tasks to prevent timeout-related failures:\\n\\n- **Loop tasks**: Always use timeout-free communication (event-driven execution)\\n- **Script tasks**: Use timeout-free communication when `timeout_seconds \u003e 180` (host timeout threshold)\\n- **Other task types**: Use standard timeout-based communication\\n\\n```yaml\\ntasks:\\n - id: short_script\\n type: script\\n timeout_seconds: 60 # Uses standard timeout-based communication\\n script: \\\"print(\u0027Short script\u0027)\\\"\\n \\n - id: long_script\\n type: script\\n timeout_seconds: 600 # Uses timeout-free communication (600 \u003e 180)\\n script: |\\n import time\\n print(\\\"\\ud83d\\ude80 This uses timeout-free communication\\\")\\n time.sleep(300) # Won\u0027t cause timeout issues\\n print(\\\"Long script completed\\\")\\n \\n - id: data_processing_loop\\n type: loop\\n timeout_seconds: \\\"1h\\\" # Always uses timeout-free communication\\n loop_type: for\\n max_iterations: 10\\n # Loop tasks automatically use timeout-free communication\\n```\\n\\n#### Loop Task Timeout Inheritance\\n\\nLoop tasks can pass timeout configuration to their inner script tasks:\\n\\n```yaml\\n- id: batch_processing_loop\\n type: loop\\n timeout_seconds: \\\"30m\\\" # Loop-level timeout\\n loop_type: for\\n max_iterations: 5\\n loop_tasks:\\n - id: process_batch\\n type: script\\n # Inherits 30-minute timeout from parent loop\\n script: |\\n print(\\\"Processing batch with inherited timeout\\\")\\n # This script will use the 30-minute timeout from the loop\\n```\\n\\n#### Best Practices for Timeout Configuration\\n\\n1. **Use duration strings** for readability: `\\\"5m\\\"` vs `300`\\n2. **Set appropriate timeouts** based on expected execution time plus buffer\\n3. **Long-running tasks**: Set `timeout_seconds \u003e 180` to enable timeout-free communication\\n4. **Loop tasks**: Always get timeout-free communication automatically\\n5. **Critical tasks**: Use shorter timeouts for fail-fast behavior\\n6. **External API calls**: Use longer timeouts to account for network latency\\n\\n```yaml\\n# Example: Well-configured timeout workflow\\ntimeout_seconds: \\\"1h\\\" # Reasonable workflow timeout\\n\\ntasks:\\n - id: api_call\\n type: http\\n timeout_seconds: \\\"30s\\\" # Quick timeout for API calls\\n url: \\\"https://api.example.com/data\\\"\\n \\n - id: data_processing\\n type: script\\n timeout_seconds: \\\"10m\\\" # Longer timeout for processing (uses timeout-free)\\n script: |\\n # Process large dataset\\n process_data()\\n \\n - id: batch_processing\\n type: loop\\n timeout_seconds: \\\"45m\\\" # Generous timeout for loop (timeout-free)\\n loop_type: for\\n max_iterations: 100\\n # Inner tasks inherit the 45-minute timeout\\n```\\n\\n## Field Validation Rules\\n\\n### ID Fields\\n\\n```yaml\\n# Valid IDs\\nid: \\\"customer_processor\\\" # \\u2705 Valid\\nid: \\\"process-data\\\" # \\u2705 Valid\\nid: \\\"task_001\\\" # \\u2705 Valid\\n\\n# Invalid IDs\\nid: \\\"process data\\\" # \\u274c Spaces not allowed\\nid: \\\"123_start\\\" # \\u274c Cannot start with number\\nid: \\\"\\\" # \\u274c Cannot be empty\\n```\\n\\n### Timeout Values\\n\\n```yaml\\n# Valid timeouts\\ntimeout_seconds: 300 # \\u2705 Positive integer\\ntimeout_seconds: \\\"5m\\\" # \\u2705 Duration string\\ntimeout_seconds: \\\"1h30m\\\" # \\u2705 Complex duration\\n\\n# Invalid timeouts\\ntimeout_seconds: 0 # \\u274c Must be positive\\ntimeout_seconds: -60 # \\u274c Cannot be negative\\ntimeout_seconds: \\\"invalid\\\" # \\u274c Invalid duration format\\n```\\n\\n### Dependency Types and Validation\\n\\n#### `depends_on` vs `previous_node`\\n\\n**`depends_on` - Execution Dependencies:**\\n- Controls actual task execution order\\n- Enables template resolution and data access\\n- Required for accessing outputs from other tasks\\n- Validates that referenced tasks exist\\n- Multiple tasks can be specified\\n\\n**`previous_node` - Task Connectivity and UI Hierarchy:**\\n- **REQUIRED for proper task connectivity in UI**\\n- Essential for workflow visualization and task linking\\n- Critical for execution flow representation\\n- Single task reference only\\n- **Without this field, tasks won\u0027t be properly connected in the UI**\\n\\n```yaml\\ntasks:\\n - id: fetch_data\\n type: http\\n url: \\\"https://api.example.com/data\\\"\\n \\n - id: validate_data\\n type: script\\n depends_on: [fetch_data] # \\u2705 EXECUTION: wait for fetch_data to complete\\n previous_node: fetch_data # \\u2705 UI: show as child of fetch_data in hierarchy\\n script: |\\n data = ${fetch_data} # \\u2705 Can access because of depends_on\\n if not validate(data):\\n raise ValueError(\\\"Invalid data\\\")\\n \\n # Required: Use outputs dict and __OUTPUTS__ marker for proper parsing\\n outputs[\u0027status\u0027] = \u0027valid\u0027\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: process_data\\n type: script\\n depends_on: [validate_data] # \\u2705 EXECUTION: wait for validation\\n previous_node: validate_data # \\u2705 UI: show as child of validate_data\\n script: |\\n validation_result = ${validate_data} # \\u2705 Accessible via depends_on\\n # Process data...\\n\\n - id: separate_task\\n type: script\\n previous_node: process_data # \\u2705 UI: appears under process_data in hierarchy\\n # NO depends_on means this can run immediately (parallel to other tasks)\\n script: |\\n # This task runs independently but appears under process_data in UI\\n # Cannot access ${process_data} because no depends_on relationship\\n print(\\\"Independent task\\\")\\n\\n### Complete Example with Proper Hierarchy\\n\\n```yaml\\ntasks:\\n - id: fetch_customer_data\\n name: \\\"Fetch Customer Data\\\"\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: validate_customer\\n name: \\\"Validate Customer Data\\\"\\n type: script\\n depends_on: [fetch_customer_data] # \\u2705 EXECUTION: Wait for data fetch\\n previous_node: fetch_customer_data # \\u2705 UI: Show under fetch_customer_data\\n script: |\\n customer = ${fetch_customer_data} # \\u2705 Can access because of depends_on\\n if not customer.get(\u0027email\u0027):\\n raise ValueError(\\\"Invalid customer data\\\")\\n print(\\\"__OUTPUTS__ {\u0027status\u0027: \u0027valid\u0027}\\\")\\n \\n - id: process_customer\\n name: \\\"Process Customer\\\"\\n type: ai_agent\\n depends_on: [validate_customer] # \\u2705 EXECUTION: Wait for validation\\n previous_node: validate_customer # \\u2705 UI: Show under validate_customer\\n config:\\n model_client_id: gpt4_model\\n user_message: \\\"Process customer: ${fetch_customer_data}\\\"\\n \\n - id: send_notification\\n name: \\\"Send Notification\\\"\\n type: script\\n depends_on: [process_customer] # \\u2705 EXECUTION: Wait for processing\\n previous_node: process_customer # \\u2705 UI: Show under process_customer\\n script: |\\n result = ${process_customer}\\n print(f\\\"Notification sent for: ${fetch_customer_data.name}\\\")\\n \\n - id: parallel_audit_log\\n name: \\\"Audit Log (Parallel)\\\"\\n type: script\\n depends_on: [fetch_customer_data] # \\u2705 EXECUTION: Only needs initial data\\n previous_node: fetch_customer_data # \\u2705 UI: Show under fetch_customer_data\\n script: |\\n # This runs in parallel with validation/processing\\n customer = ${fetch_customer_data}\\n print(f\\\"Audit: Customer {customer.id} accessed at ${current_timestamp}\\\")\\n```\\n\\n # Invalid examples\\n - id: invalid_task\\n type: script\\n depends_on: [nonexistent_task] # \\u274c VALIDATION ERROR: task doesn\u0027t exist\\n previous_node: another_missing # \\u26a0\\ufe0f UI WARNING: task doesn\u0027t exist (but not fatal)\\n script: \\\"print(\u0027Invalid\u0027)\\\"\\n```\\n\\n#### Key Differences:\\n\\n| Aspect | `depends_on` | `previous_node` |\\n|--------|--------------|-----------------|\\n| **Purpose** | Execution control | Task connectivity \u0026 UI organization |\\n| **Impact** | Blocks task until dependencies complete | **REQUIRED for UI task linking** |\\n| **Template Access** | \\u2705 Enables `${task_id}` resolution | \\u274c No template access |\\n| **Validation** | \\u274c Parser error if task missing | \\u26a0\\ufe0f Warning if task missing |\\n| **Multiple References** | \\u2705 Array of task IDs | \\u274c Single task ID only |\\n| **Execution Order** | \\u2705 Enforced | \\u274c No execution impact |\\n| **UI Connectivity** | \\u274c No UI impact | \\u2705 **ESSENTIAL for proper task connection** |\\n\\n### Circular Dependency Detection\\n\\n```yaml\\n# This will be rejected by the parser\\ntasks:\\n - id: task_a\\n depends_on: [task_b] # \\u274c Circular dependency\\n \\n - id: task_b \\n depends_on: [task_a] # \\u274c Circular dependency\\n```\\n\\n## Parser Behavior\\n\\n### Field Name Compatibility\\n\\nThe parser supports both camelCase and snake_case for field names:\\n\\n```yaml\\ntasks:\\n - id: flexible_task\\n # Both formats are accepted\\n dependsOn: [other_task] # camelCase\\n depends_on: [other_task] # snake_case\\n \\n inputMapping: # camelCase\\n source_field: \\\"target_field\\\"\\n input_mapping: # snake_case \\n source_field: \\\"target_field\\\"\\n \\n outputMapping: {} # camelCase\\n output_mapping: {} # snake_case\\n \\n workerGroup: \\\"gpu-workers\\\" # camelCase\\n worker_group: \\\"gpu-workers\\\" # snake_case\\n```\\n\\n### Default Value Assignment\\n\\n```yaml\\n# Fields with defaults are automatically populated\\nid: my_workflow\\n# name defaults to id value: \\\"my_workflow\\\"\\n# version defaults to: 1\\n# namespace defaults to: \\\"default\\\" \\n# timeout_seconds defaults to: 3600\\n# description defaults to: \\\"\\\"\\n\\ntasks:\\n - id: simple_task\\n type: script\\n # name defaults to id: \\\"simple_task\\\"\\n # timeout_seconds defaults to: 300\\n # retry.maxAttempts defaults to: 1\\n```\\n\\n### Template Resolution Context\\n\\nThe parser provides these built-in variables for template resolution:\\n\\n```yaml\\ntasks:\\n - id: context_aware_task\\n type: script\\n script: |\\n # Workflow context\\n workflow_id = \\\"${workflow_id}\\\" # Current workflow ID\\n execution_id = \\\"${execution_id}\\\" # Unique execution ID\\n namespace = \\\"${namespace}\\\" # Workflow namespace\\n \\n # Timing context\\n current_timestamp = \\\"${current_timestamp}\\\" # ISO timestamp\\n started_at = \\\"${started_at}\\\" # Workflow start time\\n \\n # Input context\\n user_input = \\\"${input_parameter}\\\" # Workflow inputs\\n \\n # Task context\\n previous_result = ${previous_task} # Previous task output\\n specific_field = ${task_id.field_name} # Specific output field\\n \\n # Route context (for conditional workflows)\\n route_info = ${_route_context} # Current route information\\n```\\n\\n### Error Handling During Parsing\\n\\n```yaml\\n# The parser will validate and report these errors:\\n\\n# 1. Missing required fields\\n# ERROR: Missing required field: \u0027id\u0027\\n\\n# 2. Invalid YAML syntax \\n# ERROR: Invalid YAML: mapping values are not allowed here\\n\\n# 3. Circular dependencies\\n# ERROR: Circular dependency detected involving task \u0027task_a\u0027\\n\\n# 4. Invalid task references\\n# ERROR: Task \u0027task_b\u0027 depends on non-existent task \u0027invalid_task\u0027\\n\\n# 5. Invalid duration formats\\n# ERROR: Invalid duration: \u0027not_a_duration\u0027\\n\\n# 6. Invalid field values\\n# ERROR: Timeout must be positive, got -10\\n```\\n\\n## Migration Guide\\n\\n### From Version 1.0 to 2.0\\n\\n```yaml\\n# OLD FORMAT (v1.0)\\nworkflow:\\n id: my_workflow\\n tasks:\\n - task_id: process_data\\n task_type: python_script\\n \\n# NEW FORMAT (v2.0) \\nid: my_workflow\\ntasks:\\n - id: process_data\\n type: script\\n language: python\\n```\\n\\n### Deprecated Fields\\n\\n| Old Field | New Field | Status |\\n|-----------|-----------|--------|\\n| `task_type` | `type` | Deprecated in v2.0 |\\n| `task_id` | `id` | Deprecated in v2.0 |\\n| `python_script` | `script` (type) | Deprecated in v2.0 |\\n| `plugin` | `mcp` (type) | Deprecated in v2.1 |\\n\\n### Backward Compatibility\\n\\nThe parser maintains backward compatibility for deprecated fields but will show warnings:\\n\\n```yaml\\n# This will work but show a warning\\ntasks:\\n - task_id: old_style_task # \\u26a0\\ufe0f Warning: Use \u0027id\u0027 instead\\n task_type: python_script # \\u26a0\\ufe0f Warning: Use \u0027type: script\u0027 instead\\n```\\n\\n## Advanced Schema Features\\n\\n### Conditional Field Requirements\\n\\nSome fields become required based on task type:\\n\\n```yaml\\n# HTTP tasks require URL\\n- id: api_call\\n type: http\\n url: \\\"https://api.example.com\\\" # \\u2705 Required for HTTP tasks\\n \\n# AI agent tasks require model_client_id\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: gpt4 # \\u2705 Required for AI agent tasks\\n```\\n\\n### Dynamic Field Validation\\n\\nField validation can depend on other field values:\\n\\n```yaml\\n# Loop tasks require loop-specific fields\\n- id: data_loop\\n type: loop\\n loop_type: \\\"for\\\" # Determines required fields\\n max_iterations: 10 # \\u2705 Required for \u0027for\u0027 loops\\n # exit_condition: not needed for \u0027for\u0027 loops\\n \\n- id: condition_loop \\n type: loop\\n loop_type: \\\"while\\\" # Determines required fields\\n exit_condition: \\\"${count} \u003e= 10\\\" # \\u2705 Required for \u0027while\u0027 loops\\n # max_iterations: optional for \u0027while\u0027 loops\\n```\\n\\n## Template Resolution and Data Flow\\n\\nTemplate resolution is a core feature that allows tasks to access data from previous tasks, workflow inputs, and built-in variables using the `${variable_name}` syntax. This is essential for creating dynamic workflows where tasks pass data to each other.\\n\\n### Basic Template Syntax\\n\\n```yaml\\n# Access workflow inputs\\nscript: |\\n customer_name = \\\"${customer_name}\\\" # From workflow inputs\\n \\n# Access previous task outputs\\nscript: |\\n previous_result = ${data_fetch_task} # Entire output from data_fetch_task\\n specific_field = ${data_fetch_task.customer_id} # Specific field\\n \\n# Access built-in variables\\nscript: |\\n workflow_id = \\\"${workflow_id}\\\"\\n execution_id = \\\"${execution_id}\\\"\\n timestamp = \\\"${current_timestamp}\\\"\\n```\\n\\n### Available Built-in Variables\\n\\n| Variable | Description | Example Value |\\n|----------|-------------|---------------|\\n| `${workflow_id}` | Current workflow identifier | `customer_processor` |\\n| `${execution_id}` | Unique execution identifier | `exec_12345_abcde` |\\n| `${namespace}` | Workflow namespace | `production` |\\n| `${current_timestamp}` | ISO timestamp | `2024-01-15T10:30:00Z` |\\n| `${started_at}` | Workflow start time | `2024-01-15T10:29:45Z` |\\n\\n### Task Output Resolution\\n\\nTasks can access outputs from other tasks they depend on:\\n\\n```yaml\\ntasks:\\n - id: fetch_customer\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: process_customer\\n type: script\\n depends_on: [fetch_customer] # Required for template access\\n previous_node: fetch_customer\\n script: |\\n # Access entire response\\n customer_data = ${fetch_customer}\\n \\n # Access specific fields\\n customer_name = \\\"${fetch_customer.name}\\\"\\n customer_email = \\\"${fetch_customer.email}\\\"\\n \\n # Process the data\\n result = {\\n \\\"processed_name\\\": customer_name.upper(),\\n \\\"domain\\\": customer_email.split(\\\"@\\\")[1]\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n```\\n\\n### AI Agent Output Resolution\\n\\nAI agent tasks return structured output that requires specific access patterns:\\n\\n```yaml\\n# AI Agent task\\n- id: ai_analysis_task\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Analyze the customer data and provide insights.\\\"\\n user_message: \\\"Customer data: ${customer_data}\\\"\\n\\n# Accessing AI agent output in subsequent tasks\\n- id: process_ai_response\\n type: script\\n depends_on: [ai_analysis_task]\\n previous_node: ai_analysis_task\\n script: |\\n # CORRECT: AI agents return structured output\\n agent_output = ${ai_analysis_task}\\n ai_response = agent_output[\u0027ai_response\u0027].strip()\\n \\n # Additional metadata available\\n token_usage = agent_output.get(\u0027token_usage\u0027, {})\\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\\n \\n # Process the AI response\\n processed_analysis = {\\n \\\"analysis\\\": ai_response,\\n \\\"tokens_used\\\": token_usage.get(\u0027total_tokens\u0027, 0),\\n \\\"model\\\": model_used,\\n \\\"processed_at\\\": \\\"${current_timestamp}\\\"\\n }\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(processed_analysis)}\\\")\\n\\n# INCORRECT: Direct access won\u0027t work\\n- id: incorrect_ai_access\\n type: script\\n depends_on: [ai_analysis_task]\\n script: |\\n # \\u274c This will not work correctly\\n ai_response = ${ai_analysis_task} # Returns full agent object\\n # ai_response is now a dict, not the actual response text\\n```\\n\\n### AI Agent Response Structure\\n\\nAI agent tasks return a structured object with the following format:\\n\\n```json\\n{\\n \\\"ai_response\\\": \\\"The actual AI-generated text response\\\",\\n \\\"model\\\": \\\"gpt-4-turbo-preview\\\",\\n \\\"token_usage\\\": {\\n \\\"prompt_tokens\\\": 150,\\n \\\"completion_tokens\\\": 75,\\n \\\"total_tokens\\\": 225\\n },\\n \\\"estimated_cost\\\": {\\n \\\"amount\\\": 0.00337,\\n \\\"currency\\\": \\\"USD\\\"\\n },\\n \\\"provider\\\": \\\"openai\\\",\\n \\\"timestamp\\\": \\\"2024-01-15T10:30:00Z\\\"\\n}\\n```\\n\\n### Loop Task Data Access and Dependencies\\n\\nLoop tasks have a special execution context that differs from regular workflow tasks. Understanding how to access data within loops is crucial for building functional workflows.\\n\\n#### Loop Task Internal Data Access\\n\\nWithin loop tasks, data access works through injected context files rather than template resolution:\\n\\n```yaml\\n# Loop task with state variables and internal task dependencies\\n- id: data_processing_loop\\n type: loop\\n loop_type: for\\n max_iterations: 3\\n state_variables:\\n processed_items: []\\n total_processed: 0\\n error_count: 0\\n loop_tasks:\\n # First task in loop\\n - id: fetch_data\\n type: script\\n script: |\\n # Access loop state variables through injected context\\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\\n processed_items = loop_state.get(\u0027processed_items\u0027, [])\\n \\n # Access workflow inputs and external task outputs\\n external_data = inputs.get(\u0027previous_workflow_task\u0027, {})\\n iteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable\\n \\n # Process data\\n batch_data = fetch_batch_data(iteration_num)\\n \\n outputs = {\\\"batch_data\\\": batch_data, \\\"batch_size\\\": len(batch_data)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n # Second task depends on first task in same loop\\n - id: process_data\\n type: script\\n depends_on: [fetch_data] # \\u2705 CORRECT: Dependencies within loop tasks work normally\\n script: |\\n # \\u2705 CORRECT: Access previous loop task through inputs context\\n fetch_result = inputs.get(\u0027fetch_data\u0027, {})\\n batch_data = fetch_result.get(\u0027batch_data\u0027, [])\\n \\n # \\u274c INCORRECT: This won\u0027t work in loop context\\n # batch_data = ${fetch_data.batch_data} # Template resolution fails in loops\\n \\n # Access and update state variables\\n current_processed = loop_state.get(\u0027processed_items\u0027, [])\\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\\n \\n # Process the batch\\n processed_batch = process_batch(batch_data)\\n current_processed.extend(processed_batch)\\n \\n # Update state variables using state_updates\\n state_updates[\u0027processed_items\u0027] = current_processed\\n state_updates[\u0027total_processed\u0027] = current_total + len(processed_batch)\\n \\n outputs = {\\\"processed_count\\\": len(processed_batch)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n print(f\\\"__STATE_UPDATES__ {json.dumps(state_updates)}\\\") # \\u2705 REQUIRED for state updates\\n```\\n\\n#### Key Differences in Loop Task Data Access\\n\\n| Access Pattern | Regular Tasks | Loop Tasks |\\n|---------------|---------------|------------|\\n| **Previous Task Output** | `${previous_task}` | `inputs.get(\u0027field_name\u0027, default)` (flattened) |\\n| **State Variables** | N/A | `loop_state.get(\u0027variable_name\u0027, default)` |\\n| **State Updates** | N/A | `state_updates[\u0027variable_name\u0027] = value` |\\n| **Workflow Inputs** | `\\\"${input_name}\\\"` | `inputs.get(\u0027input_name\u0027, \u0027default\u0027)` |\\n| **Dependencies** | `depends_on: [task_id]` | Same: `depends_on: [task_id]` |\\n\\n#### Critical: Loop Task Output Flattening\\n\\n**Important:** The loop executor flattens task outputs into individual fields in the `inputs` dictionary, rather than storing them as nested objects under the task name.\\n\\n```yaml\\n# First loop task outputs:\\noutputs = {\\\"company_name\\\": \\\"ACME Corp\\\", \\\"company_id\\\": \\\"123\\\", \\\"status\\\": \\\"success\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n\\n# Second loop task accessing the data:\\nscript: |\\n # \\u274c INCORRECT: This won\u0027t work - no nested object\\n # company_data = inputs.get(\u0027first_task\u0027, {})\\n # company_name = company_data.get(\u0027company_name\u0027)\\n \\n # \\u2705 CORRECT: Access flattened fields directly\\n company_name = inputs.get(\u0027company_name\u0027, \u0027\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\\n \\n print(f\\\"Company: {company_name}, ID: {company_id}, Status: {task_status}\\\")\\n```\\n\\n#### Complete Loop Task Example\\n\\n```yaml\\n- id: financial_processing_loop\\n type: loop\\n loop_type: for\\n max_iterations: 3\\n iteration_variable: company_index\\n depends_on: [list_companies] # External dependency\\n \\n state_variables:\\n processed_companies: []\\n total_extractions: 0\\n successful_extractions: 0\\n \\n loop_tasks:\\n - id: process_current_company\\n type: script\\n script: |\\n # Access external workflow task data\\n companies_data = inputs.get(\u0027list_companies\u0027, {})\\n company_idx = inputs.get(\u0027company_index\u0027, 0)\\n \\n # Access state variables\\n processed = loop_state.get(\u0027processed_companies\u0027, [])\\n \\n # Process company\\n company = companies_data[\u0027companies\u0027][company_idx]\\n result = process_company(company)\\n \\n outputs = {\\\"company_name\\\": company[\u0027name\u0027], \\\"status\\\": \\\"success\\\"}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: extract_financial_data\\n type: script\\n depends_on: [process_current_company] # Depend on previous loop task\\n requirements:\\n - requests\u003e=2.28.0\\n - some-ai-library\u003e=1.0.0\\n script: |\\n # \\u2705 CORRECT: Access flattened outputs from previous loop task\\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\\n \\n # \\u274c INCORRECT: Loop executor flattens outputs, no nested access\\n # company_data = inputs.get(\u0027process_current_company\u0027, {})\\n # company_name = company_data.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n \\n # \\u274c INCORRECT: Template resolution doesn\u0027t work in loops\\n # company_data = ${process_current_company}\\n \\n # Access external data\\n companies_list = inputs.get(\u0027list_companies\u0027, {})\\n \\n # Perform AI extraction\\n extraction_result = extract_with_ai({\\\"company_name\\\": company_name, \\\"company_id\\\": company_id})\\n \\n # Update state variables\\n current_processed = loop_state.get(\u0027processed_companies\u0027, [])\\n current_total = loop_state.get(\u0027total_extractions\u0027, 0)\\n current_successful = loop_state.get(\u0027successful_extractions\u0027, 0)\\n \\n current_processed.append({\\n \\\"company_name\\\": company_name,\\n \\\"extraction_result\\\": extraction_result\\n })\\n \\n # State updates are required for persistence\\n state_updates[\u0027processed_companies\u0027] = current_processed\\n state_updates[\u0027total_extractions\u0027] = current_total + 1\\n state_updates[\u0027successful_extractions\u0027] = current_successful + (1 if extraction_result else 0)\\n \\n outputs = {\\\"extracted\\\": bool(extraction_result)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n print(f\\\"__STATE_UPDATES__ {json.dumps(state_updates)}\\\") # \\u2705 REQUIRED\\n\\n# Accessing loop task outputs in subsequent workflow tasks\\n- id: process_loop_results\\n type: script\\n depends_on: [financial_processing_loop]\\n previous_node: financial_processing_loop\\n script: |\\n # \\u2705 CORRECT: Access final state of loop\\n loop_result = ${financial_processing_loop}\\n loop_final_state = loop_result.get(\u0027final_state\u0027, {})\\n \\n # Access specific state variables from final state\\n processed_companies = loop_final_state.get(\u0027processed_companies\u0027, [])\\n total_processed = loop_final_state.get(\u0027total_extractions\u0027, 0)\\n successful_count = loop_final_state.get(\u0027successful_extractions\u0027, 0)\\n \\n # Access loop execution metadata\\n iterations_completed = loop_result.get(\u0027iterations_completed\u0027, 0)\\n \\n print(f\\\"Loop completed {iterations_completed} iterations\\\")\\n print(f\\\"Processed {len(processed_companies)} companies\\\")\\n print(f\\\"Success rate: {successful_count}/{total_processed}\\\")\\n \\n outputs = {\\n \\\"companies_processed\\\": len(processed_companies),\\n \\\"success_rate\\\": f\\\"{successful_count}/{total_processed}\\\",\\n \\\"iterations\\\": iterations_completed\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n#### Loop Task Context Injection and Output Flattening\\n\\nThe loop executor injects context through temporary files and flattens task outputs for easier access:\\n\\n```python\\n# Auto-injected at the start of each loop task script:\\nimport json\\nimport os\\n\\n# Load context from temporary files (auto-generated)\\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\\n inputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\\n\\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\\n loop_state = json.load(f) # Contains current state variables\\n\\noutputs = {}\\nstate_updates = {}\\n\\n# Your script code follows...\\n```\\n\\n#### Debugging Loop Task Data Access\\n\\nWhen debugging loop tasks, always print available keys to understand what data is accessible:\\n\\n```yaml\\n- id: debug_loop_task\\n type: script\\n depends_on: [previous_loop_task]\\n script: |\\n # Debug: Print all available data\\n print(f\\\"\\ud83d\\udd0d Available inputs: {list(inputs.keys())}\\\")\\n print(f\\\"\\ud83d\\udd0d Loop state keys: {list(loop_state.keys())}\\\")\\n \\n # Debug: Print specific values\\n for key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\\n value = inputs.get(key, \u0027NOT_FOUND\u0027)\\n print(f\\\"\\ud83d\\udd0d {key}: {value}\\\")\\n \\n # Now access the flattened data\\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n \\n # Process your data...\\n outputs = {\\\"processed\\\": f\\\"Company {company_name}\\\"}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n#### Real Example: Task Output Flattening\\n\\n```yaml\\n# Task 1 outputs this structure:\\n__OUTPUTS__ {\\\"company_name\\\": \\\"LTTS\\\", \\\"company_id\\\": \\\"123\\\", \\\"fy_folders\\\": [{\\\"id\\\": \\\"456\\\", \\\"name\\\": \\\"FY24\\\"}], \\\"status\\\": \\\"success\\\"}\\n\\n# Task 2 receives flattened inputs:\\ninputs = {\\n \\\"folder_id\\\": \\\"...\\\", # Workflow input\\n \\\"gemini_api_key\\\": \\\"...\\\", # Workflow input \\n \\\"list_company_folders\\\": {...}, # External task output\\n \\\"company_index\\\": 0, # Loop iteration variable\\n \\\"company_name\\\": \\\"LTTS\\\", # \\u2705 Flattened from Task 1\\n \\\"company_id\\\": \\\"123\\\", # \\u2705 Flattened from Task 1\\n \\\"fy_folders\\\": [{\\\"id\\\": \\\"456\\\", \\\"name\\\": \\\"FY24\\\"}], # \\u2705 Flattened from Task 1\\n \\\"status\\\": \\\"success\\\", # \\u2705 Flattened from Task 1\\n \\\"total_extractions\\\": 0, # Loop state variable\\n \\\"processed_companies\\\": [] # Loop state variable\\n}\\n```\\n\\n#### Loop Task Special Output Fields\\n\\n| Field | Description | Example Access |\\n|-------|-------------|----------------|\\n| `final_state` | Final values of all state variables | `${loop_task.final_state}` |\\n| `final_state[\\\"variable_name\\\"]` | Specific state variable from final state | `${loop_task.final_state[\\\"total_processed\\\"]}` |\\n| `iterations_completed` | Number of iterations executed | `${loop_task.iterations_completed}` |\\n\\n#### Safe Template Resolution Pattern for Loop States\\n\\nWhen accessing loop final state fields, use this safe parsing pattern:\\n\\n```yaml\\nscript: |\\n try:\\n # Template resolution with safety checks\\n final_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\\n processed_count_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_count\\\"]}\u0027\u0027\u0027\\n items_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_items\\\"]}\u0027\u0027\u0027\\n iterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027\\n \\n # Safe parsing with fallbacks\\n final_state = json.loads(final_state_str) if not final_state_str.startswith(\u0027UNRESOLVED_\u0027) else {}\\n processed_count = int(processed_count_str) if processed_count_str.isdigit() else 0\\n processed_items = json.loads(items_str) if not items_str.startswith(\u0027UNRESOLVED_\u0027) else []\\n iterations = int(iterations_str) if iterations_str.isdigit() else 0\\n \\n print(f\\\"Successfully processed {processed_count} items in {iterations} iterations\\\")\\n \\n except Exception as e:\\n print(f\\\"Error accessing loop results: {str(e)}\\\")\\n # Handle error case\\n```\\n```\\n\\n### Real-World AI Agent Example\\n\\nBased on actual WhatsApp workflow patterns:\\n\\n```yaml\\n# AI greeting agent\\n- id: greeting_agent\\n type: ai_agent\\n config:\\n model_client_id: assistant_model\\n system_message: \\\"You are a helpful assistant. Generate a friendly greeting.\\\"\\n user_message: \\\"User said: ${user_message}\\\"\\n\\n# Send the AI response via API\\n- id: send_greeting\\n type: script\\n depends_on: [greeting_agent]\\n previous_node: greeting_agent\\n script: |\\n import requests\\n import json\\n \\n # CORRECT: Access AI response from structured output\\n agent_output = ${greeting_agent}\\n ai_greeting = agent_output[\u0027ai_response\u0027].strip()\\n \\n phone_number = \\\"${phone_number}\\\"\\n api_token = \\\"${API_TOKEN}\\\"\\n \\n print(f\\\"\\ud83e\\udd16 AI generated greeting: {ai_greeting}\\\")\\n \\n # Send via API\\n url = f\\\"https://api.example.com/messages/{phone_number}\\\"\\n headers = {\\\"Authorization\\\": f\\\"Bearer {api_token}\\\"}\\n \\n response = requests.post(url, headers=headers, json={\\n \\\"message\\\": ai_greeting\\n })\\n \\n result = {\\n \\\"status\\\": \\\"sent\\\",\\n \\\"ai_response\\\": ai_greeting,\\n \\\"api_response\\\": response.json(),\\n \\\"tokens_used\\\": agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\\n }\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n```\\n\\n### Complex Template Patterns\\n\\n```yaml\\n# Conditional templates with router outputs\\n- id: router_task\\n type: conditional_router\\n conditions:\\n - name: \\\"premium_customer\\\"\\n condition: \\\"${customer_data.tier} == \u0027premium\u0027\\\"\\n route: \\\"premium_processing\\\"\\n\\n- id: premium_handler\\n type: script\\n depends_on: [router_task]\\n previous_node: router_task\\n execute_on_routes: [\\\"premium_processing\\\"]\\n script: |\\n # Access router decision\\n selected_route = \\\"${router_task.selected_route}\\\"\\n route_data = ${router_task.route_data}\\n \\n # Process premium customer\\n customer = ${customer_data}\\n premium_benefits = calculate_benefits(customer)\\n\\n# Loop through arrays from previous tasks\\n- id: process_items\\n type: script\\n depends_on: [fetch_items]\\n previous_node: fetch_items\\n script: |\\n items = ${fetch_items.data}\\n processed_items = []\\n \\n for item in items:\\n processed_item = {\\n \\\"id\\\": item[\\\"id\\\"],\\n \\\"processed_at\\\": \\\"${current_timestamp}\\\",\\n \\\"status\\\": \\\"processed\\\"\\n }\\n processed_items.append(processed_item)\\n \\n print(f\\\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\\\")\\n\\n# Nested object access\\n- id: user_preferences\\n type: script\\n depends_on: [fetch_user]\\n previous_node: fetch_user\\n script: |\\n # Access nested user preferences\\n email_notifications = ${fetch_user.preferences.notifications.email}\\n theme_setting = \\\"${fetch_user.preferences.ui.theme}\\\"\\n language = \\\"${fetch_user.profile.language}\\\"\\n \\n settings = {\\n \\\"notifications\\\": email_notifications,\\n \\\"theme\\\": theme_setting,\\n \\\"locale\\\": language\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(settings)}\\\")\\n```\\n\\n### Template Resolution Rules\\n\\n1. **Dependency Requirement**: Tasks must have `depends_on` relationship to access other task outputs\\n2. **String vs Object**: Use quotes for string values, no quotes for objects/arrays\\n3. **Field Navigation**: Use dot notation for nested field access\\n4. **Error Handling**: Unresolved templates become `UNRESOLVED_variable_name`\\n\\n```yaml\\n# \\u2705 Correct template usage\\n- id: correct_task\\n depends_on: [previous_task]\\n previous_node: previous_task\\n script: |\\n string_value = \\\"${previous_task.name}\\\" # String with quotes\\n object_value = ${previous_task.data} # Object without quotes\\n nested_value = \\\"${previous_task.user.email}\\\" # Nested field\\n\\n# \\u274c Incorrect template usage \\n- id: incorrect_task\\n # Missing depends_on - template won\u0027t resolve\\n script: |\\n value = ${previous_task.name} # Will become UNRESOLVED_previous_task.name\\n```\\n\\n## Script Task Output Handling\\n\\nScript tasks must use the `__OUTPUTS__` marker pattern for proper output parsing by the workflow engine.\\n\\n### Required Output Pattern\\n\\n```yaml\\ntasks:\\n - id: example_script\\n type: script\\n script: |\\n # Your script logic here\\n data = process_data()\\n \\n # Required: Store outputs in outputs dict\\n outputs = {}\\n outputs[\u0027result\u0027] = data\\n outputs[\u0027status\u0027] = \u0027success\u0027\\n outputs[\u0027count\u0027] = len(data)\\n \\n # Required: Print with __OUTPUTS__ marker for engine parsing\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### Output Access in Subsequent Tasks\\n\\n```yaml\\ntasks:\\n - id: data_processor\\n type: script\\n script: |\\n # Process and store outputs\\n results = {\\\"processed_count\\\": 100, \\\"errors\\\": []}\\n \\n outputs = {}\\n outputs[\u0027results\u0027] = results\\n outputs[\u0027status\u0027] = \u0027completed\u0027\\n \\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: result_handler\\n type: script\\n depends_on: [data_processor]\\n script: |\\n # Access previous task outputs\\n processor_output = ${data_processor}\\n processed_count = processor_output[\u0027results\u0027][\u0027processed_count\u0027]\\n status = processor_output[\u0027status\u0027]\\n \\n print(f\\\"Previous task processed {processed_count} items with status: {status}\\\")\\n \\n # Store new outputs\\n outputs = {\u0027message\u0027: f\u0027Handled {processed_count} results\u0027}\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### Error Handling in Scripts\\n\\n```yaml\\ntasks:\\n - id: safe_script\\n type: script\\n script: |\\n try:\\n result = risky_operation()\\n outputs = {\u0027result\u0027: result, \u0027status\u0027: \u0027success\u0027}\\n except Exception as e:\\n outputs = {\u0027error\u0027: str(e), \u0027status\u0027: \u0027failed\u0027}\\n \\n # Always print outputs, even for errors\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n## Triggers and Workflow Activation\\n\\nTriggers define how workflows are started automatically based on external events. They enable event-driven workflow execution.\\n\\n### Webhook Triggers\\n\\nWebhook triggers allow external systems to start workflows by sending HTTP requests.\\n\\n```yaml\\ntriggers:\\n - name: customer_webhook\\n type: webhook\\n description: \\\"Triggered when customer data is updated\\\"\\n config:\\n path: \\\"/webhooks/customer-updated\\\" # Webhook endpoint path\\n method: POST # HTTP method (GET, POST, PUT, etc.)\\n authentication:\\n type: token # none|token|signature|basic\\n token: \\\"${WEBHOOK_SECRET}\\\" # Required if type is token\\n header_name: \\\"X-Webhook-Token\\\" # Optional: custom header name\\n validation:\\n required_fields: [\\\"customer.id\\\", \\\"customer.email\\\"] # Required payload fields\\n content_type: \\\"application/json\\\" # Expected content type\\n input_mapping:\\n - webhook_field: customer.id # Field from webhook payload\\n workflow_input: customer_id # Map to workflow input\\n - webhook_field: customer.email\\n workflow_input: customer_email\\n - webhook_field: customer.updated_at\\n workflow_input: update_timestamp\\n```\\n\\n### Scheduled Triggers\\n\\nSchedule workflows to run at specific times or intervals.\\n\\n```yaml\\ntriggers:\\n - name: daily_report\\n type: schedule\\n description: \\\"Generate daily customer report\\\"\\n config:\\n cron: \\\"0 9 * * *\\\" # Daily at 9 AM (cron format)\\n timezone: \\\"UTC\\\" # Optional: timezone\\n enabled: true # Optional: enable/disable\\n max_instances: 1 # Optional: max concurrent instances\\n input_mapping:\\n - workflow_input: report_date\\n default_value: \\\"${current_date}\\\" # Use built-in variable\\n - workflow_input: report_type\\n default_value: \\\"daily\\\"\\n\\n - name: weekly_cleanup\\n type: schedule\\n description: \\\"Weekly database cleanup\\\"\\n config:\\n cron: \\\"0 2 * * 0\\\" # Weekly on Sunday at 2 AM\\n timezone: \\\"America/New_York\\\"\\n input_mapping:\\n - workflow_input: cleanup_before_date\\n default_value: \\\"${date_minus_7_days}\\\"\\n```\\n\\n### Event Triggers\\n\\nEvent triggers respond to system events from various sources.\\n\\n```yaml\\ntriggers:\\n - name: file_uploaded\\n type: event\\n description: \\\"Process uploaded files\\\"\\n config:\\n event_type: \\\"file.uploaded\\\" # Event type to listen for\\n source: \\\"storage_service\\\" # Event source system\\n filter: # Optional: event filtering\\n file_type: [\\\"csv\\\", \\\"json\\\", \\\"xlsx\\\"]\\n file_size_max: \\\"100MB\\\"\\n path_pattern: \\\"/uploads/customers/*\\\"\\n retry: # Optional: retry configuration\\n max_attempts: 3\\n initial_delay: \\\"5s\\\"\\n input_mapping:\\n - webhook_field: file.path\\n workflow_input: file_path\\n - webhook_field: file.size\\n workflow_input: file_size\\n - webhook_field: file.type\\n workflow_input: file_type\\n\\n - name: order_status_changed\\n type: event\\n description: \\\"Handle order status changes\\\"\\n config:\\n event_type: \\\"order.status_changed\\\"\\n source: \\\"order_management\\\"\\n filter:\\n status: [\\\"completed\\\", \\\"cancelled\\\", \\\"refunded\\\"]\\n priority: [\\\"high\\\", \\\"urgent\\\"]\\n input_mapping:\\n - webhook_field: order.id\\n workflow_input: order_id\\n - webhook_field: order.status\\n workflow_input: new_status\\n - webhook_field: order.customer_id\\n workflow_input: customer_id\\n```\\n\\n### Dependency Triggers\\n\\nDependency triggers start workflows when other workflows complete.\\n\\n```yaml\\ntriggers:\\n - name: post_processing\\n type: dependency\\n description: \\\"Run after main processing completes\\\"\\n config:\\n depends_on_workflow: \\\"main_data_processing\\\" # Workflow to depend on\\n depends_on_status: \\\"SUCCESS\\\" # SUCCESS|FAILED|COMPLETED\\n namespace: \\\"production\\\" # Optional: workflow namespace\\n timeout_minutes: 60 # Optional: wait timeout\\n input_mapping:\\n - webhook_field: execution_id # From completed workflow\\n workflow_input: source_execution_id\\n - webhook_field: workflow_outputs.processed_count\\n workflow_input: items_processed\\n```\\n\\n## Model Client Configuration\\n\\nBefore defining AI-related tasks (ai_agent, chatbot, continuous_conversation), you must configure model clients. Model clients define which AI models to use and how to authenticate with them.\\n\\n### Basic Structure\\n\\n```yaml\\nmodel_clients:\\n - id: unique_identifier # Used to reference this model in tasks\\n provider: provider_name # AI provider (openai, anthropic, etc.)\\n config:\\n model: model_name # Specific model to use\\n api_key: your_api_key # Authentication key\\n # Additional provider-specific options...\\n```\\n\\n### Supported Providers\\n\\n#### OpenAI Provider\\n\\n```yaml\\nmodel_clients:\\n - id: gpt4_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview # Available: gpt-4, gpt-4-turbo, gpt-3.5-turbo\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 0.7 # Optional: 0.0-2.0\\n max_tokens: 2000 # Optional: Maximum response tokens\\n timeout: 30 # Optional: Request timeout (seconds)\\n```\\n\\n#### Anthropic Provider\\n\\n```yaml\\nmodel_clients:\\n - id: claude_model\\n provider: anthropic\\n config:\\n model: claude-3-sonnet-20240229 # Available: claude-3-opus, claude-3-sonnet, claude-3-haiku\\n api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n temperature: 0.7 # Optional: 0.0-1.0\\n max_tokens: 1500 # Optional: Maximum response tokens\\n timeout: 30 # Optional: Request timeout (seconds)\\n```\\n\\n#### OpenRouter Provider\\n\\n```yaml\\nmodel_clients:\\n - id: openrouter_model\\n provider: openrouter\\n config:\\n model: anthropic/claude-3-sonnet # Available: Various models via OpenRouter\\n api_key: \\\"sk-or-your-openrouter-api-key\\\"\\n base_url: \\\"https://openrouter.ai/api/v1\\\" # Optional: Custom endpoint\\n temperature: 0.7\\n max_tokens: 2000\\n```\\n\\n#### Multiple Model Clients\\n\\nYou can define multiple model clients and reference them by their `id` in tasks:\\n\\n```yaml\\nmodel_clients:\\n - id: fast_model\\n provider: openai\\n config:\\n model: gpt-3.5-turbo\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 0.3\\n \\n - id: smart_model\\n provider: anthropic\\n config:\\n model: claude-3-opus-20240229\\n api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n temperature: 0.7\\n \\n - id: creative_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 1.2\\n```\\n\\n### Using Model Clients in Tasks\\n\\nReference model clients in AI tasks using the `model_client_id` field:\\n\\n```yaml\\ntasks:\\n - id: analysis_task\\n type: ai_agent\\n config:\\n model_client_id: smart_model # References the claude model above\\n system_message: \\\"You are a data analyst...\\\"\\n user_message: \\\"Analyze this data: ${input_data}\\\"\\n \\n - id: creative_task\\n type: ai_agent\\n config:\\n model_client_id: creative_model # References the creative GPT-4 model\\n system_message: \\\"You are a creative writer...\\\"\\n user_message: \\\"Write a story about ${topic}\\\"\\n```\\n\\n### Environment Variables for API Keys\\n\\nFor security, use environment variables instead of hardcoding API keys:\\n\\n```yaml\\nmodel_clients:\\n - id: secure_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"${OPENAI_API_KEY}\\\" # References environment variable\\n temperature: 0.7\\n```\\n\\n### Common Configuration Options\\n\\n| Field | Description | Required | Default |\\n|-------|-------------|----------|---------|\\n| `id` | Unique identifier for referencing in tasks | Yes | - |\\n| `provider` | AI provider name | Yes | - |\\n| `config.model` | Specific model name | Yes | - |\\n| `config.api_key` | Authentication API key | Yes | - |\\n| `config.temperature` | Response randomness (0.0-2.0) | No | 0.7 |\\n| `config.max_tokens` | Maximum response tokens | No | Provider default |\\n| `config.timeout` | Request timeout in seconds | No | 30 |\\n| `config.base_url` | Custom API endpoint | No | Provider default |\\n\\n## Secrets Management and Environment Variables\\n\\nProper secrets management is crucial for security. Never hardcode sensitive information like API keys, passwords, or tokens directly in workflow YAML files.\\n\\n### Using Environment Variables\\n\\nThe recommended approach is to use environment variables for all sensitive data:\\n\\n```yaml\\n# \\u2705 RECOMMENDED: Use environment variables\\nmodel_clients:\\n - id: secure_openai\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"${OPENAI_API_KEY}\\\" # References environment variable\\n\\n# HTTP tasks with environment variables\\ntasks:\\n - id: secure_api_call\\n type: http\\n url: \\\"${API_BASE_URL}/customers\\\" # Environment variable for URL\\n headers:\\n Authorization: \\\"Bearer ${API_TOKEN}\\\" # Environment variable for token\\n X-API-Key: \\\"${EXTERNAL_API_KEY}\\\" # Environment variable for API key\\n```\\n\\n### Environment Variable Patterns\\n\\n```yaml\\n# Database connections\\n- id: database_task\\n type: storage\\n config:\\n storage_type: database\\n connection_string: \\\"${DATABASE_URL}\\\" # Full connection string\\n # or individual components:\\n host: \\\"${DB_HOST}\\\"\\n port: \\\"${DB_PORT}\\\"\\n database: \\\"${DB_NAME}\\\"\\n username: \\\"${DB_USER}\\\"\\n password: \\\"${DB_PASSWORD}\\\"\\n\\n# Third-party service credentials\\n- id: external_service\\n type: http\\n url: \\\"${SERVICE_BASE_URL}/api/v1/data\\\"\\n headers:\\n Authorization: \\\"Bearer ${SERVICE_TOKEN}\\\"\\n X-Client-ID: \\\"${SERVICE_CLIENT_ID}\\\"\\n X-Client-Secret: \\\"${SERVICE_CLIENT_SECRET}\\\"\\n\\n# MCP server credentials\\n- id: mcp_task\\n type: mcp\\n mcp_server: \\\"@agentic/mcp-notion-server\\\"\\n environment_vars:\\n NOTION_TOKEN: \\\"${NOTION_API_TOKEN}\\\"\\n WORKSPACE_ID: \\\"${NOTION_WORKSPACE_ID}\\\"\\n```\\n\\n### Security Best Practices\\n\\n#### \\u274c What NOT to Do\\n\\n```yaml\\n# DON\u0027T: Hardcode secrets\\nmodel_clients:\\n - id: insecure_example\\n provider: openai\\n config:\\n api_key: \\\"sk-actual-api-key-here\\\" # SECURITY RISK!\\n\\n# DON\u0027T: Put secrets in comments\\ntasks:\\n - id: bad_example\\n type: http\\n url: \\\"https://api.example.com\\\"\\n # API key is: sk-12345... DON\u0027T DO THIS!\\n headers:\\n Authorization: \\\"Bearer sk-real-token\\\" # SECURITY RISK!\\n```\\n\\n#### \\u2705 What TO Do\\n\\n```yaml\\n# DO: Use environment variables\\nmodel_clients:\\n - id: secure_example\\n provider: openai\\n config:\\n api_key: \\\"${OPENAI_API_KEY}\\\" # Secure approach\\n\\n# DO: Use different variables for different environments\\n- id: environment_aware_task\\n type: http\\n url: \\\"${API_BASE_URL}\\\" # Different URLs per environment\\n headers:\\n Authorization: \\\"Bearer ${API_TOKEN}\\\" # Different tokens per environment\\n X-Environment: \\\"${ENVIRONMENT_NAME}\\\" # Track environment\\n\\n# DO: Use descriptive variable names\\nenvironment_vars:\\n CUSTOMER_API_TOKEN: \\\"${CUSTOMER_SERVICE_TOKEN}\\\"\\n BILLING_API_KEY: \\\"${BILLING_SERVICE_API_KEY}\\\"\\n NOTIFICATION_WEBHOOK_SECRET: \\\"${NOTIFICATION_SECRET}\\\"\\n```\\n\\n### Environment Variable Naming Conventions\\n\\n| Pattern | Example | Use Case |\\n|---------|---------|----------|\\n| `SERVICE_API_KEY` | `OPENAI_API_KEY` | Service API keys |\\n| `SERVICE_BASE_URL` | `CUSTOMER_API_BASE_URL` | Service endpoints |\\n| `DB_*` | `DB_HOST`, `DB_PASSWORD` | Database credentials |\\n| `WEBHOOK_*` | `WEBHOOK_SECRET_TOKEN` | Webhook authentication |\\n| `*_TIMEOUT` | `HTTP_REQUEST_TIMEOUT` | Timeout configurations |\\n\\n### Dynamic Environment Variables\\n\\nUse template resolution to create dynamic environment variable names:\\n\\n```yaml\\n# Different API keys per customer\\n- id: customer_specific_api\\n type: http\\n url: \\\"https://api.partner.com/data\\\"\\n headers:\\n Authorization: \\\"Bearer ${PARTNER_API_KEY_${customer_tier}}\\\" # Resolves to PARTNER_API_KEY_PREMIUM\\n\\n# Environment-specific configurations\\n- id: env_specific_task\\n type: storage\\n config:\\n connection_string: \\\"${DATABASE_URL_${environment}}\\\" # DATABASE_URL_PRODUCTION\\n```\\n\\n### Secrets in Different Task Types\\n\\n```yaml\\n# AI Agent with multiple secure credentials\\n- id: secure_ai_task\\n type: ai_agent\\n config:\\n model_client_id: secure_model\\n system_message: \\\"You are a secure assistant with access to customer data.\\\"\\n user_message: \\\"Process customer ${customer_id}\\\"\\n environment_vars:\\n CUSTOMER_DB_TOKEN: \\\"${CUSTOMER_DATABASE_TOKEN}\\\"\\n AUDIT_LOG_KEY: \\\"${AUDIT_LOGGING_API_KEY}\\\"\\n\\n# Script task with secure file access\\n- id: secure_script_task\\n type: script\\n environment_vars:\\n AWS_ACCESS_KEY_ID: \\\"${AWS_ACCESS_KEY}\\\"\\n AWS_SECRET_ACCESS_KEY: \\\"${AWS_SECRET_KEY}\\\"\\n S3_BUCKET_NAME: \\\"${SECURE_BUCKET_NAME}\\\"\\n script: |\\n import boto3\\n import os\\n \\n # Credentials automatically available from environment\\n s3_client = boto3.client(\u0027s3\u0027)\\n bucket = os.environ[\u0027S3_BUCKET_NAME\u0027]\\n \\n # Process secure files...\\n\\n# Storage task with encrypted connections\\n- id: secure_storage_task\\n type: storage\\n config:\\n storage_type: database\\n connection_string: \\\"${ENCRYPTED_DB_CONNECTION_STRING}\\\"\\n ssl_mode: \\\"require\\\"\\n ssl_cert: \\\"${SSL_CERT_PATH}\\\"\\n ssl_key: \\\"${SSL_KEY_PATH}\\\"\\n```\\n\\n### Environment Setup Examples\\n\\n#### Development Environment (.env file)\\n```bash\\n# Development API keys (non-production)\\nOPENAI_API_KEY=sk-dev-api-key-here\\nCUSTOMER_API_BASE_URL=https://api-dev.example.com\\nDATABASE_URL=postgresql://dev:pass@localhost:5432/workflow_dev\\nENVIRONMENT_NAME=development\\n```\\n\\n#### Production Environment\\n```bash\\n# Production API keys (secure)\\nOPENAI_API_KEY=sk-prod-api-key-here\\nCUSTOMER_API_BASE_URL=https://api.example.com\\nDATABASE_URL=postgresql://prod_user:secure_pass@prod-db:5432/workflow_prod\\nENVIRONMENT_NAME=production\\n```\\n\\n## Workflow Execution and Orchestration\\n\\nUnderstanding how workflows execute and how tasks are orchestrated is essential for building reliable, efficient workflows.\\n\\n### Execution Flow\\n\\nWorkflows follow a directed acyclic graph (DAG) execution model:\\n\\n1. **Initialization**: Workflow starts with inputs and model clients\\n2. **Task Scheduling**: Tasks are scheduled based on dependencies\\n3. **Parallel Execution**: Independent tasks run simultaneously\\n4. **Data Flow**: Outputs flow between tasks via template resolution\\n5. **Completion**: Workflow completes when all tasks finish\\n\\n```yaml\\n# Example execution flow\\ntasks:\\n # Stage 1: Independent tasks (run in parallel)\\n - id: fetch_customer # Runs immediately\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: fetch_products # Runs in parallel with fetch_customer\\n type: http\\n url: \\\"https://api.example.com/products\\\"\\n \\n # Stage 2: Dependent tasks (wait for Stage 1)\\n - id: validate_customer # Waits for fetch_customer\\n type: script\\n depends_on: [fetch_customer]\\n previous_node: fetch_customer\\n script: # validation logic...\\n \\n - id: filter_products # Waits for fetch_products\\n type: script\\n depends_on: [fetch_products]\\n previous_node: fetch_products\\n script: # filtering logic...\\n \\n # Stage 3: Final processing (waits for Stage 2)\\n - id: generate_recommendations # Waits for both validation and filtering\\n type: ai_agent\\n depends_on: [validate_customer, filter_products]\\n previous_node: validate_customer # UI hierarchy under customer validation\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Generate product recommendations for customer.\\\"\\n user_message: |\\n Customer: ${validate_customer}\\n Available products: ${filter_products}\\n```\\n\\n### Task States and Lifecycle\\n\\nEach task goes through these states during execution:\\n\\n| State | Description | Next States |\\n|-------|-------------|-------------|\\n| `PENDING` | Task created, waiting for dependencies | `RUNNING`, `SKIPPED` |\\n| `RUNNING` | Task is actively executing | `COMPLETED`, `FAILED` |\\n| `COMPLETED` | Task finished successfully | - |\\n| `FAILED` | Task encountered an error | `RUNNING` (if retries) |\\n| `SKIPPED` | Task skipped due to conditions | - |\\n| `TIMEOUT` | Task exceeded timeout limit | `RUNNING` (if retries) |\\n\\n### Conditional Execution\\n\\nTasks can be executed conditionally based on routes or conditions:\\n\\n```yaml\\n# Router-based conditional execution\\n- id: data_router\\n type: conditional_router\\n depends_on: [fetch_data]\\n previous_node: fetch_data\\n conditions:\\n - name: \\\"large_dataset\\\"\\n condition: \\\"${fetch_data.record_count} \u003e 10000\\\"\\n route: \\\"batch_processing\\\"\\n - name: \\\"small_dataset\\\"\\n condition: \\\"${fetch_data.record_count} \u003c= 1000\\\"\\n route: \\\"direct_processing\\\"\\n default_route: \\\"standard_processing\\\"\\n\\n# Tasks that execute on specific routes\\n- id: batch_processor\\n type: script\\n depends_on: [data_router]\\n previous_node: data_router\\n execute_on_routes: [\\\"batch_processing\\\"] # Only runs for large datasets\\n script: |\\n # Handle large dataset in batches\\n data = ${fetch_data}\\n process_in_batches(data, batch_size=1000)\\n\\n- id: direct_processor\\n type: script\\n depends_on: [data_router]\\n previous_node: data_router\\n execute_on_routes: [\\\"direct_processing\\\"] # Only runs for small datasets\\n script: |\\n # Process small dataset directly\\n data = ${fetch_data}\\n process_directly(data)\\n```\\n\\n### Error Handling and Retries\\n\\nConfigure retry policies for resilient workflow execution:\\n\\n```yaml\\n# Workflow-level retry policy\\nretry:\\n retryOn: [\\\"TEMPORARY_FAILURE\\\", \\\"TIMEOUT\\\", \\\"NETWORK_ERROR\\\"]\\n maxAttempts: 3\\n initialDelay: \\\"1s\\\"\\n maxDelay: \\\"60s\\\"\\n backoffMultiplier: 2.0\\n\\ntasks:\\n # Task-specific retry policy (overrides workflow-level)\\n - id: external_api_call\\n type: http\\n url: \\\"https://unreliable-api.example.com/data\\\"\\n retry:\\n maxAttempts: 5 # More retries for unreliable API\\n initialDelay: \\\"2s\\\"\\n maxDelay: \\\"120s\\\"\\n backoffMultiplier: 2.0\\n retryOn: [\\\"TIMEOUT\\\", \\\"HTTP_5XX\\\", \\\"NETWORK_ERROR\\\"]\\n \\n # Task with no retries\\n - id: critical_operation\\n type: script\\n retry:\\n maxAttempts: 1 # No retries for critical operations\\n script: |\\n # This operation should not be retried if it fails\\n perform_critical_database_operation()\\n```\\n\\n### Parallel Execution Patterns\\n\\n```yaml\\n# Fan-out pattern: One task triggers multiple parallel tasks\\n- id: data_source\\n type: http\\n url: \\\"https://api.example.com/data\\\"\\n\\n- id: process_customers # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process customer data from data_source\\n\\n- id: process_orders # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process order data from data_source\\n\\n- id: process_products # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process product data from data_source\\n\\n# Fan-in pattern: Multiple tasks feed into one task\\n- id: generate_report # Waits for all parallel tasks\\n type: ai_agent\\n depends_on: [process_customers, process_orders, process_products]\\n previous_node: process_customers # UI hierarchy\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Generate comprehensive report from processed data.\\\"\\n user_message: |\\n Customer data: ${process_customers}\\n Order data: ${process_orders}\\n Product data: ${process_products}\\n```\\n\\n### Resource Management\\n\\nControl resource usage and task distribution:\\n\\n```yaml\\n# Worker group targeting\\n- id: cpu_intensive_task\\n type: script\\n worker_group: \\\"high-cpu-workers\\\" # Target specific worker group\\n script: |\\n # CPU-intensive processing\\n perform_heavy_computation()\\n\\n- id: gpu_ml_task\\n type: ai_agent\\n worker_group: \\\"gpu-workers\\\" # Target GPU workers\\n config:\\n model_client_id: large_model\\n # Large model processing...\\n\\n# Timeout configuration\\n- id: long_running_task\\n type: script\\n timeout_seconds: 3600 # 1 hour timeout\\n script: |\\n # Long-running data processing\\n process_large_dataset()\\n\\n# Workflow-level timeout\\ntimeout_seconds: 7200 # 2 hour workflow timeout\\n```\\n\\n## Task Type Configurations\\n\\nThis section provides the complete YAML configuration for each supported task type.\\n\\n**Supported Task Types:**\\n- `script` - Execute Python code with full workflow context\\n- `http` - Make HTTP requests to external APIs\\n- `ai_agent` - Integrate with AI models for intelligent processing \\n- `loop` - Execute tasks repeatedly (for/while loops)\\n- `conditional_router` - Route execution based on conditions\\n- `mcp` - Execute tools via Model Context Protocol servers\\n- `storage` - Data persistence and retrieval operations\\n- `continuous_conversation` - Interactive AI conversations\\n- `human_input` - Tasks requiring human input or approval\\n- `chatbot` - Session-based chatbot conversations\\n\\n### Script Tasks\\n\\nExecute Python code with full workflow context access.\\n\\n```yaml\\n- id: script_task\\n name: \\\"Python Script Task\\\"\\n type: script\\n description: \\\"Execute Python code with dependencies and environment\\\"\\n \\n # Script configuration - script field is directly under task, NOT under config\\n script: | # REQUIRED: Python code to execute\\n import json\\n import requests\\n \\n # Access workflow inputs\\n input_value = \\\"${workflow_input}\\\"\\n \\n # Access previous task outputs\\n previous_result = ${previous_task}\\n \\n # Your processing logic\\n result = {\\\"processed\\\": True, \\\"value\\\": input_value}\\n \\n # Output results (must be JSON serializable)\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n \\n # Alternative: use \u0027code\u0027 field instead of \u0027script\u0027\\n code: | # Alternative to \u0027script\u0027 field\\n print(\\\"Hello World\\\")\\n \\n # Package management\\n requirements: # Optional: Python packages\\n - requests==2.31.0\\n - pandas==1.5.3\\n - numpy==1.24.3\\n \\n # Environment configuration\\n environment: # Optional: Environment variables\\n API_KEY: \\\"${secrets.api_key}\\\"\\n DEBUG: \\\"true\\\"\\n CUSTOM_VAR: \\\"value\\\"\\n \\n environment_vars: # Alternative to \u0027environment\u0027\\n DATABASE_URL: \\\"${secrets.db_url}\\\"\\n \\n # Secrets configuration\\n secrets_config: # Optional: Secrets access\\n api_key: \\\"secret_name\\\"\\n db_password: \\\"db_secret\\\"\\n```\\n\\n### HTTP Tasks\\n\\nMake HTTP requests to external APIs and services.\\n\\n```yaml\\n- id: http_task\\n name: \\\"HTTP API Call\\\"\\n type: http\\n description: \\\"Call external REST API\\\"\\n \\n # HTTP configuration\\n method: POST # Default: GET\\n url: \\\"https://api.example.com/data\\\" # REQUIRED: Target URL\\n \\n # Headers configuration\\n headers: # Optional: HTTP headers\\n Content-Type: \\\"application/json\\\"\\n Authorization: \\\"Bearer ${secrets.api_token}\\\"\\n X-Custom-Header: \\\"workflow-engine\\\"\\n User-Agent: \\\"WorkflowEngine/1.0\\\"\\n \\n # Request body\\n body: | # Optional: Request body (for POST/PUT)\\n {\\n \\\"customer_id\\\": \\\"${customer_id}\\\",\\n \\\"timestamp\\\": \\\"${current_timestamp}\\\",\\n \\\"data\\\": ${previous_task}\\n }\\n \\n # Response validation\\n expected_status: [200, 201, 202] # Default: [200]\\n expectedStatus: [200, 201] # Alternative field name\\n \\n # Additional HTTP options\\n timeout_seconds: 30 # HTTP request timeout\\n follow_redirects: true # Follow HTTP redirects\\n verify_ssl: true # Verify SSL certificates\\n```\\n\\n### AI Agent Tasks\\n\\nIntegrate with AI models for intelligent processing.\\n\\n```yaml\\n- id: ai_agent_task\\n name: \\\"AI Processing Agent\\\"\\n type: ai_agent\\n description: \\\"Process data using AI model\\\"\\n \\n # AI configuration\\n config:\\n model_client_id: gpt4_model # REQUIRED: Reference to model client\\n \\n system_message: | # Optional: System prompt\\n You are a data analyst. Analyze the provided data and extract insights.\\n Return your analysis in JSON format with the following structure:\\n {\\n \\\"insights\\\": [\\\"insight1\\\", \\\"insight2\\\"],\\n \\\"recommendations\\\": [\\\"rec1\\\", \\\"rec2\\\"],\\n \\\"confidence_score\\\": 0.95\\n }\\n \\n user_message: | # REQUIRED: User prompt with template resolution\\n Please analyze this customer data:\\n ${previous_task}\\n \\n Customer ID: ${customer_id}\\n Date Range: ${date_range}\\n \\n # Model parameters\\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\\n max_tokens: 1500 # Optional: Maximum response tokens\\n top_p: 1.0 # Optional: Nucleus sampling\\n frequency_penalty: 0.0 # Optional: Frequency penalty\\n presence_penalty: 0.0 # Optional: Presence penalty\\n \\n # Tools configuration for function calling\\n tools: # Optional: List of MCP tool names\\n - current_time # Time-related tools\\n - relative_time\\n - days_in_month\\n - weather_api # Weather tools\\n - database_query # Database tools\\n - notification_service # Communication tools\\n \\n # Required for tool resolution\\n tenant_id: \\\"your-tenant-uuid\\\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\\n \\n # Input/output format\\n input_format: \\\"text\\\" # Optional: text|json|structured\\n output_format: \\\"structured\\\" # Optional: text|json|structured\\n\\n### Important: AI Agent Output Access\\n\\nAI agent tasks return structured objects, not direct text responses. Always access the AI response using this pattern:\\n\\n```yaml\\n# In subsequent script tasks\\nscript: |\\n # CORRECT: Access AI response from structured output\\n agent_output = ${ai_agent_task}\\n ai_response = agent_output[\u0027ai_response\u0027].strip()\\n \\n # Additional metadata available\\n tokens_used = agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\\n cost_estimate = agent_output.get(\u0027estimated_cost\u0027, {}).get(\u0027amount\u0027, 0.0)\\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\\n \\n print(f\\\"AI Response: {ai_response}\\\")\\n print(f\\\"Tokens used: {tokens_used}\\\")\\n print(f\\\"Estimated cost: ${cost_estimate:.6f}\\\")\\n```\\n\\n```yaml\\n# In workflow outputs - specify the ai_response field\\noutputs:\\n ai_analysis:\\n type: string\\n source: ai_agent_task.ai_response # Access the ai_response field\\n description: \\\"AI-generated analysis text\\\"\\n \\n total_tokens:\\n type: integer \\n source: ai_agent_task.token_usage.total_tokens\\n description: \\\"Total tokens consumed\\\"\\n```\\n```\\n\\n### Loop Tasks\\n\\nExecute tasks repeatedly with for-loop and while-loop patterns.\\n\\n```yaml\\n# For-Loop (Fixed Iterations)\\n- id: for_loop_task\\n name: \\\"For Loop Processor\\\"\\n type: loop\\n description: \\\"Process data in fixed batches\\\"\\n \\n # Loop configuration\\n loop_type: for # REQUIRED: for|while\\n max_iterations: 10 # REQUIRED for \u0027for\u0027 loops\\n iteration_variable: batch_index # Default: iteration\\n \\n # State variables (persistent across iterations)\\n state_variables:\\n total_processed: 0\\n error_count: 0\\n batch_results: []\\n \\n # Tasks to execute in each iteration\\n loop_tasks: # REQUIRED: Tasks to run in loop\\n - id: process_batch\\n name: \\\"Process Current Batch\\\"\\n type: script\\n script: |\\n batch_idx = ${batch_index} # Access iteration variable\\n total = ${total_processed} # Access state variable\\n \\n # Process current batch\\n result = process_batch_data(batch_idx)\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n \\n - id: update_state\\n name: \\\"Update Loop State\\\"\\n type: script\\n depends_on: [process_batch]\\n script: |\\n # Update state variables\\n new_total = ${total_processed} + ${process_batch.items_processed}\\n \\n result = {\\\"total_processed\\\": new_total}\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n\\n# While-Loop (Conditional)\\n- id: while_loop_task\\n name: \\\"While Loop Quality Processor\\\"\\n type: loop\\n description: \\\"Process until quality threshold met\\\"\\n \\n # Loop configuration\\n loop_type: while # REQUIRED: for|while\\n exit_condition: \\\"${average_quality} \u003e= 85 || ${iteration_count} \u003e= 20\\\" # REQUIRED for \u0027while\u0027\\n max_iterations: 50 # Optional: Safety limit\\n iteration_variable: iteration_count # Default: iteration\\n \\n # State variables\\n state_variables:\\n average_quality: 0\\n items_processed: 0\\n quality_scores: []\\n \\n # Loop tasks\\n loop_tasks:\\n - id: assess_quality\\n type: script\\n script: |\\n current_quality = calculate_quality()\\n print(f\\\"__OUTPUTS__ {json.dumps({\u0027quality\u0027: current_quality})}\\\")\\n```\\n\\n\\n### Conditional Router Tasks\\n\\nRoute execution to different paths based on conditions.\\n\\n```yaml\\n- id: conditional_router_task\\n name: \\\"Data Type Router\\\"\\n type: conditional_router\\n description: \\\"Route processing based on data characteristics\\\"\\n \\n # Router configuration\\n router_type: condition # Default: condition\\n \\n # Routing conditions\\n conditions: # REQUIRED: List of conditions\\n - name: \\\"large_dataset\\\"\\n condition: \\\"${data_size} \u003e 10000\\\"\\n route: \\\"batch_processing\\\"\\n - name: \\\"small_dataset\\\" \\n condition: \\\"${data_size} \u003c= 1000\\\"\\n route: \\\"direct_processing\\\"\\n - name: \\\"medium_dataset\\\"\\n condition: \\\"${data_size} \u003e 1000 \u0026\u0026 ${data_size} \u003c= 10000\\\"\\n route: \\\"standard_processing\\\"\\n \\n default_route: \\\"standard_processing\\\" # REQUIRED: Default route if no conditions match\\n \\n # Alternative: evaluation script for complex routing\\n evaluation_script: | # Optional: Custom routing logic\\n if data_type == \\\"json\\\":\\n return \\\"json_processing\\\"\\n elif data_type == \\\"xml\\\":\\n return \\\"xml_processing\\\"\\n else:\\n return \\\"default_processing\\\"\\n \\n # Route mapping for complex scenarios\\n route_mapping: # Optional: Route name mappings\\n json_processing: \\\"specialized_json\\\"\\n xml_processing: \\\"specialized_xml\\\"\\n\\n# Tasks that execute on specific routes\\n- id: batch_handler\\n name: \\\"Handle Large Dataset\\\"\\n type: script\\n execute_on_routes: [\\\"batch_processing\\\"] # REQUIRED: Routes this task runs on\\n depends_on: [conditional_router_task]\\n script: |\\n print(\\\"Processing large dataset in batches...\\\")\\n\\n- id: direct_handler\\n name: \\\"Handle Small Dataset\\\" \\n type: script\\n execute_on_routes: [\\\"direct_processing\\\"]\\n depends_on: [conditional_router_task]\\n script: |\\n print(\\\"Processing small dataset directly...\\\")\\n```\\n\\n### MCP Tasks\\n\\nExecute tools via Model Context Protocol servers.\\n\\n```yaml\\n- id: mcp_task\\n name: \\\"MCP Tool Execution\\\"\\n type: mcp\\n description: \\\"Execute external tool via MCP server\\\"\\n \\n # MCP server configuration\\n mcp_server: \\\"@agentic/mcp-notion-server\\\" # REQUIRED: MCP server package\\n tool_name: \\\"update_database\\\" # REQUIRED: Tool to execute\\n runtime: \\\"npx\\\" # REQUIRED: npx|uvx\\n \\n # Server startup\\n server_args: [] # Optional: Server startup arguments\\n server_startup_timeout: 30 # Optional: Server startup timeout (seconds)\\n \\n # Tool execution\\n tool_arguments: # REQUIRED: Arguments for the tool\\n database_id: \\\"${secrets.notion_database_id}\\\"\\n properties:\\n title: \\\"Workflow Results\\\"\\n status: \\\"Completed\\\"\\n processed_count: \\\"${previous_task.count}\\\"\\n timestamp: \\\"${current_timestamp}\\\"\\n \\n # Environment configuration\\n environment_vars: # Optional: Environment variables\\n NOTION_TOKEN: \\\"${secrets.notion_token}\\\"\\n DEBUG: \\\"true\\\"\\n \\n # Timeout configuration\\n timeout_seconds: 120 # Optional: Tool execution timeout\\n```\\n\\n### Storage Tasks\\n\\nManage data persistence and retrieval operations.\\n\\n```yaml\\n- id: storage_task\\n name: \\\"Data Storage Operation\\\"\\n type: storage\\n description: \\\"Save/load data to/from storage\\\"\\n \\n # Storage operation\\n operation: save # REQUIRED: save|load|delete|list\\n \\n # Storage configuration\\n storage_type: \\\"file\\\" # REQUIRED: file|database|cache\\n path: \\\"/data/results/${execution_id}.json\\\" # REQUIRED: Storage path\\n \\n # Data configuration\\n data: ${previous_task} # REQUIRED for \u0027save\u0027: Data to store\\n format: \\\"json\\\" # Optional: json|yaml|csv|parquet\\n compression: \\\"gzip\\\" # Optional: gzip|bz2|none\\n \\n # Metadata\\n metadata: # Optional: Additional metadata\\n workflow_id: \\\"${workflow_id}\\\"\\n created_at: \\\"${current_timestamp}\\\"\\n source_task: \\\"previous_task\\\"\\n \\n # File options\\n create_directories: true # Optional: Create parent directories\\n overwrite: true # Optional: Overwrite existing files\\n backup: false # Optional: Create backup before overwrite\\n \\n # Database options (when storage_type: \\\"database\\\")\\n table_name: \\\"workflow_results\\\" # Required for database storage\\n connection_string: \\\"${secrets.db_url}\\\" # Required for database storage\\n \\n # Cache options (when storage_type: \\\"cache\\\")\\n cache_key: \\\"result_${execution_id}\\\" # Required for cache storage\\n ttl: 3600 # Optional: Time to live (seconds)\\n```\\n\\n\\n### Continuous Conversation Tasks\\n\\nCreate interactive AI-powered conversations.\\n\\n```yaml\\n- id: conversation_task\\n name: \\\"AI Conversation Interface\\\"\\n type: continuous_conversation\\n description: \\\"Interactive conversation with AI assistant\\\"\\n \\n # Conversation configuration\\n config:\\n # AI configuration\\n ai_config:\\n model_client_id: support_assistant # REQUIRED: AI model to use\\n system_prompt: | # REQUIRED: System instructions\\n You are a helpful customer support representative.\\n Be friendly, professional, and solution-oriented.\\n \\n Guidelines:\\n - Always be polite and respectful\\n - Ask clarifying questions when needed\\n - Provide clear and actionable solutions\\n \\n temperature: 0.7 # Optional: AI creativity\\n max_tokens: 1000 # Optional: Response length limit\\n \\n # UI configuration\\n ui_config:\\n title: \\\"Customer Support Chat\\\" # Optional: Interface title\\n placeholder: \\\"How can I help you today?\\\" # Optional: Input placeholder\\n theme: \\\"light\\\" # Optional: light|dark\\n show_timestamp: true # Optional: Show message timestamps\\n \\n # Termination conditions\\n termination:\\n keywords: [\\\"goodbye\\\", \\\"bye\\\", \\\"thanks\\\", \\\"done\\\"] # Optional: End conversation keywords\\n max_turns: 50 # Optional: Maximum conversation turns\\n timeout_minutes: 30 # Optional: Conversation timeout\\n inactivity_timeout: 10 # Optional: Inactivity timeout (minutes)\\n \\n # Context configuration\\n context:\\n system_context: \\\"Customer support conversation\\\" # Optional: Context description\\n user_id: \\\"${customer_id}\\\" # Optional: User identifier\\n session_data: ${previous_task} # Optional: Additional context data\\n```\\n\\n### Human Input Tasks\\n\\nTasks requiring human input or approval.\\n\\n```yaml\\n- id: human_input_task\\n name: \\\"Human Input Required\\\"\\n type: human_input\\n description: \\\"Wait for human input or approval\\\"\\n \\n # Human input configuration\\n config:\\n prompt: \\\"Please review and approve the processed data\\\" # REQUIRED: Prompt for human\\n input_type: \\\"approval\\\" # REQUIRED: approval|text|selection\\n \\n # Input options (for selection type)\\n options: [\\\"approve\\\", \\\"reject\\\", \\\"request_changes\\\"] # Optional: For selection input\\n \\n # Timeout configuration\\n timeout_minutes: 60 # Optional: How long to wait for input (minutes)\\n \\n # Conversation settings\\n conversation_mode: false # Optional: Add to conversation history\\n next_task: \\\"process_approval\\\" # Optional: Task to trigger after input\\n end_condition: \\\"bye|exit|quit\\\" # Optional: Keywords to end conversation\\n \\n # Input validation\\n required: true # Optional: Whether input is required\\n \\n # Additional context\\n context_data: ${previous_task} # Optional: Data to show to human\\n```\\n\\n### ChatBot Tasks\\n\\nSession-based chatbot conversations with streaming support.\\n\\n```yaml\\n- id: chatbot_task\\n name: \\\"AI Chatbot\\\"\\n type: chatbot\\n description: \\\"Interactive chatbot with session management\\\"\\n \\n # Chatbot configuration\\n config:\\n # AI model configuration\\n model_client_id: gpt4_model # REQUIRED: Reference to model client\\n \\n system_message: | # REQUIRED: System prompt for chatbot\\n You are a helpful AI assistant. Be friendly, helpful, and concise.\\n Answer questions clearly and ask for clarification when needed.\\n \\n # Session management\\n session_id: \\\"${session_id}\\\" # Optional: Session identifier (auto-generated if not provided)\\n max_history_messages: 20 # Optional: Maximum conversation history to maintain\\n \\n # Model parameters\\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\\n max_tokens: 1000 # Optional: Maximum response tokens\\n stream: true # Optional: Enable streaming responses\\n \\n # Tools configuration for function calling\\n tools: # Optional: List of MCP tool names\\n - current_time # Time-related tools\\n - relative_time\\n - days_in_month\\n - weather_forecast # Weather tools\\n - send_notification # Communication tools\\n \\n # Required for tool resolution\\n tenant_id: \\\"your-tenant-uuid\\\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\\n \\n # UI configuration\\n ui_config:\\n title: \\\"AI Assistant\\\" # Optional: Chat interface title\\n placeholder: \\\"Type your message...\\\" # Optional: Input placeholder\\n theme: \\\"light\\\" # Optional: light|dark\\n show_typing_indicator: true # Optional: Show typing animation\\n \\n # Response configuration\\n response_config:\\n max_retries: 3 # Optional: Retry failed requests\\n timeout_seconds: 30 # Optional: Response timeout\\n \\n # Input message (for single-turn mode)\\n user_message: \\\"${user_input}\\\" # Optional: Direct message input\\n \\n # Context data\\n context_data: ${previous_task} # Optional: Additional context\\n```\\n\\n---\\n\\n## Common Errors and Corrections\\n\\nThis section documents frequent mistakes encountered when creating workflows and their proper corrections.\\n\\n### 1. Missing Required `id` Field\\n\\n**\\u274c Error:**\\n```yaml\\nname: \\\"My Workflow\\\"\\ndescription: \\\"A workflow without id\\\"\\ntasks:\\n - name: \\\"Task 1\\\"\\n type: script\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nid: \\\"my_workflow\\\" # REQUIRED: Unique workflow identifier\\nname: \\\"My Workflow\\\"\\ndescription: \\\"A workflow with proper id\\\"\\ntasks:\\n - id: task_1 # REQUIRED: Unique task identifier\\n name: \\\"Task 1\\\"\\n type: script\\n```\\n\\n### 2. Incorrect Task Structure (Object vs Array)\\n\\n**\\u274c Error:**\\n```yaml\\ntasks:\\n task_1: # Wrong: tasks as object mapping\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n task_2:\\n type: ai_agent\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ntasks:\\n - id: task_1 # Correct: tasks as array with id field\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n - id: task_2\\n type: ai_agent\\n```\\n\\n### 3. Incorrect Inputs Format (Object vs Array)\\n\\n**\\u274c Error:**\\n```yaml\\ninputs:\\n user_name: # Wrong: inputs as object mapping\\n type: string\\n required: true\\n age:\\n type: integer\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ninputs:\\n - name: user_name # Correct: inputs as array with name field\\n type: string\\n required: true\\n - name: age\\n type: integer\\n```\\n\\n### 4. Script Configuration Location Error\\n\\n**\\u274c Error:**\\n```yaml\\n- id: script_task\\n type: script\\n config: # Wrong: script under config\\n script: |\\n print(\\\"Hello\\\")\\n requirements:\\n - requests\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: script_task\\n type: script\\n script: | # Correct: script directly under task\\n print(\\\"Hello\\\")\\n # requirements can be at task level if needed\\n```\\n\\n### 5. Individual Task Inputs (Not Supported)\\n\\n**\\u274c Error:**\\n```yaml\\n- id: task_1\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n inputs: # Wrong: tasks don\u0027t have individual inputs\\n search_query: ${search_query}\\n prev_result: ${previous_task}\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: task_1\\n type: script\\n depends_on: [previous_task] # Use depends_on for data access\\n script: |\\n # Access workflow inputs directly\\n search_query = \\\"${search_query}\\\"\\n # Access previous task outputs via template resolution\\n prev_result = ${previous_task}\\n```\\n\\n### 6. Incorrect Script Output Handling\\n\\n**\\u274c Error:**\\n```yaml\\nscript: |\\n result = process_data()\\n outputs[\u0027result\u0027] = result # Wrong: missing __OUTPUTS__ pattern\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nscript: |\\n result = process_data()\\n \\n # Required: Use outputs dict and __OUTPUTS__ marker\\n outputs = {}\\n outputs[\u0027result\u0027] = result\\n outputs[\u0027status\u0027] = \u0027success\u0027\\n \\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### 7. AI Agent Configuration Errors\\n\\n**\\u274c Error:**\\n```yaml\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: my_model\\n prompt: \\\"Process this data: ${data}\\\" # Wrong: use user_message\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: my_model\\n system_message: \\\"You are a helpful assistant.\\\"\\n user_message: \\\"Process this data: ${data}\\\" # Correct field name\\n```\\n\\n### 8. Missing Dependencies for Template Resolution\\n\\n**\\u274c Error:**\\n```yaml\\n- id: task_2\\n type: script\\n # Missing depends_on\\n script: |\\n data = ${task_1} # Will become UNRESOLVED_task_1\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: task_2\\n type: script\\n depends_on: [task_1] # Required for template access\\n script: |\\n data = ${task_1} # Now resolves correctly\\n```\\n\\n### 9. Incorrect Model Client Configuration\\n\\n**\\u274c Error:**\\n```yaml\\nmodel_clients:\\n my_model: # Wrong: object mapping format\\n type: openai\\n api_key: \\\"sk-...\\\"\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nmodel_clients:\\n - id: my_model # Correct: array format with id field\\n provider: openai\\n config:\\n model: \\\"gpt-4\\\"\\n api_key: \\\"sk-...\\\"\\n```\\n\\n### 10. Empty String Literals Causing Python Syntax Errors\\n\\n**\\u274c Error:**\\n```yaml\\nscript: |\\n def process_data(path=\\\"\\\"): # Wrong: empty string can cause syntax issues\\n current_path = f\\\"{path}.{key}\\\" if path else key\\n \\n # This can lead to: SyntaxError: unterminated string literal\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nscript: |\\n def process_data(path=None): # Correct: use None instead of empty string\\n current_path = f\\\"{path}.{key}\\\" if path else key\\n # Or use path = path or \\\"default_value\\\"\\n```\\n\\n### 11. YAML Indentation and Structure Issues\\n\\n**\\u274c Error:**\\n```yaml\\ntasks:\\n- id: task_1\\n type: script\\nscript: | # Wrong: inconsistent indentation\\n print(\\\"hello\\\")\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ntasks:\\n- id: task_1\\n type: script\\n script: | # Correct: consistent indentation\\n print(\\\"hello\\\")\\n```\\n\\n### Best Practices Summary\\n\\n1. **Always include required `id` fields** for workflows and tasks\\n2. **Use array format** for `tasks`, `inputs`, and `model_clients`\\n3. **Script field goes directly under task**, not under `config`\\n4. **No individual task inputs** - use template resolution instead\\n5. **Use `depends_on`** to access previous task outputs\\n6. **Print `__OUTPUTS__`** in scripts for proper output parsing\\n7. **Use `user_message`** (not `prompt`) for AI agent tasks\\n8. **Avoid empty string literals** - use `None` or meaningful defaults\\n9. **Follow consistent YAML indentation** throughout the file\\n\\n## MCP Tools and Function Calling\\n\\nAI agents and chatbots can use MCP (Model Context Protocol) tools for function calling, enabling them to execute external tools and services.\\n\\n### Tools Configuration\\n\\nBoth `ai_agent` and `chatbot` tasks support tools configuration:\\n\\n```yaml\\n# AI Agent with MCP tools\\n- id: ai_with_tools\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful assistant with access to various tools.\\n Use the available tools when needed to help users.\\n user_message: \\\"What time is it and what\u0027s the weather like?\\\"\\n \\n # Tools configuration\\n tools: # List of MCP tool names\\n - current_time # Time tools\\n - relative_time\\n - days_in_month\\n - weather_forecast # Weather tools\\n - database_query # Database tools\\n - send_notification # Communication tools\\n \\n tenant_id: \\\"550e8400-e29b-41d4-a716-446655440000\\\" # Required for tool resolution\\n\\n# Chatbot with MCP tools\\n- id: chatbot_with_tools\\n type: chatbot\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful chatbot with access to tools.\\n Use tools when users ask for specific information or actions.\\n \\n # Tools configuration (same format as AI agent)\\n tools:\\n - current_time\\n - weather_api\\n - user_preferences\\n \\n tenant_id: \\\"550e8400-e29b-41d4-a716-446655440000\\\" # Required for tool resolution\\n```\\n\\n### How Tools Work\\n\\n1. **Tool Resolution**: The workflow engine resolves tool names to actual MCP server deployments using the `tenant_id`\\n2. **Function Calling**: When the AI model decides to use a tool, it calls the `tool_executor` function\\n3. **Tool Execution**: The engine executes the specified tool via the appropriate MCP server\\n4. **Response Integration**: Tool results are fed back to the AI model for response generation\\n\\n### Tool Configuration Requirements\\n\\n| Field | Required | Description |\\n|-------|----------|-------------|\\n| `tools` | No | List of MCP tool names available to the AI |\\n| `tenant_id` | **Yes** (when using tools) | Tenant UUID for tool resolution |\\n\\n### Example: Complete AI Agent with Tools\\n\\n```yaml\\nid: intelligent_assistant\\nname: \\\"Intelligent Assistant with Tools\\\"\\n\\nmodel_clients:\\n - id: gpt4_model\\n provider: openai\\n config:\\n model: gpt-4o-mini\\n api_key: \\\"${OPENAI_API_KEY}\\\"\\n temperature: 0.3\\n\\n# Tenant ID can be set at workflow level for all tasks\\ntenant_id: \\\"0572fa8d-d7c3-47db-9219-ef40b80d42b7\\\"\\n\\ntasks:\\n - id: assistant_with_tools\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful assistant with access to various tools:\\n - Time-related tools: current_time, relative_time, days_in_month\\n - Weather tools: weather_forecast\\n - Communication: send_notification\\n \\n Use these tools when users ask for:\\n - Current time or date information\\n - Weather forecasts\\n - Sending notifications\\n \\n Always explain what tool you\u0027re using and why.\\n \\n user_message: \\\"${user_query}\\\"\\n \\n # MCP tools configuration\\n tools:\\n - current_time\\n - relative_time\\n - days_in_month\\n - weather_forecast\\n - send_notification\\n \\n # Tool resolution requires tenant_id\\n tenant_id: \\\"0572fa8d-d7c3-47db-9219-ef40b80d42b7\\\"\\n \\n temperature: 0.7\\n max_tokens: 1000\\n\\ninputs:\\n - name: user_query\\n type: string\\n required: true\\n default: \\\"What time is it and can you send a notification to user123?\\\"\\n\\noutputs:\\n ai_response:\\n type: string\\n source: assistant_with_tools.ai_response\\n description: \\\"AI response using tools when needed\\\"\\n```\\n\\n### Tool Execution Flow\\n\\n1. **User Input**: User provides a query that might require tool usage\\n2. **AI Analysis**: AI model analyzes the query and decides if tools are needed\\n3. **Tool Calling**: AI calls `tool_executor(deployment_id, tool_name, arguments)`\\n4. **Tool Resolution**: Engine resolves tool name to MCP server using tenant_id\\n5. **Tool Execution**: MCP server executes the requested tool\\n6. **Result Processing**: Tool results are returned to the AI model\\n7. **Response Generation**: AI generates final response incorporating tool results\\n\\n### Best Practices for Tools\\n\\n1. **Clear System Messages**: Explain available tools and when to use them\\n2. **Tool Categories**: Group related tools (time, weather, database, etc.)\\n3. **Error Handling**: AI models handle tool execution failures gracefully\\n4. **Tool Documentation**: Include tool descriptions in system messages\\n5. **Tenant Configuration**: Ensure tenant_id matches your MCP server setup\\n\\n---\\n\\n*This comprehensive reference covers all supported task types and their YAML configurations. Use this as a quick reference when building workflows with different task types.*\", \"format\": \"markdown\"}, \"status\": \"success\", \"successful_url\": \"http://172.17.0.1:5000/api/docs/yaml-reference\"}\n",
"return_code": 0,
"status": "completed",
"stderr": "",
"successful_url": "http://172.17.0.1:5000/api/docs/yaml-reference",
"task_id": "fetch_documentation"
},
"status": "SUCCESS",
"task_outputs": {
"ai_assistant": {
"agent_type": "assistant",
"execution_details": {
"agent_type": "assistant",
"input_length": 7301,
"model": "moonshotai/kimi-k2",
"output_length": 4529,
"provider": "openrouter",
"system_message": "You are a helpful AI assistant that answers questions about workflow engine documentation.\n\nYour role is to:\n1. Analyze the provided documentation search results\n2. Answer the user\u0027s question based on the relevant documentation content\n3. Provide clear, accurate, and helpful responses\n4. Include specific examples when possible\n5. Reference the documentation sections used in your answer\n\nIf the search results don\u0027t contain enough information to answer the question, \nsay so clearly and suggest what additional information might be needed.\n"
},
"input_format": "text",
"model_client_id": "openrouter_kimi",
"output": "## Creating and Using Loop Nodes in the Workflow Engine\n\nBased on the documentation, here\u0027s a comprehensive guide on how to create and use loop nodes:\n\n### 1. Basic Loop Node Structure\n\nA loop node is defined in YAML with the following essential components:\n\n**Required Configuration:**\n```yaml\n- id: financial_processing_loop\n type: loop\n loop_type: for # REQUIRED: \u0027for\u0027 or \u0027while\u0027\n max_iterations: 3 # REQUIRED for \u0027for\u0027 loops\n iteration_variable: company_index # Optional: defaults to \u0027iteration\u0027\n```\n\n**Supported Loop Types:**\n- `loop_type: for` - Fixed iterations (documentation section: content.chunk_178)\n- `loop_type: while` - Conditional loops\n\n### 2. Complete Loop Node Example\n\nHere\u0027s a comprehensive example showing all the pieces working together:\n\n```yaml\n- id: financial_processing_loop\n type: loop\n loop_type: for\n max_iterations: 3\n iteration_variable: company_index\n depends_on: [list_companies] # External dependency to previous task\n \n # State variables to maintain across iterations\n state_variables:\n processed_companies: []\n total_extractions: 0\n successful_extractions: 0\n \n # Tasks that run in each iteration\n loop_tasks:\n - id: process_current_company\n type: script\n script: |\n # Access array data from external task\n companies_data = inputs.get(\u0027list_companies\u0027, {})\n company_idx = inputs.get(\u0027company_index\u0027, 0)\n \n # Access loop state variables\n processed = loop_state.get(\u0027processed_companies\u0027, [])\n \n # Process current company\n company = companies_data[\u0027companies\u0027][company_idx]\n result = process_company(company)\n \n outputs = {\"company_name\": company[\u0027name\u0027], \"status\": \"success\"}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n*(Documentation reference: content.chunk_62 - Complete Loop Task Example)*\n\n### 3. Data Access Within Loop Tasks\n\nLoop tasks have special data access patterns:\n\n- **Loop state variables** are accessed via injected context files\n- **External task outputs** are automatically flattened and available\n- **Iteration variables** are injected from the loop scope\n\nExample of accessing data within a loop task:\n```yaml\nloop_tasks:\n - id: debug_loop_task\n type: script\n script: |\n # Debug: Print all available data\n print(f\"Available inputs: {list(inputs.keys())}\")\n print(f\"Loop state keys: {list(loop_state.keys())}\")\n \n # Access loop state variables\n processed_items = loop_state.get(\u0027processed_items\u0027, [])\n \n # Access external task data (flattened)\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\n```\n\n*(Documentation reference: content.chunk_68 - Debugging Loop Task Data Access)*\n\n### 4. Context Injection\n\nEach loop task automatically receives context through temporary files:\n\n```python\nimport json\nimport os\n\n# Auto-injected at the start of each loop task script:\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\n inputs = json.load(f) # Contains workflow data + flattened previous outputs\n \nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\n loop_state = json.load(f) # Contains current state variables\n\noutputs = {}\nstate_updates = {}\n```\n\n*(Documentation reference: content.chunk_67 - Loop Task Context Injection and Output Flattening)*\n\n### 5. Practical Loop Patterns\n\n**For loops with arrays from previous tasks:**\n```yaml\n- id: process_items_loop\n type: loop\n loop_type: for\n max_iterations: 5\n state_variables:\n processed_count: 0\n \n loop_tasks:\n - id: process_individual_item\n type: script\n script: |\n items = inputs.get(\u0027fetch_items\u0027, {}).get(\u0027data\u0027, [])\n current_idx = inputs.get(\u0027iteration\u0027, 0)\n \n item = items[current_idx]\n processed_item = {\n \"id\": item[\"id\"],\n \"processed_at\": \"${current_timestamp}\",\n \"status\": \"processed\"\n }\n \n outputs = {\"processed_item\": processed_item}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n*(Documentation reference: content.chunk_81 - Loop through arrays from previous tasks)*\n\n### Key Points to Remember:\n\n1. **State variables** are automatically managed across iterations\n2. **External dependencies** must be specified with `depends_on`\n3. **Iteration variables** are injected into each task automatically\n4. **Outputs are auto-flattened** for downstream tasks\n5. Use **debug printing** to understand available data structure\n__OUTPUTS__ {\"ai_response\": \"## Creating and Using Loop Nodes in the Workflow Engine\\n\\nBased on the documentation, here\u0027s a comprehensive guide on how to create and use loop nodes:\\n\\n### 1. Basic Loop Node Structure\\n\\nA loop node is defined in YAML with the following essential components:\\n\\n**Required Configuration:**\\n```yaml\\n- id: financial_processing_loop\\n type: loop\\n loop_type: for # REQUIRED: \u0027for\u0027 or \u0027while\u0027\\n max_iterations: 3 # REQUIRED for \u0027for\u0027 loops\\n iteration_variable: company_index # Optional: defaults to \u0027iteration\u0027\\n```\\n\\n**Supported Loop Types:**\\n- `loop_type: for` - Fixed iterations (documentation section: content.chunk_178)\\n- `loop_type: while` - Conditional loops\\n\\n### 2. Complete Loop Node Example\\n\\nHere\u0027s a comprehensive example showing all the pieces working together:\\n\\n```yaml\\n- id: financial_processing_loop\\n type: loop\\n loop_type: for\\n max_iterations: 3\\n iteration_variable: company_index\\n depends_on: [list_companies] # External dependency to previous task\\n \\n # State variables to maintain across iterations\\n state_variables:\\n processed_companies: []\\n total_extractions: 0\\n successful_extractions: 0\\n \\n # Tasks that run in each iteration\\n loop_tasks:\\n - id: process_current_company\\n type: script\\n script: |\\n # Access array data from external task\\n companies_data = inputs.get(\u0027list_companies\u0027, {})\\n company_idx = inputs.get(\u0027company_index\u0027, 0)\\n \\n # Access loop state variables\\n processed = loop_state.get(\u0027processed_companies\u0027, [])\\n \\n # Process current company\\n company = companies_data[\u0027companies\u0027][company_idx]\\n result = process_company(company)\\n \\n outputs = {\\\"company_name\\\": company[\u0027name\u0027], \\\"status\\\": \\\"success\\\"}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n*(Documentation reference: content.chunk_62 - Complete Loop Task Example)*\\n\\n### 3. Data Access Within Loop Tasks\\n\\nLoop tasks have special data access patterns:\\n\\n- **Loop state variables** are accessed via injected context files\\n- **External task outputs** are automatically flattened and available\\n- **Iteration variables** are injected from the loop scope\\n\\nExample of accessing data within a loop task:\\n```yaml\\nloop_tasks:\\n - id: debug_loop_task\\n type: script\\n script: |\\n # Debug: Print all available data\\n print(f\\\"Available inputs: {list(inputs.keys())}\\\")\\n print(f\\\"Loop state keys: {list(loop_state.keys())}\\\")\\n \\n # Access loop state variables\\n processed_items = loop_state.get(\u0027processed_items\u0027, [])\\n \\n # Access external task data (flattened)\\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n```\\n\\n*(Documentation reference: content.chunk_68 - Debugging Loop Task Data Access)*\\n\\n### 4. Context Injection\\n\\nEach loop task automatically receives context through temporary files:\\n\\n```python\\nimport json\\nimport os\\n\\n# Auto-injected at the start of each loop task script:\\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\\n inputs = json.load(f) # Contains workflow data + flattened previous outputs\\n \\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\\n loop_state = json.load(f) # Contains current state variables\\n\\noutputs = {}\\nstate_updates = {}\\n```\\n\\n*(Documentation reference: content.chunk_67 - Loop Task Context Injection and Output Flattening)*\\n\\n### 5. Practical Loop Patterns\\n\\n**For loops with arrays from previous tasks:**\\n```yaml\\n- id: process_items_loop\\n type: loop\\n loop_type: for\\n max_iterations: 5\\n state_variables:\\n processed_count: 0\\n \\n loop_tasks:\\n - id: process_individual_item\\n type: script\\n script: |\\n items = inputs.get(\u0027fetch_items\u0027, {}).get(\u0027data\u0027, [])\\n current_idx = inputs.get(\u0027iteration\u0027, 0)\\n \\n item = items[current_idx]\\n processed_item = {\\n \\\"id\\\": item[\\\"id\\\"],\\n \\\"processed_at\\\": \\\"${current_timestamp}\\\",\\n \\\"status\\\": \\\"processed\\\"\\n }\\n \\n outputs = {\\\"processed_item\\\": processed_item}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n*(Documentation reference: content.chunk_81 - Loop through arrays from previous tasks)*\\n\\n### Key Points to Remember:\\n\\n1. **State variables** are automatically managed across iterations\\n2. **External dependencies** must be specified with `depends_on`\\n3. **Iteration variables** are injected into each task automatically\\n4. **Outputs are auto-flattened** for downstream tasks\\n5. Use **debug printing** to understand available data structure\", \"model_client_id\": \"openrouter_kimi\", \"agent_type\": \"assistant\", \"model\": \"moonshotai/kimi-k2\", \"provider\": \"openrouter\", \"status\": \"completed\", \"token_usage\": {\"prompt_tokens\": 2028, \"completion_tokens\": 1002, \"total_tokens\": 3030, \"estimated_cost\": {\"amount\": 0.12096, \"currency\": \"USD\"}, \"model\": \"moonshotai/kimi-k2\", \"provider\": \"openrouter\"}}",
"output_format": "text",
"return_code": 0,
"status": "completed",
"stderr": "",
"task_id": "ai_assistant",
"token_usage": {
"completion_tokens": 1002,
"estimated_cost": {
"amount": 0.12096,
"currency": "USD"
},
"model": "moonshotai/kimi-k2",
"prompt_tokens": 2028,
"provider": "openrouter",
"total_tokens": 3030
}
},
"fetch_documentation": {
"documentation": {
"content": "# YAML Workflow Structure Reference\n\nComplete reference for the YAML workflow definition structure supported by the Assistants Workflow Engine.\n\n## Table of Contents\n\n1. [Complete Schema Overview](#complete-schema-overview)\n2. [Required vs Optional Fields](#required-vs-optional-fields)\n3. [Data Types Reference](#data-types-reference)\n4. [Field Validation Rules](#field-validation-rules)\n5. [Parser Behavior](#parser-behavior)\n6. [Migration Guide](#migration-guide)\n\n## Complete Schema Overview\n\n```yaml\n# === WORKFLOW METADATA (Required) ===\nid: string # REQUIRED: Unique workflow identifier\nname: string # REQUIRED: Human-readable workflow name\n\n# === WORKFLOW METADATA (Optional) ===\nversion: string | integer # Default: 1\nnamespace: string # Default: \"default\"\ndescription: string # Default: \"\"\ntenant_id: string # Default: null\n\n# === EXECUTION CONFIGURATION ===\ntimeout_seconds: integer|string # Default: 3600 (1 hour) - Workflow-level timeout (supports duration strings like \"1h\", \"30m\")\nlabels: object # Default: {}\nmetadata: object # Default: {}\n\n# === DATA FLOW DEFINITIONS ===\ninputs: # Default: []\n - name: string # REQUIRED\n type: string # REQUIRED: string|integer|boolean|object|array\n description: string # Optional\n required: boolean # Default: false\n default: any # Optional default value\n\noutputs: object # Default: {}\n output_name:\n type: string # REQUIRED\n source: string # REQUIRED: task_id or task_id.field\n description: string # Optional\n\nvariables: object # Default: {}\n\n# === MODEL CLIENT CONFIGURATION (Required for AI tasks) ===\nmodel_clients: # REQUIRED: For ai_agent, chatbot, continuous_conversation tasks\n - id: string # REQUIRED: Unique identifier to reference in tasks\n provider: string # REQUIRED: openai|anthropic|openrouter|groq\n config: object # REQUIRED: Provider-specific config\n model: string # REQUIRED: Model name (e.g., gpt-4, claude-3-sonnet-20240229)\n api_key: string # REQUIRED: API key for the provider\n temperature: number # Optional: 0.0-2.0, controls randomness\n max_tokens: integer # Optional: Maximum tokens to generate\n timeout: integer # Optional: Request timeout in seconds\n base_url: string # Optional: Custom API base URL\n\n# Model Client Examples:\n# OpenAI Configuration:\n# model_clients:\n# - id: gpt4_model\n# provider: openai\n# config:\n# model: gpt-4-turbo-preview\n# api_key: \"sk-your-openai-api-key\"\n# temperature: 0.7\n# max_tokens: 2000\n#\n# Anthropic Configuration:\n# model_clients:\n# - id: claude_model\n# provider: anthropic\n# config:\n# model: claude-3-sonnet-20240229\n# api_key: \"sk-ant-your-anthropic-api-key\"\n# temperature: 0.7\n# max_tokens: 1500\n#\n# Multiple Models:\n# model_clients:\n# - id: gpt4_model\n# provider: openai\n# config:\n# model: gpt-4-turbo-preview\n# api_key: \"sk-your-openai-api-key\"\n# - id: claude_model\n# provider: anthropic\n# config:\n# model: claude-3-sonnet-20240229\n# api_key: \"sk-ant-your-anthropic-api-key\"\n\n# === TASK DEFINITIONS (Required) ===\ntasks: # REQUIRED: List of tasks\n - id: string # REQUIRED: Unique task identifier\n name: string # Optional: Human-readable name\n type: string # REQUIRED: Task type\n description: string # Optional\n \n # DEPENDENCY CONFIGURATION\n depends_on: array # Optional: Execution dependencies and template resolution\n previous_node: string # REQUIRED: Task connectivity and UI hierarchy (essential for proper workflow execution)\n \n timeout_seconds: integer|string # Optional: Task-specific timeout (supports duration strings like \"5m\", \"30s\")\n worker_group: string # Optional: Target worker group\n conditions: array # Optional: Execution conditions\n execute_on_routes: array # Optional: Routes for conditional execution\n \n # Input/Output mapping\n input_mapping: object # Optional: Map workflow context to task inputs\n output_mapping: object # Optional: Map task outputs to workflow context\n \n # Retry configuration\n retry: object # Optional: Task-specific retry policy\n maxAttempts: integer # Default: 1\n initialDelay: string|integer # Default: \"1s\"\n maxDelay: string|integer # Default: \"300s\"\n backoffMultiplier: number # Default: 2.0\n retryOn: array # Default: [\"TEMPORARY_FAILURE\"]\n \n # === TASK TYPE SPECIFIC CONFIGURATIONS ===\n # See detailed sections below for each task type\n\n# === WORKFLOW-LEVEL RETRY POLICY ===\nretry: object # Optional: Global retry policy\n retryOn: array # Default: [\"TEMPORARY_FAILURE\"]\n maxAttempts: integer # Default: 1\n initialDelay: string|integer # Default: \"1s\"\n maxDelay: string|integer # Default: \"300s\"\n backoffMultiplier: number # Default: 2.0\n\n# === TRIGGERS ===\ntriggers: # Optional: List of triggers\n - name: string # REQUIRED\n type: string # REQUIRED: webhook|schedule|event\n description: string # Optional\n config: object # REQUIRED: Trigger-specific config\n input_mapping: array # Optional: Map trigger data to workflow inputs\n - webhook_field: string # Source field\n workflow_input: string # Target workflow input\n```\n\n## Required vs Optional Fields\n\n### Required Fields (Workflow Level)\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `id` | string | Unique identifier for the workflow |\n| `tasks` | array | List of task definitions (can be empty) |\n\n### Required Fields (Task Level)\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `id` | string | Unique identifier for the task |\n| `type` | string | Task type (script, http, ai_agent, etc.) |\n| `previous_node` | string | Reference to previous task for UI connectivity (required for proper task linking, except for the first task) |\n\n### Required Fields (Input Definition)\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `name` | string | Input parameter name |\n| `type` | string | Data type (string, integer, boolean, object, array) |\n\n### Required Fields (Output Definition)\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `type` | string | Output data type |\n| `source` | string | Source task ID or task_id.field_name |\n\n## Data Types Reference\n\n### Primitive Types\n\n```yaml\n# String type\ninputs:\n - name: customer_name\n type: string\n default: \"John Doe\"\n description: \"Customer full name\"\n\n# Integer type \ninputs:\n - name: batch_size\n type: integer\n default: 100\n description: \"Number of records per batch\"\n\n# Boolean type\ninputs:\n - name: include_archived\n type: boolean\n default: false\n description: \"Include archived records\"\n```\n\n### Complex Types\n\n```yaml\n# Object type\ninputs:\n - name: config_params\n type: object\n default:\n timeout: 30\n retries: 3\n debug: false\n description: \"Configuration parameters\"\n\n# Array type\ninputs:\n - name: customer_ids\n type: array\n default: [\"cust_001\", \"cust_002\"]\n description: \"List of customer identifiers\"\n```\n\n### Duration Types\n\nDuration values can be specified as strings or integers:\n\n```yaml\n# String format (recommended)\ntimeout_seconds: \"1h\" # 1 hour\ntimeout_seconds: \"30m\" # 30 minutes \ntimeout_seconds: \"45s\" # 45 seconds\ntimeout_seconds: \"2d\" # 2 days\n\n# Integer format (seconds)\ntimeout_seconds: 3600 # 1 hour in seconds\n\n# Supported units\n# s, sec, second, seconds\n# m, min, minute, minutes \n# h, hour, hours\n# d, day, days\n```\n\n### Timeout Configuration and Execution Behavior\n\nThe workflow engine supports configurable timeouts at both workflow and task levels with intelligent timeout-free communication for long-running tasks.\n\n#### Timeout Hierarchy\n\nTimeout values are resolved in this order of precedence:\n1. **Task-level `timeout_seconds`**: Specific timeout for individual tasks\n2. **Workflow-level `timeout_seconds`**: Default for all tasks in the workflow\n3. **System default**: 300 seconds (5 minutes) for tasks, 3600 seconds (1 hour) for workflows\n\n```yaml\n# Workflow-level timeout (applies to all tasks by default)\ntimeout_seconds: \"2h\" # 2 hour workflow timeout\n\ntasks:\n - id: quick_task\n type: script\n # Uses workflow-level timeout (2h)\n script: \"print(\u0027Quick task\u0027)\"\n \n - id: custom_timeout_task\n type: script\n timeout_seconds: \"30m\" # Override with 30 minute timeout\n script: \"print(\u0027Custom timeout task\u0027)\"\n \n - id: long_running_task\n type: script\n timeout_seconds: 600 # 10 minutes (integer format)\n script: |\n import time\n time.sleep(300) # 5 minute processing\n print(\"Long task completed\")\n```\n\n#### Timeout-Free Communication\n\nThe workflow engine automatically uses **timeout-free communication** for long-running tasks to prevent timeout-related failures:\n\n- **Loop tasks**: Always use timeout-free communication (event-driven execution)\n- **Script tasks**: Use timeout-free communication when `timeout_seconds \u003e 180` (host timeout threshold)\n- **Other task types**: Use standard timeout-based communication\n\n```yaml\ntasks:\n - id: short_script\n type: script\n timeout_seconds: 60 # Uses standard timeout-based communication\n script: \"print(\u0027Short script\u0027)\"\n \n - id: long_script\n type: script\n timeout_seconds: 600 # Uses timeout-free communication (600 \u003e 180)\n script: |\n import time\n print(\"\ud83d\ude80 This uses timeout-free communication\")\n time.sleep(300) # Won\u0027t cause timeout issues\n print(\"Long script completed\")\n \n - id: data_processing_loop\n type: loop\n timeout_seconds: \"1h\" # Always uses timeout-free communication\n loop_type: for\n max_iterations: 10\n # Loop tasks automatically use timeout-free communication\n```\n\n#### Loop Task Timeout Inheritance\n\nLoop tasks can pass timeout configuration to their inner script tasks:\n\n```yaml\n- id: batch_processing_loop\n type: loop\n timeout_seconds: \"30m\" # Loop-level timeout\n loop_type: for\n max_iterations: 5\n loop_tasks:\n - id: process_batch\n type: script\n # Inherits 30-minute timeout from parent loop\n script: |\n print(\"Processing batch with inherited timeout\")\n # This script will use the 30-minute timeout from the loop\n```\n\n#### Best Practices for Timeout Configuration\n\n1. **Use duration strings** for readability: `\"5m\"` vs `300`\n2. **Set appropriate timeouts** based on expected execution time plus buffer\n3. **Long-running tasks**: Set `timeout_seconds \u003e 180` to enable timeout-free communication\n4. **Loop tasks**: Always get timeout-free communication automatically\n5. **Critical tasks**: Use shorter timeouts for fail-fast behavior\n6. **External API calls**: Use longer timeouts to account for network latency\n\n```yaml\n# Example: Well-configured timeout workflow\ntimeout_seconds: \"1h\" # Reasonable workflow timeout\n\ntasks:\n - id: api_call\n type: http\n timeout_seconds: \"30s\" # Quick timeout for API calls\n url: \"https://api.example.com/data\"\n \n - id: data_processing\n type: script\n timeout_seconds: \"10m\" # Longer timeout for processing (uses timeout-free)\n script: |\n # Process large dataset\n process_data()\n \n - id: batch_processing\n type: loop\n timeout_seconds: \"45m\" # Generous timeout for loop (timeout-free)\n loop_type: for\n max_iterations: 100\n # Inner tasks inherit the 45-minute timeout\n```\n\n## Field Validation Rules\n\n### ID Fields\n\n```yaml\n# Valid IDs\nid: \"customer_processor\" # \u2705 Valid\nid: \"process-data\" # \u2705 Valid\nid: \"task_001\" # \u2705 Valid\n\n# Invalid IDs\nid: \"process data\" # \u274c Spaces not allowed\nid: \"123_start\" # \u274c Cannot start with number\nid: \"\" # \u274c Cannot be empty\n```\n\n### Timeout Values\n\n```yaml\n# Valid timeouts\ntimeout_seconds: 300 # \u2705 Positive integer\ntimeout_seconds: \"5m\" # \u2705 Duration string\ntimeout_seconds: \"1h30m\" # \u2705 Complex duration\n\n# Invalid timeouts\ntimeout_seconds: 0 # \u274c Must be positive\ntimeout_seconds: -60 # \u274c Cannot be negative\ntimeout_seconds: \"invalid\" # \u274c Invalid duration format\n```\n\n### Dependency Types and Validation\n\n#### `depends_on` vs `previous_node`\n\n**`depends_on` - Execution Dependencies:**\n- Controls actual task execution order\n- Enables template resolution and data access\n- Required for accessing outputs from other tasks\n- Validates that referenced tasks exist\n- Multiple tasks can be specified\n\n**`previous_node` - Task Connectivity and UI Hierarchy:**\n- **REQUIRED for proper task connectivity in UI**\n- Essential for workflow visualization and task linking\n- Critical for execution flow representation\n- Single task reference only\n- **Without this field, tasks won\u0027t be properly connected in the UI**\n\n```yaml\ntasks:\n - id: fetch_data\n type: http\n url: \"https://api.example.com/data\"\n \n - id: validate_data\n type: script\n depends_on: [fetch_data] # \u2705 EXECUTION: wait for fetch_data to complete\n previous_node: fetch_data # \u2705 UI: show as child of fetch_data in hierarchy\n script: |\n data = ${fetch_data} # \u2705 Can access because of depends_on\n if not validate(data):\n raise ValueError(\"Invalid data\")\n \n # Required: Use outputs dict and __OUTPUTS__ marker for proper parsing\n outputs[\u0027status\u0027] = \u0027valid\u0027\n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n \n - id: process_data\n type: script\n depends_on: [validate_data] # \u2705 EXECUTION: wait for validation\n previous_node: validate_data # \u2705 UI: show as child of validate_data\n script: |\n validation_result = ${validate_data} # \u2705 Accessible via depends_on\n # Process data...\n\n - id: separate_task\n type: script\n previous_node: process_data # \u2705 UI: appears under process_data in hierarchy\n # NO depends_on means this can run immediately (parallel to other tasks)\n script: |\n # This task runs independently but appears under process_data in UI\n # Cannot access ${process_data} because no depends_on relationship\n print(\"Independent task\")\n\n### Complete Example with Proper Hierarchy\n\n```yaml\ntasks:\n - id: fetch_customer_data\n name: \"Fetch Customer Data\"\n type: http\n url: \"https://api.example.com/customers/${customer_id}\"\n \n - id: validate_customer\n name: \"Validate Customer Data\"\n type: script\n depends_on: [fetch_customer_data] # \u2705 EXECUTION: Wait for data fetch\n previous_node: fetch_customer_data # \u2705 UI: Show under fetch_customer_data\n script: |\n customer = ${fetch_customer_data} # \u2705 Can access because of depends_on\n if not customer.get(\u0027email\u0027):\n raise ValueError(\"Invalid customer data\")\n print(\"__OUTPUTS__ {\u0027status\u0027: \u0027valid\u0027}\")\n \n - id: process_customer\n name: \"Process Customer\"\n type: ai_agent\n depends_on: [validate_customer] # \u2705 EXECUTION: Wait for validation\n previous_node: validate_customer # \u2705 UI: Show under validate_customer\n config:\n model_client_id: gpt4_model\n user_message: \"Process customer: ${fetch_customer_data}\"\n \n - id: send_notification\n name: \"Send Notification\"\n type: script\n depends_on: [process_customer] # \u2705 EXECUTION: Wait for processing\n previous_node: process_customer # \u2705 UI: Show under process_customer\n script: |\n result = ${process_customer}\n print(f\"Notification sent for: ${fetch_customer_data.name}\")\n \n - id: parallel_audit_log\n name: \"Audit Log (Parallel)\"\n type: script\n depends_on: [fetch_customer_data] # \u2705 EXECUTION: Only needs initial data\n previous_node: fetch_customer_data # \u2705 UI: Show under fetch_customer_data\n script: |\n # This runs in parallel with validation/processing\n customer = ${fetch_customer_data}\n print(f\"Audit: Customer {customer.id} accessed at ${current_timestamp}\")\n```\n\n # Invalid examples\n - id: invalid_task\n type: script\n depends_on: [nonexistent_task] # \u274c VALIDATION ERROR: task doesn\u0027t exist\n previous_node: another_missing # \u26a0\ufe0f UI WARNING: task doesn\u0027t exist (but not fatal)\n script: \"print(\u0027Invalid\u0027)\"\n```\n\n#### Key Differences:\n\n| Aspect | `depends_on` | `previous_node` |\n|--------|--------------|-----------------|\n| **Purpose** | Execution control | Task connectivity \u0026 UI organization |\n| **Impact** | Blocks task until dependencies complete | **REQUIRED for UI task linking** |\n| **Template Access** | \u2705 Enables `${task_id}` resolution | \u274c No template access |\n| **Validation** | \u274c Parser error if task missing | \u26a0\ufe0f Warning if task missing |\n| **Multiple References** | \u2705 Array of task IDs | \u274c Single task ID only |\n| **Execution Order** | \u2705 Enforced | \u274c No execution impact |\n| **UI Connectivity** | \u274c No UI impact | \u2705 **ESSENTIAL for proper task connection** |\n\n### Circular Dependency Detection\n\n```yaml\n# This will be rejected by the parser\ntasks:\n - id: task_a\n depends_on: [task_b] # \u274c Circular dependency\n \n - id: task_b \n depends_on: [task_a] # \u274c Circular dependency\n```\n\n## Parser Behavior\n\n### Field Name Compatibility\n\nThe parser supports both camelCase and snake_case for field names:\n\n```yaml\ntasks:\n - id: flexible_task\n # Both formats are accepted\n dependsOn: [other_task] # camelCase\n depends_on: [other_task] # snake_case\n \n inputMapping: # camelCase\n source_field: \"target_field\"\n input_mapping: # snake_case \n source_field: \"target_field\"\n \n outputMapping: {} # camelCase\n output_mapping: {} # snake_case\n \n workerGroup: \"gpu-workers\" # camelCase\n worker_group: \"gpu-workers\" # snake_case\n```\n\n### Default Value Assignment\n\n```yaml\n# Fields with defaults are automatically populated\nid: my_workflow\n# name defaults to id value: \"my_workflow\"\n# version defaults to: 1\n# namespace defaults to: \"default\" \n# timeout_seconds defaults to: 3600\n# description defaults to: \"\"\n\ntasks:\n - id: simple_task\n type: script\n # name defaults to id: \"simple_task\"\n # timeout_seconds defaults to: 300\n # retry.maxAttempts defaults to: 1\n```\n\n### Template Resolution Context\n\nThe parser provides these built-in variables for template resolution:\n\n```yaml\ntasks:\n - id: context_aware_task\n type: script\n script: |\n # Workflow context\n workflow_id = \"${workflow_id}\" # Current workflow ID\n execution_id = \"${execution_id}\" # Unique execution ID\n namespace = \"${namespace}\" # Workflow namespace\n \n # Timing context\n current_timestamp = \"${current_timestamp}\" # ISO timestamp\n started_at = \"${started_at}\" # Workflow start time\n \n # Input context\n user_input = \"${input_parameter}\" # Workflow inputs\n \n # Task context\n previous_result = ${previous_task} # Previous task output\n specific_field = ${task_id.field_name} # Specific output field\n \n # Route context (for conditional workflows)\n route_info = ${_route_context} # Current route information\n```\n\n### Error Handling During Parsing\n\n```yaml\n# The parser will validate and report these errors:\n\n# 1. Missing required fields\n# ERROR: Missing required field: \u0027id\u0027\n\n# 2. Invalid YAML syntax \n# ERROR: Invalid YAML: mapping values are not allowed here\n\n# 3. Circular dependencies\n# ERROR: Circular dependency detected involving task \u0027task_a\u0027\n\n# 4. Invalid task references\n# ERROR: Task \u0027task_b\u0027 depends on non-existent task \u0027invalid_task\u0027\n\n# 5. Invalid duration formats\n# ERROR: Invalid duration: \u0027not_a_duration\u0027\n\n# 6. Invalid field values\n# ERROR: Timeout must be positive, got -10\n```\n\n## Migration Guide\n\n### From Version 1.0 to 2.0\n\n```yaml\n# OLD FORMAT (v1.0)\nworkflow:\n id: my_workflow\n tasks:\n - task_id: process_data\n task_type: python_script\n \n# NEW FORMAT (v2.0) \nid: my_workflow\ntasks:\n - id: process_data\n type: script\n language: python\n```\n\n### Deprecated Fields\n\n| Old Field | New Field | Status |\n|-----------|-----------|--------|\n| `task_type` | `type` | Deprecated in v2.0 |\n| `task_id` | `id` | Deprecated in v2.0 |\n| `python_script` | `script` (type) | Deprecated in v2.0 |\n| `plugin` | `mcp` (type) | Deprecated in v2.1 |\n\n### Backward Compatibility\n\nThe parser maintains backward compatibility for deprecated fields but will show warnings:\n\n```yaml\n# This will work but show a warning\ntasks:\n - task_id: old_style_task # \u26a0\ufe0f Warning: Use \u0027id\u0027 instead\n task_type: python_script # \u26a0\ufe0f Warning: Use \u0027type: script\u0027 instead\n```\n\n## Advanced Schema Features\n\n### Conditional Field Requirements\n\nSome fields become required based on task type:\n\n```yaml\n# HTTP tasks require URL\n- id: api_call\n type: http\n url: \"https://api.example.com\" # \u2705 Required for HTTP tasks\n \n# AI agent tasks require model_client_id\n- id: ai_task\n type: ai_agent\n config:\n model_client_id: gpt4 # \u2705 Required for AI agent tasks\n```\n\n### Dynamic Field Validation\n\nField validation can depend on other field values:\n\n```yaml\n# Loop tasks require loop-specific fields\n- id: data_loop\n type: loop\n loop_type: \"for\" # Determines required fields\n max_iterations: 10 # \u2705 Required for \u0027for\u0027 loops\n # exit_condition: not needed for \u0027for\u0027 loops\n \n- id: condition_loop \n type: loop\n loop_type: \"while\" # Determines required fields\n exit_condition: \"${count} \u003e= 10\" # \u2705 Required for \u0027while\u0027 loops\n # max_iterations: optional for \u0027while\u0027 loops\n```\n\n## Template Resolution and Data Flow\n\nTemplate resolution is a core feature that allows tasks to access data from previous tasks, workflow inputs, and built-in variables using the `${variable_name}` syntax. This is essential for creating dynamic workflows where tasks pass data to each other.\n\n### Basic Template Syntax\n\n```yaml\n# Access workflow inputs\nscript: |\n customer_name = \"${customer_name}\" # From workflow inputs\n \n# Access previous task outputs\nscript: |\n previous_result = ${data_fetch_task} # Entire output from data_fetch_task\n specific_field = ${data_fetch_task.customer_id} # Specific field\n \n# Access built-in variables\nscript: |\n workflow_id = \"${workflow_id}\"\n execution_id = \"${execution_id}\"\n timestamp = \"${current_timestamp}\"\n```\n\n### Available Built-in Variables\n\n| Variable | Description | Example Value |\n|----------|-------------|---------------|\n| `${workflow_id}` | Current workflow identifier | `customer_processor` |\n| `${execution_id}` | Unique execution identifier | `exec_12345_abcde` |\n| `${namespace}` | Workflow namespace | `production` |\n| `${current_timestamp}` | ISO timestamp | `2024-01-15T10:30:00Z` |\n| `${started_at}` | Workflow start time | `2024-01-15T10:29:45Z` |\n\n### Task Output Resolution\n\nTasks can access outputs from other tasks they depend on:\n\n```yaml\ntasks:\n - id: fetch_customer\n type: http\n url: \"https://api.example.com/customers/${customer_id}\"\n \n - id: process_customer\n type: script\n depends_on: [fetch_customer] # Required for template access\n previous_node: fetch_customer\n script: |\n # Access entire response\n customer_data = ${fetch_customer}\n \n # Access specific fields\n customer_name = \"${fetch_customer.name}\"\n customer_email = \"${fetch_customer.email}\"\n \n # Process the data\n result = {\n \"processed_name\": customer_name.upper(),\n \"domain\": customer_email.split(\"@\")[1]\n }\n print(f\"__OUTPUTS__ {json.dumps(result)}\")\n```\n\n### AI Agent Output Resolution\n\nAI agent tasks return structured output that requires specific access patterns:\n\n```yaml\n# AI Agent task\n- id: ai_analysis_task\n type: ai_agent\n config:\n model_client_id: gpt4_model\n system_message: \"Analyze the customer data and provide insights.\"\n user_message: \"Customer data: ${customer_data}\"\n\n# Accessing AI agent output in subsequent tasks\n- id: process_ai_response\n type: script\n depends_on: [ai_analysis_task]\n previous_node: ai_analysis_task\n script: |\n # CORRECT: AI agents return structured output\n agent_output = ${ai_analysis_task}\n ai_response = agent_output[\u0027ai_response\u0027].strip()\n \n # Additional metadata available\n token_usage = agent_output.get(\u0027token_usage\u0027, {})\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\n \n # Process the AI response\n processed_analysis = {\n \"analysis\": ai_response,\n \"tokens_used\": token_usage.get(\u0027total_tokens\u0027, 0),\n \"model\": model_used,\n \"processed_at\": \"${current_timestamp}\"\n }\n \n print(f\"__OUTPUTS__ {json.dumps(processed_analysis)}\")\n\n# INCORRECT: Direct access won\u0027t work\n- id: incorrect_ai_access\n type: script\n depends_on: [ai_analysis_task]\n script: |\n # \u274c This will not work correctly\n ai_response = ${ai_analysis_task} # Returns full agent object\n # ai_response is now a dict, not the actual response text\n```\n\n### AI Agent Response Structure\n\nAI agent tasks return a structured object with the following format:\n\n```json\n{\n \"ai_response\": \"The actual AI-generated text response\",\n \"model\": \"gpt-4-turbo-preview\",\n \"token_usage\": {\n \"prompt_tokens\": 150,\n \"completion_tokens\": 75,\n \"total_tokens\": 225\n },\n \"estimated_cost\": {\n \"amount\": 0.00337,\n \"currency\": \"USD\"\n },\n \"provider\": \"openai\",\n \"timestamp\": \"2024-01-15T10:30:00Z\"\n}\n```\n\n### Loop Task Data Access and Dependencies\n\nLoop tasks have a special execution context that differs from regular workflow tasks. Understanding how to access data within loops is crucial for building functional workflows.\n\n#### Loop Task Internal Data Access\n\nWithin loop tasks, data access works through injected context files rather than template resolution:\n\n```yaml\n# Loop task with state variables and internal task dependencies\n- id: data_processing_loop\n type: loop\n loop_type: for\n max_iterations: 3\n state_variables:\n processed_items: []\n total_processed: 0\n error_count: 0\n loop_tasks:\n # First task in loop\n - id: fetch_data\n type: script\n script: |\n # Access loop state variables through injected context\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\n processed_items = loop_state.get(\u0027processed_items\u0027, [])\n \n # Access workflow inputs and external task outputs\n external_data = inputs.get(\u0027previous_workflow_task\u0027, {})\n iteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable\n \n # Process data\n batch_data = fetch_batch_data(iteration_num)\n \n outputs = {\"batch_data\": batch_data, \"batch_size\": len(batch_data)}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n \n # Second task depends on first task in same loop\n - id: process_data\n type: script\n depends_on: [fetch_data] # \u2705 CORRECT: Dependencies within loop tasks work normally\n script: |\n # \u2705 CORRECT: Access previous loop task through inputs context\n fetch_result = inputs.get(\u0027fetch_data\u0027, {})\n batch_data = fetch_result.get(\u0027batch_data\u0027, [])\n \n # \u274c INCORRECT: This won\u0027t work in loop context\n # batch_data = ${fetch_data.batch_data} # Template resolution fails in loops\n \n # Access and update state variables\n current_processed = loop_state.get(\u0027processed_items\u0027, [])\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\n \n # Process the batch\n processed_batch = process_batch(batch_data)\n current_processed.extend(processed_batch)\n \n # Update state variables using state_updates\n state_updates[\u0027processed_items\u0027] = current_processed\n state_updates[\u0027total_processed\u0027] = current_total + len(processed_batch)\n \n outputs = {\"processed_count\": len(processed_batch)}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n print(f\"__STATE_UPDATES__ {json.dumps(state_updates)}\") # \u2705 REQUIRED for state updates\n```\n\n#### Key Differences in Loop Task Data Access\n\n| Access Pattern | Regular Tasks | Loop Tasks |\n|---------------|---------------|------------|\n| **Previous Task Output** | `${previous_task}` | `inputs.get(\u0027field_name\u0027, default)` (flattened) |\n| **State Variables** | N/A | `loop_state.get(\u0027variable_name\u0027, default)` |\n| **State Updates** | N/A | `state_updates[\u0027variable_name\u0027] = value` |\n| **Workflow Inputs** | `\"${input_name}\"` | `inputs.get(\u0027input_name\u0027, \u0027default\u0027)` |\n| **Dependencies** | `depends_on: [task_id]` | Same: `depends_on: [task_id]` |\n\n#### Critical: Loop Task Output Flattening\n\n**Important:** The loop executor flattens task outputs into individual fields in the `inputs` dictionary, rather than storing them as nested objects under the task name.\n\n```yaml\n# First loop task outputs:\noutputs = {\"company_name\": \"ACME Corp\", \"company_id\": \"123\", \"status\": \"success\"}\nprint(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n\n# Second loop task accessing the data:\nscript: |\n # \u274c INCORRECT: This won\u0027t work - no nested object\n # company_data = inputs.get(\u0027first_task\u0027, {})\n # company_name = company_data.get(\u0027company_name\u0027)\n \n # \u2705 CORRECT: Access flattened fields directly\n company_name = inputs.get(\u0027company_name\u0027, \u0027\u0027)\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\n \n print(f\"Company: {company_name}, ID: {company_id}, Status: {task_status}\")\n```\n\n#### Complete Loop Task Example\n\n```yaml\n- id: financial_processing_loop\n type: loop\n loop_type: for\n max_iterations: 3\n iteration_variable: company_index\n depends_on: [list_companies] # External dependency\n \n state_variables:\n processed_companies: []\n total_extractions: 0\n successful_extractions: 0\n \n loop_tasks:\n - id: process_current_company\n type: script\n script: |\n # Access external workflow task data\n companies_data = inputs.get(\u0027list_companies\u0027, {})\n company_idx = inputs.get(\u0027company_index\u0027, 0)\n \n # Access state variables\n processed = loop_state.get(\u0027processed_companies\u0027, [])\n \n # Process company\n company = companies_data[\u0027companies\u0027][company_idx]\n result = process_company(company)\n \n outputs = {\"company_name\": company[\u0027name\u0027], \"status\": \"success\"}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n \n - id: extract_financial_data\n type: script\n depends_on: [process_current_company] # Depend on previous loop task\n requirements:\n - requests\u003e=2.28.0\n - some-ai-library\u003e=1.0.0\n script: |\n # \u2705 CORRECT: Access flattened outputs from previous loop task\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\n \n # \u274c INCORRECT: Loop executor flattens outputs, no nested access\n # company_data = inputs.get(\u0027process_current_company\u0027, {})\n # company_name = company_data.get(\u0027company_name\u0027, \u0027Unknown\u0027)\n \n # \u274c INCORRECT: Template resolution doesn\u0027t work in loops\n # company_data = ${process_current_company}\n \n # Access external data\n companies_list = inputs.get(\u0027list_companies\u0027, {})\n \n # Perform AI extraction\n extraction_result = extract_with_ai({\"company_name\": company_name, \"company_id\": company_id})\n \n # Update state variables\n current_processed = loop_state.get(\u0027processed_companies\u0027, [])\n current_total = loop_state.get(\u0027total_extractions\u0027, 0)\n current_successful = loop_state.get(\u0027successful_extractions\u0027, 0)\n \n current_processed.append({\n \"company_name\": company_name,\n \"extraction_result\": extraction_result\n })\n \n # State updates are required for persistence\n state_updates[\u0027processed_companies\u0027] = current_processed\n state_updates[\u0027total_extractions\u0027] = current_total + 1\n state_updates[\u0027successful_extractions\u0027] = current_successful + (1 if extraction_result else 0)\n \n outputs = {\"extracted\": bool(extraction_result)}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n print(f\"__STATE_UPDATES__ {json.dumps(state_updates)}\") # \u2705 REQUIRED\n\n# Accessing loop task outputs in subsequent workflow tasks\n- id: process_loop_results\n type: script\n depends_on: [financial_processing_loop]\n previous_node: financial_processing_loop\n script: |\n # \u2705 CORRECT: Access final state of loop\n loop_result = ${financial_processing_loop}\n loop_final_state = loop_result.get(\u0027final_state\u0027, {})\n \n # Access specific state variables from final state\n processed_companies = loop_final_state.get(\u0027processed_companies\u0027, [])\n total_processed = loop_final_state.get(\u0027total_extractions\u0027, 0)\n successful_count = loop_final_state.get(\u0027successful_extractions\u0027, 0)\n \n # Access loop execution metadata\n iterations_completed = loop_result.get(\u0027iterations_completed\u0027, 0)\n \n print(f\"Loop completed {iterations_completed} iterations\")\n print(f\"Processed {len(processed_companies)} companies\")\n print(f\"Success rate: {successful_count}/{total_processed}\")\n \n outputs = {\n \"companies_processed\": len(processed_companies),\n \"success_rate\": f\"{successful_count}/{total_processed}\",\n \"iterations\": iterations_completed\n }\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n#### Loop Task Context Injection and Output Flattening\n\nThe loop executor injects context through temporary files and flattens task outputs for easier access:\n\n```python\n# Auto-injected at the start of each loop task script:\nimport json\nimport os\n\n# Load context from temporary files (auto-generated)\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\n inputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\n\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\n loop_state = json.load(f) # Contains current state variables\n\noutputs = {}\nstate_updates = {}\n\n# Your script code follows...\n```\n\n#### Debugging Loop Task Data Access\n\nWhen debugging loop tasks, always print available keys to understand what data is accessible:\n\n```yaml\n- id: debug_loop_task\n type: script\n depends_on: [previous_loop_task]\n script: |\n # Debug: Print all available data\n print(f\"\ud83d\udd0d Available inputs: {list(inputs.keys())}\")\n print(f\"\ud83d\udd0d Loop state keys: {list(loop_state.keys())}\")\n \n # Debug: Print specific values\n for key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\n value = inputs.get(key, \u0027NOT_FOUND\u0027)\n print(f\"\ud83d\udd0d {key}: {value}\")\n \n # Now access the flattened data\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\n \n # Process your data...\n outputs = {\"processed\": f\"Company {company_name}\"}\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n#### Real Example: Task Output Flattening\n\n```yaml\n# Task 1 outputs this structure:\n__OUTPUTS__ {\"company_name\": \"LTTS\", \"company_id\": \"123\", \"fy_folders\": [{\"id\": \"456\", \"name\": \"FY24\"}], \"status\": \"success\"}\n\n# Task 2 receives flattened inputs:\ninputs = {\n \"folder_id\": \"...\", # Workflow input\n \"gemini_api_key\": \"...\", # Workflow input \n \"list_company_folders\": {...}, # External task output\n \"company_index\": 0, # Loop iteration variable\n \"company_name\": \"LTTS\", # \u2705 Flattened from Task 1\n \"company_id\": \"123\", # \u2705 Flattened from Task 1\n \"fy_folders\": [{\"id\": \"456\", \"name\": \"FY24\"}], # \u2705 Flattened from Task 1\n \"status\": \"success\", # \u2705 Flattened from Task 1\n \"total_extractions\": 0, # Loop state variable\n \"processed_companies\": [] # Loop state variable\n}\n```\n\n#### Loop Task Special Output Fields\n\n| Field | Description | Example Access |\n|-------|-------------|----------------|\n| `final_state` | Final values of all state variables | `${loop_task.final_state}` |\n| `final_state[\"variable_name\"]` | Specific state variable from final state | `${loop_task.final_state[\"total_processed\"]}` |\n| `iterations_completed` | Number of iterations executed | `${loop_task.iterations_completed}` |\n\n#### Safe Template Resolution Pattern for Loop States\n\nWhen accessing loop final state fields, use this safe parsing pattern:\n\n```yaml\nscript: |\n try:\n # Template resolution with safety checks\n final_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\n processed_count_str = \u0027\u0027\u0027${loop_task.final_state[\"processed_count\"]}\u0027\u0027\u0027\n items_str = \u0027\u0027\u0027${loop_task.final_state[\"processed_items\"]}\u0027\u0027\u0027\n iterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027\n \n # Safe parsing with fallbacks\n final_state = json.loads(final_state_str) if not final_state_str.startswith(\u0027UNRESOLVED_\u0027) else {}\n processed_count = int(processed_count_str) if processed_count_str.isdigit() else 0\n processed_items = json.loads(items_str) if not items_str.startswith(\u0027UNRESOLVED_\u0027) else []\n iterations = int(iterations_str) if iterations_str.isdigit() else 0\n \n print(f\"Successfully processed {processed_count} items in {iterations} iterations\")\n \n except Exception as e:\n print(f\"Error accessing loop results: {str(e)}\")\n # Handle error case\n```\n```\n\n### Real-World AI Agent Example\n\nBased on actual WhatsApp workflow patterns:\n\n```yaml\n# AI greeting agent\n- id: greeting_agent\n type: ai_agent\n config:\n model_client_id: assistant_model\n system_message: \"You are a helpful assistant. Generate a friendly greeting.\"\n user_message: \"User said: ${user_message}\"\n\n# Send the AI response via API\n- id: send_greeting\n type: script\n depends_on: [greeting_agent]\n previous_node: greeting_agent\n script: |\n import requests\n import json\n \n # CORRECT: Access AI response from structured output\n agent_output = ${greeting_agent}\n ai_greeting = agent_output[\u0027ai_response\u0027].strip()\n \n phone_number = \"${phone_number}\"\n api_token = \"${API_TOKEN}\"\n \n print(f\"\ud83e\udd16 AI generated greeting: {ai_greeting}\")\n \n # Send via API\n url = f\"https://api.example.com/messages/{phone_number}\"\n headers = {\"Authorization\": f\"Bearer {api_token}\"}\n \n response = requests.post(url, headers=headers, json={\n \"message\": ai_greeting\n })\n \n result = {\n \"status\": \"sent\",\n \"ai_response\": ai_greeting,\n \"api_response\": response.json(),\n \"tokens_used\": agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\n }\n \n print(f\"__OUTPUTS__ {json.dumps(result)}\")\n```\n\n### Complex Template Patterns\n\n```yaml\n# Conditional templates with router outputs\n- id: router_task\n type: conditional_router\n conditions:\n - name: \"premium_customer\"\n condition: \"${customer_data.tier} == \u0027premium\u0027\"\n route: \"premium_processing\"\n\n- id: premium_handler\n type: script\n depends_on: [router_task]\n previous_node: router_task\n execute_on_routes: [\"premium_processing\"]\n script: |\n # Access router decision\n selected_route = \"${router_task.selected_route}\"\n route_data = ${router_task.route_data}\n \n # Process premium customer\n customer = ${customer_data}\n premium_benefits = calculate_benefits(customer)\n\n# Loop through arrays from previous tasks\n- id: process_items\n type: script\n depends_on: [fetch_items]\n previous_node: fetch_items\n script: |\n items = ${fetch_items.data}\n processed_items = []\n \n for item in items:\n processed_item = {\n \"id\": item[\"id\"],\n \"processed_at\": \"${current_timestamp}\",\n \"status\": \"processed\"\n }\n processed_items.append(processed_item)\n \n print(f\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\")\n\n# Nested object access\n- id: user_preferences\n type: script\n depends_on: [fetch_user]\n previous_node: fetch_user\n script: |\n # Access nested user preferences\n email_notifications = ${fetch_user.preferences.notifications.email}\n theme_setting = \"${fetch_user.preferences.ui.theme}\"\n language = \"${fetch_user.profile.language}\"\n \n settings = {\n \"notifications\": email_notifications,\n \"theme\": theme_setting,\n \"locale\": language\n }\n print(f\"__OUTPUTS__ {json.dumps(settings)}\")\n```\n\n### Template Resolution Rules\n\n1. **Dependency Requirement**: Tasks must have `depends_on` relationship to access other task outputs\n2. **String vs Object**: Use quotes for string values, no quotes for objects/arrays\n3. **Field Navigation**: Use dot notation for nested field access\n4. **Error Handling**: Unresolved templates become `UNRESOLVED_variable_name`\n\n```yaml\n# \u2705 Correct template usage\n- id: correct_task\n depends_on: [previous_task]\n previous_node: previous_task\n script: |\n string_value = \"${previous_task.name}\" # String with quotes\n object_value = ${previous_task.data} # Object without quotes\n nested_value = \"${previous_task.user.email}\" # Nested field\n\n# \u274c Incorrect template usage \n- id: incorrect_task\n # Missing depends_on - template won\u0027t resolve\n script: |\n value = ${previous_task.name} # Will become UNRESOLVED_previous_task.name\n```\n\n## Script Task Output Handling\n\nScript tasks must use the `__OUTPUTS__` marker pattern for proper output parsing by the workflow engine.\n\n### Required Output Pattern\n\n```yaml\ntasks:\n - id: example_script\n type: script\n script: |\n # Your script logic here\n data = process_data()\n \n # Required: Store outputs in outputs dict\n outputs = {}\n outputs[\u0027result\u0027] = data\n outputs[\u0027status\u0027] = \u0027success\u0027\n outputs[\u0027count\u0027] = len(data)\n \n # Required: Print with __OUTPUTS__ marker for engine parsing\n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n### Output Access in Subsequent Tasks\n\n```yaml\ntasks:\n - id: data_processor\n type: script\n script: |\n # Process and store outputs\n results = {\"processed_count\": 100, \"errors\": []}\n \n outputs = {}\n outputs[\u0027results\u0027] = results\n outputs[\u0027status\u0027] = \u0027completed\u0027\n \n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n \n - id: result_handler\n type: script\n depends_on: [data_processor]\n script: |\n # Access previous task outputs\n processor_output = ${data_processor}\n processed_count = processor_output[\u0027results\u0027][\u0027processed_count\u0027]\n status = processor_output[\u0027status\u0027]\n \n print(f\"Previous task processed {processed_count} items with status: {status}\")\n \n # Store new outputs\n outputs = {\u0027message\u0027: f\u0027Handled {processed_count} results\u0027}\n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n### Error Handling in Scripts\n\n```yaml\ntasks:\n - id: safe_script\n type: script\n script: |\n try:\n result = risky_operation()\n outputs = {\u0027result\u0027: result, \u0027status\u0027: \u0027success\u0027}\n except Exception as e:\n outputs = {\u0027error\u0027: str(e), \u0027status\u0027: \u0027failed\u0027}\n \n # Always print outputs, even for errors\n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n## Triggers and Workflow Activation\n\nTriggers define how workflows are started automatically based on external events. They enable event-driven workflow execution.\n\n### Webhook Triggers\n\nWebhook triggers allow external systems to start workflows by sending HTTP requests.\n\n```yaml\ntriggers:\n - name: customer_webhook\n type: webhook\n description: \"Triggered when customer data is updated\"\n config:\n path: \"/webhooks/customer-updated\" # Webhook endpoint path\n method: POST # HTTP method (GET, POST, PUT, etc.)\n authentication:\n type: token # none|token|signature|basic\n token: \"${WEBHOOK_SECRET}\" # Required if type is token\n header_name: \"X-Webhook-Token\" # Optional: custom header name\n validation:\n required_fields: [\"customer.id\", \"customer.email\"] # Required payload fields\n content_type: \"application/json\" # Expected content type\n input_mapping:\n - webhook_field: customer.id # Field from webhook payload\n workflow_input: customer_id # Map to workflow input\n - webhook_field: customer.email\n workflow_input: customer_email\n - webhook_field: customer.updated_at\n workflow_input: update_timestamp\n```\n\n### Scheduled Triggers\n\nSchedule workflows to run at specific times or intervals.\n\n```yaml\ntriggers:\n - name: daily_report\n type: schedule\n description: \"Generate daily customer report\"\n config:\n cron: \"0 9 * * *\" # Daily at 9 AM (cron format)\n timezone: \"UTC\" # Optional: timezone\n enabled: true # Optional: enable/disable\n max_instances: 1 # Optional: max concurrent instances\n input_mapping:\n - workflow_input: report_date\n default_value: \"${current_date}\" # Use built-in variable\n - workflow_input: report_type\n default_value: \"daily\"\n\n - name: weekly_cleanup\n type: schedule\n description: \"Weekly database cleanup\"\n config:\n cron: \"0 2 * * 0\" # Weekly on Sunday at 2 AM\n timezone: \"America/New_York\"\n input_mapping:\n - workflow_input: cleanup_before_date\n default_value: \"${date_minus_7_days}\"\n```\n\n### Event Triggers\n\nEvent triggers respond to system events from various sources.\n\n```yaml\ntriggers:\n - name: file_uploaded\n type: event\n description: \"Process uploaded files\"\n config:\n event_type: \"file.uploaded\" # Event type to listen for\n source: \"storage_service\" # Event source system\n filter: # Optional: event filtering\n file_type: [\"csv\", \"json\", \"xlsx\"]\n file_size_max: \"100MB\"\n path_pattern: \"/uploads/customers/*\"\n retry: # Optional: retry configuration\n max_attempts: 3\n initial_delay: \"5s\"\n input_mapping:\n - webhook_field: file.path\n workflow_input: file_path\n - webhook_field: file.size\n workflow_input: file_size\n - webhook_field: file.type\n workflow_input: file_type\n\n - name: order_status_changed\n type: event\n description: \"Handle order status changes\"\n config:\n event_type: \"order.status_changed\"\n source: \"order_management\"\n filter:\n status: [\"completed\", \"cancelled\", \"refunded\"]\n priority: [\"high\", \"urgent\"]\n input_mapping:\n - webhook_field: order.id\n workflow_input: order_id\n - webhook_field: order.status\n workflow_input: new_status\n - webhook_field: order.customer_id\n workflow_input: customer_id\n```\n\n### Dependency Triggers\n\nDependency triggers start workflows when other workflows complete.\n\n```yaml\ntriggers:\n - name: post_processing\n type: dependency\n description: \"Run after main processing completes\"\n config:\n depends_on_workflow: \"main_data_processing\" # Workflow to depend on\n depends_on_status: \"SUCCESS\" # SUCCESS|FAILED|COMPLETED\n namespace: \"production\" # Optional: workflow namespace\n timeout_minutes: 60 # Optional: wait timeout\n input_mapping:\n - webhook_field: execution_id # From completed workflow\n workflow_input: source_execution_id\n - webhook_field: workflow_outputs.processed_count\n workflow_input: items_processed\n```\n\n## Model Client Configuration\n\nBefore defining AI-related tasks (ai_agent, chatbot, continuous_conversation), you must configure model clients. Model clients define which AI models to use and how to authenticate with them.\n\n### Basic Structure\n\n```yaml\nmodel_clients:\n - id: unique_identifier # Used to reference this model in tasks\n provider: provider_name # AI provider (openai, anthropic, etc.)\n config:\n model: model_name # Specific model to use\n api_key: your_api_key # Authentication key\n # Additional provider-specific options...\n```\n\n### Supported Providers\n\n#### OpenAI Provider\n\n```yaml\nmodel_clients:\n - id: gpt4_model\n provider: openai\n config:\n model: gpt-4-turbo-preview # Available: gpt-4, gpt-4-turbo, gpt-3.5-turbo\n api_key: \"sk-your-openai-api-key\"\n temperature: 0.7 # Optional: 0.0-2.0\n max_tokens: 2000 # Optional: Maximum response tokens\n timeout: 30 # Optional: Request timeout (seconds)\n```\n\n#### Anthropic Provider\n\n```yaml\nmodel_clients:\n - id: claude_model\n provider: anthropic\n config:\n model: claude-3-sonnet-20240229 # Available: claude-3-opus, claude-3-sonnet, claude-3-haiku\n api_key: \"sk-ant-your-anthropic-api-key\"\n temperature: 0.7 # Optional: 0.0-1.0\n max_tokens: 1500 # Optional: Maximum response tokens\n timeout: 30 # Optional: Request timeout (seconds)\n```\n\n#### OpenRouter Provider\n\n```yaml\nmodel_clients:\n - id: openrouter_model\n provider: openrouter\n config:\n model: anthropic/claude-3-sonnet # Available: Various models via OpenRouter\n api_key: \"sk-or-your-openrouter-api-key\"\n base_url: \"https://openrouter.ai/api/v1\" # Optional: Custom endpoint\n temperature: 0.7\n max_tokens: 2000\n```\n\n#### Multiple Model Clients\n\nYou can define multiple model clients and reference them by their `id` in tasks:\n\n```yaml\nmodel_clients:\n - id: fast_model\n provider: openai\n config:\n model: gpt-3.5-turbo\n api_key: \"sk-your-openai-api-key\"\n temperature: 0.3\n \n - id: smart_model\n provider: anthropic\n config:\n model: claude-3-opus-20240229\n api_key: \"sk-ant-your-anthropic-api-key\"\n temperature: 0.7\n \n - id: creative_model\n provider: openai\n config:\n model: gpt-4-turbo-preview\n api_key: \"sk-your-openai-api-key\"\n temperature: 1.2\n```\n\n### Using Model Clients in Tasks\n\nReference model clients in AI tasks using the `model_client_id` field:\n\n```yaml\ntasks:\n - id: analysis_task\n type: ai_agent\n config:\n model_client_id: smart_model # References the claude model above\n system_message: \"You are a data analyst...\"\n user_message: \"Analyze this data: ${input_data}\"\n \n - id: creative_task\n type: ai_agent\n config:\n model_client_id: creative_model # References the creative GPT-4 model\n system_message: \"You are a creative writer...\"\n user_message: \"Write a story about ${topic}\"\n```\n\n### Environment Variables for API Keys\n\nFor security, use environment variables instead of hardcoding API keys:\n\n```yaml\nmodel_clients:\n - id: secure_model\n provider: openai\n config:\n model: gpt-4-turbo-preview\n api_key: \"${OPENAI_API_KEY}\" # References environment variable\n temperature: 0.7\n```\n\n### Common Configuration Options\n\n| Field | Description | Required | Default |\n|-------|-------------|----------|---------|\n| `id` | Unique identifier for referencing in tasks | Yes | - |\n| `provider` | AI provider name | Yes | - |\n| `config.model` | Specific model name | Yes | - |\n| `config.api_key` | Authentication API key | Yes | - |\n| `config.temperature` | Response randomness (0.0-2.0) | No | 0.7 |\n| `config.max_tokens` | Maximum response tokens | No | Provider default |\n| `config.timeout` | Request timeout in seconds | No | 30 |\n| `config.base_url` | Custom API endpoint | No | Provider default |\n\n## Secrets Management and Environment Variables\n\nProper secrets management is crucial for security. Never hardcode sensitive information like API keys, passwords, or tokens directly in workflow YAML files.\n\n### Using Environment Variables\n\nThe recommended approach is to use environment variables for all sensitive data:\n\n```yaml\n# \u2705 RECOMMENDED: Use environment variables\nmodel_clients:\n - id: secure_openai\n provider: openai\n config:\n model: gpt-4-turbo-preview\n api_key: \"${OPENAI_API_KEY}\" # References environment variable\n\n# HTTP tasks with environment variables\ntasks:\n - id: secure_api_call\n type: http\n url: \"${API_BASE_URL}/customers\" # Environment variable for URL\n headers:\n Authorization: \"Bearer ${API_TOKEN}\" # Environment variable for token\n X-API-Key: \"${EXTERNAL_API_KEY}\" # Environment variable for API key\n```\n\n### Environment Variable Patterns\n\n```yaml\n# Database connections\n- id: database_task\n type: storage\n config:\n storage_type: database\n connection_string: \"${DATABASE_URL}\" # Full connection string\n # or individual components:\n host: \"${DB_HOST}\"\n port: \"${DB_PORT}\"\n database: \"${DB_NAME}\"\n username: \"${DB_USER}\"\n password: \"${DB_PASSWORD}\"\n\n# Third-party service credentials\n- id: external_service\n type: http\n url: \"${SERVICE_BASE_URL}/api/v1/data\"\n headers:\n Authorization: \"Bearer ${SERVICE_TOKEN}\"\n X-Client-ID: \"${SERVICE_CLIENT_ID}\"\n X-Client-Secret: \"${SERVICE_CLIENT_SECRET}\"\n\n# MCP server credentials\n- id: mcp_task\n type: mcp\n mcp_server: \"@agentic/mcp-notion-server\"\n environment_vars:\n NOTION_TOKEN: \"${NOTION_API_TOKEN}\"\n WORKSPACE_ID: \"${NOTION_WORKSPACE_ID}\"\n```\n\n### Security Best Practices\n\n#### \u274c What NOT to Do\n\n```yaml\n# DON\u0027T: Hardcode secrets\nmodel_clients:\n - id: insecure_example\n provider: openai\n config:\n api_key: \"sk-actual-api-key-here\" # SECURITY RISK!\n\n# DON\u0027T: Put secrets in comments\ntasks:\n - id: bad_example\n type: http\n url: \"https://api.example.com\"\n # API key is: sk-12345... DON\u0027T DO THIS!\n headers:\n Authorization: \"Bearer sk-real-token\" # SECURITY RISK!\n```\n\n#### \u2705 What TO Do\n\n```yaml\n# DO: Use environment variables\nmodel_clients:\n - id: secure_example\n provider: openai\n config:\n api_key: \"${OPENAI_API_KEY}\" # Secure approach\n\n# DO: Use different variables for different environments\n- id: environment_aware_task\n type: http\n url: \"${API_BASE_URL}\" # Different URLs per environment\n headers:\n Authorization: \"Bearer ${API_TOKEN}\" # Different tokens per environment\n X-Environment: \"${ENVIRONMENT_NAME}\" # Track environment\n\n# DO: Use descriptive variable names\nenvironment_vars:\n CUSTOMER_API_TOKEN: \"${CUSTOMER_SERVICE_TOKEN}\"\n BILLING_API_KEY: \"${BILLING_SERVICE_API_KEY}\"\n NOTIFICATION_WEBHOOK_SECRET: \"${NOTIFICATION_SECRET}\"\n```\n\n### Environment Variable Naming Conventions\n\n| Pattern | Example | Use Case |\n|---------|---------|----------|\n| `SERVICE_API_KEY` | `OPENAI_API_KEY` | Service API keys |\n| `SERVICE_BASE_URL` | `CUSTOMER_API_BASE_URL` | Service endpoints |\n| `DB_*` | `DB_HOST`, `DB_PASSWORD` | Database credentials |\n| `WEBHOOK_*` | `WEBHOOK_SECRET_TOKEN` | Webhook authentication |\n| `*_TIMEOUT` | `HTTP_REQUEST_TIMEOUT` | Timeout configurations |\n\n### Dynamic Environment Variables\n\nUse template resolution to create dynamic environment variable names:\n\n```yaml\n# Different API keys per customer\n- id: customer_specific_api\n type: http\n url: \"https://api.partner.com/data\"\n headers:\n Authorization: \"Bearer ${PARTNER_API_KEY_${customer_tier}}\" # Resolves to PARTNER_API_KEY_PREMIUM\n\n# Environment-specific configurations\n- id: env_specific_task\n type: storage\n config:\n connection_string: \"${DATABASE_URL_${environment}}\" # DATABASE_URL_PRODUCTION\n```\n\n### Secrets in Different Task Types\n\n```yaml\n# AI Agent with multiple secure credentials\n- id: secure_ai_task\n type: ai_agent\n config:\n model_client_id: secure_model\n system_message: \"You are a secure assistant with access to customer data.\"\n user_message: \"Process customer ${customer_id}\"\n environment_vars:\n CUSTOMER_DB_TOKEN: \"${CUSTOMER_DATABASE_TOKEN}\"\n AUDIT_LOG_KEY: \"${AUDIT_LOGGING_API_KEY}\"\n\n# Script task with secure file access\n- id: secure_script_task\n type: script\n environment_vars:\n AWS_ACCESS_KEY_ID: \"${AWS_ACCESS_KEY}\"\n AWS_SECRET_ACCESS_KEY: \"${AWS_SECRET_KEY}\"\n S3_BUCKET_NAME: \"${SECURE_BUCKET_NAME}\"\n script: |\n import boto3\n import os\n \n # Credentials automatically available from environment\n s3_client = boto3.client(\u0027s3\u0027)\n bucket = os.environ[\u0027S3_BUCKET_NAME\u0027]\n \n # Process secure files...\n\n# Storage task with encrypted connections\n- id: secure_storage_task\n type: storage\n config:\n storage_type: database\n connection_string: \"${ENCRYPTED_DB_CONNECTION_STRING}\"\n ssl_mode: \"require\"\n ssl_cert: \"${SSL_CERT_PATH}\"\n ssl_key: \"${SSL_KEY_PATH}\"\n```\n\n### Environment Setup Examples\n\n#### Development Environment (.env file)\n```bash\n# Development API keys (non-production)\nOPENAI_API_KEY=sk-dev-api-key-here\nCUSTOMER_API_BASE_URL=https://api-dev.example.com\nDATABASE_URL=postgresql://dev:pass@localhost:5432/workflow_dev\nENVIRONMENT_NAME=development\n```\n\n#### Production Environment\n```bash\n# Production API keys (secure)\nOPENAI_API_KEY=sk-prod-api-key-here\nCUSTOMER_API_BASE_URL=https://api.example.com\nDATABASE_URL=postgresql://prod_user:secure_pass@prod-db:5432/workflow_prod\nENVIRONMENT_NAME=production\n```\n\n## Workflow Execution and Orchestration\n\nUnderstanding how workflows execute and how tasks are orchestrated is essential for building reliable, efficient workflows.\n\n### Execution Flow\n\nWorkflows follow a directed acyclic graph (DAG) execution model:\n\n1. **Initialization**: Workflow starts with inputs and model clients\n2. **Task Scheduling**: Tasks are scheduled based on dependencies\n3. **Parallel Execution**: Independent tasks run simultaneously\n4. **Data Flow**: Outputs flow between tasks via template resolution\n5. **Completion**: Workflow completes when all tasks finish\n\n```yaml\n# Example execution flow\ntasks:\n # Stage 1: Independent tasks (run in parallel)\n - id: fetch_customer # Runs immediately\n type: http\n url: \"https://api.example.com/customers/${customer_id}\"\n \n - id: fetch_products # Runs in parallel with fetch_customer\n type: http\n url: \"https://api.example.com/products\"\n \n # Stage 2: Dependent tasks (wait for Stage 1)\n - id: validate_customer # Waits for fetch_customer\n type: script\n depends_on: [fetch_customer]\n previous_node: fetch_customer\n script: # validation logic...\n \n - id: filter_products # Waits for fetch_products\n type: script\n depends_on: [fetch_products]\n previous_node: fetch_products\n script: # filtering logic...\n \n # Stage 3: Final processing (waits for Stage 2)\n - id: generate_recommendations # Waits for both validation and filtering\n type: ai_agent\n depends_on: [validate_customer, filter_products]\n previous_node: validate_customer # UI hierarchy under customer validation\n config:\n model_client_id: gpt4_model\n system_message: \"Generate product recommendations for customer.\"\n user_message: |\n Customer: ${validate_customer}\n Available products: ${filter_products}\n```\n\n### Task States and Lifecycle\n\nEach task goes through these states during execution:\n\n| State | Description | Next States |\n|-------|-------------|-------------|\n| `PENDING` | Task created, waiting for dependencies | `RUNNING`, `SKIPPED` |\n| `RUNNING` | Task is actively executing | `COMPLETED`, `FAILED` |\n| `COMPLETED` | Task finished successfully | - |\n| `FAILED` | Task encountered an error | `RUNNING` (if retries) |\n| `SKIPPED` | Task skipped due to conditions | - |\n| `TIMEOUT` | Task exceeded timeout limit | `RUNNING` (if retries) |\n\n### Conditional Execution\n\nTasks can be executed conditionally based on routes or conditions:\n\n```yaml\n# Router-based conditional execution\n- id: data_router\n type: conditional_router\n depends_on: [fetch_data]\n previous_node: fetch_data\n conditions:\n - name: \"large_dataset\"\n condition: \"${fetch_data.record_count} \u003e 10000\"\n route: \"batch_processing\"\n - name: \"small_dataset\"\n condition: \"${fetch_data.record_count} \u003c= 1000\"\n route: \"direct_processing\"\n default_route: \"standard_processing\"\n\n# Tasks that execute on specific routes\n- id: batch_processor\n type: script\n depends_on: [data_router]\n previous_node: data_router\n execute_on_routes: [\"batch_processing\"] # Only runs for large datasets\n script: |\n # Handle large dataset in batches\n data = ${fetch_data}\n process_in_batches(data, batch_size=1000)\n\n- id: direct_processor\n type: script\n depends_on: [data_router]\n previous_node: data_router\n execute_on_routes: [\"direct_processing\"] # Only runs for small datasets\n script: |\n # Process small dataset directly\n data = ${fetch_data}\n process_directly(data)\n```\n\n### Error Handling and Retries\n\nConfigure retry policies for resilient workflow execution:\n\n```yaml\n# Workflow-level retry policy\nretry:\n retryOn: [\"TEMPORARY_FAILURE\", \"TIMEOUT\", \"NETWORK_ERROR\"]\n maxAttempts: 3\n initialDelay: \"1s\"\n maxDelay: \"60s\"\n backoffMultiplier: 2.0\n\ntasks:\n # Task-specific retry policy (overrides workflow-level)\n - id: external_api_call\n type: http\n url: \"https://unreliable-api.example.com/data\"\n retry:\n maxAttempts: 5 # More retries for unreliable API\n initialDelay: \"2s\"\n maxDelay: \"120s\"\n backoffMultiplier: 2.0\n retryOn: [\"TIMEOUT\", \"HTTP_5XX\", \"NETWORK_ERROR\"]\n \n # Task with no retries\n - id: critical_operation\n type: script\n retry:\n maxAttempts: 1 # No retries for critical operations\n script: |\n # This operation should not be retried if it fails\n perform_critical_database_operation()\n```\n\n### Parallel Execution Patterns\n\n```yaml\n# Fan-out pattern: One task triggers multiple parallel tasks\n- id: data_source\n type: http\n url: \"https://api.example.com/data\"\n\n- id: process_customers # Parallel processing\n type: script\n depends_on: [data_source]\n previous_node: data_source\n script: # process customer data from data_source\n\n- id: process_orders # Parallel processing\n type: script\n depends_on: [data_source]\n previous_node: data_source\n script: # process order data from data_source\n\n- id: process_products # Parallel processing\n type: script\n depends_on: [data_source]\n previous_node: data_source\n script: # process product data from data_source\n\n# Fan-in pattern: Multiple tasks feed into one task\n- id: generate_report # Waits for all parallel tasks\n type: ai_agent\n depends_on: [process_customers, process_orders, process_products]\n previous_node: process_customers # UI hierarchy\n config:\n model_client_id: gpt4_model\n system_message: \"Generate comprehensive report from processed data.\"\n user_message: |\n Customer data: ${process_customers}\n Order data: ${process_orders}\n Product data: ${process_products}\n```\n\n### Resource Management\n\nControl resource usage and task distribution:\n\n```yaml\n# Worker group targeting\n- id: cpu_intensive_task\n type: script\n worker_group: \"high-cpu-workers\" # Target specific worker group\n script: |\n # CPU-intensive processing\n perform_heavy_computation()\n\n- id: gpu_ml_task\n type: ai_agent\n worker_group: \"gpu-workers\" # Target GPU workers\n config:\n model_client_id: large_model\n # Large model processing...\n\n# Timeout configuration\n- id: long_running_task\n type: script\n timeout_seconds: 3600 # 1 hour timeout\n script: |\n # Long-running data processing\n process_large_dataset()\n\n# Workflow-level timeout\ntimeout_seconds: 7200 # 2 hour workflow timeout\n```\n\n## Task Type Configurations\n\nThis section provides the complete YAML configuration for each supported task type.\n\n**Supported Task Types:**\n- `script` - Execute Python code with full workflow context\n- `http` - Make HTTP requests to external APIs\n- `ai_agent` - Integrate with AI models for intelligent processing \n- `loop` - Execute tasks repeatedly (for/while loops)\n- `conditional_router` - Route execution based on conditions\n- `mcp` - Execute tools via Model Context Protocol servers\n- `storage` - Data persistence and retrieval operations\n- `continuous_conversation` - Interactive AI conversations\n- `human_input` - Tasks requiring human input or approval\n- `chatbot` - Session-based chatbot conversations\n\n### Script Tasks\n\nExecute Python code with full workflow context access.\n\n```yaml\n- id: script_task\n name: \"Python Script Task\"\n type: script\n description: \"Execute Python code with dependencies and environment\"\n \n # Script configuration - script field is directly under task, NOT under config\n script: | # REQUIRED: Python code to execute\n import json\n import requests\n \n # Access workflow inputs\n input_value = \"${workflow_input}\"\n \n # Access previous task outputs\n previous_result = ${previous_task}\n \n # Your processing logic\n result = {\"processed\": True, \"value\": input_value}\n \n # Output results (must be JSON serializable)\n print(f\"__OUTPUTS__ {json.dumps(result)}\")\n \n # Alternative: use \u0027code\u0027 field instead of \u0027script\u0027\n code: | # Alternative to \u0027script\u0027 field\n print(\"Hello World\")\n \n # Package management\n requirements: # Optional: Python packages\n - requests==2.31.0\n - pandas==1.5.3\n - numpy==1.24.3\n \n # Environment configuration\n environment: # Optional: Environment variables\n API_KEY: \"${secrets.api_key}\"\n DEBUG: \"true\"\n CUSTOM_VAR: \"value\"\n \n environment_vars: # Alternative to \u0027environment\u0027\n DATABASE_URL: \"${secrets.db_url}\"\n \n # Secrets configuration\n secrets_config: # Optional: Secrets access\n api_key: \"secret_name\"\n db_password: \"db_secret\"\n```\n\n### HTTP Tasks\n\nMake HTTP requests to external APIs and services.\n\n```yaml\n- id: http_task\n name: \"HTTP API Call\"\n type: http\n description: \"Call external REST API\"\n \n # HTTP configuration\n method: POST # Default: GET\n url: \"https://api.example.com/data\" # REQUIRED: Target URL\n \n # Headers configuration\n headers: # Optional: HTTP headers\n Content-Type: \"application/json\"\n Authorization: \"Bearer ${secrets.api_token}\"\n X-Custom-Header: \"workflow-engine\"\n User-Agent: \"WorkflowEngine/1.0\"\n \n # Request body\n body: | # Optional: Request body (for POST/PUT)\n {\n \"customer_id\": \"${customer_id}\",\n \"timestamp\": \"${current_timestamp}\",\n \"data\": ${previous_task}\n }\n \n # Response validation\n expected_status: [200, 201, 202] # Default: [200]\n expectedStatus: [200, 201] # Alternative field name\n \n # Additional HTTP options\n timeout_seconds: 30 # HTTP request timeout\n follow_redirects: true # Follow HTTP redirects\n verify_ssl: true # Verify SSL certificates\n```\n\n### AI Agent Tasks\n\nIntegrate with AI models for intelligent processing.\n\n```yaml\n- id: ai_agent_task\n name: \"AI Processing Agent\"\n type: ai_agent\n description: \"Process data using AI model\"\n \n # AI configuration\n config:\n model_client_id: gpt4_model # REQUIRED: Reference to model client\n \n system_message: | # Optional: System prompt\n You are a data analyst. Analyze the provided data and extract insights.\n Return your analysis in JSON format with the following structure:\n {\n \"insights\": [\"insight1\", \"insight2\"],\n \"recommendations\": [\"rec1\", \"rec2\"],\n \"confidence_score\": 0.95\n }\n \n user_message: | # REQUIRED: User prompt with template resolution\n Please analyze this customer data:\n ${previous_task}\n \n Customer ID: ${customer_id}\n Date Range: ${date_range}\n \n # Model parameters\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\n max_tokens: 1500 # Optional: Maximum response tokens\n top_p: 1.0 # Optional: Nucleus sampling\n frequency_penalty: 0.0 # Optional: Frequency penalty\n presence_penalty: 0.0 # Optional: Presence penalty\n \n # Tools configuration for function calling\n tools: # Optional: List of MCP tool names\n - current_time # Time-related tools\n - relative_time\n - days_in_month\n - weather_api # Weather tools\n - database_query # Database tools\n - notification_service # Communication tools\n \n # Required for tool resolution\n tenant_id: \"your-tenant-uuid\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\n \n # Input/output format\n input_format: \"text\" # Optional: text|json|structured\n output_format: \"structured\" # Optional: text|json|structured\n\n### Important: AI Agent Output Access\n\nAI agent tasks return structured objects, not direct text responses. Always access the AI response using this pattern:\n\n```yaml\n# In subsequent script tasks\nscript: |\n # CORRECT: Access AI response from structured output\n agent_output = ${ai_agent_task}\n ai_response = agent_output[\u0027ai_response\u0027].strip()\n \n # Additional metadata available\n tokens_used = agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\n cost_estimate = agent_output.get(\u0027estimated_cost\u0027, {}).get(\u0027amount\u0027, 0.0)\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\n \n print(f\"AI Response: {ai_response}\")\n print(f\"Tokens used: {tokens_used}\")\n print(f\"Estimated cost: ${cost_estimate:.6f}\")\n```\n\n```yaml\n# In workflow outputs - specify the ai_response field\noutputs:\n ai_analysis:\n type: string\n source: ai_agent_task.ai_response # Access the ai_response field\n description: \"AI-generated analysis text\"\n \n total_tokens:\n type: integer \n source: ai_agent_task.token_usage.total_tokens\n description: \"Total tokens consumed\"\n```\n```\n\n### Loop Tasks\n\nExecute tasks repeatedly with for-loop and while-loop patterns.\n\n```yaml\n# For-Loop (Fixed Iterations)\n- id: for_loop_task\n name: \"For Loop Processor\"\n type: loop\n description: \"Process data in fixed batches\"\n \n # Loop configuration\n loop_type: for # REQUIRED: for|while\n max_iterations: 10 # REQUIRED for \u0027for\u0027 loops\n iteration_variable: batch_index # Default: iteration\n \n # State variables (persistent across iterations)\n state_variables:\n total_processed: 0\n error_count: 0\n batch_results: []\n \n # Tasks to execute in each iteration\n loop_tasks: # REQUIRED: Tasks to run in loop\n - id: process_batch\n name: \"Process Current Batch\"\n type: script\n script: |\n batch_idx = ${batch_index} # Access iteration variable\n total = ${total_processed} # Access state variable\n \n # Process current batch\n result = process_batch_data(batch_idx)\n \n print(f\"__OUTPUTS__ {json.dumps(result)}\")\n \n - id: update_state\n name: \"Update Loop State\"\n type: script\n depends_on: [process_batch]\n script: |\n # Update state variables\n new_total = ${total_processed} + ${process_batch.items_processed}\n \n result = {\"total_processed\": new_total}\n print(f\"__OUTPUTS__ {json.dumps(result)}\")\n\n# While-Loop (Conditional)\n- id: while_loop_task\n name: \"While Loop Quality Processor\"\n type: loop\n description: \"Process until quality threshold met\"\n \n # Loop configuration\n loop_type: while # REQUIRED: for|while\n exit_condition: \"${average_quality} \u003e= 85 || ${iteration_count} \u003e= 20\" # REQUIRED for \u0027while\u0027\n max_iterations: 50 # Optional: Safety limit\n iteration_variable: iteration_count # Default: iteration\n \n # State variables\n state_variables:\n average_quality: 0\n items_processed: 0\n quality_scores: []\n \n # Loop tasks\n loop_tasks:\n - id: assess_quality\n type: script\n script: |\n current_quality = calculate_quality()\n print(f\"__OUTPUTS__ {json.dumps({\u0027quality\u0027: current_quality})}\")\n```\n\n\n### Conditional Router Tasks\n\nRoute execution to different paths based on conditions.\n\n```yaml\n- id: conditional_router_task\n name: \"Data Type Router\"\n type: conditional_router\n description: \"Route processing based on data characteristics\"\n \n # Router configuration\n router_type: condition # Default: condition\n \n # Routing conditions\n conditions: # REQUIRED: List of conditions\n - name: \"large_dataset\"\n condition: \"${data_size} \u003e 10000\"\n route: \"batch_processing\"\n - name: \"small_dataset\" \n condition: \"${data_size} \u003c= 1000\"\n route: \"direct_processing\"\n - name: \"medium_dataset\"\n condition: \"${data_size} \u003e 1000 \u0026\u0026 ${data_size} \u003c= 10000\"\n route: \"standard_processing\"\n \n default_route: \"standard_processing\" # REQUIRED: Default route if no conditions match\n \n # Alternative: evaluation script for complex routing\n evaluation_script: | # Optional: Custom routing logic\n if data_type == \"json\":\n return \"json_processing\"\n elif data_type == \"xml\":\n return \"xml_processing\"\n else:\n return \"default_processing\"\n \n # Route mapping for complex scenarios\n route_mapping: # Optional: Route name mappings\n json_processing: \"specialized_json\"\n xml_processing: \"specialized_xml\"\n\n# Tasks that execute on specific routes\n- id: batch_handler\n name: \"Handle Large Dataset\"\n type: script\n execute_on_routes: [\"batch_processing\"] # REQUIRED: Routes this task runs on\n depends_on: [conditional_router_task]\n script: |\n print(\"Processing large dataset in batches...\")\n\n- id: direct_handler\n name: \"Handle Small Dataset\" \n type: script\n execute_on_routes: [\"direct_processing\"]\n depends_on: [conditional_router_task]\n script: |\n print(\"Processing small dataset directly...\")\n```\n\n### MCP Tasks\n\nExecute tools via Model Context Protocol servers.\n\n```yaml\n- id: mcp_task\n name: \"MCP Tool Execution\"\n type: mcp\n description: \"Execute external tool via MCP server\"\n \n # MCP server configuration\n mcp_server: \"@agentic/mcp-notion-server\" # REQUIRED: MCP server package\n tool_name: \"update_database\" # REQUIRED: Tool to execute\n runtime: \"npx\" # REQUIRED: npx|uvx\n \n # Server startup\n server_args: [] # Optional: Server startup arguments\n server_startup_timeout: 30 # Optional: Server startup timeout (seconds)\n \n # Tool execution\n tool_arguments: # REQUIRED: Arguments for the tool\n database_id: \"${secrets.notion_database_id}\"\n properties:\n title: \"Workflow Results\"\n status: \"Completed\"\n processed_count: \"${previous_task.count}\"\n timestamp: \"${current_timestamp}\"\n \n # Environment configuration\n environment_vars: # Optional: Environment variables\n NOTION_TOKEN: \"${secrets.notion_token}\"\n DEBUG: \"true\"\n \n # Timeout configuration\n timeout_seconds: 120 # Optional: Tool execution timeout\n```\n\n### Storage Tasks\n\nManage data persistence and retrieval operations.\n\n```yaml\n- id: storage_task\n name: \"Data Storage Operation\"\n type: storage\n description: \"Save/load data to/from storage\"\n \n # Storage operation\n operation: save # REQUIRED: save|load|delete|list\n \n # Storage configuration\n storage_type: \"file\" # REQUIRED: file|database|cache\n path: \"/data/results/${execution_id}.json\" # REQUIRED: Storage path\n \n # Data configuration\n data: ${previous_task} # REQUIRED for \u0027save\u0027: Data to store\n format: \"json\" # Optional: json|yaml|csv|parquet\n compression: \"gzip\" # Optional: gzip|bz2|none\n \n # Metadata\n metadata: # Optional: Additional metadata\n workflow_id: \"${workflow_id}\"\n created_at: \"${current_timestamp}\"\n source_task: \"previous_task\"\n \n # File options\n create_directories: true # Optional: Create parent directories\n overwrite: true # Optional: Overwrite existing files\n backup: false # Optional: Create backup before overwrite\n \n # Database options (when storage_type: \"database\")\n table_name: \"workflow_results\" # Required for database storage\n connection_string: \"${secrets.db_url}\" # Required for database storage\n \n # Cache options (when storage_type: \"cache\")\n cache_key: \"result_${execution_id}\" # Required for cache storage\n ttl: 3600 # Optional: Time to live (seconds)\n```\n\n\n### Continuous Conversation Tasks\n\nCreate interactive AI-powered conversations.\n\n```yaml\n- id: conversation_task\n name: \"AI Conversation Interface\"\n type: continuous_conversation\n description: \"Interactive conversation with AI assistant\"\n \n # Conversation configuration\n config:\n # AI configuration\n ai_config:\n model_client_id: support_assistant # REQUIRED: AI model to use\n system_prompt: | # REQUIRED: System instructions\n You are a helpful customer support representative.\n Be friendly, professional, and solution-oriented.\n \n Guidelines:\n - Always be polite and respectful\n - Ask clarifying questions when needed\n - Provide clear and actionable solutions\n \n temperature: 0.7 # Optional: AI creativity\n max_tokens: 1000 # Optional: Response length limit\n \n # UI configuration\n ui_config:\n title: \"Customer Support Chat\" # Optional: Interface title\n placeholder: \"How can I help you today?\" # Optional: Input placeholder\n theme: \"light\" # Optional: light|dark\n show_timestamp: true # Optional: Show message timestamps\n \n # Termination conditions\n termination:\n keywords: [\"goodbye\", \"bye\", \"thanks\", \"done\"] # Optional: End conversation keywords\n max_turns: 50 # Optional: Maximum conversation turns\n timeout_minutes: 30 # Optional: Conversation timeout\n inactivity_timeout: 10 # Optional: Inactivity timeout (minutes)\n \n # Context configuration\n context:\n system_context: \"Customer support conversation\" # Optional: Context description\n user_id: \"${customer_id}\" # Optional: User identifier\n session_data: ${previous_task} # Optional: Additional context data\n```\n\n### Human Input Tasks\n\nTasks requiring human input or approval.\n\n```yaml\n- id: human_input_task\n name: \"Human Input Required\"\n type: human_input\n description: \"Wait for human input or approval\"\n \n # Human input configuration\n config:\n prompt: \"Please review and approve the processed data\" # REQUIRED: Prompt for human\n input_type: \"approval\" # REQUIRED: approval|text|selection\n \n # Input options (for selection type)\n options: [\"approve\", \"reject\", \"request_changes\"] # Optional: For selection input\n \n # Timeout configuration\n timeout_minutes: 60 # Optional: How long to wait for input (minutes)\n \n # Conversation settings\n conversation_mode: false # Optional: Add to conversation history\n next_task: \"process_approval\" # Optional: Task to trigger after input\n end_condition: \"bye|exit|quit\" # Optional: Keywords to end conversation\n \n # Input validation\n required: true # Optional: Whether input is required\n \n # Additional context\n context_data: ${previous_task} # Optional: Data to show to human\n```\n\n### ChatBot Tasks\n\nSession-based chatbot conversations with streaming support.\n\n```yaml\n- id: chatbot_task\n name: \"AI Chatbot\"\n type: chatbot\n description: \"Interactive chatbot with session management\"\n \n # Chatbot configuration\n config:\n # AI model configuration\n model_client_id: gpt4_model # REQUIRED: Reference to model client\n \n system_message: | # REQUIRED: System prompt for chatbot\n You are a helpful AI assistant. Be friendly, helpful, and concise.\n Answer questions clearly and ask for clarification when needed.\n \n # Session management\n session_id: \"${session_id}\" # Optional: Session identifier (auto-generated if not provided)\n max_history_messages: 20 # Optional: Maximum conversation history to maintain\n \n # Model parameters\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\n max_tokens: 1000 # Optional: Maximum response tokens\n stream: true # Optional: Enable streaming responses\n \n # Tools configuration for function calling\n tools: # Optional: List of MCP tool names\n - current_time # Time-related tools\n - relative_time\n - days_in_month\n - weather_forecast # Weather tools\n - send_notification # Communication tools\n \n # Required for tool resolution\n tenant_id: \"your-tenant-uuid\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\n \n # UI configuration\n ui_config:\n title: \"AI Assistant\" # Optional: Chat interface title\n placeholder: \"Type your message...\" # Optional: Input placeholder\n theme: \"light\" # Optional: light|dark\n show_typing_indicator: true # Optional: Show typing animation\n \n # Response configuration\n response_config:\n max_retries: 3 # Optional: Retry failed requests\n timeout_seconds: 30 # Optional: Response timeout\n \n # Input message (for single-turn mode)\n user_message: \"${user_input}\" # Optional: Direct message input\n \n # Context data\n context_data: ${previous_task} # Optional: Additional context\n```\n\n---\n\n## Common Errors and Corrections\n\nThis section documents frequent mistakes encountered when creating workflows and their proper corrections.\n\n### 1. Missing Required `id` Field\n\n**\u274c Error:**\n```yaml\nname: \"My Workflow\"\ndescription: \"A workflow without id\"\ntasks:\n - name: \"Task 1\"\n type: script\n```\n\n**\u2705 Correct:**\n```yaml\nid: \"my_workflow\" # REQUIRED: Unique workflow identifier\nname: \"My Workflow\"\ndescription: \"A workflow with proper id\"\ntasks:\n - id: task_1 # REQUIRED: Unique task identifier\n name: \"Task 1\"\n type: script\n```\n\n### 2. Incorrect Task Structure (Object vs Array)\n\n**\u274c Error:**\n```yaml\ntasks:\n task_1: # Wrong: tasks as object mapping\n type: script\n script: \"print(\u0027hello\u0027)\"\n task_2:\n type: ai_agent\n```\n\n**\u2705 Correct:**\n```yaml\ntasks:\n - id: task_1 # Correct: tasks as array with id field\n type: script\n script: \"print(\u0027hello\u0027)\"\n - id: task_2\n type: ai_agent\n```\n\n### 3. Incorrect Inputs Format (Object vs Array)\n\n**\u274c Error:**\n```yaml\ninputs:\n user_name: # Wrong: inputs as object mapping\n type: string\n required: true\n age:\n type: integer\n```\n\n**\u2705 Correct:**\n```yaml\ninputs:\n - name: user_name # Correct: inputs as array with name field\n type: string\n required: true\n - name: age\n type: integer\n```\n\n### 4. Script Configuration Location Error\n\n**\u274c Error:**\n```yaml\n- id: script_task\n type: script\n config: # Wrong: script under config\n script: |\n print(\"Hello\")\n requirements:\n - requests\n```\n\n**\u2705 Correct:**\n```yaml\n- id: script_task\n type: script\n script: | # Correct: script directly under task\n print(\"Hello\")\n # requirements can be at task level if needed\n```\n\n### 5. Individual Task Inputs (Not Supported)\n\n**\u274c Error:**\n```yaml\n- id: task_1\n type: script\n script: \"print(\u0027hello\u0027)\"\n inputs: # Wrong: tasks don\u0027t have individual inputs\n search_query: ${search_query}\n prev_result: ${previous_task}\n```\n\n**\u2705 Correct:**\n```yaml\n- id: task_1\n type: script\n depends_on: [previous_task] # Use depends_on for data access\n script: |\n # Access workflow inputs directly\n search_query = \"${search_query}\"\n # Access previous task outputs via template resolution\n prev_result = ${previous_task}\n```\n\n### 6. Incorrect Script Output Handling\n\n**\u274c Error:**\n```yaml\nscript: |\n result = process_data()\n outputs[\u0027result\u0027] = result # Wrong: missing __OUTPUTS__ pattern\n```\n\n**\u2705 Correct:**\n```yaml\nscript: |\n result = process_data()\n \n # Required: Use outputs dict and __OUTPUTS__ marker\n outputs = {}\n outputs[\u0027result\u0027] = result\n outputs[\u0027status\u0027] = \u0027success\u0027\n \n import json\n print(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```\n\n### 7. AI Agent Configuration Errors\n\n**\u274c Error:**\n```yaml\n- id: ai_task\n type: ai_agent\n config:\n model_client_id: my_model\n prompt: \"Process this data: ${data}\" # Wrong: use user_message\n```\n\n**\u2705 Correct:**\n```yaml\n- id: ai_task\n type: ai_agent\n config:\n model_client_id: my_model\n system_message: \"You are a helpful assistant.\"\n user_message: \"Process this data: ${data}\" # Correct field name\n```\n\n### 8. Missing Dependencies for Template Resolution\n\n**\u274c Error:**\n```yaml\n- id: task_2\n type: script\n # Missing depends_on\n script: |\n data = ${task_1} # Will become UNRESOLVED_task_1\n```\n\n**\u2705 Correct:**\n```yaml\n- id: task_2\n type: script\n depends_on: [task_1] # Required for template access\n script: |\n data = ${task_1} # Now resolves correctly\n```\n\n### 9. Incorrect Model Client Configuration\n\n**\u274c Error:**\n```yaml\nmodel_clients:\n my_model: # Wrong: object mapping format\n type: openai\n api_key: \"sk-...\"\n```\n\n**\u2705 Correct:**\n```yaml\nmodel_clients:\n - id: my_model # Correct: array format with id field\n provider: openai\n config:\n model: \"gpt-4\"\n api_key: \"sk-...\"\n```\n\n### 10. Empty String Literals Causing Python Syntax Errors\n\n**\u274c Error:**\n```yaml\nscript: |\n def process_data(path=\"\"): # Wrong: empty string can cause syntax issues\n current_path = f\"{path}.{key}\" if path else key\n \n # This can lead to: SyntaxError: unterminated string literal\n```\n\n**\u2705 Correct:**\n```yaml\nscript: |\n def process_data(path=None): # Correct: use None instead of empty string\n current_path = f\"{path}.{key}\" if path else key\n # Or use path = path or \"default_value\"\n```\n\n### 11. YAML Indentation and Structure Issues\n\n**\u274c Error:**\n```yaml\ntasks:\n- id: task_1\n type: script\nscript: | # Wrong: inconsistent indentation\n print(\"hello\")\n```\n\n**\u2705 Correct:**\n```yaml\ntasks:\n- id: task_1\n type: script\n script: | # Correct: consistent indentation\n print(\"hello\")\n```\n\n### Best Practices Summary\n\n1. **Always include required `id` fields** for workflows and tasks\n2. **Use array format** for `tasks`, `inputs`, and `model_clients`\n3. **Script field goes directly under task**, not under `config`\n4. **No individual task inputs** - use template resolution instead\n5. **Use `depends_on`** to access previous task outputs\n6. **Print `__OUTPUTS__`** in scripts for proper output parsing\n7. **Use `user_message`** (not `prompt`) for AI agent tasks\n8. **Avoid empty string literals** - use `None` or meaningful defaults\n9. **Follow consistent YAML indentation** throughout the file\n\n## MCP Tools and Function Calling\n\nAI agents and chatbots can use MCP (Model Context Protocol) tools for function calling, enabling them to execute external tools and services.\n\n### Tools Configuration\n\nBoth `ai_agent` and `chatbot` tasks support tools configuration:\n\n```yaml\n# AI Agent with MCP tools\n- id: ai_with_tools\n type: ai_agent\n config:\n model_client_id: gpt4_model\n system_message: |\n You are a helpful assistant with access to various tools.\n Use the available tools when needed to help users.\n user_message: \"What time is it and what\u0027s the weather like?\"\n \n # Tools configuration\n tools: # List of MCP tool names\n - current_time # Time tools\n - relative_time\n - days_in_month\n - weather_forecast # Weather tools\n - database_query # Database tools\n - send_notification # Communication tools\n \n tenant_id: \"550e8400-e29b-41d4-a716-446655440000\" # Required for tool resolution\n\n# Chatbot with MCP tools\n- id: chatbot_with_tools\n type: chatbot\n config:\n model_client_id: gpt4_model\n system_message: |\n You are a helpful chatbot with access to tools.\n Use tools when users ask for specific information or actions.\n \n # Tools configuration (same format as AI agent)\n tools:\n - current_time\n - weather_api\n - user_preferences\n \n tenant_id: \"550e8400-e29b-41d4-a716-446655440000\" # Required for tool resolution\n```\n\n### How Tools Work\n\n1. **Tool Resolution**: The workflow engine resolves tool names to actual MCP server deployments using the `tenant_id`\n2. **Function Calling**: When the AI model decides to use a tool, it calls the `tool_executor` function\n3. **Tool Execution**: The engine executes the specified tool via the appropriate MCP server\n4. **Response Integration**: Tool results are fed back to the AI model for response generation\n\n### Tool Configuration Requirements\n\n| Field | Required | Description |\n|-------|----------|-------------|\n| `tools` | No | List of MCP tool names available to the AI |\n| `tenant_id` | **Yes** (when using tools) | Tenant UUID for tool resolution |\n\n### Example: Complete AI Agent with Tools\n\n```yaml\nid: intelligent_assistant\nname: \"Intelligent Assistant with Tools\"\n\nmodel_clients:\n - id: gpt4_model\n provider: openai\n config:\n model: gpt-4o-mini\n api_key: \"${OPENAI_API_KEY}\"\n temperature: 0.3\n\n# Tenant ID can be set at workflow level for all tasks\ntenant_id: \"0572fa8d-d7c3-47db-9219-ef40b80d42b7\"\n\ntasks:\n - id: assistant_with_tools\n type: ai_agent\n config:\n model_client_id: gpt4_model\n system_message: |\n You are a helpful assistant with access to various tools:\n - Time-related tools: current_time, relative_time, days_in_month\n - Weather tools: weather_forecast\n - Communication: send_notification\n \n Use these tools when users ask for:\n - Current time or date information\n - Weather forecasts\n - Sending notifications\n \n Always explain what tool you\u0027re using and why.\n \n user_message: \"${user_query}\"\n \n # MCP tools configuration\n tools:\n - current_time\n - relative_time\n - days_in_month\n - weather_forecast\n - send_notification\n \n # Tool resolution requires tenant_id\n tenant_id: \"0572fa8d-d7c3-47db-9219-ef40b80d42b7\"\n \n temperature: 0.7\n max_tokens: 1000\n\ninputs:\n - name: user_query\n type: string\n required: true\n default: \"What time is it and can you send a notification to user123?\"\n\noutputs:\n ai_response:\n type: string\n source: assistant_with_tools.ai_response\n description: \"AI response using tools when needed\"\n```\n\n### Tool Execution Flow\n\n1. **User Input**: User provides a query that might require tool usage\n2. **AI Analysis**: AI model analyzes the query and decides if tools are needed\n3. **Tool Calling**: AI calls `tool_executor(deployment_id, tool_name, arguments)`\n4. **Tool Resolution**: Engine resolves tool name to MCP server using tenant_id\n5. **Tool Execution**: MCP server executes the requested tool\n6. **Result Processing**: Tool results are returned to the AI model\n7. **Response Generation**: AI generates final response incorporating tool results\n\n### Best Practices for Tools\n\n1. **Clear System Messages**: Explain available tools and when to use them\n2. **Tool Categories**: Group related tools (time, weather, database, etc.)\n3. **Error Handling**: AI models handle tool execution failures gracefully\n4. **Tool Documentation**: Include tool descriptions in system messages\n5. **Tenant Configuration**: Ensure tenant_id matches your MCP server setup\n\n---\n\n*This comprehensive reference covers all supported task types and their YAML configurations. Use this as a quick reference when building workflows with different task types.*",
"format": "markdown"
},
"execution_details": {
"actual_result": {
"output": "Trying to fetch documentation from: http://host.docker.internal:5000/api/docs/yaml-reference\n\u274c Connection failed: http://host.docker.internal:5000/api/docs/yaml-reference - HTTPConnectionPool(host=\u0027host.docker.internal\u0027, port=5000): Max retries exceeded with url: /api/docs/yaml-reference (Caused by NameResolutionError(\"\u003curllib3.connection.HTTPConnection object at 0x7d0629aa9010\u003e: Failed to resolve \u0027host.docker.internal\u0027 ([Errno -2] Name or service not known)\"))\nTrying to fetch documentation from: http://172.17.0.1:5000/api/docs/yaml-reference\n\u2705 Successfully connected to: http://172.17.0.1:5000/api/docs/yaml-reference\nSuccessfully fetched documentation from http://172.17.0.1:5000/api/docs/yaml-reference: 97047 characters\n__OUTPUTS__ {\"documentation\": {\"content\": \"# YAML Workflow Structure Reference\\n\\nComplete reference for the YAML workflow definition structure supported by the Assistants Workflow Engine.\\n\\n## Table of Contents\\n\\n1. [Complete Schema Overview](#complete-schema-overview)\\n2. [Required vs Optional Fields](#required-vs-optional-fields)\\n3. [Data Types Reference](#data-types-reference)\\n4. [Field Validation Rules](#field-validation-rules)\\n5. [Parser Behavior](#parser-behavior)\\n6. [Migration Guide](#migration-guide)\\n\\n## Complete Schema Overview\\n\\n```yaml\\n# === WORKFLOW METADATA (Required) ===\\nid: string # REQUIRED: Unique workflow identifier\\nname: string # REQUIRED: Human-readable workflow name\\n\\n# === WORKFLOW METADATA (Optional) ===\\nversion: string | integer # Default: 1\\nnamespace: string # Default: \\\"default\\\"\\ndescription: string # Default: \\\"\\\"\\ntenant_id: string # Default: null\\n\\n# === EXECUTION CONFIGURATION ===\\ntimeout_seconds: integer|string # Default: 3600 (1 hour) - Workflow-level timeout (supports duration strings like \\\"1h\\\", \\\"30m\\\")\\nlabels: object # Default: {}\\nmetadata: object # Default: {}\\n\\n# === DATA FLOW DEFINITIONS ===\\ninputs: # Default: []\\n - name: string # REQUIRED\\n type: string # REQUIRED: string|integer|boolean|object|array\\n description: string # Optional\\n required: boolean # Default: false\\n default: any # Optional default value\\n\\noutputs: object # Default: {}\\n output_name:\\n type: string # REQUIRED\\n source: string # REQUIRED: task_id or task_id.field\\n description: string # Optional\\n\\nvariables: object # Default: {}\\n\\n# === MODEL CLIENT CONFIGURATION (Required for AI tasks) ===\\nmodel_clients: # REQUIRED: For ai_agent, chatbot, continuous_conversation tasks\\n - id: string # REQUIRED: Unique identifier to reference in tasks\\n provider: string # REQUIRED: openai|anthropic|openrouter|groq\\n config: object # REQUIRED: Provider-specific config\\n model: string # REQUIRED: Model name (e.g., gpt-4, claude-3-sonnet-20240229)\\n api_key: string # REQUIRED: API key for the provider\\n temperature: number # Optional: 0.0-2.0, controls randomness\\n max_tokens: integer # Optional: Maximum tokens to generate\\n timeout: integer # Optional: Request timeout in seconds\\n base_url: string # Optional: Custom API base URL\\n\\n# Model Client Examples:\\n# OpenAI Configuration:\\n# model_clients:\\n# - id: gpt4_model\\n# provider: openai\\n# config:\\n# model: gpt-4-turbo-preview\\n# api_key: \\\"sk-your-openai-api-key\\\"\\n# temperature: 0.7\\n# max_tokens: 2000\\n#\\n# Anthropic Configuration:\\n# model_clients:\\n# - id: claude_model\\n# provider: anthropic\\n# config:\\n# model: claude-3-sonnet-20240229\\n# api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n# temperature: 0.7\\n# max_tokens: 1500\\n#\\n# Multiple Models:\\n# model_clients:\\n# - id: gpt4_model\\n# provider: openai\\n# config:\\n# model: gpt-4-turbo-preview\\n# api_key: \\\"sk-your-openai-api-key\\\"\\n# - id: claude_model\\n# provider: anthropic\\n# config:\\n# model: claude-3-sonnet-20240229\\n# api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n\\n# === TASK DEFINITIONS (Required) ===\\ntasks: # REQUIRED: List of tasks\\n - id: string # REQUIRED: Unique task identifier\\n name: string # Optional: Human-readable name\\n type: string # REQUIRED: Task type\\n description: string # Optional\\n \\n # DEPENDENCY CONFIGURATION\\n depends_on: array # Optional: Execution dependencies and template resolution\\n previous_node: string # REQUIRED: Task connectivity and UI hierarchy (essential for proper workflow execution)\\n \\n timeout_seconds: integer|string # Optional: Task-specific timeout (supports duration strings like \\\"5m\\\", \\\"30s\\\")\\n worker_group: string # Optional: Target worker group\\n conditions: array # Optional: Execution conditions\\n execute_on_routes: array # Optional: Routes for conditional execution\\n \\n # Input/Output mapping\\n input_mapping: object # Optional: Map workflow context to task inputs\\n output_mapping: object # Optional: Map task outputs to workflow context\\n \\n # Retry configuration\\n retry: object # Optional: Task-specific retry policy\\n maxAttempts: integer # Default: 1\\n initialDelay: string|integer # Default: \\\"1s\\\"\\n maxDelay: string|integer # Default: \\\"300s\\\"\\n backoffMultiplier: number # Default: 2.0\\n retryOn: array # Default: [\\\"TEMPORARY_FAILURE\\\"]\\n \\n # === TASK TYPE SPECIFIC CONFIGURATIONS ===\\n # See detailed sections below for each task type\\n\\n# === WORKFLOW-LEVEL RETRY POLICY ===\\nretry: object # Optional: Global retry policy\\n retryOn: array # Default: [\\\"TEMPORARY_FAILURE\\\"]\\n maxAttempts: integer # Default: 1\\n initialDelay: string|integer # Default: \\\"1s\\\"\\n maxDelay: string|integer # Default: \\\"300s\\\"\\n backoffMultiplier: number # Default: 2.0\\n\\n# === TRIGGERS ===\\ntriggers: # Optional: List of triggers\\n - name: string # REQUIRED\\n type: string # REQUIRED: webhook|schedule|event\\n description: string # Optional\\n config: object # REQUIRED: Trigger-specific config\\n input_mapping: array # Optional: Map trigger data to workflow inputs\\n - webhook_field: string # Source field\\n workflow_input: string # Target workflow input\\n```\\n\\n## Required vs Optional Fields\\n\\n### Required Fields (Workflow Level)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `id` | string | Unique identifier for the workflow |\\n| `tasks` | array | List of task definitions (can be empty) |\\n\\n### Required Fields (Task Level)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `id` | string | Unique identifier for the task |\\n| `type` | string | Task type (script, http, ai_agent, etc.) |\\n| `previous_node` | string | Reference to previous task for UI connectivity (required for proper task linking, except for the first task) |\\n\\n### Required Fields (Input Definition)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `name` | string | Input parameter name |\\n| `type` | string | Data type (string, integer, boolean, object, array) |\\n\\n### Required Fields (Output Definition)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `type` | string | Output data type |\\n| `source` | string | Source task ID or task_id.field_name |\\n\\n## Data Types Reference\\n\\n### Primitive Types\\n\\n```yaml\\n# String type\\ninputs:\\n - name: customer_name\\n type: string\\n default: \\\"John Doe\\\"\\n description: \\\"Customer full name\\\"\\n\\n# Integer type \\ninputs:\\n - name: batch_size\\n type: integer\\n default: 100\\n description: \\\"Number of records per batch\\\"\\n\\n# Boolean type\\ninputs:\\n - name: include_archived\\n type: boolean\\n default: false\\n description: \\\"Include archived records\\\"\\n```\\n\\n### Complex Types\\n\\n```yaml\\n# Object type\\ninputs:\\n - name: config_params\\n type: object\\n default:\\n timeout: 30\\n retries: 3\\n debug: false\\n description: \\\"Configuration parameters\\\"\\n\\n# Array type\\ninputs:\\n - name: customer_ids\\n type: array\\n default: [\\\"cust_001\\\", \\\"cust_002\\\"]\\n description: \\\"List of customer identifiers\\\"\\n```\\n\\n### Duration Types\\n\\nDuration values can be specified as strings or integers:\\n\\n```yaml\\n# String format (recommended)\\ntimeout_seconds: \\\"1h\\\" # 1 hour\\ntimeout_seconds: \\\"30m\\\" # 30 minutes \\ntimeout_seconds: \\\"45s\\\" # 45 seconds\\ntimeout_seconds: \\\"2d\\\" # 2 days\\n\\n# Integer format (seconds)\\ntimeout_seconds: 3600 # 1 hour in seconds\\n\\n# Supported units\\n# s, sec, second, seconds\\n# m, min, minute, minutes \\n# h, hour, hours\\n# d, day, days\\n```\\n\\n### Timeout Configuration and Execution Behavior\\n\\nThe workflow engine supports configurable timeouts at both workflow and task levels with intelligent timeout-free communication for long-running tasks.\\n\\n#### Timeout Hierarchy\\n\\nTimeout values are resolved in this order of precedence:\\n1. **Task-level `timeout_seconds`**: Specific timeout for individual tasks\\n2. **Workflow-level `timeout_seconds`**: Default for all tasks in the workflow\\n3. **System default**: 300 seconds (5 minutes) for tasks, 3600 seconds (1 hour) for workflows\\n\\n```yaml\\n# Workflow-level timeout (applies to all tasks by default)\\ntimeout_seconds: \\\"2h\\\" # 2 hour workflow timeout\\n\\ntasks:\\n - id: quick_task\\n type: script\\n # Uses workflow-level timeout (2h)\\n script: \\\"print(\u0027Quick task\u0027)\\\"\\n \\n - id: custom_timeout_task\\n type: script\\n timeout_seconds: \\\"30m\\\" # Override with 30 minute timeout\\n script: \\\"print(\u0027Custom timeout task\u0027)\\\"\\n \\n - id: long_running_task\\n type: script\\n timeout_seconds: 600 # 10 minutes (integer format)\\n script: |\\n import time\\n time.sleep(300) # 5 minute processing\\n print(\\\"Long task completed\\\")\\n```\\n\\n#### Timeout-Free Communication\\n\\nThe workflow engine automatically uses **timeout-free communication** for long-running tasks to prevent timeout-related failures:\\n\\n- **Loop tasks**: Always use timeout-free communication (event-driven execution)\\n- **Script tasks**: Use timeout-free communication when `timeout_seconds \u003e 180` (host timeout threshold)\\n- **Other task types**: Use standard timeout-based communication\\n\\n```yaml\\ntasks:\\n - id: short_script\\n type: script\\n timeout_seconds: 60 # Uses standard timeout-based communication\\n script: \\\"print(\u0027Short script\u0027)\\\"\\n \\n - id: long_script\\n type: script\\n timeout_seconds: 600 # Uses timeout-free communication (600 \u003e 180)\\n script: |\\n import time\\n print(\\\"\\ud83d\\ude80 This uses timeout-free communication\\\")\\n time.sleep(300) # Won\u0027t cause timeout issues\\n print(\\\"Long script completed\\\")\\n \\n - id: data_processing_loop\\n type: loop\\n timeout_seconds: \\\"1h\\\" # Always uses timeout-free communication\\n loop_type: for\\n max_iterations: 10\\n # Loop tasks automatically use timeout-free communication\\n```\\n\\n#### Loop Task Timeout Inheritance\\n\\nLoop tasks can pass timeout configuration to their inner script tasks:\\n\\n```yaml\\n- id: batch_processing_loop\\n type: loop\\n timeout_seconds: \\\"30m\\\" # Loop-level timeout\\n loop_type: for\\n max_iterations: 5\\n loop_tasks:\\n - id: process_batch\\n type: script\\n # Inherits 30-minute timeout from parent loop\\n script: |\\n print(\\\"Processing batch with inherited timeout\\\")\\n # This script will use the 30-minute timeout from the loop\\n```\\n\\n#### Best Practices for Timeout Configuration\\n\\n1. **Use duration strings** for readability: `\\\"5m\\\"` vs `300`\\n2. **Set appropriate timeouts** based on expected execution time plus buffer\\n3. **Long-running tasks**: Set `timeout_seconds \u003e 180` to enable timeout-free communication\\n4. **Loop tasks**: Always get timeout-free communication automatically\\n5. **Critical tasks**: Use shorter timeouts for fail-fast behavior\\n6. **External API calls**: Use longer timeouts to account for network latency\\n\\n```yaml\\n# Example: Well-configured timeout workflow\\ntimeout_seconds: \\\"1h\\\" # Reasonable workflow timeout\\n\\ntasks:\\n - id: api_call\\n type: http\\n timeout_seconds: \\\"30s\\\" # Quick timeout for API calls\\n url: \\\"https://api.example.com/data\\\"\\n \\n - id: data_processing\\n type: script\\n timeout_seconds: \\\"10m\\\" # Longer timeout for processing (uses timeout-free)\\n script: |\\n # Process large dataset\\n process_data()\\n \\n - id: batch_processing\\n type: loop\\n timeout_seconds: \\\"45m\\\" # Generous timeout for loop (timeout-free)\\n loop_type: for\\n max_iterations: 100\\n # Inner tasks inherit the 45-minute timeout\\n```\\n\\n## Field Validation Rules\\n\\n### ID Fields\\n\\n```yaml\\n# Valid IDs\\nid: \\\"customer_processor\\\" # \\u2705 Valid\\nid: \\\"process-data\\\" # \\u2705 Valid\\nid: \\\"task_001\\\" # \\u2705 Valid\\n\\n# Invalid IDs\\nid: \\\"process data\\\" # \\u274c Spaces not allowed\\nid: \\\"123_start\\\" # \\u274c Cannot start with number\\nid: \\\"\\\" # \\u274c Cannot be empty\\n```\\n\\n### Timeout Values\\n\\n```yaml\\n# Valid timeouts\\ntimeout_seconds: 300 # \\u2705 Positive integer\\ntimeout_seconds: \\\"5m\\\" # \\u2705 Duration string\\ntimeout_seconds: \\\"1h30m\\\" # \\u2705 Complex duration\\n\\n# Invalid timeouts\\ntimeout_seconds: 0 # \\u274c Must be positive\\ntimeout_seconds: -60 # \\u274c Cannot be negative\\ntimeout_seconds: \\\"invalid\\\" # \\u274c Invalid duration format\\n```\\n\\n### Dependency Types and Validation\\n\\n#### `depends_on` vs `previous_node`\\n\\n**`depends_on` - Execution Dependencies:**\\n- Controls actual task execution order\\n- Enables template resolution and data access\\n- Required for accessing outputs from other tasks\\n- Validates that referenced tasks exist\\n- Multiple tasks can be specified\\n\\n**`previous_node` - Task Connectivity and UI Hierarchy:**\\n- **REQUIRED for proper task connectivity in UI**\\n- Essential for workflow visualization and task linking\\n- Critical for execution flow representation\\n- Single task reference only\\n- **Without this field, tasks won\u0027t be properly connected in the UI**\\n\\n```yaml\\ntasks:\\n - id: fetch_data\\n type: http\\n url: \\\"https://api.example.com/data\\\"\\n \\n - id: validate_data\\n type: script\\n depends_on: [fetch_data] # \\u2705 EXECUTION: wait for fetch_data to complete\\n previous_node: fetch_data # \\u2705 UI: show as child of fetch_data in hierarchy\\n script: |\\n data = ${fetch_data} # \\u2705 Can access because of depends_on\\n if not validate(data):\\n raise ValueError(\\\"Invalid data\\\")\\n \\n # Required: Use outputs dict and __OUTPUTS__ marker for proper parsing\\n outputs[\u0027status\u0027] = \u0027valid\u0027\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: process_data\\n type: script\\n depends_on: [validate_data] # \\u2705 EXECUTION: wait for validation\\n previous_node: validate_data # \\u2705 UI: show as child of validate_data\\n script: |\\n validation_result = ${validate_data} # \\u2705 Accessible via depends_on\\n # Process data...\\n\\n - id: separate_task\\n type: script\\n previous_node: process_data # \\u2705 UI: appears under process_data in hierarchy\\n # NO depends_on means this can run immediately (parallel to other tasks)\\n script: |\\n # This task runs independently but appears under process_data in UI\\n # Cannot access ${process_data} because no depends_on relationship\\n print(\\\"Independent task\\\")\\n\\n### Complete Example with Proper Hierarchy\\n\\n```yaml\\ntasks:\\n - id: fetch_customer_data\\n name: \\\"Fetch Customer Data\\\"\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: validate_customer\\n name: \\\"Validate Customer Data\\\"\\n type: script\\n depends_on: [fetch_customer_data] # \\u2705 EXECUTION: Wait for data fetch\\n previous_node: fetch_customer_data # \\u2705 UI: Show under fetch_customer_data\\n script: |\\n customer = ${fetch_customer_data} # \\u2705 Can access because of depends_on\\n if not customer.get(\u0027email\u0027):\\n raise ValueError(\\\"Invalid customer data\\\")\\n print(\\\"__OUTPUTS__ {\u0027status\u0027: \u0027valid\u0027}\\\")\\n \\n - id: process_customer\\n name: \\\"Process Customer\\\"\\n type: ai_agent\\n depends_on: [validate_customer] # \\u2705 EXECUTION: Wait for validation\\n previous_node: validate_customer # \\u2705 UI: Show under validate_customer\\n config:\\n model_client_id: gpt4_model\\n user_message: \\\"Process customer: ${fetch_customer_data}\\\"\\n \\n - id: send_notification\\n name: \\\"Send Notification\\\"\\n type: script\\n depends_on: [process_customer] # \\u2705 EXECUTION: Wait for processing\\n previous_node: process_customer # \\u2705 UI: Show under process_customer\\n script: |\\n result = ${process_customer}\\n print(f\\\"Notification sent for: ${fetch_customer_data.name}\\\")\\n \\n - id: parallel_audit_log\\n name: \\\"Audit Log (Parallel)\\\"\\n type: script\\n depends_on: [fetch_customer_data] # \\u2705 EXECUTION: Only needs initial data\\n previous_node: fetch_customer_data # \\u2705 UI: Show under fetch_customer_data\\n script: |\\n # This runs in parallel with validation/processing\\n customer = ${fetch_customer_data}\\n print(f\\\"Audit: Customer {customer.id} accessed at ${current_timestamp}\\\")\\n```\\n\\n # Invalid examples\\n - id: invalid_task\\n type: script\\n depends_on: [nonexistent_task] # \\u274c VALIDATION ERROR: task doesn\u0027t exist\\n previous_node: another_missing # \\u26a0\\ufe0f UI WARNING: task doesn\u0027t exist (but not fatal)\\n script: \\\"print(\u0027Invalid\u0027)\\\"\\n```\\n\\n#### Key Differences:\\n\\n| Aspect | `depends_on` | `previous_node` |\\n|--------|--------------|-----------------|\\n| **Purpose** | Execution control | Task connectivity \u0026 UI organization |\\n| **Impact** | Blocks task until dependencies complete | **REQUIRED for UI task linking** |\\n| **Template Access** | \\u2705 Enables `${task_id}` resolution | \\u274c No template access |\\n| **Validation** | \\u274c Parser error if task missing | \\u26a0\\ufe0f Warning if task missing |\\n| **Multiple References** | \\u2705 Array of task IDs | \\u274c Single task ID only |\\n| **Execution Order** | \\u2705 Enforced | \\u274c No execution impact |\\n| **UI Connectivity** | \\u274c No UI impact | \\u2705 **ESSENTIAL for proper task connection** |\\n\\n### Circular Dependency Detection\\n\\n```yaml\\n# This will be rejected by the parser\\ntasks:\\n - id: task_a\\n depends_on: [task_b] # \\u274c Circular dependency\\n \\n - id: task_b \\n depends_on: [task_a] # \\u274c Circular dependency\\n```\\n\\n## Parser Behavior\\n\\n### Field Name Compatibility\\n\\nThe parser supports both camelCase and snake_case for field names:\\n\\n```yaml\\ntasks:\\n - id: flexible_task\\n # Both formats are accepted\\n dependsOn: [other_task] # camelCase\\n depends_on: [other_task] # snake_case\\n \\n inputMapping: # camelCase\\n source_field: \\\"target_field\\\"\\n input_mapping: # snake_case \\n source_field: \\\"target_field\\\"\\n \\n outputMapping: {} # camelCase\\n output_mapping: {} # snake_case\\n \\n workerGroup: \\\"gpu-workers\\\" # camelCase\\n worker_group: \\\"gpu-workers\\\" # snake_case\\n```\\n\\n### Default Value Assignment\\n\\n```yaml\\n# Fields with defaults are automatically populated\\nid: my_workflow\\n# name defaults to id value: \\\"my_workflow\\\"\\n# version defaults to: 1\\n# namespace defaults to: \\\"default\\\" \\n# timeout_seconds defaults to: 3600\\n# description defaults to: \\\"\\\"\\n\\ntasks:\\n - id: simple_task\\n type: script\\n # name defaults to id: \\\"simple_task\\\"\\n # timeout_seconds defaults to: 300\\n # retry.maxAttempts defaults to: 1\\n```\\n\\n### Template Resolution Context\\n\\nThe parser provides these built-in variables for template resolution:\\n\\n```yaml\\ntasks:\\n - id: context_aware_task\\n type: script\\n script: |\\n # Workflow context\\n workflow_id = \\\"${workflow_id}\\\" # Current workflow ID\\n execution_id = \\\"${execution_id}\\\" # Unique execution ID\\n namespace = \\\"${namespace}\\\" # Workflow namespace\\n \\n # Timing context\\n current_timestamp = \\\"${current_timestamp}\\\" # ISO timestamp\\n started_at = \\\"${started_at}\\\" # Workflow start time\\n \\n # Input context\\n user_input = \\\"${input_parameter}\\\" # Workflow inputs\\n \\n # Task context\\n previous_result = ${previous_task} # Previous task output\\n specific_field = ${task_id.field_name} # Specific output field\\n \\n # Route context (for conditional workflows)\\n route_info = ${_route_context} # Current route information\\n```\\n\\n### Error Handling During Parsing\\n\\n```yaml\\n# The parser will validate and report these errors:\\n\\n# 1. Missing required fields\\n# ERROR: Missing required field: \u0027id\u0027\\n\\n# 2. Invalid YAML syntax \\n# ERROR: Invalid YAML: mapping values are not allowed here\\n\\n# 3. Circular dependencies\\n# ERROR: Circular dependency detected involving task \u0027task_a\u0027\\n\\n# 4. Invalid task references\\n# ERROR: Task \u0027task_b\u0027 depends on non-existent task \u0027invalid_task\u0027\\n\\n# 5. Invalid duration formats\\n# ERROR: Invalid duration: \u0027not_a_duration\u0027\\n\\n# 6. Invalid field values\\n# ERROR: Timeout must be positive, got -10\\n```\\n\\n## Migration Guide\\n\\n### From Version 1.0 to 2.0\\n\\n```yaml\\n# OLD FORMAT (v1.0)\\nworkflow:\\n id: my_workflow\\n tasks:\\n - task_id: process_data\\n task_type: python_script\\n \\n# NEW FORMAT (v2.0) \\nid: my_workflow\\ntasks:\\n - id: process_data\\n type: script\\n language: python\\n```\\n\\n### Deprecated Fields\\n\\n| Old Field | New Field | Status |\\n|-----------|-----------|--------|\\n| `task_type` | `type` | Deprecated in v2.0 |\\n| `task_id` | `id` | Deprecated in v2.0 |\\n| `python_script` | `script` (type) | Deprecated in v2.0 |\\n| `plugin` | `mcp` (type) | Deprecated in v2.1 |\\n\\n### Backward Compatibility\\n\\nThe parser maintains backward compatibility for deprecated fields but will show warnings:\\n\\n```yaml\\n# This will work but show a warning\\ntasks:\\n - task_id: old_style_task # \\u26a0\\ufe0f Warning: Use \u0027id\u0027 instead\\n task_type: python_script # \\u26a0\\ufe0f Warning: Use \u0027type: script\u0027 instead\\n```\\n\\n## Advanced Schema Features\\n\\n### Conditional Field Requirements\\n\\nSome fields become required based on task type:\\n\\n```yaml\\n# HTTP tasks require URL\\n- id: api_call\\n type: http\\n url: \\\"https://api.example.com\\\" # \\u2705 Required for HTTP tasks\\n \\n# AI agent tasks require model_client_id\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: gpt4 # \\u2705 Required for AI agent tasks\\n```\\n\\n### Dynamic Field Validation\\n\\nField validation can depend on other field values:\\n\\n```yaml\\n# Loop tasks require loop-specific fields\\n- id: data_loop\\n type: loop\\n loop_type: \\\"for\\\" # Determines required fields\\n max_iterations: 10 # \\u2705 Required for \u0027for\u0027 loops\\n # exit_condition: not needed for \u0027for\u0027 loops\\n \\n- id: condition_loop \\n type: loop\\n loop_type: \\\"while\\\" # Determines required fields\\n exit_condition: \\\"${count} \u003e= 10\\\" # \\u2705 Required for \u0027while\u0027 loops\\n # max_iterations: optional for \u0027while\u0027 loops\\n```\\n\\n## Template Resolution and Data Flow\\n\\nTemplate resolution is a core feature that allows tasks to access data from previous tasks, workflow inputs, and built-in variables using the `${variable_name}` syntax. This is essential for creating dynamic workflows where tasks pass data to each other.\\n\\n### Basic Template Syntax\\n\\n```yaml\\n# Access workflow inputs\\nscript: |\\n customer_name = \\\"${customer_name}\\\" # From workflow inputs\\n \\n# Access previous task outputs\\nscript: |\\n previous_result = ${data_fetch_task} # Entire output from data_fetch_task\\n specific_field = ${data_fetch_task.customer_id} # Specific field\\n \\n# Access built-in variables\\nscript: |\\n workflow_id = \\\"${workflow_id}\\\"\\n execution_id = \\\"${execution_id}\\\"\\n timestamp = \\\"${current_timestamp}\\\"\\n```\\n\\n### Available Built-in Variables\\n\\n| Variable | Description | Example Value |\\n|----------|-------------|---------------|\\n| `${workflow_id}` | Current workflow identifier | `customer_processor` |\\n| `${execution_id}` | Unique execution identifier | `exec_12345_abcde` |\\n| `${namespace}` | Workflow namespace | `production` |\\n| `${current_timestamp}` | ISO timestamp | `2024-01-15T10:30:00Z` |\\n| `${started_at}` | Workflow start time | `2024-01-15T10:29:45Z` |\\n\\n### Task Output Resolution\\n\\nTasks can access outputs from other tasks they depend on:\\n\\n```yaml\\ntasks:\\n - id: fetch_customer\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: process_customer\\n type: script\\n depends_on: [fetch_customer] # Required for template access\\n previous_node: fetch_customer\\n script: |\\n # Access entire response\\n customer_data = ${fetch_customer}\\n \\n # Access specific fields\\n customer_name = \\\"${fetch_customer.name}\\\"\\n customer_email = \\\"${fetch_customer.email}\\\"\\n \\n # Process the data\\n result = {\\n \\\"processed_name\\\": customer_name.upper(),\\n \\\"domain\\\": customer_email.split(\\\"@\\\")[1]\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n```\\n\\n### AI Agent Output Resolution\\n\\nAI agent tasks return structured output that requires specific access patterns:\\n\\n```yaml\\n# AI Agent task\\n- id: ai_analysis_task\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Analyze the customer data and provide insights.\\\"\\n user_message: \\\"Customer data: ${customer_data}\\\"\\n\\n# Accessing AI agent output in subsequent tasks\\n- id: process_ai_response\\n type: script\\n depends_on: [ai_analysis_task]\\n previous_node: ai_analysis_task\\n script: |\\n # CORRECT: AI agents return structured output\\n agent_output = ${ai_analysis_task}\\n ai_response = agent_output[\u0027ai_response\u0027].strip()\\n \\n # Additional metadata available\\n token_usage = agent_output.get(\u0027token_usage\u0027, {})\\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\\n \\n # Process the AI response\\n processed_analysis = {\\n \\\"analysis\\\": ai_response,\\n \\\"tokens_used\\\": token_usage.get(\u0027total_tokens\u0027, 0),\\n \\\"model\\\": model_used,\\n \\\"processed_at\\\": \\\"${current_timestamp}\\\"\\n }\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(processed_analysis)}\\\")\\n\\n# INCORRECT: Direct access won\u0027t work\\n- id: incorrect_ai_access\\n type: script\\n depends_on: [ai_analysis_task]\\n script: |\\n # \\u274c This will not work correctly\\n ai_response = ${ai_analysis_task} # Returns full agent object\\n # ai_response is now a dict, not the actual response text\\n```\\n\\n### AI Agent Response Structure\\n\\nAI agent tasks return a structured object with the following format:\\n\\n```json\\n{\\n \\\"ai_response\\\": \\\"The actual AI-generated text response\\\",\\n \\\"model\\\": \\\"gpt-4-turbo-preview\\\",\\n \\\"token_usage\\\": {\\n \\\"prompt_tokens\\\": 150,\\n \\\"completion_tokens\\\": 75,\\n \\\"total_tokens\\\": 225\\n },\\n \\\"estimated_cost\\\": {\\n \\\"amount\\\": 0.00337,\\n \\\"currency\\\": \\\"USD\\\"\\n },\\n \\\"provider\\\": \\\"openai\\\",\\n \\\"timestamp\\\": \\\"2024-01-15T10:30:00Z\\\"\\n}\\n```\\n\\n### Loop Task Data Access and Dependencies\\n\\nLoop tasks have a special execution context that differs from regular workflow tasks. Understanding how to access data within loops is crucial for building functional workflows.\\n\\n#### Loop Task Internal Data Access\\n\\nWithin loop tasks, data access works through injected context files rather than template resolution:\\n\\n```yaml\\n# Loop task with state variables and internal task dependencies\\n- id: data_processing_loop\\n type: loop\\n loop_type: for\\n max_iterations: 3\\n state_variables:\\n processed_items: []\\n total_processed: 0\\n error_count: 0\\n loop_tasks:\\n # First task in loop\\n - id: fetch_data\\n type: script\\n script: |\\n # Access loop state variables through injected context\\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\\n processed_items = loop_state.get(\u0027processed_items\u0027, [])\\n \\n # Access workflow inputs and external task outputs\\n external_data = inputs.get(\u0027previous_workflow_task\u0027, {})\\n iteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable\\n \\n # Process data\\n batch_data = fetch_batch_data(iteration_num)\\n \\n outputs = {\\\"batch_data\\\": batch_data, \\\"batch_size\\\": len(batch_data)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n # Second task depends on first task in same loop\\n - id: process_data\\n type: script\\n depends_on: [fetch_data] # \\u2705 CORRECT: Dependencies within loop tasks work normally\\n script: |\\n # \\u2705 CORRECT: Access previous loop task through inputs context\\n fetch_result = inputs.get(\u0027fetch_data\u0027, {})\\n batch_data = fetch_result.get(\u0027batch_data\u0027, [])\\n \\n # \\u274c INCORRECT: This won\u0027t work in loop context\\n # batch_data = ${fetch_data.batch_data} # Template resolution fails in loops\\n \\n # Access and update state variables\\n current_processed = loop_state.get(\u0027processed_items\u0027, [])\\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\\n \\n # Process the batch\\n processed_batch = process_batch(batch_data)\\n current_processed.extend(processed_batch)\\n \\n # Update state variables using state_updates\\n state_updates[\u0027processed_items\u0027] = current_processed\\n state_updates[\u0027total_processed\u0027] = current_total + len(processed_batch)\\n \\n outputs = {\\\"processed_count\\\": len(processed_batch)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n print(f\\\"__STATE_UPDATES__ {json.dumps(state_updates)}\\\") # \\u2705 REQUIRED for state updates\\n```\\n\\n#### Key Differences in Loop Task Data Access\\n\\n| Access Pattern | Regular Tasks | Loop Tasks |\\n|---------------|---------------|------------|\\n| **Previous Task Output** | `${previous_task}` | `inputs.get(\u0027field_name\u0027, default)` (flattened) |\\n| **State Variables** | N/A | `loop_state.get(\u0027variable_name\u0027, default)` |\\n| **State Updates** | N/A | `state_updates[\u0027variable_name\u0027] = value` |\\n| **Workflow Inputs** | `\\\"${input_name}\\\"` | `inputs.get(\u0027input_name\u0027, \u0027default\u0027)` |\\n| **Dependencies** | `depends_on: [task_id]` | Same: `depends_on: [task_id]` |\\n\\n#### Critical: Loop Task Output Flattening\\n\\n**Important:** The loop executor flattens task outputs into individual fields in the `inputs` dictionary, rather than storing them as nested objects under the task name.\\n\\n```yaml\\n# First loop task outputs:\\noutputs = {\\\"company_name\\\": \\\"ACME Corp\\\", \\\"company_id\\\": \\\"123\\\", \\\"status\\\": \\\"success\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n\\n# Second loop task accessing the data:\\nscript: |\\n # \\u274c INCORRECT: This won\u0027t work - no nested object\\n # company_data = inputs.get(\u0027first_task\u0027, {})\\n # company_name = company_data.get(\u0027company_name\u0027)\\n \\n # \\u2705 CORRECT: Access flattened fields directly\\n company_name = inputs.get(\u0027company_name\u0027, \u0027\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\\n \\n print(f\\\"Company: {company_name}, ID: {company_id}, Status: {task_status}\\\")\\n```\\n\\n#### Complete Loop Task Example\\n\\n```yaml\\n- id: financial_processing_loop\\n type: loop\\n loop_type: for\\n max_iterations: 3\\n iteration_variable: company_index\\n depends_on: [list_companies] # External dependency\\n \\n state_variables:\\n processed_companies: []\\n total_extractions: 0\\n successful_extractions: 0\\n \\n loop_tasks:\\n - id: process_current_company\\n type: script\\n script: |\\n # Access external workflow task data\\n companies_data = inputs.get(\u0027list_companies\u0027, {})\\n company_idx = inputs.get(\u0027company_index\u0027, 0)\\n \\n # Access state variables\\n processed = loop_state.get(\u0027processed_companies\u0027, [])\\n \\n # Process company\\n company = companies_data[\u0027companies\u0027][company_idx]\\n result = process_company(company)\\n \\n outputs = {\\\"company_name\\\": company[\u0027name\u0027], \\\"status\\\": \\\"success\\\"}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: extract_financial_data\\n type: script\\n depends_on: [process_current_company] # Depend on previous loop task\\n requirements:\\n - requests\u003e=2.28.0\\n - some-ai-library\u003e=1.0.0\\n script: |\\n # \\u2705 CORRECT: Access flattened outputs from previous loop task\\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\\n \\n # \\u274c INCORRECT: Loop executor flattens outputs, no nested access\\n # company_data = inputs.get(\u0027process_current_company\u0027, {})\\n # company_name = company_data.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n \\n # \\u274c INCORRECT: Template resolution doesn\u0027t work in loops\\n # company_data = ${process_current_company}\\n \\n # Access external data\\n companies_list = inputs.get(\u0027list_companies\u0027, {})\\n \\n # Perform AI extraction\\n extraction_result = extract_with_ai({\\\"company_name\\\": company_name, \\\"company_id\\\": company_id})\\n \\n # Update state variables\\n current_processed = loop_state.get(\u0027processed_companies\u0027, [])\\n current_total = loop_state.get(\u0027total_extractions\u0027, 0)\\n current_successful = loop_state.get(\u0027successful_extractions\u0027, 0)\\n \\n current_processed.append({\\n \\\"company_name\\\": company_name,\\n \\\"extraction_result\\\": extraction_result\\n })\\n \\n # State updates are required for persistence\\n state_updates[\u0027processed_companies\u0027] = current_processed\\n state_updates[\u0027total_extractions\u0027] = current_total + 1\\n state_updates[\u0027successful_extractions\u0027] = current_successful + (1 if extraction_result else 0)\\n \\n outputs = {\\\"extracted\\\": bool(extraction_result)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n print(f\\\"__STATE_UPDATES__ {json.dumps(state_updates)}\\\") # \\u2705 REQUIRED\\n\\n# Accessing loop task outputs in subsequent workflow tasks\\n- id: process_loop_results\\n type: script\\n depends_on: [financial_processing_loop]\\n previous_node: financial_processing_loop\\n script: |\\n # \\u2705 CORRECT: Access final state of loop\\n loop_result = ${financial_processing_loop}\\n loop_final_state = loop_result.get(\u0027final_state\u0027, {})\\n \\n # Access specific state variables from final state\\n processed_companies = loop_final_state.get(\u0027processed_companies\u0027, [])\\n total_processed = loop_final_state.get(\u0027total_extractions\u0027, 0)\\n successful_count = loop_final_state.get(\u0027successful_extractions\u0027, 0)\\n \\n # Access loop execution metadata\\n iterations_completed = loop_result.get(\u0027iterations_completed\u0027, 0)\\n \\n print(f\\\"Loop completed {iterations_completed} iterations\\\")\\n print(f\\\"Processed {len(processed_companies)} companies\\\")\\n print(f\\\"Success rate: {successful_count}/{total_processed}\\\")\\n \\n outputs = {\\n \\\"companies_processed\\\": len(processed_companies),\\n \\\"success_rate\\\": f\\\"{successful_count}/{total_processed}\\\",\\n \\\"iterations\\\": iterations_completed\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n#### Loop Task Context Injection and Output Flattening\\n\\nThe loop executor injects context through temporary files and flattens task outputs for easier access:\\n\\n```python\\n# Auto-injected at the start of each loop task script:\\nimport json\\nimport os\\n\\n# Load context from temporary files (auto-generated)\\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\\n inputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\\n\\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\\n loop_state = json.load(f) # Contains current state variables\\n\\noutputs = {}\\nstate_updates = {}\\n\\n# Your script code follows...\\n```\\n\\n#### Debugging Loop Task Data Access\\n\\nWhen debugging loop tasks, always print available keys to understand what data is accessible:\\n\\n```yaml\\n- id: debug_loop_task\\n type: script\\n depends_on: [previous_loop_task]\\n script: |\\n # Debug: Print all available data\\n print(f\\\"\\ud83d\\udd0d Available inputs: {list(inputs.keys())}\\\")\\n print(f\\\"\\ud83d\\udd0d Loop state keys: {list(loop_state.keys())}\\\")\\n \\n # Debug: Print specific values\\n for key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\\n value = inputs.get(key, \u0027NOT_FOUND\u0027)\\n print(f\\\"\\ud83d\\udd0d {key}: {value}\\\")\\n \\n # Now access the flattened data\\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n \\n # Process your data...\\n outputs = {\\\"processed\\\": f\\\"Company {company_name}\\\"}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n#### Real Example: Task Output Flattening\\n\\n```yaml\\n# Task 1 outputs this structure:\\n__OUTPUTS__ {\\\"company_name\\\": \\\"LTTS\\\", \\\"company_id\\\": \\\"123\\\", \\\"fy_folders\\\": [{\\\"id\\\": \\\"456\\\", \\\"name\\\": \\\"FY24\\\"}], \\\"status\\\": \\\"success\\\"}\\n\\n# Task 2 receives flattened inputs:\\ninputs = {\\n \\\"folder_id\\\": \\\"...\\\", # Workflow input\\n \\\"gemini_api_key\\\": \\\"...\\\", # Workflow input \\n \\\"list_company_folders\\\": {...}, # External task output\\n \\\"company_index\\\": 0, # Loop iteration variable\\n \\\"company_name\\\": \\\"LTTS\\\", # \\u2705 Flattened from Task 1\\n \\\"company_id\\\": \\\"123\\\", # \\u2705 Flattened from Task 1\\n \\\"fy_folders\\\": [{\\\"id\\\": \\\"456\\\", \\\"name\\\": \\\"FY24\\\"}], # \\u2705 Flattened from Task 1\\n \\\"status\\\": \\\"success\\\", # \\u2705 Flattened from Task 1\\n \\\"total_extractions\\\": 0, # Loop state variable\\n \\\"processed_companies\\\": [] # Loop state variable\\n}\\n```\\n\\n#### Loop Task Special Output Fields\\n\\n| Field | Description | Example Access |\\n|-------|-------------|----------------|\\n| `final_state` | Final values of all state variables | `${loop_task.final_state}` |\\n| `final_state[\\\"variable_name\\\"]` | Specific state variable from final state | `${loop_task.final_state[\\\"total_processed\\\"]}` |\\n| `iterations_completed` | Number of iterations executed | `${loop_task.iterations_completed}` |\\n\\n#### Safe Template Resolution Pattern for Loop States\\n\\nWhen accessing loop final state fields, use this safe parsing pattern:\\n\\n```yaml\\nscript: |\\n try:\\n # Template resolution with safety checks\\n final_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\\n processed_count_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_count\\\"]}\u0027\u0027\u0027\\n items_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_items\\\"]}\u0027\u0027\u0027\\n iterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027\\n \\n # Safe parsing with fallbacks\\n final_state = json.loads(final_state_str) if not final_state_str.startswith(\u0027UNRESOLVED_\u0027) else {}\\n processed_count = int(processed_count_str) if processed_count_str.isdigit() else 0\\n processed_items = json.loads(items_str) if not items_str.startswith(\u0027UNRESOLVED_\u0027) else []\\n iterations = int(iterations_str) if iterations_str.isdigit() else 0\\n \\n print(f\\\"Successfully processed {processed_count} items in {iterations} iterations\\\")\\n \\n except Exception as e:\\n print(f\\\"Error accessing loop results: {str(e)}\\\")\\n # Handle error case\\n```\\n```\\n\\n### Real-World AI Agent Example\\n\\nBased on actual WhatsApp workflow patterns:\\n\\n```yaml\\n# AI greeting agent\\n- id: greeting_agent\\n type: ai_agent\\n config:\\n model_client_id: assistant_model\\n system_message: \\\"You are a helpful assistant. Generate a friendly greeting.\\\"\\n user_message: \\\"User said: ${user_message}\\\"\\n\\n# Send the AI response via API\\n- id: send_greeting\\n type: script\\n depends_on: [greeting_agent]\\n previous_node: greeting_agent\\n script: |\\n import requests\\n import json\\n \\n # CORRECT: Access AI response from structured output\\n agent_output = ${greeting_agent}\\n ai_greeting = agent_output[\u0027ai_response\u0027].strip()\\n \\n phone_number = \\\"${phone_number}\\\"\\n api_token = \\\"${API_TOKEN}\\\"\\n \\n print(f\\\"\\ud83e\\udd16 AI generated greeting: {ai_greeting}\\\")\\n \\n # Send via API\\n url = f\\\"https://api.example.com/messages/{phone_number}\\\"\\n headers = {\\\"Authorization\\\": f\\\"Bearer {api_token}\\\"}\\n \\n response = requests.post(url, headers=headers, json={\\n \\\"message\\\": ai_greeting\\n })\\n \\n result = {\\n \\\"status\\\": \\\"sent\\\",\\n \\\"ai_response\\\": ai_greeting,\\n \\\"api_response\\\": response.json(),\\n \\\"tokens_used\\\": agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\\n }\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n```\\n\\n### Complex Template Patterns\\n\\n```yaml\\n# Conditional templates with router outputs\\n- id: router_task\\n type: conditional_router\\n conditions:\\n - name: \\\"premium_customer\\\"\\n condition: \\\"${customer_data.tier} == \u0027premium\u0027\\\"\\n route: \\\"premium_processing\\\"\\n\\n- id: premium_handler\\n type: script\\n depends_on: [router_task]\\n previous_node: router_task\\n execute_on_routes: [\\\"premium_processing\\\"]\\n script: |\\n # Access router decision\\n selected_route = \\\"${router_task.selected_route}\\\"\\n route_data = ${router_task.route_data}\\n \\n # Process premium customer\\n customer = ${customer_data}\\n premium_benefits = calculate_benefits(customer)\\n\\n# Loop through arrays from previous tasks\\n- id: process_items\\n type: script\\n depends_on: [fetch_items]\\n previous_node: fetch_items\\n script: |\\n items = ${fetch_items.data}\\n processed_items = []\\n \\n for item in items:\\n processed_item = {\\n \\\"id\\\": item[\\\"id\\\"],\\n \\\"processed_at\\\": \\\"${current_timestamp}\\\",\\n \\\"status\\\": \\\"processed\\\"\\n }\\n processed_items.append(processed_item)\\n \\n print(f\\\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\\\")\\n\\n# Nested object access\\n- id: user_preferences\\n type: script\\n depends_on: [fetch_user]\\n previous_node: fetch_user\\n script: |\\n # Access nested user preferences\\n email_notifications = ${fetch_user.preferences.notifications.email}\\n theme_setting = \\\"${fetch_user.preferences.ui.theme}\\\"\\n language = \\\"${fetch_user.profile.language}\\\"\\n \\n settings = {\\n \\\"notifications\\\": email_notifications,\\n \\\"theme\\\": theme_setting,\\n \\\"locale\\\": language\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(settings)}\\\")\\n```\\n\\n### Template Resolution Rules\\n\\n1. **Dependency Requirement**: Tasks must have `depends_on` relationship to access other task outputs\\n2. **String vs Object**: Use quotes for string values, no quotes for objects/arrays\\n3. **Field Navigation**: Use dot notation for nested field access\\n4. **Error Handling**: Unresolved templates become `UNRESOLVED_variable_name`\\n\\n```yaml\\n# \\u2705 Correct template usage\\n- id: correct_task\\n depends_on: [previous_task]\\n previous_node: previous_task\\n script: |\\n string_value = \\\"${previous_task.name}\\\" # String with quotes\\n object_value = ${previous_task.data} # Object without quotes\\n nested_value = \\\"${previous_task.user.email}\\\" # Nested field\\n\\n# \\u274c Incorrect template usage \\n- id: incorrect_task\\n # Missing depends_on - template won\u0027t resolve\\n script: |\\n value = ${previous_task.name} # Will become UNRESOLVED_previous_task.name\\n```\\n\\n## Script Task Output Handling\\n\\nScript tasks must use the `__OUTPUTS__` marker pattern for proper output parsing by the workflow engine.\\n\\n### Required Output Pattern\\n\\n```yaml\\ntasks:\\n - id: example_script\\n type: script\\n script: |\\n # Your script logic here\\n data = process_data()\\n \\n # Required: Store outputs in outputs dict\\n outputs = {}\\n outputs[\u0027result\u0027] = data\\n outputs[\u0027status\u0027] = \u0027success\u0027\\n outputs[\u0027count\u0027] = len(data)\\n \\n # Required: Print with __OUTPUTS__ marker for engine parsing\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### Output Access in Subsequent Tasks\\n\\n```yaml\\ntasks:\\n - id: data_processor\\n type: script\\n script: |\\n # Process and store outputs\\n results = {\\\"processed_count\\\": 100, \\\"errors\\\": []}\\n \\n outputs = {}\\n outputs[\u0027results\u0027] = results\\n outputs[\u0027status\u0027] = \u0027completed\u0027\\n \\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: result_handler\\n type: script\\n depends_on: [data_processor]\\n script: |\\n # Access previous task outputs\\n processor_output = ${data_processor}\\n processed_count = processor_output[\u0027results\u0027][\u0027processed_count\u0027]\\n status = processor_output[\u0027status\u0027]\\n \\n print(f\\\"Previous task processed {processed_count} items with status: {status}\\\")\\n \\n # Store new outputs\\n outputs = {\u0027message\u0027: f\u0027Handled {processed_count} results\u0027}\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### Error Handling in Scripts\\n\\n```yaml\\ntasks:\\n - id: safe_script\\n type: script\\n script: |\\n try:\\n result = risky_operation()\\n outputs = {\u0027result\u0027: result, \u0027status\u0027: \u0027success\u0027}\\n except Exception as e:\\n outputs = {\u0027error\u0027: str(e), \u0027status\u0027: \u0027failed\u0027}\\n \\n # Always print outputs, even for errors\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n## Triggers and Workflow Activation\\n\\nTriggers define how workflows are started automatically based on external events. They enable event-driven workflow execution.\\n\\n### Webhook Triggers\\n\\nWebhook triggers allow external systems to start workflows by sending HTTP requests.\\n\\n```yaml\\ntriggers:\\n - name: customer_webhook\\n type: webhook\\n description: \\\"Triggered when customer data is updated\\\"\\n config:\\n path: \\\"/webhooks/customer-updated\\\" # Webhook endpoint path\\n method: POST # HTTP method (GET, POST, PUT, etc.)\\n authentication:\\n type: token # none|token|signature|basic\\n token: \\\"${WEBHOOK_SECRET}\\\" # Required if type is token\\n header_name: \\\"X-Webhook-Token\\\" # Optional: custom header name\\n validation:\\n required_fields: [\\\"customer.id\\\", \\\"customer.email\\\"] # Required payload fields\\n content_type: \\\"application/json\\\" # Expected content type\\n input_mapping:\\n - webhook_field: customer.id # Field from webhook payload\\n workflow_input: customer_id # Map to workflow input\\n - webhook_field: customer.email\\n workflow_input: customer_email\\n - webhook_field: customer.updated_at\\n workflow_input: update_timestamp\\n```\\n\\n### Scheduled Triggers\\n\\nSchedule workflows to run at specific times or intervals.\\n\\n```yaml\\ntriggers:\\n - name: daily_report\\n type: schedule\\n description: \\\"Generate daily customer report\\\"\\n config:\\n cron: \\\"0 9 * * *\\\" # Daily at 9 AM (cron format)\\n timezone: \\\"UTC\\\" # Optional: timezone\\n enabled: true # Optional: enable/disable\\n max_instances: 1 # Optional: max concurrent instances\\n input_mapping:\\n - workflow_input: report_date\\n default_value: \\\"${current_date}\\\" # Use built-in variable\\n - workflow_input: report_type\\n default_value: \\\"daily\\\"\\n\\n - name: weekly_cleanup\\n type: schedule\\n description: \\\"Weekly database cleanup\\\"\\n config:\\n cron: \\\"0 2 * * 0\\\" # Weekly on Sunday at 2 AM\\n timezone: \\\"America/New_York\\\"\\n input_mapping:\\n - workflow_input: cleanup_before_date\\n default_value: \\\"${date_minus_7_days}\\\"\\n```\\n\\n### Event Triggers\\n\\nEvent triggers respond to system events from various sources.\\n\\n```yaml\\ntriggers:\\n - name: file_uploaded\\n type: event\\n description: \\\"Process uploaded files\\\"\\n config:\\n event_type: \\\"file.uploaded\\\" # Event type to listen for\\n source: \\\"storage_service\\\" # Event source system\\n filter: # Optional: event filtering\\n file_type: [\\\"csv\\\", \\\"json\\\", \\\"xlsx\\\"]\\n file_size_max: \\\"100MB\\\"\\n path_pattern: \\\"/uploads/customers/*\\\"\\n retry: # Optional: retry configuration\\n max_attempts: 3\\n initial_delay: \\\"5s\\\"\\n input_mapping:\\n - webhook_field: file.path\\n workflow_input: file_path\\n - webhook_field: file.size\\n workflow_input: file_size\\n - webhook_field: file.type\\n workflow_input: file_type\\n\\n - name: order_status_changed\\n type: event\\n description: \\\"Handle order status changes\\\"\\n config:\\n event_type: \\\"order.status_changed\\\"\\n source: \\\"order_management\\\"\\n filter:\\n status: [\\\"completed\\\", \\\"cancelled\\\", \\\"refunded\\\"]\\n priority: [\\\"high\\\", \\\"urgent\\\"]\\n input_mapping:\\n - webhook_field: order.id\\n workflow_input: order_id\\n - webhook_field: order.status\\n workflow_input: new_status\\n - webhook_field: order.customer_id\\n workflow_input: customer_id\\n```\\n\\n### Dependency Triggers\\n\\nDependency triggers start workflows when other workflows complete.\\n\\n```yaml\\ntriggers:\\n - name: post_processing\\n type: dependency\\n description: \\\"Run after main processing completes\\\"\\n config:\\n depends_on_workflow: \\\"main_data_processing\\\" # Workflow to depend on\\n depends_on_status: \\\"SUCCESS\\\" # SUCCESS|FAILED|COMPLETED\\n namespace: \\\"production\\\" # Optional: workflow namespace\\n timeout_minutes: 60 # Optional: wait timeout\\n input_mapping:\\n - webhook_field: execution_id # From completed workflow\\n workflow_input: source_execution_id\\n - webhook_field: workflow_outputs.processed_count\\n workflow_input: items_processed\\n```\\n\\n## Model Client Configuration\\n\\nBefore defining AI-related tasks (ai_agent, chatbot, continuous_conversation), you must configure model clients. Model clients define which AI models to use and how to authenticate with them.\\n\\n### Basic Structure\\n\\n```yaml\\nmodel_clients:\\n - id: unique_identifier # Used to reference this model in tasks\\n provider: provider_name # AI provider (openai, anthropic, etc.)\\n config:\\n model: model_name # Specific model to use\\n api_key: your_api_key # Authentication key\\n # Additional provider-specific options...\\n```\\n\\n### Supported Providers\\n\\n#### OpenAI Provider\\n\\n```yaml\\nmodel_clients:\\n - id: gpt4_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview # Available: gpt-4, gpt-4-turbo, gpt-3.5-turbo\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 0.7 # Optional: 0.0-2.0\\n max_tokens: 2000 # Optional: Maximum response tokens\\n timeout: 30 # Optional: Request timeout (seconds)\\n```\\n\\n#### Anthropic Provider\\n\\n```yaml\\nmodel_clients:\\n - id: claude_model\\n provider: anthropic\\n config:\\n model: claude-3-sonnet-20240229 # Available: claude-3-opus, claude-3-sonnet, claude-3-haiku\\n api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n temperature: 0.7 # Optional: 0.0-1.0\\n max_tokens: 1500 # Optional: Maximum response tokens\\n timeout: 30 # Optional: Request timeout (seconds)\\n```\\n\\n#### OpenRouter Provider\\n\\n```yaml\\nmodel_clients:\\n - id: openrouter_model\\n provider: openrouter\\n config:\\n model: anthropic/claude-3-sonnet # Available: Various models via OpenRouter\\n api_key: \\\"sk-or-your-openrouter-api-key\\\"\\n base_url: \\\"https://openrouter.ai/api/v1\\\" # Optional: Custom endpoint\\n temperature: 0.7\\n max_tokens: 2000\\n```\\n\\n#### Multiple Model Clients\\n\\nYou can define multiple model clients and reference them by their `id` in tasks:\\n\\n```yaml\\nmodel_clients:\\n - id: fast_model\\n provider: openai\\n config:\\n model: gpt-3.5-turbo\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 0.3\\n \\n - id: smart_model\\n provider: anthropic\\n config:\\n model: claude-3-opus-20240229\\n api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n temperature: 0.7\\n \\n - id: creative_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 1.2\\n```\\n\\n### Using Model Clients in Tasks\\n\\nReference model clients in AI tasks using the `model_client_id` field:\\n\\n```yaml\\ntasks:\\n - id: analysis_task\\n type: ai_agent\\n config:\\n model_client_id: smart_model # References the claude model above\\n system_message: \\\"You are a data analyst...\\\"\\n user_message: \\\"Analyze this data: ${input_data}\\\"\\n \\n - id: creative_task\\n type: ai_agent\\n config:\\n model_client_id: creative_model # References the creative GPT-4 model\\n system_message: \\\"You are a creative writer...\\\"\\n user_message: \\\"Write a story about ${topic}\\\"\\n```\\n\\n### Environment Variables for API Keys\\n\\nFor security, use environment variables instead of hardcoding API keys:\\n\\n```yaml\\nmodel_clients:\\n - id: secure_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"${OPENAI_API_KEY}\\\" # References environment variable\\n temperature: 0.7\\n```\\n\\n### Common Configuration Options\\n\\n| Field | Description | Required | Default |\\n|-------|-------------|----------|---------|\\n| `id` | Unique identifier for referencing in tasks | Yes | - |\\n| `provider` | AI provider name | Yes | - |\\n| `config.model` | Specific model name | Yes | - |\\n| `config.api_key` | Authentication API key | Yes | - |\\n| `config.temperature` | Response randomness (0.0-2.0) | No | 0.7 |\\n| `config.max_tokens` | Maximum response tokens | No | Provider default |\\n| `config.timeout` | Request timeout in seconds | No | 30 |\\n| `config.base_url` | Custom API endpoint | No | Provider default |\\n\\n## Secrets Management and Environment Variables\\n\\nProper secrets management is crucial for security. Never hardcode sensitive information like API keys, passwords, or tokens directly in workflow YAML files.\\n\\n### Using Environment Variables\\n\\nThe recommended approach is to use environment variables for all sensitive data:\\n\\n```yaml\\n# \\u2705 RECOMMENDED: Use environment variables\\nmodel_clients:\\n - id: secure_openai\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"${OPENAI_API_KEY}\\\" # References environment variable\\n\\n# HTTP tasks with environment variables\\ntasks:\\n - id: secure_api_call\\n type: http\\n url: \\\"${API_BASE_URL}/customers\\\" # Environment variable for URL\\n headers:\\n Authorization: \\\"Bearer ${API_TOKEN}\\\" # Environment variable for token\\n X-API-Key: \\\"${EXTERNAL_API_KEY}\\\" # Environment variable for API key\\n```\\n\\n### Environment Variable Patterns\\n\\n```yaml\\n# Database connections\\n- id: database_task\\n type: storage\\n config:\\n storage_type: database\\n connection_string: \\\"${DATABASE_URL}\\\" # Full connection string\\n # or individual components:\\n host: \\\"${DB_HOST}\\\"\\n port: \\\"${DB_PORT}\\\"\\n database: \\\"${DB_NAME}\\\"\\n username: \\\"${DB_USER}\\\"\\n password: \\\"${DB_PASSWORD}\\\"\\n\\n# Third-party service credentials\\n- id: external_service\\n type: http\\n url: \\\"${SERVICE_BASE_URL}/api/v1/data\\\"\\n headers:\\n Authorization: \\\"Bearer ${SERVICE_TOKEN}\\\"\\n X-Client-ID: \\\"${SERVICE_CLIENT_ID}\\\"\\n X-Client-Secret: \\\"${SERVICE_CLIENT_SECRET}\\\"\\n\\n# MCP server credentials\\n- id: mcp_task\\n type: mcp\\n mcp_server: \\\"@agentic/mcp-notion-server\\\"\\n environment_vars:\\n NOTION_TOKEN: \\\"${NOTION_API_TOKEN}\\\"\\n WORKSPACE_ID: \\\"${NOTION_WORKSPACE_ID}\\\"\\n```\\n\\n### Security Best Practices\\n\\n#### \\u274c What NOT to Do\\n\\n```yaml\\n# DON\u0027T: Hardcode secrets\\nmodel_clients:\\n - id: insecure_example\\n provider: openai\\n config:\\n api_key: \\\"sk-actual-api-key-here\\\" # SECURITY RISK!\\n\\n# DON\u0027T: Put secrets in comments\\ntasks:\\n - id: bad_example\\n type: http\\n url: \\\"https://api.example.com\\\"\\n # API key is: sk-12345... DON\u0027T DO THIS!\\n headers:\\n Authorization: \\\"Bearer sk-real-token\\\" # SECURITY RISK!\\n```\\n\\n#### \\u2705 What TO Do\\n\\n```yaml\\n# DO: Use environment variables\\nmodel_clients:\\n - id: secure_example\\n provider: openai\\n config:\\n api_key: \\\"${OPENAI_API_KEY}\\\" # Secure approach\\n\\n# DO: Use different variables for different environments\\n- id: environment_aware_task\\n type: http\\n url: \\\"${API_BASE_URL}\\\" # Different URLs per environment\\n headers:\\n Authorization: \\\"Bearer ${API_TOKEN}\\\" # Different tokens per environment\\n X-Environment: \\\"${ENVIRONMENT_NAME}\\\" # Track environment\\n\\n# DO: Use descriptive variable names\\nenvironment_vars:\\n CUSTOMER_API_TOKEN: \\\"${CUSTOMER_SERVICE_TOKEN}\\\"\\n BILLING_API_KEY: \\\"${BILLING_SERVICE_API_KEY}\\\"\\n NOTIFICATION_WEBHOOK_SECRET: \\\"${NOTIFICATION_SECRET}\\\"\\n```\\n\\n### Environment Variable Naming Conventions\\n\\n| Pattern | Example | Use Case |\\n|---------|---------|----------|\\n| `SERVICE_API_KEY` | `OPENAI_API_KEY` | Service API keys |\\n| `SERVICE_BASE_URL` | `CUSTOMER_API_BASE_URL` | Service endpoints |\\n| `DB_*` | `DB_HOST`, `DB_PASSWORD` | Database credentials |\\n| `WEBHOOK_*` | `WEBHOOK_SECRET_TOKEN` | Webhook authentication |\\n| `*_TIMEOUT` | `HTTP_REQUEST_TIMEOUT` | Timeout configurations |\\n\\n### Dynamic Environment Variables\\n\\nUse template resolution to create dynamic environment variable names:\\n\\n```yaml\\n# Different API keys per customer\\n- id: customer_specific_api\\n type: http\\n url: \\\"https://api.partner.com/data\\\"\\n headers:\\n Authorization: \\\"Bearer ${PARTNER_API_KEY_${customer_tier}}\\\" # Resolves to PARTNER_API_KEY_PREMIUM\\n\\n# Environment-specific configurations\\n- id: env_specific_task\\n type: storage\\n config:\\n connection_string: \\\"${DATABASE_URL_${environment}}\\\" # DATABASE_URL_PRODUCTION\\n```\\n\\n### Secrets in Different Task Types\\n\\n```yaml\\n# AI Agent with multiple secure credentials\\n- id: secure_ai_task\\n type: ai_agent\\n config:\\n model_client_id: secure_model\\n system_message: \\\"You are a secure assistant with access to customer data.\\\"\\n user_message: \\\"Process customer ${customer_id}\\\"\\n environment_vars:\\n CUSTOMER_DB_TOKEN: \\\"${CUSTOMER_DATABASE_TOKEN}\\\"\\n AUDIT_LOG_KEY: \\\"${AUDIT_LOGGING_API_KEY}\\\"\\n\\n# Script task with secure file access\\n- id: secure_script_task\\n type: script\\n environment_vars:\\n AWS_ACCESS_KEY_ID: \\\"${AWS_ACCESS_KEY}\\\"\\n AWS_SECRET_ACCESS_KEY: \\\"${AWS_SECRET_KEY}\\\"\\n S3_BUCKET_NAME: \\\"${SECURE_BUCKET_NAME}\\\"\\n script: |\\n import boto3\\n import os\\n \\n # Credentials automatically available from environment\\n s3_client = boto3.client(\u0027s3\u0027)\\n bucket = os.environ[\u0027S3_BUCKET_NAME\u0027]\\n \\n # Process secure files...\\n\\n# Storage task with encrypted connections\\n- id: secure_storage_task\\n type: storage\\n config:\\n storage_type: database\\n connection_string: \\\"${ENCRYPTED_DB_CONNECTION_STRING}\\\"\\n ssl_mode: \\\"require\\\"\\n ssl_cert: \\\"${SSL_CERT_PATH}\\\"\\n ssl_key: \\\"${SSL_KEY_PATH}\\\"\\n```\\n\\n### Environment Setup Examples\\n\\n#### Development Environment (.env file)\\n```bash\\n# Development API keys (non-production)\\nOPENAI_API_KEY=sk-dev-api-key-here\\nCUSTOMER_API_BASE_URL=https://api-dev.example.com\\nDATABASE_URL=postgresql://dev:pass@localhost:5432/workflow_dev\\nENVIRONMENT_NAME=development\\n```\\n\\n#### Production Environment\\n```bash\\n# Production API keys (secure)\\nOPENAI_API_KEY=sk-prod-api-key-here\\nCUSTOMER_API_BASE_URL=https://api.example.com\\nDATABASE_URL=postgresql://prod_user:secure_pass@prod-db:5432/workflow_prod\\nENVIRONMENT_NAME=production\\n```\\n\\n## Workflow Execution and Orchestration\\n\\nUnderstanding how workflows execute and how tasks are orchestrated is essential for building reliable, efficient workflows.\\n\\n### Execution Flow\\n\\nWorkflows follow a directed acyclic graph (DAG) execution model:\\n\\n1. **Initialization**: Workflow starts with inputs and model clients\\n2. **Task Scheduling**: Tasks are scheduled based on dependencies\\n3. **Parallel Execution**: Independent tasks run simultaneously\\n4. **Data Flow**: Outputs flow between tasks via template resolution\\n5. **Completion**: Workflow completes when all tasks finish\\n\\n```yaml\\n# Example execution flow\\ntasks:\\n # Stage 1: Independent tasks (run in parallel)\\n - id: fetch_customer # Runs immediately\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: fetch_products # Runs in parallel with fetch_customer\\n type: http\\n url: \\\"https://api.example.com/products\\\"\\n \\n # Stage 2: Dependent tasks (wait for Stage 1)\\n - id: validate_customer # Waits for fetch_customer\\n type: script\\n depends_on: [fetch_customer]\\n previous_node: fetch_customer\\n script: # validation logic...\\n \\n - id: filter_products # Waits for fetch_products\\n type: script\\n depends_on: [fetch_products]\\n previous_node: fetch_products\\n script: # filtering logic...\\n \\n # Stage 3: Final processing (waits for Stage 2)\\n - id: generate_recommendations # Waits for both validation and filtering\\n type: ai_agent\\n depends_on: [validate_customer, filter_products]\\n previous_node: validate_customer # UI hierarchy under customer validation\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Generate product recommendations for customer.\\\"\\n user_message: |\\n Customer: ${validate_customer}\\n Available products: ${filter_products}\\n```\\n\\n### Task States and Lifecycle\\n\\nEach task goes through these states during execution:\\n\\n| State | Description | Next States |\\n|-------|-------------|-------------|\\n| `PENDING` | Task created, waiting for dependencies | `RUNNING`, `SKIPPED` |\\n| `RUNNING` | Task is actively executing | `COMPLETED`, `FAILED` |\\n| `COMPLETED` | Task finished successfully | - |\\n| `FAILED` | Task encountered an error | `RUNNING` (if retries) |\\n| `SKIPPED` | Task skipped due to conditions | - |\\n| `TIMEOUT` | Task exceeded timeout limit | `RUNNING` (if retries) |\\n\\n### Conditional Execution\\n\\nTasks can be executed conditionally based on routes or conditions:\\n\\n```yaml\\n# Router-based conditional execution\\n- id: data_router\\n type: conditional_router\\n depends_on: [fetch_data]\\n previous_node: fetch_data\\n conditions:\\n - name: \\\"large_dataset\\\"\\n condition: \\\"${fetch_data.record_count} \u003e 10000\\\"\\n route: \\\"batch_processing\\\"\\n - name: \\\"small_dataset\\\"\\n condition: \\\"${fetch_data.record_count} \u003c= 1000\\\"\\n route: \\\"direct_processing\\\"\\n default_route: \\\"standard_processing\\\"\\n\\n# Tasks that execute on specific routes\\n- id: batch_processor\\n type: script\\n depends_on: [data_router]\\n previous_node: data_router\\n execute_on_routes: [\\\"batch_processing\\\"] # Only runs for large datasets\\n script: |\\n # Handle large dataset in batches\\n data = ${fetch_data}\\n process_in_batches(data, batch_size=1000)\\n\\n- id: direct_processor\\n type: script\\n depends_on: [data_router]\\n previous_node: data_router\\n execute_on_routes: [\\\"direct_processing\\\"] # Only runs for small datasets\\n script: |\\n # Process small dataset directly\\n data = ${fetch_data}\\n process_directly(data)\\n```\\n\\n### Error Handling and Retries\\n\\nConfigure retry policies for resilient workflow execution:\\n\\n```yaml\\n# Workflow-level retry policy\\nretry:\\n retryOn: [\\\"TEMPORARY_FAILURE\\\", \\\"TIMEOUT\\\", \\\"NETWORK_ERROR\\\"]\\n maxAttempts: 3\\n initialDelay: \\\"1s\\\"\\n maxDelay: \\\"60s\\\"\\n backoffMultiplier: 2.0\\n\\ntasks:\\n # Task-specific retry policy (overrides workflow-level)\\n - id: external_api_call\\n type: http\\n url: \\\"https://unreliable-api.example.com/data\\\"\\n retry:\\n maxAttempts: 5 # More retries for unreliable API\\n initialDelay: \\\"2s\\\"\\n maxDelay: \\\"120s\\\"\\n backoffMultiplier: 2.0\\n retryOn: [\\\"TIMEOUT\\\", \\\"HTTP_5XX\\\", \\\"NETWORK_ERROR\\\"]\\n \\n # Task with no retries\\n - id: critical_operation\\n type: script\\n retry:\\n maxAttempts: 1 # No retries for critical operations\\n script: |\\n # This operation should not be retried if it fails\\n perform_critical_database_operation()\\n```\\n\\n### Parallel Execution Patterns\\n\\n```yaml\\n# Fan-out pattern: One task triggers multiple parallel tasks\\n- id: data_source\\n type: http\\n url: \\\"https://api.example.com/data\\\"\\n\\n- id: process_customers # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process customer data from data_source\\n\\n- id: process_orders # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process order data from data_source\\n\\n- id: process_products # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process product data from data_source\\n\\n# Fan-in pattern: Multiple tasks feed into one task\\n- id: generate_report # Waits for all parallel tasks\\n type: ai_agent\\n depends_on: [process_customers, process_orders, process_products]\\n previous_node: process_customers # UI hierarchy\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Generate comprehensive report from processed data.\\\"\\n user_message: |\\n Customer data: ${process_customers}\\n Order data: ${process_orders}\\n Product data: ${process_products}\\n```\\n\\n### Resource Management\\n\\nControl resource usage and task distribution:\\n\\n```yaml\\n# Worker group targeting\\n- id: cpu_intensive_task\\n type: script\\n worker_group: \\\"high-cpu-workers\\\" # Target specific worker group\\n script: |\\n # CPU-intensive processing\\n perform_heavy_computation()\\n\\n- id: gpu_ml_task\\n type: ai_agent\\n worker_group: \\\"gpu-workers\\\" # Target GPU workers\\n config:\\n model_client_id: large_model\\n # Large model processing...\\n\\n# Timeout configuration\\n- id: long_running_task\\n type: script\\n timeout_seconds: 3600 # 1 hour timeout\\n script: |\\n # Long-running data processing\\n process_large_dataset()\\n\\n# Workflow-level timeout\\ntimeout_seconds: 7200 # 2 hour workflow timeout\\n```\\n\\n## Task Type Configurations\\n\\nThis section provides the complete YAML configuration for each supported task type.\\n\\n**Supported Task Types:**\\n- `script` - Execute Python code with full workflow context\\n- `http` - Make HTTP requests to external APIs\\n- `ai_agent` - Integrate with AI models for intelligent processing \\n- `loop` - Execute tasks repeatedly (for/while loops)\\n- `conditional_router` - Route execution based on conditions\\n- `mcp` - Execute tools via Model Context Protocol servers\\n- `storage` - Data persistence and retrieval operations\\n- `continuous_conversation` - Interactive AI conversations\\n- `human_input` - Tasks requiring human input or approval\\n- `chatbot` - Session-based chatbot conversations\\n\\n### Script Tasks\\n\\nExecute Python code with full workflow context access.\\n\\n```yaml\\n- id: script_task\\n name: \\\"Python Script Task\\\"\\n type: script\\n description: \\\"Execute Python code with dependencies and environment\\\"\\n \\n # Script configuration - script field is directly under task, NOT under config\\n script: | # REQUIRED: Python code to execute\\n import json\\n import requests\\n \\n # Access workflow inputs\\n input_value = \\\"${workflow_input}\\\"\\n \\n # Access previous task outputs\\n previous_result = ${previous_task}\\n \\n # Your processing logic\\n result = {\\\"processed\\\": True, \\\"value\\\": input_value}\\n \\n # Output results (must be JSON serializable)\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n \\n # Alternative: use \u0027code\u0027 field instead of \u0027script\u0027\\n code: | # Alternative to \u0027script\u0027 field\\n print(\\\"Hello World\\\")\\n \\n # Package management\\n requirements: # Optional: Python packages\\n - requests==2.31.0\\n - pandas==1.5.3\\n - numpy==1.24.3\\n \\n # Environment configuration\\n environment: # Optional: Environment variables\\n API_KEY: \\\"${secrets.api_key}\\\"\\n DEBUG: \\\"true\\\"\\n CUSTOM_VAR: \\\"value\\\"\\n \\n environment_vars: # Alternative to \u0027environment\u0027\\n DATABASE_URL: \\\"${secrets.db_url}\\\"\\n \\n # Secrets configuration\\n secrets_config: # Optional: Secrets access\\n api_key: \\\"secret_name\\\"\\n db_password: \\\"db_secret\\\"\\n```\\n\\n### HTTP Tasks\\n\\nMake HTTP requests to external APIs and services.\\n\\n```yaml\\n- id: http_task\\n name: \\\"HTTP API Call\\\"\\n type: http\\n description: \\\"Call external REST API\\\"\\n \\n # HTTP configuration\\n method: POST # Default: GET\\n url: \\\"https://api.example.com/data\\\" # REQUIRED: Target URL\\n \\n # Headers configuration\\n headers: # Optional: HTTP headers\\n Content-Type: \\\"application/json\\\"\\n Authorization: \\\"Bearer ${secrets.api_token}\\\"\\n X-Custom-Header: \\\"workflow-engine\\\"\\n User-Agent: \\\"WorkflowEngine/1.0\\\"\\n \\n # Request body\\n body: | # Optional: Request body (for POST/PUT)\\n {\\n \\\"customer_id\\\": \\\"${customer_id}\\\",\\n \\\"timestamp\\\": \\\"${current_timestamp}\\\",\\n \\\"data\\\": ${previous_task}\\n }\\n \\n # Response validation\\n expected_status: [200, 201, 202] # Default: [200]\\n expectedStatus: [200, 201] # Alternative field name\\n \\n # Additional HTTP options\\n timeout_seconds: 30 # HTTP request timeout\\n follow_redirects: true # Follow HTTP redirects\\n verify_ssl: true # Verify SSL certificates\\n```\\n\\n### AI Agent Tasks\\n\\nIntegrate with AI models for intelligent processing.\\n\\n```yaml\\n- id: ai_agent_task\\n name: \\\"AI Processing Agent\\\"\\n type: ai_agent\\n description: \\\"Process data using AI model\\\"\\n \\n # AI configuration\\n config:\\n model_client_id: gpt4_model # REQUIRED: Reference to model client\\n \\n system_message: | # Optional: System prompt\\n You are a data analyst. Analyze the provided data and extract insights.\\n Return your analysis in JSON format with the following structure:\\n {\\n \\\"insights\\\": [\\\"insight1\\\", \\\"insight2\\\"],\\n \\\"recommendations\\\": [\\\"rec1\\\", \\\"rec2\\\"],\\n \\\"confidence_score\\\": 0.95\\n }\\n \\n user_message: | # REQUIRED: User prompt with template resolution\\n Please analyze this customer data:\\n ${previous_task}\\n \\n Customer ID: ${customer_id}\\n Date Range: ${date_range}\\n \\n # Model parameters\\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\\n max_tokens: 1500 # Optional: Maximum response tokens\\n top_p: 1.0 # Optional: Nucleus sampling\\n frequency_penalty: 0.0 # Optional: Frequency penalty\\n presence_penalty: 0.0 # Optional: Presence penalty\\n \\n # Tools configuration for function calling\\n tools: # Optional: List of MCP tool names\\n - current_time # Time-related tools\\n - relative_time\\n - days_in_month\\n - weather_api # Weather tools\\n - database_query # Database tools\\n - notification_service # Communication tools\\n \\n # Required for tool resolution\\n tenant_id: \\\"your-tenant-uuid\\\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\\n \\n # Input/output format\\n input_format: \\\"text\\\" # Optional: text|json|structured\\n output_format: \\\"structured\\\" # Optional: text|json|structured\\n\\n### Important: AI Agent Output Access\\n\\nAI agent tasks return structured objects, not direct text responses. Always access the AI response using this pattern:\\n\\n```yaml\\n# In subsequent script tasks\\nscript: |\\n # CORRECT: Access AI response from structured output\\n agent_output = ${ai_agent_task}\\n ai_response = agent_output[\u0027ai_response\u0027].strip()\\n \\n # Additional metadata available\\n tokens_used = agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\\n cost_estimate = agent_output.get(\u0027estimated_cost\u0027, {}).get(\u0027amount\u0027, 0.0)\\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\\n \\n print(f\\\"AI Response: {ai_response}\\\")\\n print(f\\\"Tokens used: {tokens_used}\\\")\\n print(f\\\"Estimated cost: ${cost_estimate:.6f}\\\")\\n```\\n\\n```yaml\\n# In workflow outputs - specify the ai_response field\\noutputs:\\n ai_analysis:\\n type: string\\n source: ai_agent_task.ai_response # Access the ai_response field\\n description: \\\"AI-generated analysis text\\\"\\n \\n total_tokens:\\n type: integer \\n source: ai_agent_task.token_usage.total_tokens\\n description: \\\"Total tokens consumed\\\"\\n```\\n```\\n\\n### Loop Tasks\\n\\nExecute tasks repeatedly with for-loop and while-loop patterns.\\n\\n```yaml\\n# For-Loop (Fixed Iterations)\\n- id: for_loop_task\\n name: \\\"For Loop Processor\\\"\\n type: loop\\n description: \\\"Process data in fixed batches\\\"\\n \\n # Loop configuration\\n loop_type: for # REQUIRED: for|while\\n max_iterations: 10 # REQUIRED for \u0027for\u0027 loops\\n iteration_variable: batch_index # Default: iteration\\n \\n # State variables (persistent across iterations)\\n state_variables:\\n total_processed: 0\\n error_count: 0\\n batch_results: []\\n \\n # Tasks to execute in each iteration\\n loop_tasks: # REQUIRED: Tasks to run in loop\\n - id: process_batch\\n name: \\\"Process Current Batch\\\"\\n type: script\\n script: |\\n batch_idx = ${batch_index} # Access iteration variable\\n total = ${total_processed} # Access state variable\\n \\n # Process current batch\\n result = process_batch_data(batch_idx)\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n \\n - id: update_state\\n name: \\\"Update Loop State\\\"\\n type: script\\n depends_on: [process_batch]\\n script: |\\n # Update state variables\\n new_total = ${total_processed} + ${process_batch.items_processed}\\n \\n result = {\\\"total_processed\\\": new_total}\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n\\n# While-Loop (Conditional)\\n- id: while_loop_task\\n name: \\\"While Loop Quality Processor\\\"\\n type: loop\\n description: \\\"Process until quality threshold met\\\"\\n \\n # Loop configuration\\n loop_type: while # REQUIRED: for|while\\n exit_condition: \\\"${average_quality} \u003e= 85 || ${iteration_count} \u003e= 20\\\" # REQUIRED for \u0027while\u0027\\n max_iterations: 50 # Optional: Safety limit\\n iteration_variable: iteration_count # Default: iteration\\n \\n # State variables\\n state_variables:\\n average_quality: 0\\n items_processed: 0\\n quality_scores: []\\n \\n # Loop tasks\\n loop_tasks:\\n - id: assess_quality\\n type: script\\n script: |\\n current_quality = calculate_quality()\\n print(f\\\"__OUTPUTS__ {json.dumps({\u0027quality\u0027: current_quality})}\\\")\\n```\\n\\n\\n### Conditional Router Tasks\\n\\nRoute execution to different paths based on conditions.\\n\\n```yaml\\n- id: conditional_router_task\\n name: \\\"Data Type Router\\\"\\n type: conditional_router\\n description: \\\"Route processing based on data characteristics\\\"\\n \\n # Router configuration\\n router_type: condition # Default: condition\\n \\n # Routing conditions\\n conditions: # REQUIRED: List of conditions\\n - name: \\\"large_dataset\\\"\\n condition: \\\"${data_size} \u003e 10000\\\"\\n route: \\\"batch_processing\\\"\\n - name: \\\"small_dataset\\\" \\n condition: \\\"${data_size} \u003c= 1000\\\"\\n route: \\\"direct_processing\\\"\\n - name: \\\"medium_dataset\\\"\\n condition: \\\"${data_size} \u003e 1000 \u0026\u0026 ${data_size} \u003c= 10000\\\"\\n route: \\\"standard_processing\\\"\\n \\n default_route: \\\"standard_processing\\\" # REQUIRED: Default route if no conditions match\\n \\n # Alternative: evaluation script for complex routing\\n evaluation_script: | # Optional: Custom routing logic\\n if data_type == \\\"json\\\":\\n return \\\"json_processing\\\"\\n elif data_type == \\\"xml\\\":\\n return \\\"xml_processing\\\"\\n else:\\n return \\\"default_processing\\\"\\n \\n # Route mapping for complex scenarios\\n route_mapping: # Optional: Route name mappings\\n json_processing: \\\"specialized_json\\\"\\n xml_processing: \\\"specialized_xml\\\"\\n\\n# Tasks that execute on specific routes\\n- id: batch_handler\\n name: \\\"Handle Large Dataset\\\"\\n type: script\\n execute_on_routes: [\\\"batch_processing\\\"] # REQUIRED: Routes this task runs on\\n depends_on: [conditional_router_task]\\n script: |\\n print(\\\"Processing large dataset in batches...\\\")\\n\\n- id: direct_handler\\n name: \\\"Handle Small Dataset\\\" \\n type: script\\n execute_on_routes: [\\\"direct_processing\\\"]\\n depends_on: [conditional_router_task]\\n script: |\\n print(\\\"Processing small dataset directly...\\\")\\n```\\n\\n### MCP Tasks\\n\\nExecute tools via Model Context Protocol servers.\\n\\n```yaml\\n- id: mcp_task\\n name: \\\"MCP Tool Execution\\\"\\n type: mcp\\n description: \\\"Execute external tool via MCP server\\\"\\n \\n # MCP server configuration\\n mcp_server: \\\"@agentic/mcp-notion-server\\\" # REQUIRED: MCP server package\\n tool_name: \\\"update_database\\\" # REQUIRED: Tool to execute\\n runtime: \\\"npx\\\" # REQUIRED: npx|uvx\\n \\n # Server startup\\n server_args: [] # Optional: Server startup arguments\\n server_startup_timeout: 30 # Optional: Server startup timeout (seconds)\\n \\n # Tool execution\\n tool_arguments: # REQUIRED: Arguments for the tool\\n database_id: \\\"${secrets.notion_database_id}\\\"\\n properties:\\n title: \\\"Workflow Results\\\"\\n status: \\\"Completed\\\"\\n processed_count: \\\"${previous_task.count}\\\"\\n timestamp: \\\"${current_timestamp}\\\"\\n \\n # Environment configuration\\n environment_vars: # Optional: Environment variables\\n NOTION_TOKEN: \\\"${secrets.notion_token}\\\"\\n DEBUG: \\\"true\\\"\\n \\n # Timeout configuration\\n timeout_seconds: 120 # Optional: Tool execution timeout\\n```\\n\\n### Storage Tasks\\n\\nManage data persistence and retrieval operations.\\n\\n```yaml\\n- id: storage_task\\n name: \\\"Data Storage Operation\\\"\\n type: storage\\n description: \\\"Save/load data to/from storage\\\"\\n \\n # Storage operation\\n operation: save # REQUIRED: save|load|delete|list\\n \\n # Storage configuration\\n storage_type: \\\"file\\\" # REQUIRED: file|database|cache\\n path: \\\"/data/results/${execution_id}.json\\\" # REQUIRED: Storage path\\n \\n # Data configuration\\n data: ${previous_task} # REQUIRED for \u0027save\u0027: Data to store\\n format: \\\"json\\\" # Optional: json|yaml|csv|parquet\\n compression: \\\"gzip\\\" # Optional: gzip|bz2|none\\n \\n # Metadata\\n metadata: # Optional: Additional metadata\\n workflow_id: \\\"${workflow_id}\\\"\\n created_at: \\\"${current_timestamp}\\\"\\n source_task: \\\"previous_task\\\"\\n \\n # File options\\n create_directories: true # Optional: Create parent directories\\n overwrite: true # Optional: Overwrite existing files\\n backup: false # Optional: Create backup before overwrite\\n \\n # Database options (when storage_type: \\\"database\\\")\\n table_name: \\\"workflow_results\\\" # Required for database storage\\n connection_string: \\\"${secrets.db_url}\\\" # Required for database storage\\n \\n # Cache options (when storage_type: \\\"cache\\\")\\n cache_key: \\\"result_${execution_id}\\\" # Required for cache storage\\n ttl: 3600 # Optional: Time to live (seconds)\\n```\\n\\n\\n### Continuous Conversation Tasks\\n\\nCreate interactive AI-powered conversations.\\n\\n```yaml\\n- id: conversation_task\\n name: \\\"AI Conversation Interface\\\"\\n type: continuous_conversation\\n description: \\\"Interactive conversation with AI assistant\\\"\\n \\n # Conversation configuration\\n config:\\n # AI configuration\\n ai_config:\\n model_client_id: support_assistant # REQUIRED: AI model to use\\n system_prompt: | # REQUIRED: System instructions\\n You are a helpful customer support representative.\\n Be friendly, professional, and solution-oriented.\\n \\n Guidelines:\\n - Always be polite and respectful\\n - Ask clarifying questions when needed\\n - Provide clear and actionable solutions\\n \\n temperature: 0.7 # Optional: AI creativity\\n max_tokens: 1000 # Optional: Response length limit\\n \\n # UI configuration\\n ui_config:\\n title: \\\"Customer Support Chat\\\" # Optional: Interface title\\n placeholder: \\\"How can I help you today?\\\" # Optional: Input placeholder\\n theme: \\\"light\\\" # Optional: light|dark\\n show_timestamp: true # Optional: Show message timestamps\\n \\n # Termination conditions\\n termination:\\n keywords: [\\\"goodbye\\\", \\\"bye\\\", \\\"thanks\\\", \\\"done\\\"] # Optional: End conversation keywords\\n max_turns: 50 # Optional: Maximum conversation turns\\n timeout_minutes: 30 # Optional: Conversation timeout\\n inactivity_timeout: 10 # Optional: Inactivity timeout (minutes)\\n \\n # Context configuration\\n context:\\n system_context: \\\"Customer support conversation\\\" # Optional: Context description\\n user_id: \\\"${customer_id}\\\" # Optional: User identifier\\n session_data: ${previous_task} # Optional: Additional context data\\n```\\n\\n### Human Input Tasks\\n\\nTasks requiring human input or approval.\\n\\n```yaml\\n- id: human_input_task\\n name: \\\"Human Input Required\\\"\\n type: human_input\\n description: \\\"Wait for human input or approval\\\"\\n \\n # Human input configuration\\n config:\\n prompt: \\\"Please review and approve the processed data\\\" # REQUIRED: Prompt for human\\n input_type: \\\"approval\\\" # REQUIRED: approval|text|selection\\n \\n # Input options (for selection type)\\n options: [\\\"approve\\\", \\\"reject\\\", \\\"request_changes\\\"] # Optional: For selection input\\n \\n # Timeout configuration\\n timeout_minutes: 60 # Optional: How long to wait for input (minutes)\\n \\n # Conversation settings\\n conversation_mode: false # Optional: Add to conversation history\\n next_task: \\\"process_approval\\\" # Optional: Task to trigger after input\\n end_condition: \\\"bye|exit|quit\\\" # Optional: Keywords to end conversation\\n \\n # Input validation\\n required: true # Optional: Whether input is required\\n \\n # Additional context\\n context_data: ${previous_task} # Optional: Data to show to human\\n```\\n\\n### ChatBot Tasks\\n\\nSession-based chatbot conversations with streaming support.\\n\\n```yaml\\n- id: chatbot_task\\n name: \\\"AI Chatbot\\\"\\n type: chatbot\\n description: \\\"Interactive chatbot with session management\\\"\\n \\n # Chatbot configuration\\n config:\\n # AI model configuration\\n model_client_id: gpt4_model # REQUIRED: Reference to model client\\n \\n system_message: | # REQUIRED: System prompt for chatbot\\n You are a helpful AI assistant. Be friendly, helpful, and concise.\\n Answer questions clearly and ask for clarification when needed.\\n \\n # Session management\\n session_id: \\\"${session_id}\\\" # Optional: Session identifier (auto-generated if not provided)\\n max_history_messages: 20 # Optional: Maximum conversation history to maintain\\n \\n # Model parameters\\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\\n max_tokens: 1000 # Optional: Maximum response tokens\\n stream: true # Optional: Enable streaming responses\\n \\n # Tools configuration for function calling\\n tools: # Optional: List of MCP tool names\\n - current_time # Time-related tools\\n - relative_time\\n - days_in_month\\n - weather_forecast # Weather tools\\n - send_notification # Communication tools\\n \\n # Required for tool resolution\\n tenant_id: \\\"your-tenant-uuid\\\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\\n \\n # UI configuration\\n ui_config:\\n title: \\\"AI Assistant\\\" # Optional: Chat interface title\\n placeholder: \\\"Type your message...\\\" # Optional: Input placeholder\\n theme: \\\"light\\\" # Optional: light|dark\\n show_typing_indicator: true # Optional: Show typing animation\\n \\n # Response configuration\\n response_config:\\n max_retries: 3 # Optional: Retry failed requests\\n timeout_seconds: 30 # Optional: Response timeout\\n \\n # Input message (for single-turn mode)\\n user_message: \\\"${user_input}\\\" # Optional: Direct message input\\n \\n # Context data\\n context_data: ${previous_task} # Optional: Additional context\\n```\\n\\n---\\n\\n## Common Errors and Corrections\\n\\nThis section documents frequent mistakes encountered when creating workflows and their proper corrections.\\n\\n### 1. Missing Required `id` Field\\n\\n**\\u274c Error:**\\n```yaml\\nname: \\\"My Workflow\\\"\\ndescription: \\\"A workflow without id\\\"\\ntasks:\\n - name: \\\"Task 1\\\"\\n type: script\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nid: \\\"my_workflow\\\" # REQUIRED: Unique workflow identifier\\nname: \\\"My Workflow\\\"\\ndescription: \\\"A workflow with proper id\\\"\\ntasks:\\n - id: task_1 # REQUIRED: Unique task identifier\\n name: \\\"Task 1\\\"\\n type: script\\n```\\n\\n### 2. Incorrect Task Structure (Object vs Array)\\n\\n**\\u274c Error:**\\n```yaml\\ntasks:\\n task_1: # Wrong: tasks as object mapping\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n task_2:\\n type: ai_agent\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ntasks:\\n - id: task_1 # Correct: tasks as array with id field\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n - id: task_2\\n type: ai_agent\\n```\\n\\n### 3. Incorrect Inputs Format (Object vs Array)\\n\\n**\\u274c Error:**\\n```yaml\\ninputs:\\n user_name: # Wrong: inputs as object mapping\\n type: string\\n required: true\\n age:\\n type: integer\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ninputs:\\n - name: user_name # Correct: inputs as array with name field\\n type: string\\n required: true\\n - name: age\\n type: integer\\n```\\n\\n### 4. Script Configuration Location Error\\n\\n**\\u274c Error:**\\n```yaml\\n- id: script_task\\n type: script\\n config: # Wrong: script under config\\n script: |\\n print(\\\"Hello\\\")\\n requirements:\\n - requests\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: script_task\\n type: script\\n script: | # Correct: script directly under task\\n print(\\\"Hello\\\")\\n # requirements can be at task level if needed\\n```\\n\\n### 5. Individual Task Inputs (Not Supported)\\n\\n**\\u274c Error:**\\n```yaml\\n- id: task_1\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n inputs: # Wrong: tasks don\u0027t have individual inputs\\n search_query: ${search_query}\\n prev_result: ${previous_task}\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: task_1\\n type: script\\n depends_on: [previous_task] # Use depends_on for data access\\n script: |\\n # Access workflow inputs directly\\n search_query = \\\"${search_query}\\\"\\n # Access previous task outputs via template resolution\\n prev_result = ${previous_task}\\n```\\n\\n### 6. Incorrect Script Output Handling\\n\\n**\\u274c Error:**\\n```yaml\\nscript: |\\n result = process_data()\\n outputs[\u0027result\u0027] = result # Wrong: missing __OUTPUTS__ pattern\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nscript: |\\n result = process_data()\\n \\n # Required: Use outputs dict and __OUTPUTS__ marker\\n outputs = {}\\n outputs[\u0027result\u0027] = result\\n outputs[\u0027status\u0027] = \u0027success\u0027\\n \\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### 7. AI Agent Configuration Errors\\n\\n**\\u274c Error:**\\n```yaml\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: my_model\\n prompt: \\\"Process this data: ${data}\\\" # Wrong: use user_message\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: my_model\\n system_message: \\\"You are a helpful assistant.\\\"\\n user_message: \\\"Process this data: ${data}\\\" # Correct field name\\n```\\n\\n### 8. Missing Dependencies for Template Resolution\\n\\n**\\u274c Error:**\\n```yaml\\n- id: task_2\\n type: script\\n # Missing depends_on\\n script: |\\n data = ${task_1} # Will become UNRESOLVED_task_1\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: task_2\\n type: script\\n depends_on: [task_1] # Required for template access\\n script: |\\n data = ${task_1} # Now resolves correctly\\n```\\n\\n### 9. Incorrect Model Client Configuration\\n\\n**\\u274c Error:**\\n```yaml\\nmodel_clients:\\n my_model: # Wrong: object mapping format\\n type: openai\\n api_key: \\\"sk-...\\\"\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nmodel_clients:\\n - id: my_model # Correct: array format with id field\\n provider: openai\\n config:\\n model: \\\"gpt-4\\\"\\n api_key: \\\"sk-...\\\"\\n```\\n\\n### 10. Empty String Literals Causing Python Syntax Errors\\n\\n**\\u274c Error:**\\n```yaml\\nscript: |\\n def process_data(path=\\\"\\\"): # Wrong: empty string can cause syntax issues\\n current_path = f\\\"{path}.{key}\\\" if path else key\\n \\n # This can lead to: SyntaxError: unterminated string literal\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nscript: |\\n def process_data(path=None): # Correct: use None instead of empty string\\n current_path = f\\\"{path}.{key}\\\" if path else key\\n # Or use path = path or \\\"default_value\\\"\\n```\\n\\n### 11. YAML Indentation and Structure Issues\\n\\n**\\u274c Error:**\\n```yaml\\ntasks:\\n- id: task_1\\n type: script\\nscript: | # Wrong: inconsistent indentation\\n print(\\\"hello\\\")\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ntasks:\\n- id: task_1\\n type: script\\n script: | # Correct: consistent indentation\\n print(\\\"hello\\\")\\n```\\n\\n### Best Practices Summary\\n\\n1. **Always include required `id` fields** for workflows and tasks\\n2. **Use array format** for `tasks`, `inputs`, and `model_clients`\\n3. **Script field goes directly under task**, not under `config`\\n4. **No individual task inputs** - use template resolution instead\\n5. **Use `depends_on`** to access previous task outputs\\n6. **Print `__OUTPUTS__`** in scripts for proper output parsing\\n7. **Use `user_message`** (not `prompt`) for AI agent tasks\\n8. **Avoid empty string literals** - use `None` or meaningful defaults\\n9. **Follow consistent YAML indentation** throughout the file\\n\\n## MCP Tools and Function Calling\\n\\nAI agents and chatbots can use MCP (Model Context Protocol) tools for function calling, enabling them to execute external tools and services.\\n\\n### Tools Configuration\\n\\nBoth `ai_agent` and `chatbot` tasks support tools configuration:\\n\\n```yaml\\n# AI Agent with MCP tools\\n- id: ai_with_tools\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful assistant with access to various tools.\\n Use the available tools when needed to help users.\\n user_message: \\\"What time is it and what\u0027s the weather like?\\\"\\n \\n # Tools configuration\\n tools: # List of MCP tool names\\n - current_time # Time tools\\n - relative_time\\n - days_in_month\\n - weather_forecast # Weather tools\\n - database_query # Database tools\\n - send_notification # Communication tools\\n \\n tenant_id: \\\"550e8400-e29b-41d4-a716-446655440000\\\" # Required for tool resolution\\n\\n# Chatbot with MCP tools\\n- id: chatbot_with_tools\\n type: chatbot\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful chatbot with access to tools.\\n Use tools when users ask for specific information or actions.\\n \\n # Tools configuration (same format as AI agent)\\n tools:\\n - current_time\\n - weather_api\\n - user_preferences\\n \\n tenant_id: \\\"550e8400-e29b-41d4-a716-446655440000\\\" # Required for tool resolution\\n```\\n\\n### How Tools Work\\n\\n1. **Tool Resolution**: The workflow engine resolves tool names to actual MCP server deployments using the `tenant_id`\\n2. **Function Calling**: When the AI model decides to use a tool, it calls the `tool_executor` function\\n3. **Tool Execution**: The engine executes the specified tool via the appropriate MCP server\\n4. **Response Integration**: Tool results are fed back to the AI model for response generation\\n\\n### Tool Configuration Requirements\\n\\n| Field | Required | Description |\\n|-------|----------|-------------|\\n| `tools` | No | List of MCP tool names available to the AI |\\n| `tenant_id` | **Yes** (when using tools) | Tenant UUID for tool resolution |\\n\\n### Example: Complete AI Agent with Tools\\n\\n```yaml\\nid: intelligent_assistant\\nname: \\\"Intelligent Assistant with Tools\\\"\\n\\nmodel_clients:\\n - id: gpt4_model\\n provider: openai\\n config:\\n model: gpt-4o-mini\\n api_key: \\\"${OPENAI_API_KEY}\\\"\\n temperature: 0.3\\n\\n# Tenant ID can be set at workflow level for all tasks\\ntenant_id: \\\"0572fa8d-d7c3-47db-9219-ef40b80d42b7\\\"\\n\\ntasks:\\n - id: assistant_with_tools\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful assistant with access to various tools:\\n - Time-related tools: current_time, relative_time, days_in_month\\n - Weather tools: weather_forecast\\n - Communication: send_notification\\n \\n Use these tools when users ask for:\\n - Current time or date information\\n - Weather forecasts\\n - Sending notifications\\n \\n Always explain what tool you\u0027re using and why.\\n \\n user_message: \\\"${user_query}\\\"\\n \\n # MCP tools configuration\\n tools:\\n - current_time\\n - relative_time\\n - days_in_month\\n - weather_forecast\\n - send_notification\\n \\n # Tool resolution requires tenant_id\\n tenant_id: \\\"0572fa8d-d7c3-47db-9219-ef40b80d42b7\\\"\\n \\n temperature: 0.7\\n max_tokens: 1000\\n\\ninputs:\\n - name: user_query\\n type: string\\n required: true\\n default: \\\"What time is it and can you send a notification to user123?\\\"\\n\\noutputs:\\n ai_response:\\n type: string\\n source: assistant_with_tools.ai_response\\n description: \\\"AI response using tools when needed\\\"\\n```\\n\\n### Tool Execution Flow\\n\\n1. **User Input**: User provides a query that might require tool usage\\n2. **AI Analysis**: AI model analyzes the query and decides if tools are needed\\n3. **Tool Calling**: AI calls `tool_executor(deployment_id, tool_name, arguments)`\\n4. **Tool Resolution**: Engine resolves tool name to MCP server using tenant_id\\n5. **Tool Execution**: MCP server executes the requested tool\\n6. **Result Processing**: Tool results are returned to the AI model\\n7. **Response Generation**: AI generates final response incorporating tool results\\n\\n### Best Practices for Tools\\n\\n1. **Clear System Messages**: Explain available tools and when to use them\\n2. **Tool Categories**: Group related tools (time, weather, database, etc.)\\n3. **Error Handling**: AI models handle tool execution failures gracefully\\n4. **Tool Documentation**: Include tool descriptions in system messages\\n5. **Tenant Configuration**: Ensure tenant_id matches your MCP server setup\\n\\n---\\n\\n*This comprehensive reference covers all supported task types and their YAML configurations. Use this as a quick reference when building workflows with different task types.*\", \"format\": \"markdown\"}, \"status\": \"success\", \"successful_url\": \"http://172.17.0.1:5000/api/docs/yaml-reference\"}\n",
"return_code": 0,
"status": "completed",
"stderr": ""
},
"duration_seconds": 0.820081,
"end_time": "2025-08-14T11:23:07.670722",
"message_sent": true,
"start_time": "2025-08-14T11:23:06.850641",
"timestamp": "2025-08-14T11:23:07.670722",
"worker_executed": true,
"workers_notified": true
},
"output": "Trying to fetch documentation from: http://host.docker.internal:5000/api/docs/yaml-reference\n\u274c Connection failed: http://host.docker.internal:5000/api/docs/yaml-reference - HTTPConnectionPool(host=\u0027host.docker.internal\u0027, port=5000): Max retries exceeded with url: /api/docs/yaml-reference (Caused by NameResolutionError(\"\u003curllib3.connection.HTTPConnection object at 0x7d0629aa9010\u003e: Failed to resolve \u0027host.docker.internal\u0027 ([Errno -2] Name or service not known)\"))\nTrying to fetch documentation from: http://172.17.0.1:5000/api/docs/yaml-reference\n\u2705 Successfully connected to: http://172.17.0.1:5000/api/docs/yaml-reference\nSuccessfully fetched documentation from http://172.17.0.1:5000/api/docs/yaml-reference: 97047 characters\n__OUTPUTS__ {\"documentation\": {\"content\": \"# YAML Workflow Structure Reference\\n\\nComplete reference for the YAML workflow definition structure supported by the Assistants Workflow Engine.\\n\\n## Table of Contents\\n\\n1. [Complete Schema Overview](#complete-schema-overview)\\n2. [Required vs Optional Fields](#required-vs-optional-fields)\\n3. [Data Types Reference](#data-types-reference)\\n4. [Field Validation Rules](#field-validation-rules)\\n5. [Parser Behavior](#parser-behavior)\\n6. [Migration Guide](#migration-guide)\\n\\n## Complete Schema Overview\\n\\n```yaml\\n# === WORKFLOW METADATA (Required) ===\\nid: string # REQUIRED: Unique workflow identifier\\nname: string # REQUIRED: Human-readable workflow name\\n\\n# === WORKFLOW METADATA (Optional) ===\\nversion: string | integer # Default: 1\\nnamespace: string # Default: \\\"default\\\"\\ndescription: string # Default: \\\"\\\"\\ntenant_id: string # Default: null\\n\\n# === EXECUTION CONFIGURATION ===\\ntimeout_seconds: integer|string # Default: 3600 (1 hour) - Workflow-level timeout (supports duration strings like \\\"1h\\\", \\\"30m\\\")\\nlabels: object # Default: {}\\nmetadata: object # Default: {}\\n\\n# === DATA FLOW DEFINITIONS ===\\ninputs: # Default: []\\n - name: string # REQUIRED\\n type: string # REQUIRED: string|integer|boolean|object|array\\n description: string # Optional\\n required: boolean # Default: false\\n default: any # Optional default value\\n\\noutputs: object # Default: {}\\n output_name:\\n type: string # REQUIRED\\n source: string # REQUIRED: task_id or task_id.field\\n description: string # Optional\\n\\nvariables: object # Default: {}\\n\\n# === MODEL CLIENT CONFIGURATION (Required for AI tasks) ===\\nmodel_clients: # REQUIRED: For ai_agent, chatbot, continuous_conversation tasks\\n - id: string # REQUIRED: Unique identifier to reference in tasks\\n provider: string # REQUIRED: openai|anthropic|openrouter|groq\\n config: object # REQUIRED: Provider-specific config\\n model: string # REQUIRED: Model name (e.g., gpt-4, claude-3-sonnet-20240229)\\n api_key: string # REQUIRED: API key for the provider\\n temperature: number # Optional: 0.0-2.0, controls randomness\\n max_tokens: integer # Optional: Maximum tokens to generate\\n timeout: integer # Optional: Request timeout in seconds\\n base_url: string # Optional: Custom API base URL\\n\\n# Model Client Examples:\\n# OpenAI Configuration:\\n# model_clients:\\n# - id: gpt4_model\\n# provider: openai\\n# config:\\n# model: gpt-4-turbo-preview\\n# api_key: \\\"sk-your-openai-api-key\\\"\\n# temperature: 0.7\\n# max_tokens: 2000\\n#\\n# Anthropic Configuration:\\n# model_clients:\\n# - id: claude_model\\n# provider: anthropic\\n# config:\\n# model: claude-3-sonnet-20240229\\n# api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n# temperature: 0.7\\n# max_tokens: 1500\\n#\\n# Multiple Models:\\n# model_clients:\\n# - id: gpt4_model\\n# provider: openai\\n# config:\\n# model: gpt-4-turbo-preview\\n# api_key: \\\"sk-your-openai-api-key\\\"\\n# - id: claude_model\\n# provider: anthropic\\n# config:\\n# model: claude-3-sonnet-20240229\\n# api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n\\n# === TASK DEFINITIONS (Required) ===\\ntasks: # REQUIRED: List of tasks\\n - id: string # REQUIRED: Unique task identifier\\n name: string # Optional: Human-readable name\\n type: string # REQUIRED: Task type\\n description: string # Optional\\n \\n # DEPENDENCY CONFIGURATION\\n depends_on: array # Optional: Execution dependencies and template resolution\\n previous_node: string # REQUIRED: Task connectivity and UI hierarchy (essential for proper workflow execution)\\n \\n timeout_seconds: integer|string # Optional: Task-specific timeout (supports duration strings like \\\"5m\\\", \\\"30s\\\")\\n worker_group: string # Optional: Target worker group\\n conditions: array # Optional: Execution conditions\\n execute_on_routes: array # Optional: Routes for conditional execution\\n \\n # Input/Output mapping\\n input_mapping: object # Optional: Map workflow context to task inputs\\n output_mapping: object # Optional: Map task outputs to workflow context\\n \\n # Retry configuration\\n retry: object # Optional: Task-specific retry policy\\n maxAttempts: integer # Default: 1\\n initialDelay: string|integer # Default: \\\"1s\\\"\\n maxDelay: string|integer # Default: \\\"300s\\\"\\n backoffMultiplier: number # Default: 2.0\\n retryOn: array # Default: [\\\"TEMPORARY_FAILURE\\\"]\\n \\n # === TASK TYPE SPECIFIC CONFIGURATIONS ===\\n # See detailed sections below for each task type\\n\\n# === WORKFLOW-LEVEL RETRY POLICY ===\\nretry: object # Optional: Global retry policy\\n retryOn: array # Default: [\\\"TEMPORARY_FAILURE\\\"]\\n maxAttempts: integer # Default: 1\\n initialDelay: string|integer # Default: \\\"1s\\\"\\n maxDelay: string|integer # Default: \\\"300s\\\"\\n backoffMultiplier: number # Default: 2.0\\n\\n# === TRIGGERS ===\\ntriggers: # Optional: List of triggers\\n - name: string # REQUIRED\\n type: string # REQUIRED: webhook|schedule|event\\n description: string # Optional\\n config: object # REQUIRED: Trigger-specific config\\n input_mapping: array # Optional: Map trigger data to workflow inputs\\n - webhook_field: string # Source field\\n workflow_input: string # Target workflow input\\n```\\n\\n## Required vs Optional Fields\\n\\n### Required Fields (Workflow Level)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `id` | string | Unique identifier for the workflow |\\n| `tasks` | array | List of task definitions (can be empty) |\\n\\n### Required Fields (Task Level)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `id` | string | Unique identifier for the task |\\n| `type` | string | Task type (script, http, ai_agent, etc.) |\\n| `previous_node` | string | Reference to previous task for UI connectivity (required for proper task linking, except for the first task) |\\n\\n### Required Fields (Input Definition)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `name` | string | Input parameter name |\\n| `type` | string | Data type (string, integer, boolean, object, array) |\\n\\n### Required Fields (Output Definition)\\n\\n| Field | Type | Description |\\n|-------|------|-------------|\\n| `type` | string | Output data type |\\n| `source` | string | Source task ID or task_id.field_name |\\n\\n## Data Types Reference\\n\\n### Primitive Types\\n\\n```yaml\\n# String type\\ninputs:\\n - name: customer_name\\n type: string\\n default: \\\"John Doe\\\"\\n description: \\\"Customer full name\\\"\\n\\n# Integer type \\ninputs:\\n - name: batch_size\\n type: integer\\n default: 100\\n description: \\\"Number of records per batch\\\"\\n\\n# Boolean type\\ninputs:\\n - name: include_archived\\n type: boolean\\n default: false\\n description: \\\"Include archived records\\\"\\n```\\n\\n### Complex Types\\n\\n```yaml\\n# Object type\\ninputs:\\n - name: config_params\\n type: object\\n default:\\n timeout: 30\\n retries: 3\\n debug: false\\n description: \\\"Configuration parameters\\\"\\n\\n# Array type\\ninputs:\\n - name: customer_ids\\n type: array\\n default: [\\\"cust_001\\\", \\\"cust_002\\\"]\\n description: \\\"List of customer identifiers\\\"\\n```\\n\\n### Duration Types\\n\\nDuration values can be specified as strings or integers:\\n\\n```yaml\\n# String format (recommended)\\ntimeout_seconds: \\\"1h\\\" # 1 hour\\ntimeout_seconds: \\\"30m\\\" # 30 minutes \\ntimeout_seconds: \\\"45s\\\" # 45 seconds\\ntimeout_seconds: \\\"2d\\\" # 2 days\\n\\n# Integer format (seconds)\\ntimeout_seconds: 3600 # 1 hour in seconds\\n\\n# Supported units\\n# s, sec, second, seconds\\n# m, min, minute, minutes \\n# h, hour, hours\\n# d, day, days\\n```\\n\\n### Timeout Configuration and Execution Behavior\\n\\nThe workflow engine supports configurable timeouts at both workflow and task levels with intelligent timeout-free communication for long-running tasks.\\n\\n#### Timeout Hierarchy\\n\\nTimeout values are resolved in this order of precedence:\\n1. **Task-level `timeout_seconds`**: Specific timeout for individual tasks\\n2. **Workflow-level `timeout_seconds`**: Default for all tasks in the workflow\\n3. **System default**: 300 seconds (5 minutes) for tasks, 3600 seconds (1 hour) for workflows\\n\\n```yaml\\n# Workflow-level timeout (applies to all tasks by default)\\ntimeout_seconds: \\\"2h\\\" # 2 hour workflow timeout\\n\\ntasks:\\n - id: quick_task\\n type: script\\n # Uses workflow-level timeout (2h)\\n script: \\\"print(\u0027Quick task\u0027)\\\"\\n \\n - id: custom_timeout_task\\n type: script\\n timeout_seconds: \\\"30m\\\" # Override with 30 minute timeout\\n script: \\\"print(\u0027Custom timeout task\u0027)\\\"\\n \\n - id: long_running_task\\n type: script\\n timeout_seconds: 600 # 10 minutes (integer format)\\n script: |\\n import time\\n time.sleep(300) # 5 minute processing\\n print(\\\"Long task completed\\\")\\n```\\n\\n#### Timeout-Free Communication\\n\\nThe workflow engine automatically uses **timeout-free communication** for long-running tasks to prevent timeout-related failures:\\n\\n- **Loop tasks**: Always use timeout-free communication (event-driven execution)\\n- **Script tasks**: Use timeout-free communication when `timeout_seconds \u003e 180` (host timeout threshold)\\n- **Other task types**: Use standard timeout-based communication\\n\\n```yaml\\ntasks:\\n - id: short_script\\n type: script\\n timeout_seconds: 60 # Uses standard timeout-based communication\\n script: \\\"print(\u0027Short script\u0027)\\\"\\n \\n - id: long_script\\n type: script\\n timeout_seconds: 600 # Uses timeout-free communication (600 \u003e 180)\\n script: |\\n import time\\n print(\\\"\\ud83d\\ude80 This uses timeout-free communication\\\")\\n time.sleep(300) # Won\u0027t cause timeout issues\\n print(\\\"Long script completed\\\")\\n \\n - id: data_processing_loop\\n type: loop\\n timeout_seconds: \\\"1h\\\" # Always uses timeout-free communication\\n loop_type: for\\n max_iterations: 10\\n # Loop tasks automatically use timeout-free communication\\n```\\n\\n#### Loop Task Timeout Inheritance\\n\\nLoop tasks can pass timeout configuration to their inner script tasks:\\n\\n```yaml\\n- id: batch_processing_loop\\n type: loop\\n timeout_seconds: \\\"30m\\\" # Loop-level timeout\\n loop_type: for\\n max_iterations: 5\\n loop_tasks:\\n - id: process_batch\\n type: script\\n # Inherits 30-minute timeout from parent loop\\n script: |\\n print(\\\"Processing batch with inherited timeout\\\")\\n # This script will use the 30-minute timeout from the loop\\n```\\n\\n#### Best Practices for Timeout Configuration\\n\\n1. **Use duration strings** for readability: `\\\"5m\\\"` vs `300`\\n2. **Set appropriate timeouts** based on expected execution time plus buffer\\n3. **Long-running tasks**: Set `timeout_seconds \u003e 180` to enable timeout-free communication\\n4. **Loop tasks**: Always get timeout-free communication automatically\\n5. **Critical tasks**: Use shorter timeouts for fail-fast behavior\\n6. **External API calls**: Use longer timeouts to account for network latency\\n\\n```yaml\\n# Example: Well-configured timeout workflow\\ntimeout_seconds: \\\"1h\\\" # Reasonable workflow timeout\\n\\ntasks:\\n - id: api_call\\n type: http\\n timeout_seconds: \\\"30s\\\" # Quick timeout for API calls\\n url: \\\"https://api.example.com/data\\\"\\n \\n - id: data_processing\\n type: script\\n timeout_seconds: \\\"10m\\\" # Longer timeout for processing (uses timeout-free)\\n script: |\\n # Process large dataset\\n process_data()\\n \\n - id: batch_processing\\n type: loop\\n timeout_seconds: \\\"45m\\\" # Generous timeout for loop (timeout-free)\\n loop_type: for\\n max_iterations: 100\\n # Inner tasks inherit the 45-minute timeout\\n```\\n\\n## Field Validation Rules\\n\\n### ID Fields\\n\\n```yaml\\n# Valid IDs\\nid: \\\"customer_processor\\\" # \\u2705 Valid\\nid: \\\"process-data\\\" # \\u2705 Valid\\nid: \\\"task_001\\\" # \\u2705 Valid\\n\\n# Invalid IDs\\nid: \\\"process data\\\" # \\u274c Spaces not allowed\\nid: \\\"123_start\\\" # \\u274c Cannot start with number\\nid: \\\"\\\" # \\u274c Cannot be empty\\n```\\n\\n### Timeout Values\\n\\n```yaml\\n# Valid timeouts\\ntimeout_seconds: 300 # \\u2705 Positive integer\\ntimeout_seconds: \\\"5m\\\" # \\u2705 Duration string\\ntimeout_seconds: \\\"1h30m\\\" # \\u2705 Complex duration\\n\\n# Invalid timeouts\\ntimeout_seconds: 0 # \\u274c Must be positive\\ntimeout_seconds: -60 # \\u274c Cannot be negative\\ntimeout_seconds: \\\"invalid\\\" # \\u274c Invalid duration format\\n```\\n\\n### Dependency Types and Validation\\n\\n#### `depends_on` vs `previous_node`\\n\\n**`depends_on` - Execution Dependencies:**\\n- Controls actual task execution order\\n- Enables template resolution and data access\\n- Required for accessing outputs from other tasks\\n- Validates that referenced tasks exist\\n- Multiple tasks can be specified\\n\\n**`previous_node` - Task Connectivity and UI Hierarchy:**\\n- **REQUIRED for proper task connectivity in UI**\\n- Essential for workflow visualization and task linking\\n- Critical for execution flow representation\\n- Single task reference only\\n- **Without this field, tasks won\u0027t be properly connected in the UI**\\n\\n```yaml\\ntasks:\\n - id: fetch_data\\n type: http\\n url: \\\"https://api.example.com/data\\\"\\n \\n - id: validate_data\\n type: script\\n depends_on: [fetch_data] # \\u2705 EXECUTION: wait for fetch_data to complete\\n previous_node: fetch_data # \\u2705 UI: show as child of fetch_data in hierarchy\\n script: |\\n data = ${fetch_data} # \\u2705 Can access because of depends_on\\n if not validate(data):\\n raise ValueError(\\\"Invalid data\\\")\\n \\n # Required: Use outputs dict and __OUTPUTS__ marker for proper parsing\\n outputs[\u0027status\u0027] = \u0027valid\u0027\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: process_data\\n type: script\\n depends_on: [validate_data] # \\u2705 EXECUTION: wait for validation\\n previous_node: validate_data # \\u2705 UI: show as child of validate_data\\n script: |\\n validation_result = ${validate_data} # \\u2705 Accessible via depends_on\\n # Process data...\\n\\n - id: separate_task\\n type: script\\n previous_node: process_data # \\u2705 UI: appears under process_data in hierarchy\\n # NO depends_on means this can run immediately (parallel to other tasks)\\n script: |\\n # This task runs independently but appears under process_data in UI\\n # Cannot access ${process_data} because no depends_on relationship\\n print(\\\"Independent task\\\")\\n\\n### Complete Example with Proper Hierarchy\\n\\n```yaml\\ntasks:\\n - id: fetch_customer_data\\n name: \\\"Fetch Customer Data\\\"\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: validate_customer\\n name: \\\"Validate Customer Data\\\"\\n type: script\\n depends_on: [fetch_customer_data] # \\u2705 EXECUTION: Wait for data fetch\\n previous_node: fetch_customer_data # \\u2705 UI: Show under fetch_customer_data\\n script: |\\n customer = ${fetch_customer_data} # \\u2705 Can access because of depends_on\\n if not customer.get(\u0027email\u0027):\\n raise ValueError(\\\"Invalid customer data\\\")\\n print(\\\"__OUTPUTS__ {\u0027status\u0027: \u0027valid\u0027}\\\")\\n \\n - id: process_customer\\n name: \\\"Process Customer\\\"\\n type: ai_agent\\n depends_on: [validate_customer] # \\u2705 EXECUTION: Wait for validation\\n previous_node: validate_customer # \\u2705 UI: Show under validate_customer\\n config:\\n model_client_id: gpt4_model\\n user_message: \\\"Process customer: ${fetch_customer_data}\\\"\\n \\n - id: send_notification\\n name: \\\"Send Notification\\\"\\n type: script\\n depends_on: [process_customer] # \\u2705 EXECUTION: Wait for processing\\n previous_node: process_customer # \\u2705 UI: Show under process_customer\\n script: |\\n result = ${process_customer}\\n print(f\\\"Notification sent for: ${fetch_customer_data.name}\\\")\\n \\n - id: parallel_audit_log\\n name: \\\"Audit Log (Parallel)\\\"\\n type: script\\n depends_on: [fetch_customer_data] # \\u2705 EXECUTION: Only needs initial data\\n previous_node: fetch_customer_data # \\u2705 UI: Show under fetch_customer_data\\n script: |\\n # This runs in parallel with validation/processing\\n customer = ${fetch_customer_data}\\n print(f\\\"Audit: Customer {customer.id} accessed at ${current_timestamp}\\\")\\n```\\n\\n # Invalid examples\\n - id: invalid_task\\n type: script\\n depends_on: [nonexistent_task] # \\u274c VALIDATION ERROR: task doesn\u0027t exist\\n previous_node: another_missing # \\u26a0\\ufe0f UI WARNING: task doesn\u0027t exist (but not fatal)\\n script: \\\"print(\u0027Invalid\u0027)\\\"\\n```\\n\\n#### Key Differences:\\n\\n| Aspect | `depends_on` | `previous_node` |\\n|--------|--------------|-----------------|\\n| **Purpose** | Execution control | Task connectivity \u0026 UI organization |\\n| **Impact** | Blocks task until dependencies complete | **REQUIRED for UI task linking** |\\n| **Template Access** | \\u2705 Enables `${task_id}` resolution | \\u274c No template access |\\n| **Validation** | \\u274c Parser error if task missing | \\u26a0\\ufe0f Warning if task missing |\\n| **Multiple References** | \\u2705 Array of task IDs | \\u274c Single task ID only |\\n| **Execution Order** | \\u2705 Enforced | \\u274c No execution impact |\\n| **UI Connectivity** | \\u274c No UI impact | \\u2705 **ESSENTIAL for proper task connection** |\\n\\n### Circular Dependency Detection\\n\\n```yaml\\n# This will be rejected by the parser\\ntasks:\\n - id: task_a\\n depends_on: [task_b] # \\u274c Circular dependency\\n \\n - id: task_b \\n depends_on: [task_a] # \\u274c Circular dependency\\n```\\n\\n## Parser Behavior\\n\\n### Field Name Compatibility\\n\\nThe parser supports both camelCase and snake_case for field names:\\n\\n```yaml\\ntasks:\\n - id: flexible_task\\n # Both formats are accepted\\n dependsOn: [other_task] # camelCase\\n depends_on: [other_task] # snake_case\\n \\n inputMapping: # camelCase\\n source_field: \\\"target_field\\\"\\n input_mapping: # snake_case \\n source_field: \\\"target_field\\\"\\n \\n outputMapping: {} # camelCase\\n output_mapping: {} # snake_case\\n \\n workerGroup: \\\"gpu-workers\\\" # camelCase\\n worker_group: \\\"gpu-workers\\\" # snake_case\\n```\\n\\n### Default Value Assignment\\n\\n```yaml\\n# Fields with defaults are automatically populated\\nid: my_workflow\\n# name defaults to id value: \\\"my_workflow\\\"\\n# version defaults to: 1\\n# namespace defaults to: \\\"default\\\" \\n# timeout_seconds defaults to: 3600\\n# description defaults to: \\\"\\\"\\n\\ntasks:\\n - id: simple_task\\n type: script\\n # name defaults to id: \\\"simple_task\\\"\\n # timeout_seconds defaults to: 300\\n # retry.maxAttempts defaults to: 1\\n```\\n\\n### Template Resolution Context\\n\\nThe parser provides these built-in variables for template resolution:\\n\\n```yaml\\ntasks:\\n - id: context_aware_task\\n type: script\\n script: |\\n # Workflow context\\n workflow_id = \\\"${workflow_id}\\\" # Current workflow ID\\n execution_id = \\\"${execution_id}\\\" # Unique execution ID\\n namespace = \\\"${namespace}\\\" # Workflow namespace\\n \\n # Timing context\\n current_timestamp = \\\"${current_timestamp}\\\" # ISO timestamp\\n started_at = \\\"${started_at}\\\" # Workflow start time\\n \\n # Input context\\n user_input = \\\"${input_parameter}\\\" # Workflow inputs\\n \\n # Task context\\n previous_result = ${previous_task} # Previous task output\\n specific_field = ${task_id.field_name} # Specific output field\\n \\n # Route context (for conditional workflows)\\n route_info = ${_route_context} # Current route information\\n```\\n\\n### Error Handling During Parsing\\n\\n```yaml\\n# The parser will validate and report these errors:\\n\\n# 1. Missing required fields\\n# ERROR: Missing required field: \u0027id\u0027\\n\\n# 2. Invalid YAML syntax \\n# ERROR: Invalid YAML: mapping values are not allowed here\\n\\n# 3. Circular dependencies\\n# ERROR: Circular dependency detected involving task \u0027task_a\u0027\\n\\n# 4. Invalid task references\\n# ERROR: Task \u0027task_b\u0027 depends on non-existent task \u0027invalid_task\u0027\\n\\n# 5. Invalid duration formats\\n# ERROR: Invalid duration: \u0027not_a_duration\u0027\\n\\n# 6. Invalid field values\\n# ERROR: Timeout must be positive, got -10\\n```\\n\\n## Migration Guide\\n\\n### From Version 1.0 to 2.0\\n\\n```yaml\\n# OLD FORMAT (v1.0)\\nworkflow:\\n id: my_workflow\\n tasks:\\n - task_id: process_data\\n task_type: python_script\\n \\n# NEW FORMAT (v2.0) \\nid: my_workflow\\ntasks:\\n - id: process_data\\n type: script\\n language: python\\n```\\n\\n### Deprecated Fields\\n\\n| Old Field | New Field | Status |\\n|-----------|-----------|--------|\\n| `task_type` | `type` | Deprecated in v2.0 |\\n| `task_id` | `id` | Deprecated in v2.0 |\\n| `python_script` | `script` (type) | Deprecated in v2.0 |\\n| `plugin` | `mcp` (type) | Deprecated in v2.1 |\\n\\n### Backward Compatibility\\n\\nThe parser maintains backward compatibility for deprecated fields but will show warnings:\\n\\n```yaml\\n# This will work but show a warning\\ntasks:\\n - task_id: old_style_task # \\u26a0\\ufe0f Warning: Use \u0027id\u0027 instead\\n task_type: python_script # \\u26a0\\ufe0f Warning: Use \u0027type: script\u0027 instead\\n```\\n\\n## Advanced Schema Features\\n\\n### Conditional Field Requirements\\n\\nSome fields become required based on task type:\\n\\n```yaml\\n# HTTP tasks require URL\\n- id: api_call\\n type: http\\n url: \\\"https://api.example.com\\\" # \\u2705 Required for HTTP tasks\\n \\n# AI agent tasks require model_client_id\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: gpt4 # \\u2705 Required for AI agent tasks\\n```\\n\\n### Dynamic Field Validation\\n\\nField validation can depend on other field values:\\n\\n```yaml\\n# Loop tasks require loop-specific fields\\n- id: data_loop\\n type: loop\\n loop_type: \\\"for\\\" # Determines required fields\\n max_iterations: 10 # \\u2705 Required for \u0027for\u0027 loops\\n # exit_condition: not needed for \u0027for\u0027 loops\\n \\n- id: condition_loop \\n type: loop\\n loop_type: \\\"while\\\" # Determines required fields\\n exit_condition: \\\"${count} \u003e= 10\\\" # \\u2705 Required for \u0027while\u0027 loops\\n # max_iterations: optional for \u0027while\u0027 loops\\n```\\n\\n## Template Resolution and Data Flow\\n\\nTemplate resolution is a core feature that allows tasks to access data from previous tasks, workflow inputs, and built-in variables using the `${variable_name}` syntax. This is essential for creating dynamic workflows where tasks pass data to each other.\\n\\n### Basic Template Syntax\\n\\n```yaml\\n# Access workflow inputs\\nscript: |\\n customer_name = \\\"${customer_name}\\\" # From workflow inputs\\n \\n# Access previous task outputs\\nscript: |\\n previous_result = ${data_fetch_task} # Entire output from data_fetch_task\\n specific_field = ${data_fetch_task.customer_id} # Specific field\\n \\n# Access built-in variables\\nscript: |\\n workflow_id = \\\"${workflow_id}\\\"\\n execution_id = \\\"${execution_id}\\\"\\n timestamp = \\\"${current_timestamp}\\\"\\n```\\n\\n### Available Built-in Variables\\n\\n| Variable | Description | Example Value |\\n|----------|-------------|---------------|\\n| `${workflow_id}` | Current workflow identifier | `customer_processor` |\\n| `${execution_id}` | Unique execution identifier | `exec_12345_abcde` |\\n| `${namespace}` | Workflow namespace | `production` |\\n| `${current_timestamp}` | ISO timestamp | `2024-01-15T10:30:00Z` |\\n| `${started_at}` | Workflow start time | `2024-01-15T10:29:45Z` |\\n\\n### Task Output Resolution\\n\\nTasks can access outputs from other tasks they depend on:\\n\\n```yaml\\ntasks:\\n - id: fetch_customer\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: process_customer\\n type: script\\n depends_on: [fetch_customer] # Required for template access\\n previous_node: fetch_customer\\n script: |\\n # Access entire response\\n customer_data = ${fetch_customer}\\n \\n # Access specific fields\\n customer_name = \\\"${fetch_customer.name}\\\"\\n customer_email = \\\"${fetch_customer.email}\\\"\\n \\n # Process the data\\n result = {\\n \\\"processed_name\\\": customer_name.upper(),\\n \\\"domain\\\": customer_email.split(\\\"@\\\")[1]\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n```\\n\\n### AI Agent Output Resolution\\n\\nAI agent tasks return structured output that requires specific access patterns:\\n\\n```yaml\\n# AI Agent task\\n- id: ai_analysis_task\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Analyze the customer data and provide insights.\\\"\\n user_message: \\\"Customer data: ${customer_data}\\\"\\n\\n# Accessing AI agent output in subsequent tasks\\n- id: process_ai_response\\n type: script\\n depends_on: [ai_analysis_task]\\n previous_node: ai_analysis_task\\n script: |\\n # CORRECT: AI agents return structured output\\n agent_output = ${ai_analysis_task}\\n ai_response = agent_output[\u0027ai_response\u0027].strip()\\n \\n # Additional metadata available\\n token_usage = agent_output.get(\u0027token_usage\u0027, {})\\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\\n \\n # Process the AI response\\n processed_analysis = {\\n \\\"analysis\\\": ai_response,\\n \\\"tokens_used\\\": token_usage.get(\u0027total_tokens\u0027, 0),\\n \\\"model\\\": model_used,\\n \\\"processed_at\\\": \\\"${current_timestamp}\\\"\\n }\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(processed_analysis)}\\\")\\n\\n# INCORRECT: Direct access won\u0027t work\\n- id: incorrect_ai_access\\n type: script\\n depends_on: [ai_analysis_task]\\n script: |\\n # \\u274c This will not work correctly\\n ai_response = ${ai_analysis_task} # Returns full agent object\\n # ai_response is now a dict, not the actual response text\\n```\\n\\n### AI Agent Response Structure\\n\\nAI agent tasks return a structured object with the following format:\\n\\n```json\\n{\\n \\\"ai_response\\\": \\\"The actual AI-generated text response\\\",\\n \\\"model\\\": \\\"gpt-4-turbo-preview\\\",\\n \\\"token_usage\\\": {\\n \\\"prompt_tokens\\\": 150,\\n \\\"completion_tokens\\\": 75,\\n \\\"total_tokens\\\": 225\\n },\\n \\\"estimated_cost\\\": {\\n \\\"amount\\\": 0.00337,\\n \\\"currency\\\": \\\"USD\\\"\\n },\\n \\\"provider\\\": \\\"openai\\\",\\n \\\"timestamp\\\": \\\"2024-01-15T10:30:00Z\\\"\\n}\\n```\\n\\n### Loop Task Data Access and Dependencies\\n\\nLoop tasks have a special execution context that differs from regular workflow tasks. Understanding how to access data within loops is crucial for building functional workflows.\\n\\n#### Loop Task Internal Data Access\\n\\nWithin loop tasks, data access works through injected context files rather than template resolution:\\n\\n```yaml\\n# Loop task with state variables and internal task dependencies\\n- id: data_processing_loop\\n type: loop\\n loop_type: for\\n max_iterations: 3\\n state_variables:\\n processed_items: []\\n total_processed: 0\\n error_count: 0\\n loop_tasks:\\n # First task in loop\\n - id: fetch_data\\n type: script\\n script: |\\n # Access loop state variables through injected context\\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\\n processed_items = loop_state.get(\u0027processed_items\u0027, [])\\n \\n # Access workflow inputs and external task outputs\\n external_data = inputs.get(\u0027previous_workflow_task\u0027, {})\\n iteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable\\n \\n # Process data\\n batch_data = fetch_batch_data(iteration_num)\\n \\n outputs = {\\\"batch_data\\\": batch_data, \\\"batch_size\\\": len(batch_data)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n # Second task depends on first task in same loop\\n - id: process_data\\n type: script\\n depends_on: [fetch_data] # \\u2705 CORRECT: Dependencies within loop tasks work normally\\n script: |\\n # \\u2705 CORRECT: Access previous loop task through inputs context\\n fetch_result = inputs.get(\u0027fetch_data\u0027, {})\\n batch_data = fetch_result.get(\u0027batch_data\u0027, [])\\n \\n # \\u274c INCORRECT: This won\u0027t work in loop context\\n # batch_data = ${fetch_data.batch_data} # Template resolution fails in loops\\n \\n # Access and update state variables\\n current_processed = loop_state.get(\u0027processed_items\u0027, [])\\n current_total = loop_state.get(\u0027total_processed\u0027, 0)\\n \\n # Process the batch\\n processed_batch = process_batch(batch_data)\\n current_processed.extend(processed_batch)\\n \\n # Update state variables using state_updates\\n state_updates[\u0027processed_items\u0027] = current_processed\\n state_updates[\u0027total_processed\u0027] = current_total + len(processed_batch)\\n \\n outputs = {\\\"processed_count\\\": len(processed_batch)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n print(f\\\"__STATE_UPDATES__ {json.dumps(state_updates)}\\\") # \\u2705 REQUIRED for state updates\\n```\\n\\n#### Key Differences in Loop Task Data Access\\n\\n| Access Pattern | Regular Tasks | Loop Tasks |\\n|---------------|---------------|------------|\\n| **Previous Task Output** | `${previous_task}` | `inputs.get(\u0027field_name\u0027, default)` (flattened) |\\n| **State Variables** | N/A | `loop_state.get(\u0027variable_name\u0027, default)` |\\n| **State Updates** | N/A | `state_updates[\u0027variable_name\u0027] = value` |\\n| **Workflow Inputs** | `\\\"${input_name}\\\"` | `inputs.get(\u0027input_name\u0027, \u0027default\u0027)` |\\n| **Dependencies** | `depends_on: [task_id]` | Same: `depends_on: [task_id]` |\\n\\n#### Critical: Loop Task Output Flattening\\n\\n**Important:** The loop executor flattens task outputs into individual fields in the `inputs` dictionary, rather than storing them as nested objects under the task name.\\n\\n```yaml\\n# First loop task outputs:\\noutputs = {\\\"company_name\\\": \\\"ACME Corp\\\", \\\"company_id\\\": \\\"123\\\", \\\"status\\\": \\\"success\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n\\n# Second loop task accessing the data:\\nscript: |\\n # \\u274c INCORRECT: This won\u0027t work - no nested object\\n # company_data = inputs.get(\u0027first_task\u0027, {})\\n # company_name = company_data.get(\u0027company_name\u0027)\\n \\n # \\u2705 CORRECT: Access flattened fields directly\\n company_name = inputs.get(\u0027company_name\u0027, \u0027\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\\n \\n print(f\\\"Company: {company_name}, ID: {company_id}, Status: {task_status}\\\")\\n```\\n\\n#### Complete Loop Task Example\\n\\n```yaml\\n- id: financial_processing_loop\\n type: loop\\n loop_type: for\\n max_iterations: 3\\n iteration_variable: company_index\\n depends_on: [list_companies] # External dependency\\n \\n state_variables:\\n processed_companies: []\\n total_extractions: 0\\n successful_extractions: 0\\n \\n loop_tasks:\\n - id: process_current_company\\n type: script\\n script: |\\n # Access external workflow task data\\n companies_data = inputs.get(\u0027list_companies\u0027, {})\\n company_idx = inputs.get(\u0027company_index\u0027, 0)\\n \\n # Access state variables\\n processed = loop_state.get(\u0027processed_companies\u0027, [])\\n \\n # Process company\\n company = companies_data[\u0027companies\u0027][company_idx]\\n result = process_company(company)\\n \\n outputs = {\\\"company_name\\\": company[\u0027name\u0027], \\\"status\\\": \\\"success\\\"}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: extract_financial_data\\n type: script\\n depends_on: [process_current_company] # Depend on previous loop task\\n requirements:\\n - requests\u003e=2.28.0\\n - some-ai-library\u003e=1.0.0\\n script: |\\n # \\u2705 CORRECT: Access flattened outputs from previous loop task\\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n task_status = inputs.get(\u0027status\u0027, \u0027\u0027)\\n \\n # \\u274c INCORRECT: Loop executor flattens outputs, no nested access\\n # company_data = inputs.get(\u0027process_current_company\u0027, {})\\n # company_name = company_data.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n \\n # \\u274c INCORRECT: Template resolution doesn\u0027t work in loops\\n # company_data = ${process_current_company}\\n \\n # Access external data\\n companies_list = inputs.get(\u0027list_companies\u0027, {})\\n \\n # Perform AI extraction\\n extraction_result = extract_with_ai({\\\"company_name\\\": company_name, \\\"company_id\\\": company_id})\\n \\n # Update state variables\\n current_processed = loop_state.get(\u0027processed_companies\u0027, [])\\n current_total = loop_state.get(\u0027total_extractions\u0027, 0)\\n current_successful = loop_state.get(\u0027successful_extractions\u0027, 0)\\n \\n current_processed.append({\\n \\\"company_name\\\": company_name,\\n \\\"extraction_result\\\": extraction_result\\n })\\n \\n # State updates are required for persistence\\n state_updates[\u0027processed_companies\u0027] = current_processed\\n state_updates[\u0027total_extractions\u0027] = current_total + 1\\n state_updates[\u0027successful_extractions\u0027] = current_successful + (1 if extraction_result else 0)\\n \\n outputs = {\\\"extracted\\\": bool(extraction_result)}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n print(f\\\"__STATE_UPDATES__ {json.dumps(state_updates)}\\\") # \\u2705 REQUIRED\\n\\n# Accessing loop task outputs in subsequent workflow tasks\\n- id: process_loop_results\\n type: script\\n depends_on: [financial_processing_loop]\\n previous_node: financial_processing_loop\\n script: |\\n # \\u2705 CORRECT: Access final state of loop\\n loop_result = ${financial_processing_loop}\\n loop_final_state = loop_result.get(\u0027final_state\u0027, {})\\n \\n # Access specific state variables from final state\\n processed_companies = loop_final_state.get(\u0027processed_companies\u0027, [])\\n total_processed = loop_final_state.get(\u0027total_extractions\u0027, 0)\\n successful_count = loop_final_state.get(\u0027successful_extractions\u0027, 0)\\n \\n # Access loop execution metadata\\n iterations_completed = loop_result.get(\u0027iterations_completed\u0027, 0)\\n \\n print(f\\\"Loop completed {iterations_completed} iterations\\\")\\n print(f\\\"Processed {len(processed_companies)} companies\\\")\\n print(f\\\"Success rate: {successful_count}/{total_processed}\\\")\\n \\n outputs = {\\n \\\"companies_processed\\\": len(processed_companies),\\n \\\"success_rate\\\": f\\\"{successful_count}/{total_processed}\\\",\\n \\\"iterations\\\": iterations_completed\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n#### Loop Task Context Injection and Output Flattening\\n\\nThe loop executor injects context through temporary files and flattens task outputs for easier access:\\n\\n```python\\n# Auto-injected at the start of each loop task script:\\nimport json\\nimport os\\n\\n# Load context from temporary files (auto-generated)\\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\\n inputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\\n\\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\\n loop_state = json.load(f) # Contains current state variables\\n\\noutputs = {}\\nstate_updates = {}\\n\\n# Your script code follows...\\n```\\n\\n#### Debugging Loop Task Data Access\\n\\nWhen debugging loop tasks, always print available keys to understand what data is accessible:\\n\\n```yaml\\n- id: debug_loop_task\\n type: script\\n depends_on: [previous_loop_task]\\n script: |\\n # Debug: Print all available data\\n print(f\\\"\\ud83d\\udd0d Available inputs: {list(inputs.keys())}\\\")\\n print(f\\\"\\ud83d\\udd0d Loop state keys: {list(loop_state.keys())}\\\")\\n \\n # Debug: Print specific values\\n for key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\\n value = inputs.get(key, \u0027NOT_FOUND\u0027)\\n print(f\\\"\\ud83d\\udd0d {key}: {value}\\\")\\n \\n # Now access the flattened data\\n company_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\n company_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n \\n # Process your data...\\n outputs = {\\\"processed\\\": f\\\"Company {company_name}\\\"}\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n#### Real Example: Task Output Flattening\\n\\n```yaml\\n# Task 1 outputs this structure:\\n__OUTPUTS__ {\\\"company_name\\\": \\\"LTTS\\\", \\\"company_id\\\": \\\"123\\\", \\\"fy_folders\\\": [{\\\"id\\\": \\\"456\\\", \\\"name\\\": \\\"FY24\\\"}], \\\"status\\\": \\\"success\\\"}\\n\\n# Task 2 receives flattened inputs:\\ninputs = {\\n \\\"folder_id\\\": \\\"...\\\", # Workflow input\\n \\\"gemini_api_key\\\": \\\"...\\\", # Workflow input \\n \\\"list_company_folders\\\": {...}, # External task output\\n \\\"company_index\\\": 0, # Loop iteration variable\\n \\\"company_name\\\": \\\"LTTS\\\", # \\u2705 Flattened from Task 1\\n \\\"company_id\\\": \\\"123\\\", # \\u2705 Flattened from Task 1\\n \\\"fy_folders\\\": [{\\\"id\\\": \\\"456\\\", \\\"name\\\": \\\"FY24\\\"}], # \\u2705 Flattened from Task 1\\n \\\"status\\\": \\\"success\\\", # \\u2705 Flattened from Task 1\\n \\\"total_extractions\\\": 0, # Loop state variable\\n \\\"processed_companies\\\": [] # Loop state variable\\n}\\n```\\n\\n#### Loop Task Special Output Fields\\n\\n| Field | Description | Example Access |\\n|-------|-------------|----------------|\\n| `final_state` | Final values of all state variables | `${loop_task.final_state}` |\\n| `final_state[\\\"variable_name\\\"]` | Specific state variable from final state | `${loop_task.final_state[\\\"total_processed\\\"]}` |\\n| `iterations_completed` | Number of iterations executed | `${loop_task.iterations_completed}` |\\n\\n#### Safe Template Resolution Pattern for Loop States\\n\\nWhen accessing loop final state fields, use this safe parsing pattern:\\n\\n```yaml\\nscript: |\\n try:\\n # Template resolution with safety checks\\n final_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\\n processed_count_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_count\\\"]}\u0027\u0027\u0027\\n items_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_items\\\"]}\u0027\u0027\u0027\\n iterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027\\n \\n # Safe parsing with fallbacks\\n final_state = json.loads(final_state_str) if not final_state_str.startswith(\u0027UNRESOLVED_\u0027) else {}\\n processed_count = int(processed_count_str) if processed_count_str.isdigit() else 0\\n processed_items = json.loads(items_str) if not items_str.startswith(\u0027UNRESOLVED_\u0027) else []\\n iterations = int(iterations_str) if iterations_str.isdigit() else 0\\n \\n print(f\\\"Successfully processed {processed_count} items in {iterations} iterations\\\")\\n \\n except Exception as e:\\n print(f\\\"Error accessing loop results: {str(e)}\\\")\\n # Handle error case\\n```\\n```\\n\\n### Real-World AI Agent Example\\n\\nBased on actual WhatsApp workflow patterns:\\n\\n```yaml\\n# AI greeting agent\\n- id: greeting_agent\\n type: ai_agent\\n config:\\n model_client_id: assistant_model\\n system_message: \\\"You are a helpful assistant. Generate a friendly greeting.\\\"\\n user_message: \\\"User said: ${user_message}\\\"\\n\\n# Send the AI response via API\\n- id: send_greeting\\n type: script\\n depends_on: [greeting_agent]\\n previous_node: greeting_agent\\n script: |\\n import requests\\n import json\\n \\n # CORRECT: Access AI response from structured output\\n agent_output = ${greeting_agent}\\n ai_greeting = agent_output[\u0027ai_response\u0027].strip()\\n \\n phone_number = \\\"${phone_number}\\\"\\n api_token = \\\"${API_TOKEN}\\\"\\n \\n print(f\\\"\\ud83e\\udd16 AI generated greeting: {ai_greeting}\\\")\\n \\n # Send via API\\n url = f\\\"https://api.example.com/messages/{phone_number}\\\"\\n headers = {\\\"Authorization\\\": f\\\"Bearer {api_token}\\\"}\\n \\n response = requests.post(url, headers=headers, json={\\n \\\"message\\\": ai_greeting\\n })\\n \\n result = {\\n \\\"status\\\": \\\"sent\\\",\\n \\\"ai_response\\\": ai_greeting,\\n \\\"api_response\\\": response.json(),\\n \\\"tokens_used\\\": agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\\n }\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n```\\n\\n### Complex Template Patterns\\n\\n```yaml\\n# Conditional templates with router outputs\\n- id: router_task\\n type: conditional_router\\n conditions:\\n - name: \\\"premium_customer\\\"\\n condition: \\\"${customer_data.tier} == \u0027premium\u0027\\\"\\n route: \\\"premium_processing\\\"\\n\\n- id: premium_handler\\n type: script\\n depends_on: [router_task]\\n previous_node: router_task\\n execute_on_routes: [\\\"premium_processing\\\"]\\n script: |\\n # Access router decision\\n selected_route = \\\"${router_task.selected_route}\\\"\\n route_data = ${router_task.route_data}\\n \\n # Process premium customer\\n customer = ${customer_data}\\n premium_benefits = calculate_benefits(customer)\\n\\n# Loop through arrays from previous tasks\\n- id: process_items\\n type: script\\n depends_on: [fetch_items]\\n previous_node: fetch_items\\n script: |\\n items = ${fetch_items.data}\\n processed_items = []\\n \\n for item in items:\\n processed_item = {\\n \\\"id\\\": item[\\\"id\\\"],\\n \\\"processed_at\\\": \\\"${current_timestamp}\\\",\\n \\\"status\\\": \\\"processed\\\"\\n }\\n processed_items.append(processed_item)\\n \\n print(f\\\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\\\")\\n\\n# Nested object access\\n- id: user_preferences\\n type: script\\n depends_on: [fetch_user]\\n previous_node: fetch_user\\n script: |\\n # Access nested user preferences\\n email_notifications = ${fetch_user.preferences.notifications.email}\\n theme_setting = \\\"${fetch_user.preferences.ui.theme}\\\"\\n language = \\\"${fetch_user.profile.language}\\\"\\n \\n settings = {\\n \\\"notifications\\\": email_notifications,\\n \\\"theme\\\": theme_setting,\\n \\\"locale\\\": language\\n }\\n print(f\\\"__OUTPUTS__ {json.dumps(settings)}\\\")\\n```\\n\\n### Template Resolution Rules\\n\\n1. **Dependency Requirement**: Tasks must have `depends_on` relationship to access other task outputs\\n2. **String vs Object**: Use quotes for string values, no quotes for objects/arrays\\n3. **Field Navigation**: Use dot notation for nested field access\\n4. **Error Handling**: Unresolved templates become `UNRESOLVED_variable_name`\\n\\n```yaml\\n# \\u2705 Correct template usage\\n- id: correct_task\\n depends_on: [previous_task]\\n previous_node: previous_task\\n script: |\\n string_value = \\\"${previous_task.name}\\\" # String with quotes\\n object_value = ${previous_task.data} # Object without quotes\\n nested_value = \\\"${previous_task.user.email}\\\" # Nested field\\n\\n# \\u274c Incorrect template usage \\n- id: incorrect_task\\n # Missing depends_on - template won\u0027t resolve\\n script: |\\n value = ${previous_task.name} # Will become UNRESOLVED_previous_task.name\\n```\\n\\n## Script Task Output Handling\\n\\nScript tasks must use the `__OUTPUTS__` marker pattern for proper output parsing by the workflow engine.\\n\\n### Required Output Pattern\\n\\n```yaml\\ntasks:\\n - id: example_script\\n type: script\\n script: |\\n # Your script logic here\\n data = process_data()\\n \\n # Required: Store outputs in outputs dict\\n outputs = {}\\n outputs[\u0027result\u0027] = data\\n outputs[\u0027status\u0027] = \u0027success\u0027\\n outputs[\u0027count\u0027] = len(data)\\n \\n # Required: Print with __OUTPUTS__ marker for engine parsing\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### Output Access in Subsequent Tasks\\n\\n```yaml\\ntasks:\\n - id: data_processor\\n type: script\\n script: |\\n # Process and store outputs\\n results = {\\\"processed_count\\\": 100, \\\"errors\\\": []}\\n \\n outputs = {}\\n outputs[\u0027results\u0027] = results\\n outputs[\u0027status\u0027] = \u0027completed\u0027\\n \\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n \\n - id: result_handler\\n type: script\\n depends_on: [data_processor]\\n script: |\\n # Access previous task outputs\\n processor_output = ${data_processor}\\n processed_count = processor_output[\u0027results\u0027][\u0027processed_count\u0027]\\n status = processor_output[\u0027status\u0027]\\n \\n print(f\\\"Previous task processed {processed_count} items with status: {status}\\\")\\n \\n # Store new outputs\\n outputs = {\u0027message\u0027: f\u0027Handled {processed_count} results\u0027}\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### Error Handling in Scripts\\n\\n```yaml\\ntasks:\\n - id: safe_script\\n type: script\\n script: |\\n try:\\n result = risky_operation()\\n outputs = {\u0027result\u0027: result, \u0027status\u0027: \u0027success\u0027}\\n except Exception as e:\\n outputs = {\u0027error\u0027: str(e), \u0027status\u0027: \u0027failed\u0027}\\n \\n # Always print outputs, even for errors\\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n## Triggers and Workflow Activation\\n\\nTriggers define how workflows are started automatically based on external events. They enable event-driven workflow execution.\\n\\n### Webhook Triggers\\n\\nWebhook triggers allow external systems to start workflows by sending HTTP requests.\\n\\n```yaml\\ntriggers:\\n - name: customer_webhook\\n type: webhook\\n description: \\\"Triggered when customer data is updated\\\"\\n config:\\n path: \\\"/webhooks/customer-updated\\\" # Webhook endpoint path\\n method: POST # HTTP method (GET, POST, PUT, etc.)\\n authentication:\\n type: token # none|token|signature|basic\\n token: \\\"${WEBHOOK_SECRET}\\\" # Required if type is token\\n header_name: \\\"X-Webhook-Token\\\" # Optional: custom header name\\n validation:\\n required_fields: [\\\"customer.id\\\", \\\"customer.email\\\"] # Required payload fields\\n content_type: \\\"application/json\\\" # Expected content type\\n input_mapping:\\n - webhook_field: customer.id # Field from webhook payload\\n workflow_input: customer_id # Map to workflow input\\n - webhook_field: customer.email\\n workflow_input: customer_email\\n - webhook_field: customer.updated_at\\n workflow_input: update_timestamp\\n```\\n\\n### Scheduled Triggers\\n\\nSchedule workflows to run at specific times or intervals.\\n\\n```yaml\\ntriggers:\\n - name: daily_report\\n type: schedule\\n description: \\\"Generate daily customer report\\\"\\n config:\\n cron: \\\"0 9 * * *\\\" # Daily at 9 AM (cron format)\\n timezone: \\\"UTC\\\" # Optional: timezone\\n enabled: true # Optional: enable/disable\\n max_instances: 1 # Optional: max concurrent instances\\n input_mapping:\\n - workflow_input: report_date\\n default_value: \\\"${current_date}\\\" # Use built-in variable\\n - workflow_input: report_type\\n default_value: \\\"daily\\\"\\n\\n - name: weekly_cleanup\\n type: schedule\\n description: \\\"Weekly database cleanup\\\"\\n config:\\n cron: \\\"0 2 * * 0\\\" # Weekly on Sunday at 2 AM\\n timezone: \\\"America/New_York\\\"\\n input_mapping:\\n - workflow_input: cleanup_before_date\\n default_value: \\\"${date_minus_7_days}\\\"\\n```\\n\\n### Event Triggers\\n\\nEvent triggers respond to system events from various sources.\\n\\n```yaml\\ntriggers:\\n - name: file_uploaded\\n type: event\\n description: \\\"Process uploaded files\\\"\\n config:\\n event_type: \\\"file.uploaded\\\" # Event type to listen for\\n source: \\\"storage_service\\\" # Event source system\\n filter: # Optional: event filtering\\n file_type: [\\\"csv\\\", \\\"json\\\", \\\"xlsx\\\"]\\n file_size_max: \\\"100MB\\\"\\n path_pattern: \\\"/uploads/customers/*\\\"\\n retry: # Optional: retry configuration\\n max_attempts: 3\\n initial_delay: \\\"5s\\\"\\n input_mapping:\\n - webhook_field: file.path\\n workflow_input: file_path\\n - webhook_field: file.size\\n workflow_input: file_size\\n - webhook_field: file.type\\n workflow_input: file_type\\n\\n - name: order_status_changed\\n type: event\\n description: \\\"Handle order status changes\\\"\\n config:\\n event_type: \\\"order.status_changed\\\"\\n source: \\\"order_management\\\"\\n filter:\\n status: [\\\"completed\\\", \\\"cancelled\\\", \\\"refunded\\\"]\\n priority: [\\\"high\\\", \\\"urgent\\\"]\\n input_mapping:\\n - webhook_field: order.id\\n workflow_input: order_id\\n - webhook_field: order.status\\n workflow_input: new_status\\n - webhook_field: order.customer_id\\n workflow_input: customer_id\\n```\\n\\n### Dependency Triggers\\n\\nDependency triggers start workflows when other workflows complete.\\n\\n```yaml\\ntriggers:\\n - name: post_processing\\n type: dependency\\n description: \\\"Run after main processing completes\\\"\\n config:\\n depends_on_workflow: \\\"main_data_processing\\\" # Workflow to depend on\\n depends_on_status: \\\"SUCCESS\\\" # SUCCESS|FAILED|COMPLETED\\n namespace: \\\"production\\\" # Optional: workflow namespace\\n timeout_minutes: 60 # Optional: wait timeout\\n input_mapping:\\n - webhook_field: execution_id # From completed workflow\\n workflow_input: source_execution_id\\n - webhook_field: workflow_outputs.processed_count\\n workflow_input: items_processed\\n```\\n\\n## Model Client Configuration\\n\\nBefore defining AI-related tasks (ai_agent, chatbot, continuous_conversation), you must configure model clients. Model clients define which AI models to use and how to authenticate with them.\\n\\n### Basic Structure\\n\\n```yaml\\nmodel_clients:\\n - id: unique_identifier # Used to reference this model in tasks\\n provider: provider_name # AI provider (openai, anthropic, etc.)\\n config:\\n model: model_name # Specific model to use\\n api_key: your_api_key # Authentication key\\n # Additional provider-specific options...\\n```\\n\\n### Supported Providers\\n\\n#### OpenAI Provider\\n\\n```yaml\\nmodel_clients:\\n - id: gpt4_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview # Available: gpt-4, gpt-4-turbo, gpt-3.5-turbo\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 0.7 # Optional: 0.0-2.0\\n max_tokens: 2000 # Optional: Maximum response tokens\\n timeout: 30 # Optional: Request timeout (seconds)\\n```\\n\\n#### Anthropic Provider\\n\\n```yaml\\nmodel_clients:\\n - id: claude_model\\n provider: anthropic\\n config:\\n model: claude-3-sonnet-20240229 # Available: claude-3-opus, claude-3-sonnet, claude-3-haiku\\n api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n temperature: 0.7 # Optional: 0.0-1.0\\n max_tokens: 1500 # Optional: Maximum response tokens\\n timeout: 30 # Optional: Request timeout (seconds)\\n```\\n\\n#### OpenRouter Provider\\n\\n```yaml\\nmodel_clients:\\n - id: openrouter_model\\n provider: openrouter\\n config:\\n model: anthropic/claude-3-sonnet # Available: Various models via OpenRouter\\n api_key: \\\"sk-or-your-openrouter-api-key\\\"\\n base_url: \\\"https://openrouter.ai/api/v1\\\" # Optional: Custom endpoint\\n temperature: 0.7\\n max_tokens: 2000\\n```\\n\\n#### Multiple Model Clients\\n\\nYou can define multiple model clients and reference them by their `id` in tasks:\\n\\n```yaml\\nmodel_clients:\\n - id: fast_model\\n provider: openai\\n config:\\n model: gpt-3.5-turbo\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 0.3\\n \\n - id: smart_model\\n provider: anthropic\\n config:\\n model: claude-3-opus-20240229\\n api_key: \\\"sk-ant-your-anthropic-api-key\\\"\\n temperature: 0.7\\n \\n - id: creative_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"sk-your-openai-api-key\\\"\\n temperature: 1.2\\n```\\n\\n### Using Model Clients in Tasks\\n\\nReference model clients in AI tasks using the `model_client_id` field:\\n\\n```yaml\\ntasks:\\n - id: analysis_task\\n type: ai_agent\\n config:\\n model_client_id: smart_model # References the claude model above\\n system_message: \\\"You are a data analyst...\\\"\\n user_message: \\\"Analyze this data: ${input_data}\\\"\\n \\n - id: creative_task\\n type: ai_agent\\n config:\\n model_client_id: creative_model # References the creative GPT-4 model\\n system_message: \\\"You are a creative writer...\\\"\\n user_message: \\\"Write a story about ${topic}\\\"\\n```\\n\\n### Environment Variables for API Keys\\n\\nFor security, use environment variables instead of hardcoding API keys:\\n\\n```yaml\\nmodel_clients:\\n - id: secure_model\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"${OPENAI_API_KEY}\\\" # References environment variable\\n temperature: 0.7\\n```\\n\\n### Common Configuration Options\\n\\n| Field | Description | Required | Default |\\n|-------|-------------|----------|---------|\\n| `id` | Unique identifier for referencing in tasks | Yes | - |\\n| `provider` | AI provider name | Yes | - |\\n| `config.model` | Specific model name | Yes | - |\\n| `config.api_key` | Authentication API key | Yes | - |\\n| `config.temperature` | Response randomness (0.0-2.0) | No | 0.7 |\\n| `config.max_tokens` | Maximum response tokens | No | Provider default |\\n| `config.timeout` | Request timeout in seconds | No | 30 |\\n| `config.base_url` | Custom API endpoint | No | Provider default |\\n\\n## Secrets Management and Environment Variables\\n\\nProper secrets management is crucial for security. Never hardcode sensitive information like API keys, passwords, or tokens directly in workflow YAML files.\\n\\n### Using Environment Variables\\n\\nThe recommended approach is to use environment variables for all sensitive data:\\n\\n```yaml\\n# \\u2705 RECOMMENDED: Use environment variables\\nmodel_clients:\\n - id: secure_openai\\n provider: openai\\n config:\\n model: gpt-4-turbo-preview\\n api_key: \\\"${OPENAI_API_KEY}\\\" # References environment variable\\n\\n# HTTP tasks with environment variables\\ntasks:\\n - id: secure_api_call\\n type: http\\n url: \\\"${API_BASE_URL}/customers\\\" # Environment variable for URL\\n headers:\\n Authorization: \\\"Bearer ${API_TOKEN}\\\" # Environment variable for token\\n X-API-Key: \\\"${EXTERNAL_API_KEY}\\\" # Environment variable for API key\\n```\\n\\n### Environment Variable Patterns\\n\\n```yaml\\n# Database connections\\n- id: database_task\\n type: storage\\n config:\\n storage_type: database\\n connection_string: \\\"${DATABASE_URL}\\\" # Full connection string\\n # or individual components:\\n host: \\\"${DB_HOST}\\\"\\n port: \\\"${DB_PORT}\\\"\\n database: \\\"${DB_NAME}\\\"\\n username: \\\"${DB_USER}\\\"\\n password: \\\"${DB_PASSWORD}\\\"\\n\\n# Third-party service credentials\\n- id: external_service\\n type: http\\n url: \\\"${SERVICE_BASE_URL}/api/v1/data\\\"\\n headers:\\n Authorization: \\\"Bearer ${SERVICE_TOKEN}\\\"\\n X-Client-ID: \\\"${SERVICE_CLIENT_ID}\\\"\\n X-Client-Secret: \\\"${SERVICE_CLIENT_SECRET}\\\"\\n\\n# MCP server credentials\\n- id: mcp_task\\n type: mcp\\n mcp_server: \\\"@agentic/mcp-notion-server\\\"\\n environment_vars:\\n NOTION_TOKEN: \\\"${NOTION_API_TOKEN}\\\"\\n WORKSPACE_ID: \\\"${NOTION_WORKSPACE_ID}\\\"\\n```\\n\\n### Security Best Practices\\n\\n#### \\u274c What NOT to Do\\n\\n```yaml\\n# DON\u0027T: Hardcode secrets\\nmodel_clients:\\n - id: insecure_example\\n provider: openai\\n config:\\n api_key: \\\"sk-actual-api-key-here\\\" # SECURITY RISK!\\n\\n# DON\u0027T: Put secrets in comments\\ntasks:\\n - id: bad_example\\n type: http\\n url: \\\"https://api.example.com\\\"\\n # API key is: sk-12345... DON\u0027T DO THIS!\\n headers:\\n Authorization: \\\"Bearer sk-real-token\\\" # SECURITY RISK!\\n```\\n\\n#### \\u2705 What TO Do\\n\\n```yaml\\n# DO: Use environment variables\\nmodel_clients:\\n - id: secure_example\\n provider: openai\\n config:\\n api_key: \\\"${OPENAI_API_KEY}\\\" # Secure approach\\n\\n# DO: Use different variables for different environments\\n- id: environment_aware_task\\n type: http\\n url: \\\"${API_BASE_URL}\\\" # Different URLs per environment\\n headers:\\n Authorization: \\\"Bearer ${API_TOKEN}\\\" # Different tokens per environment\\n X-Environment: \\\"${ENVIRONMENT_NAME}\\\" # Track environment\\n\\n# DO: Use descriptive variable names\\nenvironment_vars:\\n CUSTOMER_API_TOKEN: \\\"${CUSTOMER_SERVICE_TOKEN}\\\"\\n BILLING_API_KEY: \\\"${BILLING_SERVICE_API_KEY}\\\"\\n NOTIFICATION_WEBHOOK_SECRET: \\\"${NOTIFICATION_SECRET}\\\"\\n```\\n\\n### Environment Variable Naming Conventions\\n\\n| Pattern | Example | Use Case |\\n|---------|---------|----------|\\n| `SERVICE_API_KEY` | `OPENAI_API_KEY` | Service API keys |\\n| `SERVICE_BASE_URL` | `CUSTOMER_API_BASE_URL` | Service endpoints |\\n| `DB_*` | `DB_HOST`, `DB_PASSWORD` | Database credentials |\\n| `WEBHOOK_*` | `WEBHOOK_SECRET_TOKEN` | Webhook authentication |\\n| `*_TIMEOUT` | `HTTP_REQUEST_TIMEOUT` | Timeout configurations |\\n\\n### Dynamic Environment Variables\\n\\nUse template resolution to create dynamic environment variable names:\\n\\n```yaml\\n# Different API keys per customer\\n- id: customer_specific_api\\n type: http\\n url: \\\"https://api.partner.com/data\\\"\\n headers:\\n Authorization: \\\"Bearer ${PARTNER_API_KEY_${customer_tier}}\\\" # Resolves to PARTNER_API_KEY_PREMIUM\\n\\n# Environment-specific configurations\\n- id: env_specific_task\\n type: storage\\n config:\\n connection_string: \\\"${DATABASE_URL_${environment}}\\\" # DATABASE_URL_PRODUCTION\\n```\\n\\n### Secrets in Different Task Types\\n\\n```yaml\\n# AI Agent with multiple secure credentials\\n- id: secure_ai_task\\n type: ai_agent\\n config:\\n model_client_id: secure_model\\n system_message: \\\"You are a secure assistant with access to customer data.\\\"\\n user_message: \\\"Process customer ${customer_id}\\\"\\n environment_vars:\\n CUSTOMER_DB_TOKEN: \\\"${CUSTOMER_DATABASE_TOKEN}\\\"\\n AUDIT_LOG_KEY: \\\"${AUDIT_LOGGING_API_KEY}\\\"\\n\\n# Script task with secure file access\\n- id: secure_script_task\\n type: script\\n environment_vars:\\n AWS_ACCESS_KEY_ID: \\\"${AWS_ACCESS_KEY}\\\"\\n AWS_SECRET_ACCESS_KEY: \\\"${AWS_SECRET_KEY}\\\"\\n S3_BUCKET_NAME: \\\"${SECURE_BUCKET_NAME}\\\"\\n script: |\\n import boto3\\n import os\\n \\n # Credentials automatically available from environment\\n s3_client = boto3.client(\u0027s3\u0027)\\n bucket = os.environ[\u0027S3_BUCKET_NAME\u0027]\\n \\n # Process secure files...\\n\\n# Storage task with encrypted connections\\n- id: secure_storage_task\\n type: storage\\n config:\\n storage_type: database\\n connection_string: \\\"${ENCRYPTED_DB_CONNECTION_STRING}\\\"\\n ssl_mode: \\\"require\\\"\\n ssl_cert: \\\"${SSL_CERT_PATH}\\\"\\n ssl_key: \\\"${SSL_KEY_PATH}\\\"\\n```\\n\\n### Environment Setup Examples\\n\\n#### Development Environment (.env file)\\n```bash\\n# Development API keys (non-production)\\nOPENAI_API_KEY=sk-dev-api-key-here\\nCUSTOMER_API_BASE_URL=https://api-dev.example.com\\nDATABASE_URL=postgresql://dev:pass@localhost:5432/workflow_dev\\nENVIRONMENT_NAME=development\\n```\\n\\n#### Production Environment\\n```bash\\n# Production API keys (secure)\\nOPENAI_API_KEY=sk-prod-api-key-here\\nCUSTOMER_API_BASE_URL=https://api.example.com\\nDATABASE_URL=postgresql://prod_user:secure_pass@prod-db:5432/workflow_prod\\nENVIRONMENT_NAME=production\\n```\\n\\n## Workflow Execution and Orchestration\\n\\nUnderstanding how workflows execute and how tasks are orchestrated is essential for building reliable, efficient workflows.\\n\\n### Execution Flow\\n\\nWorkflows follow a directed acyclic graph (DAG) execution model:\\n\\n1. **Initialization**: Workflow starts with inputs and model clients\\n2. **Task Scheduling**: Tasks are scheduled based on dependencies\\n3. **Parallel Execution**: Independent tasks run simultaneously\\n4. **Data Flow**: Outputs flow between tasks via template resolution\\n5. **Completion**: Workflow completes when all tasks finish\\n\\n```yaml\\n# Example execution flow\\ntasks:\\n # Stage 1: Independent tasks (run in parallel)\\n - id: fetch_customer # Runs immediately\\n type: http\\n url: \\\"https://api.example.com/customers/${customer_id}\\\"\\n \\n - id: fetch_products # Runs in parallel with fetch_customer\\n type: http\\n url: \\\"https://api.example.com/products\\\"\\n \\n # Stage 2: Dependent tasks (wait for Stage 1)\\n - id: validate_customer # Waits for fetch_customer\\n type: script\\n depends_on: [fetch_customer]\\n previous_node: fetch_customer\\n script: # validation logic...\\n \\n - id: filter_products # Waits for fetch_products\\n type: script\\n depends_on: [fetch_products]\\n previous_node: fetch_products\\n script: # filtering logic...\\n \\n # Stage 3: Final processing (waits for Stage 2)\\n - id: generate_recommendations # Waits for both validation and filtering\\n type: ai_agent\\n depends_on: [validate_customer, filter_products]\\n previous_node: validate_customer # UI hierarchy under customer validation\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Generate product recommendations for customer.\\\"\\n user_message: |\\n Customer: ${validate_customer}\\n Available products: ${filter_products}\\n```\\n\\n### Task States and Lifecycle\\n\\nEach task goes through these states during execution:\\n\\n| State | Description | Next States |\\n|-------|-------------|-------------|\\n| `PENDING` | Task created, waiting for dependencies | `RUNNING`, `SKIPPED` |\\n| `RUNNING` | Task is actively executing | `COMPLETED`, `FAILED` |\\n| `COMPLETED` | Task finished successfully | - |\\n| `FAILED` | Task encountered an error | `RUNNING` (if retries) |\\n| `SKIPPED` | Task skipped due to conditions | - |\\n| `TIMEOUT` | Task exceeded timeout limit | `RUNNING` (if retries) |\\n\\n### Conditional Execution\\n\\nTasks can be executed conditionally based on routes or conditions:\\n\\n```yaml\\n# Router-based conditional execution\\n- id: data_router\\n type: conditional_router\\n depends_on: [fetch_data]\\n previous_node: fetch_data\\n conditions:\\n - name: \\\"large_dataset\\\"\\n condition: \\\"${fetch_data.record_count} \u003e 10000\\\"\\n route: \\\"batch_processing\\\"\\n - name: \\\"small_dataset\\\"\\n condition: \\\"${fetch_data.record_count} \u003c= 1000\\\"\\n route: \\\"direct_processing\\\"\\n default_route: \\\"standard_processing\\\"\\n\\n# Tasks that execute on specific routes\\n- id: batch_processor\\n type: script\\n depends_on: [data_router]\\n previous_node: data_router\\n execute_on_routes: [\\\"batch_processing\\\"] # Only runs for large datasets\\n script: |\\n # Handle large dataset in batches\\n data = ${fetch_data}\\n process_in_batches(data, batch_size=1000)\\n\\n- id: direct_processor\\n type: script\\n depends_on: [data_router]\\n previous_node: data_router\\n execute_on_routes: [\\\"direct_processing\\\"] # Only runs for small datasets\\n script: |\\n # Process small dataset directly\\n data = ${fetch_data}\\n process_directly(data)\\n```\\n\\n### Error Handling and Retries\\n\\nConfigure retry policies for resilient workflow execution:\\n\\n```yaml\\n# Workflow-level retry policy\\nretry:\\n retryOn: [\\\"TEMPORARY_FAILURE\\\", \\\"TIMEOUT\\\", \\\"NETWORK_ERROR\\\"]\\n maxAttempts: 3\\n initialDelay: \\\"1s\\\"\\n maxDelay: \\\"60s\\\"\\n backoffMultiplier: 2.0\\n\\ntasks:\\n # Task-specific retry policy (overrides workflow-level)\\n - id: external_api_call\\n type: http\\n url: \\\"https://unreliable-api.example.com/data\\\"\\n retry:\\n maxAttempts: 5 # More retries for unreliable API\\n initialDelay: \\\"2s\\\"\\n maxDelay: \\\"120s\\\"\\n backoffMultiplier: 2.0\\n retryOn: [\\\"TIMEOUT\\\", \\\"HTTP_5XX\\\", \\\"NETWORK_ERROR\\\"]\\n \\n # Task with no retries\\n - id: critical_operation\\n type: script\\n retry:\\n maxAttempts: 1 # No retries for critical operations\\n script: |\\n # This operation should not be retried if it fails\\n perform_critical_database_operation()\\n```\\n\\n### Parallel Execution Patterns\\n\\n```yaml\\n# Fan-out pattern: One task triggers multiple parallel tasks\\n- id: data_source\\n type: http\\n url: \\\"https://api.example.com/data\\\"\\n\\n- id: process_customers # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process customer data from data_source\\n\\n- id: process_orders # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process order data from data_source\\n\\n- id: process_products # Parallel processing\\n type: script\\n depends_on: [data_source]\\n previous_node: data_source\\n script: # process product data from data_source\\n\\n# Fan-in pattern: Multiple tasks feed into one task\\n- id: generate_report # Waits for all parallel tasks\\n type: ai_agent\\n depends_on: [process_customers, process_orders, process_products]\\n previous_node: process_customers # UI hierarchy\\n config:\\n model_client_id: gpt4_model\\n system_message: \\\"Generate comprehensive report from processed data.\\\"\\n user_message: |\\n Customer data: ${process_customers}\\n Order data: ${process_orders}\\n Product data: ${process_products}\\n```\\n\\n### Resource Management\\n\\nControl resource usage and task distribution:\\n\\n```yaml\\n# Worker group targeting\\n- id: cpu_intensive_task\\n type: script\\n worker_group: \\\"high-cpu-workers\\\" # Target specific worker group\\n script: |\\n # CPU-intensive processing\\n perform_heavy_computation()\\n\\n- id: gpu_ml_task\\n type: ai_agent\\n worker_group: \\\"gpu-workers\\\" # Target GPU workers\\n config:\\n model_client_id: large_model\\n # Large model processing...\\n\\n# Timeout configuration\\n- id: long_running_task\\n type: script\\n timeout_seconds: 3600 # 1 hour timeout\\n script: |\\n # Long-running data processing\\n process_large_dataset()\\n\\n# Workflow-level timeout\\ntimeout_seconds: 7200 # 2 hour workflow timeout\\n```\\n\\n## Task Type Configurations\\n\\nThis section provides the complete YAML configuration for each supported task type.\\n\\n**Supported Task Types:**\\n- `script` - Execute Python code with full workflow context\\n- `http` - Make HTTP requests to external APIs\\n- `ai_agent` - Integrate with AI models for intelligent processing \\n- `loop` - Execute tasks repeatedly (for/while loops)\\n- `conditional_router` - Route execution based on conditions\\n- `mcp` - Execute tools via Model Context Protocol servers\\n- `storage` - Data persistence and retrieval operations\\n- `continuous_conversation` - Interactive AI conversations\\n- `human_input` - Tasks requiring human input or approval\\n- `chatbot` - Session-based chatbot conversations\\n\\n### Script Tasks\\n\\nExecute Python code with full workflow context access.\\n\\n```yaml\\n- id: script_task\\n name: \\\"Python Script Task\\\"\\n type: script\\n description: \\\"Execute Python code with dependencies and environment\\\"\\n \\n # Script configuration - script field is directly under task, NOT under config\\n script: | # REQUIRED: Python code to execute\\n import json\\n import requests\\n \\n # Access workflow inputs\\n input_value = \\\"${workflow_input}\\\"\\n \\n # Access previous task outputs\\n previous_result = ${previous_task}\\n \\n # Your processing logic\\n result = {\\\"processed\\\": True, \\\"value\\\": input_value}\\n \\n # Output results (must be JSON serializable)\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n \\n # Alternative: use \u0027code\u0027 field instead of \u0027script\u0027\\n code: | # Alternative to \u0027script\u0027 field\\n print(\\\"Hello World\\\")\\n \\n # Package management\\n requirements: # Optional: Python packages\\n - requests==2.31.0\\n - pandas==1.5.3\\n - numpy==1.24.3\\n \\n # Environment configuration\\n environment: # Optional: Environment variables\\n API_KEY: \\\"${secrets.api_key}\\\"\\n DEBUG: \\\"true\\\"\\n CUSTOM_VAR: \\\"value\\\"\\n \\n environment_vars: # Alternative to \u0027environment\u0027\\n DATABASE_URL: \\\"${secrets.db_url}\\\"\\n \\n # Secrets configuration\\n secrets_config: # Optional: Secrets access\\n api_key: \\\"secret_name\\\"\\n db_password: \\\"db_secret\\\"\\n```\\n\\n### HTTP Tasks\\n\\nMake HTTP requests to external APIs and services.\\n\\n```yaml\\n- id: http_task\\n name: \\\"HTTP API Call\\\"\\n type: http\\n description: \\\"Call external REST API\\\"\\n \\n # HTTP configuration\\n method: POST # Default: GET\\n url: \\\"https://api.example.com/data\\\" # REQUIRED: Target URL\\n \\n # Headers configuration\\n headers: # Optional: HTTP headers\\n Content-Type: \\\"application/json\\\"\\n Authorization: \\\"Bearer ${secrets.api_token}\\\"\\n X-Custom-Header: \\\"workflow-engine\\\"\\n User-Agent: \\\"WorkflowEngine/1.0\\\"\\n \\n # Request body\\n body: | # Optional: Request body (for POST/PUT)\\n {\\n \\\"customer_id\\\": \\\"${customer_id}\\\",\\n \\\"timestamp\\\": \\\"${current_timestamp}\\\",\\n \\\"data\\\": ${previous_task}\\n }\\n \\n # Response validation\\n expected_status: [200, 201, 202] # Default: [200]\\n expectedStatus: [200, 201] # Alternative field name\\n \\n # Additional HTTP options\\n timeout_seconds: 30 # HTTP request timeout\\n follow_redirects: true # Follow HTTP redirects\\n verify_ssl: true # Verify SSL certificates\\n```\\n\\n### AI Agent Tasks\\n\\nIntegrate with AI models for intelligent processing.\\n\\n```yaml\\n- id: ai_agent_task\\n name: \\\"AI Processing Agent\\\"\\n type: ai_agent\\n description: \\\"Process data using AI model\\\"\\n \\n # AI configuration\\n config:\\n model_client_id: gpt4_model # REQUIRED: Reference to model client\\n \\n system_message: | # Optional: System prompt\\n You are a data analyst. Analyze the provided data and extract insights.\\n Return your analysis in JSON format with the following structure:\\n {\\n \\\"insights\\\": [\\\"insight1\\\", \\\"insight2\\\"],\\n \\\"recommendations\\\": [\\\"rec1\\\", \\\"rec2\\\"],\\n \\\"confidence_score\\\": 0.95\\n }\\n \\n user_message: | # REQUIRED: User prompt with template resolution\\n Please analyze this customer data:\\n ${previous_task}\\n \\n Customer ID: ${customer_id}\\n Date Range: ${date_range}\\n \\n # Model parameters\\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\\n max_tokens: 1500 # Optional: Maximum response tokens\\n top_p: 1.0 # Optional: Nucleus sampling\\n frequency_penalty: 0.0 # Optional: Frequency penalty\\n presence_penalty: 0.0 # Optional: Presence penalty\\n \\n # Tools configuration for function calling\\n tools: # Optional: List of MCP tool names\\n - current_time # Time-related tools\\n - relative_time\\n - days_in_month\\n - weather_api # Weather tools\\n - database_query # Database tools\\n - notification_service # Communication tools\\n \\n # Required for tool resolution\\n tenant_id: \\\"your-tenant-uuid\\\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\\n \\n # Input/output format\\n input_format: \\\"text\\\" # Optional: text|json|structured\\n output_format: \\\"structured\\\" # Optional: text|json|structured\\n\\n### Important: AI Agent Output Access\\n\\nAI agent tasks return structured objects, not direct text responses. Always access the AI response using this pattern:\\n\\n```yaml\\n# In subsequent script tasks\\nscript: |\\n # CORRECT: Access AI response from structured output\\n agent_output = ${ai_agent_task}\\n ai_response = agent_output[\u0027ai_response\u0027].strip()\\n \\n # Additional metadata available\\n tokens_used = agent_output.get(\u0027token_usage\u0027, {}).get(\u0027total_tokens\u0027, 0)\\n cost_estimate = agent_output.get(\u0027estimated_cost\u0027, {}).get(\u0027amount\u0027, 0.0)\\n model_used = agent_output.get(\u0027model\u0027, \u0027unknown\u0027)\\n \\n print(f\\\"AI Response: {ai_response}\\\")\\n print(f\\\"Tokens used: {tokens_used}\\\")\\n print(f\\\"Estimated cost: ${cost_estimate:.6f}\\\")\\n```\\n\\n```yaml\\n# In workflow outputs - specify the ai_response field\\noutputs:\\n ai_analysis:\\n type: string\\n source: ai_agent_task.ai_response # Access the ai_response field\\n description: \\\"AI-generated analysis text\\\"\\n \\n total_tokens:\\n type: integer \\n source: ai_agent_task.token_usage.total_tokens\\n description: \\\"Total tokens consumed\\\"\\n```\\n```\\n\\n### Loop Tasks\\n\\nExecute tasks repeatedly with for-loop and while-loop patterns.\\n\\n```yaml\\n# For-Loop (Fixed Iterations)\\n- id: for_loop_task\\n name: \\\"For Loop Processor\\\"\\n type: loop\\n description: \\\"Process data in fixed batches\\\"\\n \\n # Loop configuration\\n loop_type: for # REQUIRED: for|while\\n max_iterations: 10 # REQUIRED for \u0027for\u0027 loops\\n iteration_variable: batch_index # Default: iteration\\n \\n # State variables (persistent across iterations)\\n state_variables:\\n total_processed: 0\\n error_count: 0\\n batch_results: []\\n \\n # Tasks to execute in each iteration\\n loop_tasks: # REQUIRED: Tasks to run in loop\\n - id: process_batch\\n name: \\\"Process Current Batch\\\"\\n type: script\\n script: |\\n batch_idx = ${batch_index} # Access iteration variable\\n total = ${total_processed} # Access state variable\\n \\n # Process current batch\\n result = process_batch_data(batch_idx)\\n \\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n \\n - id: update_state\\n name: \\\"Update Loop State\\\"\\n type: script\\n depends_on: [process_batch]\\n script: |\\n # Update state variables\\n new_total = ${total_processed} + ${process_batch.items_processed}\\n \\n result = {\\\"total_processed\\\": new_total}\\n print(f\\\"__OUTPUTS__ {json.dumps(result)}\\\")\\n\\n# While-Loop (Conditional)\\n- id: while_loop_task\\n name: \\\"While Loop Quality Processor\\\"\\n type: loop\\n description: \\\"Process until quality threshold met\\\"\\n \\n # Loop configuration\\n loop_type: while # REQUIRED: for|while\\n exit_condition: \\\"${average_quality} \u003e= 85 || ${iteration_count} \u003e= 20\\\" # REQUIRED for \u0027while\u0027\\n max_iterations: 50 # Optional: Safety limit\\n iteration_variable: iteration_count # Default: iteration\\n \\n # State variables\\n state_variables:\\n average_quality: 0\\n items_processed: 0\\n quality_scores: []\\n \\n # Loop tasks\\n loop_tasks:\\n - id: assess_quality\\n type: script\\n script: |\\n current_quality = calculate_quality()\\n print(f\\\"__OUTPUTS__ {json.dumps({\u0027quality\u0027: current_quality})}\\\")\\n```\\n\\n\\n### Conditional Router Tasks\\n\\nRoute execution to different paths based on conditions.\\n\\n```yaml\\n- id: conditional_router_task\\n name: \\\"Data Type Router\\\"\\n type: conditional_router\\n description: \\\"Route processing based on data characteristics\\\"\\n \\n # Router configuration\\n router_type: condition # Default: condition\\n \\n # Routing conditions\\n conditions: # REQUIRED: List of conditions\\n - name: \\\"large_dataset\\\"\\n condition: \\\"${data_size} \u003e 10000\\\"\\n route: \\\"batch_processing\\\"\\n - name: \\\"small_dataset\\\" \\n condition: \\\"${data_size} \u003c= 1000\\\"\\n route: \\\"direct_processing\\\"\\n - name: \\\"medium_dataset\\\"\\n condition: \\\"${data_size} \u003e 1000 \u0026\u0026 ${data_size} \u003c= 10000\\\"\\n route: \\\"standard_processing\\\"\\n \\n default_route: \\\"standard_processing\\\" # REQUIRED: Default route if no conditions match\\n \\n # Alternative: evaluation script for complex routing\\n evaluation_script: | # Optional: Custom routing logic\\n if data_type == \\\"json\\\":\\n return \\\"json_processing\\\"\\n elif data_type == \\\"xml\\\":\\n return \\\"xml_processing\\\"\\n else:\\n return \\\"default_processing\\\"\\n \\n # Route mapping for complex scenarios\\n route_mapping: # Optional: Route name mappings\\n json_processing: \\\"specialized_json\\\"\\n xml_processing: \\\"specialized_xml\\\"\\n\\n# Tasks that execute on specific routes\\n- id: batch_handler\\n name: \\\"Handle Large Dataset\\\"\\n type: script\\n execute_on_routes: [\\\"batch_processing\\\"] # REQUIRED: Routes this task runs on\\n depends_on: [conditional_router_task]\\n script: |\\n print(\\\"Processing large dataset in batches...\\\")\\n\\n- id: direct_handler\\n name: \\\"Handle Small Dataset\\\" \\n type: script\\n execute_on_routes: [\\\"direct_processing\\\"]\\n depends_on: [conditional_router_task]\\n script: |\\n print(\\\"Processing small dataset directly...\\\")\\n```\\n\\n### MCP Tasks\\n\\nExecute tools via Model Context Protocol servers.\\n\\n```yaml\\n- id: mcp_task\\n name: \\\"MCP Tool Execution\\\"\\n type: mcp\\n description: \\\"Execute external tool via MCP server\\\"\\n \\n # MCP server configuration\\n mcp_server: \\\"@agentic/mcp-notion-server\\\" # REQUIRED: MCP server package\\n tool_name: \\\"update_database\\\" # REQUIRED: Tool to execute\\n runtime: \\\"npx\\\" # REQUIRED: npx|uvx\\n \\n # Server startup\\n server_args: [] # Optional: Server startup arguments\\n server_startup_timeout: 30 # Optional: Server startup timeout (seconds)\\n \\n # Tool execution\\n tool_arguments: # REQUIRED: Arguments for the tool\\n database_id: \\\"${secrets.notion_database_id}\\\"\\n properties:\\n title: \\\"Workflow Results\\\"\\n status: \\\"Completed\\\"\\n processed_count: \\\"${previous_task.count}\\\"\\n timestamp: \\\"${current_timestamp}\\\"\\n \\n # Environment configuration\\n environment_vars: # Optional: Environment variables\\n NOTION_TOKEN: \\\"${secrets.notion_token}\\\"\\n DEBUG: \\\"true\\\"\\n \\n # Timeout configuration\\n timeout_seconds: 120 # Optional: Tool execution timeout\\n```\\n\\n### Storage Tasks\\n\\nManage data persistence and retrieval operations.\\n\\n```yaml\\n- id: storage_task\\n name: \\\"Data Storage Operation\\\"\\n type: storage\\n description: \\\"Save/load data to/from storage\\\"\\n \\n # Storage operation\\n operation: save # REQUIRED: save|load|delete|list\\n \\n # Storage configuration\\n storage_type: \\\"file\\\" # REQUIRED: file|database|cache\\n path: \\\"/data/results/${execution_id}.json\\\" # REQUIRED: Storage path\\n \\n # Data configuration\\n data: ${previous_task} # REQUIRED for \u0027save\u0027: Data to store\\n format: \\\"json\\\" # Optional: json|yaml|csv|parquet\\n compression: \\\"gzip\\\" # Optional: gzip|bz2|none\\n \\n # Metadata\\n metadata: # Optional: Additional metadata\\n workflow_id: \\\"${workflow_id}\\\"\\n created_at: \\\"${current_timestamp}\\\"\\n source_task: \\\"previous_task\\\"\\n \\n # File options\\n create_directories: true # Optional: Create parent directories\\n overwrite: true # Optional: Overwrite existing files\\n backup: false # Optional: Create backup before overwrite\\n \\n # Database options (when storage_type: \\\"database\\\")\\n table_name: \\\"workflow_results\\\" # Required for database storage\\n connection_string: \\\"${secrets.db_url}\\\" # Required for database storage\\n \\n # Cache options (when storage_type: \\\"cache\\\")\\n cache_key: \\\"result_${execution_id}\\\" # Required for cache storage\\n ttl: 3600 # Optional: Time to live (seconds)\\n```\\n\\n\\n### Continuous Conversation Tasks\\n\\nCreate interactive AI-powered conversations.\\n\\n```yaml\\n- id: conversation_task\\n name: \\\"AI Conversation Interface\\\"\\n type: continuous_conversation\\n description: \\\"Interactive conversation with AI assistant\\\"\\n \\n # Conversation configuration\\n config:\\n # AI configuration\\n ai_config:\\n model_client_id: support_assistant # REQUIRED: AI model to use\\n system_prompt: | # REQUIRED: System instructions\\n You are a helpful customer support representative.\\n Be friendly, professional, and solution-oriented.\\n \\n Guidelines:\\n - Always be polite and respectful\\n - Ask clarifying questions when needed\\n - Provide clear and actionable solutions\\n \\n temperature: 0.7 # Optional: AI creativity\\n max_tokens: 1000 # Optional: Response length limit\\n \\n # UI configuration\\n ui_config:\\n title: \\\"Customer Support Chat\\\" # Optional: Interface title\\n placeholder: \\\"How can I help you today?\\\" # Optional: Input placeholder\\n theme: \\\"light\\\" # Optional: light|dark\\n show_timestamp: true # Optional: Show message timestamps\\n \\n # Termination conditions\\n termination:\\n keywords: [\\\"goodbye\\\", \\\"bye\\\", \\\"thanks\\\", \\\"done\\\"] # Optional: End conversation keywords\\n max_turns: 50 # Optional: Maximum conversation turns\\n timeout_minutes: 30 # Optional: Conversation timeout\\n inactivity_timeout: 10 # Optional: Inactivity timeout (minutes)\\n \\n # Context configuration\\n context:\\n system_context: \\\"Customer support conversation\\\" # Optional: Context description\\n user_id: \\\"${customer_id}\\\" # Optional: User identifier\\n session_data: ${previous_task} # Optional: Additional context data\\n```\\n\\n### Human Input Tasks\\n\\nTasks requiring human input or approval.\\n\\n```yaml\\n- id: human_input_task\\n name: \\\"Human Input Required\\\"\\n type: human_input\\n description: \\\"Wait for human input or approval\\\"\\n \\n # Human input configuration\\n config:\\n prompt: \\\"Please review and approve the processed data\\\" # REQUIRED: Prompt for human\\n input_type: \\\"approval\\\" # REQUIRED: approval|text|selection\\n \\n # Input options (for selection type)\\n options: [\\\"approve\\\", \\\"reject\\\", \\\"request_changes\\\"] # Optional: For selection input\\n \\n # Timeout configuration\\n timeout_minutes: 60 # Optional: How long to wait for input (minutes)\\n \\n # Conversation settings\\n conversation_mode: false # Optional: Add to conversation history\\n next_task: \\\"process_approval\\\" # Optional: Task to trigger after input\\n end_condition: \\\"bye|exit|quit\\\" # Optional: Keywords to end conversation\\n \\n # Input validation\\n required: true # Optional: Whether input is required\\n \\n # Additional context\\n context_data: ${previous_task} # Optional: Data to show to human\\n```\\n\\n### ChatBot Tasks\\n\\nSession-based chatbot conversations with streaming support.\\n\\n```yaml\\n- id: chatbot_task\\n name: \\\"AI Chatbot\\\"\\n type: chatbot\\n description: \\\"Interactive chatbot with session management\\\"\\n \\n # Chatbot configuration\\n config:\\n # AI model configuration\\n model_client_id: gpt4_model # REQUIRED: Reference to model client\\n \\n system_message: | # REQUIRED: System prompt for chatbot\\n You are a helpful AI assistant. Be friendly, helpful, and concise.\\n Answer questions clearly and ask for clarification when needed.\\n \\n # Session management\\n session_id: \\\"${session_id}\\\" # Optional: Session identifier (auto-generated if not provided)\\n max_history_messages: 20 # Optional: Maximum conversation history to maintain\\n \\n # Model parameters\\n temperature: 0.7 # Optional: Model creativity (0.0-2.0)\\n max_tokens: 1000 # Optional: Maximum response tokens\\n stream: true # Optional: Enable streaming responses\\n \\n # Tools configuration for function calling\\n tools: # Optional: List of MCP tool names\\n - current_time # Time-related tools\\n - relative_time\\n - days_in_month\\n - weather_forecast # Weather tools\\n - send_notification # Communication tools\\n \\n # Required for tool resolution\\n tenant_id: \\\"your-tenant-uuid\\\" # REQUIRED when using tools: Tenant ID for MCP tool resolution\\n \\n # UI configuration\\n ui_config:\\n title: \\\"AI Assistant\\\" # Optional: Chat interface title\\n placeholder: \\\"Type your message...\\\" # Optional: Input placeholder\\n theme: \\\"light\\\" # Optional: light|dark\\n show_typing_indicator: true # Optional: Show typing animation\\n \\n # Response configuration\\n response_config:\\n max_retries: 3 # Optional: Retry failed requests\\n timeout_seconds: 30 # Optional: Response timeout\\n \\n # Input message (for single-turn mode)\\n user_message: \\\"${user_input}\\\" # Optional: Direct message input\\n \\n # Context data\\n context_data: ${previous_task} # Optional: Additional context\\n```\\n\\n---\\n\\n## Common Errors and Corrections\\n\\nThis section documents frequent mistakes encountered when creating workflows and their proper corrections.\\n\\n### 1. Missing Required `id` Field\\n\\n**\\u274c Error:**\\n```yaml\\nname: \\\"My Workflow\\\"\\ndescription: \\\"A workflow without id\\\"\\ntasks:\\n - name: \\\"Task 1\\\"\\n type: script\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nid: \\\"my_workflow\\\" # REQUIRED: Unique workflow identifier\\nname: \\\"My Workflow\\\"\\ndescription: \\\"A workflow with proper id\\\"\\ntasks:\\n - id: task_1 # REQUIRED: Unique task identifier\\n name: \\\"Task 1\\\"\\n type: script\\n```\\n\\n### 2. Incorrect Task Structure (Object vs Array)\\n\\n**\\u274c Error:**\\n```yaml\\ntasks:\\n task_1: # Wrong: tasks as object mapping\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n task_2:\\n type: ai_agent\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ntasks:\\n - id: task_1 # Correct: tasks as array with id field\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n - id: task_2\\n type: ai_agent\\n```\\n\\n### 3. Incorrect Inputs Format (Object vs Array)\\n\\n**\\u274c Error:**\\n```yaml\\ninputs:\\n user_name: # Wrong: inputs as object mapping\\n type: string\\n required: true\\n age:\\n type: integer\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ninputs:\\n - name: user_name # Correct: inputs as array with name field\\n type: string\\n required: true\\n - name: age\\n type: integer\\n```\\n\\n### 4. Script Configuration Location Error\\n\\n**\\u274c Error:**\\n```yaml\\n- id: script_task\\n type: script\\n config: # Wrong: script under config\\n script: |\\n print(\\\"Hello\\\")\\n requirements:\\n - requests\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: script_task\\n type: script\\n script: | # Correct: script directly under task\\n print(\\\"Hello\\\")\\n # requirements can be at task level if needed\\n```\\n\\n### 5. Individual Task Inputs (Not Supported)\\n\\n**\\u274c Error:**\\n```yaml\\n- id: task_1\\n type: script\\n script: \\\"print(\u0027hello\u0027)\\\"\\n inputs: # Wrong: tasks don\u0027t have individual inputs\\n search_query: ${search_query}\\n prev_result: ${previous_task}\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: task_1\\n type: script\\n depends_on: [previous_task] # Use depends_on for data access\\n script: |\\n # Access workflow inputs directly\\n search_query = \\\"${search_query}\\\"\\n # Access previous task outputs via template resolution\\n prev_result = ${previous_task}\\n```\\n\\n### 6. Incorrect Script Output Handling\\n\\n**\\u274c Error:**\\n```yaml\\nscript: |\\n result = process_data()\\n outputs[\u0027result\u0027] = result # Wrong: missing __OUTPUTS__ pattern\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nscript: |\\n result = process_data()\\n \\n # Required: Use outputs dict and __OUTPUTS__ marker\\n outputs = {}\\n outputs[\u0027result\u0027] = result\\n outputs[\u0027status\u0027] = \u0027success\u0027\\n \\n import json\\n print(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\\n\\n### 7. AI Agent Configuration Errors\\n\\n**\\u274c Error:**\\n```yaml\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: my_model\\n prompt: \\\"Process this data: ${data}\\\" # Wrong: use user_message\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: ai_task\\n type: ai_agent\\n config:\\n model_client_id: my_model\\n system_message: \\\"You are a helpful assistant.\\\"\\n user_message: \\\"Process this data: ${data}\\\" # Correct field name\\n```\\n\\n### 8. Missing Dependencies for Template Resolution\\n\\n**\\u274c Error:**\\n```yaml\\n- id: task_2\\n type: script\\n # Missing depends_on\\n script: |\\n data = ${task_1} # Will become UNRESOLVED_task_1\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\n- id: task_2\\n type: script\\n depends_on: [task_1] # Required for template access\\n script: |\\n data = ${task_1} # Now resolves correctly\\n```\\n\\n### 9. Incorrect Model Client Configuration\\n\\n**\\u274c Error:**\\n```yaml\\nmodel_clients:\\n my_model: # Wrong: object mapping format\\n type: openai\\n api_key: \\\"sk-...\\\"\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nmodel_clients:\\n - id: my_model # Correct: array format with id field\\n provider: openai\\n config:\\n model: \\\"gpt-4\\\"\\n api_key: \\\"sk-...\\\"\\n```\\n\\n### 10. Empty String Literals Causing Python Syntax Errors\\n\\n**\\u274c Error:**\\n```yaml\\nscript: |\\n def process_data(path=\\\"\\\"): # Wrong: empty string can cause syntax issues\\n current_path = f\\\"{path}.{key}\\\" if path else key\\n \\n # This can lead to: SyntaxError: unterminated string literal\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\nscript: |\\n def process_data(path=None): # Correct: use None instead of empty string\\n current_path = f\\\"{path}.{key}\\\" if path else key\\n # Or use path = path or \\\"default_value\\\"\\n```\\n\\n### 11. YAML Indentation and Structure Issues\\n\\n**\\u274c Error:**\\n```yaml\\ntasks:\\n- id: task_1\\n type: script\\nscript: | # Wrong: inconsistent indentation\\n print(\\\"hello\\\")\\n```\\n\\n**\\u2705 Correct:**\\n```yaml\\ntasks:\\n- id: task_1\\n type: script\\n script: | # Correct: consistent indentation\\n print(\\\"hello\\\")\\n```\\n\\n### Best Practices Summary\\n\\n1. **Always include required `id` fields** for workflows and tasks\\n2. **Use array format** for `tasks`, `inputs`, and `model_clients`\\n3. **Script field goes directly under task**, not under `config`\\n4. **No individual task inputs** - use template resolution instead\\n5. **Use `depends_on`** to access previous task outputs\\n6. **Print `__OUTPUTS__`** in scripts for proper output parsing\\n7. **Use `user_message`** (not `prompt`) for AI agent tasks\\n8. **Avoid empty string literals** - use `None` or meaningful defaults\\n9. **Follow consistent YAML indentation** throughout the file\\n\\n## MCP Tools and Function Calling\\n\\nAI agents and chatbots can use MCP (Model Context Protocol) tools for function calling, enabling them to execute external tools and services.\\n\\n### Tools Configuration\\n\\nBoth `ai_agent` and `chatbot` tasks support tools configuration:\\n\\n```yaml\\n# AI Agent with MCP tools\\n- id: ai_with_tools\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful assistant with access to various tools.\\n Use the available tools when needed to help users.\\n user_message: \\\"What time is it and what\u0027s the weather like?\\\"\\n \\n # Tools configuration\\n tools: # List of MCP tool names\\n - current_time # Time tools\\n - relative_time\\n - days_in_month\\n - weather_forecast # Weather tools\\n - database_query # Database tools\\n - send_notification # Communication tools\\n \\n tenant_id: \\\"550e8400-e29b-41d4-a716-446655440000\\\" # Required for tool resolution\\n\\n# Chatbot with MCP tools\\n- id: chatbot_with_tools\\n type: chatbot\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful chatbot with access to tools.\\n Use tools when users ask for specific information or actions.\\n \\n # Tools configuration (same format as AI agent)\\n tools:\\n - current_time\\n - weather_api\\n - user_preferences\\n \\n tenant_id: \\\"550e8400-e29b-41d4-a716-446655440000\\\" # Required for tool resolution\\n```\\n\\n### How Tools Work\\n\\n1. **Tool Resolution**: The workflow engine resolves tool names to actual MCP server deployments using the `tenant_id`\\n2. **Function Calling**: When the AI model decides to use a tool, it calls the `tool_executor` function\\n3. **Tool Execution**: The engine executes the specified tool via the appropriate MCP server\\n4. **Response Integration**: Tool results are fed back to the AI model for response generation\\n\\n### Tool Configuration Requirements\\n\\n| Field | Required | Description |\\n|-------|----------|-------------|\\n| `tools` | No | List of MCP tool names available to the AI |\\n| `tenant_id` | **Yes** (when using tools) | Tenant UUID for tool resolution |\\n\\n### Example: Complete AI Agent with Tools\\n\\n```yaml\\nid: intelligent_assistant\\nname: \\\"Intelligent Assistant with Tools\\\"\\n\\nmodel_clients:\\n - id: gpt4_model\\n provider: openai\\n config:\\n model: gpt-4o-mini\\n api_key: \\\"${OPENAI_API_KEY}\\\"\\n temperature: 0.3\\n\\n# Tenant ID can be set at workflow level for all tasks\\ntenant_id: \\\"0572fa8d-d7c3-47db-9219-ef40b80d42b7\\\"\\n\\ntasks:\\n - id: assistant_with_tools\\n type: ai_agent\\n config:\\n model_client_id: gpt4_model\\n system_message: |\\n You are a helpful assistant with access to various tools:\\n - Time-related tools: current_time, relative_time, days_in_month\\n - Weather tools: weather_forecast\\n - Communication: send_notification\\n \\n Use these tools when users ask for:\\n - Current time or date information\\n - Weather forecasts\\n - Sending notifications\\n \\n Always explain what tool you\u0027re using and why.\\n \\n user_message: \\\"${user_query}\\\"\\n \\n # MCP tools configuration\\n tools:\\n - current_time\\n - relative_time\\n - days_in_month\\n - weather_forecast\\n - send_notification\\n \\n # Tool resolution requires tenant_id\\n tenant_id: \\\"0572fa8d-d7c3-47db-9219-ef40b80d42b7\\\"\\n \\n temperature: 0.7\\n max_tokens: 1000\\n\\ninputs:\\n - name: user_query\\n type: string\\n required: true\\n default: \\\"What time is it and can you send a notification to user123?\\\"\\n\\noutputs:\\n ai_response:\\n type: string\\n source: assistant_with_tools.ai_response\\n description: \\\"AI response using tools when needed\\\"\\n```\\n\\n### Tool Execution Flow\\n\\n1. **User Input**: User provides a query that might require tool usage\\n2. **AI Analysis**: AI model analyzes the query and decides if tools are needed\\n3. **Tool Calling**: AI calls `tool_executor(deployment_id, tool_name, arguments)`\\n4. **Tool Resolution**: Engine resolves tool name to MCP server using tenant_id\\n5. **Tool Execution**: MCP server executes the requested tool\\n6. **Result Processing**: Tool results are returned to the AI model\\n7. **Response Generation**: AI generates final response incorporating tool results\\n\\n### Best Practices for Tools\\n\\n1. **Clear System Messages**: Explain available tools and when to use them\\n2. **Tool Categories**: Group related tools (time, weather, database, etc.)\\n3. **Error Handling**: AI models handle tool execution failures gracefully\\n4. **Tool Documentation**: Include tool descriptions in system messages\\n5. **Tenant Configuration**: Ensure tenant_id matches your MCP server setup\\n\\n---\\n\\n*This comprehensive reference covers all supported task types and their YAML configurations. Use this as a quick reference when building workflows with different task types.*\", \"format\": \"markdown\"}, \"status\": \"success\", \"successful_url\": \"http://172.17.0.1:5000/api/docs/yaml-reference\"}\n",
"return_code": 0,
"status": "completed",
"stderr": "",
"successful_url": "http://172.17.0.1:5000/api/docs/yaml-reference",
"task_id": "fetch_documentation"
},
"vector_search": {
"execution_details": {
"actual_result": {
"output": "Processing vector search for query: \u0027how to create a loop node and use it?\u0027\nExtracted 1 content sections from documentation\nSplit into 246 semantic chunks\nInitializing embeddings...\nEmbedding 246 text chunks...\nPerforming semantic similarity search...\nFound 12 semantic matches\nReturning 8 most relevant results:\n 1. content... [Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Context Injection and Output Flattening] (score: 0.580)\n 2. content... [Header 1: For-Loop (Fixed Iterations)] (score: 0.560)\n 3. content... [Header 1: Loop configuration] (score: 0.555)\n__OUTPUTS__ {\"search_results\": [{\"path\": \"content.chunk_67\", \"content\": \"#### Loop Task Context Injection and Output Flattening \\nThe loop executor injects context through temporary files and flattens task outputs for easier access: \\n```python\\n# Auto-injected at the start of each loop task script:\\nimport json\\nimport os\\n\\n# Load context from temporary files (auto-generated)\\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\\ninputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\\n\\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\\nloop_state = json.load(f) # Contains current state variables\\n\\noutputs = {}\\nstate_updates = {}\\n\\n# Your script code follows...\\n```\", \"relevance_score\": 0.5802012712597091, \"distance_score\": 0.7235398292541504, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Context Injection and Output Flattening\", \"source\": \"content\", \"content_length\": 662}, {\"path\": \"content.chunk_177\", \"content\": \"# For-Loop (Fixed Iterations)\\n- id: for_loop_task\\nname: \\\"For Loop Processor\\\"\\ntype: loop\\ndescription: \\\"Process data in fixed batches\\\"\", \"relevance_score\": 0.5599206689699311, \"distance_score\": 0.7859672904014587, \"headers\": \"Header 1: For-Loop (Fixed Iterations)\", \"source\": \"content\", \"content_length\": 132}, {\"path\": \"content.chunk_178\", \"content\": \"# Loop configuration\\nloop_type: for # REQUIRED: for|while\\nmax_iterations: 10 # REQUIRED for \u0027for\u0027 loops\\niteration_variable: batch_index # Default: iteration\", \"relevance_score\": 0.5549294896027557, \"distance_score\": 0.8020307421684265, \"headers\": \"Header 1: Loop configuration\", \"source\": \"content\", \"content_length\": 198}, {\"path\": \"content.chunk_68\", \"content\": \"#### Debugging Loop Task Data Access \\nWhen debugging loop tasks, always print available keys to understand what data is accessible: \\n```yaml\\n- id: debug_loop_task\\ntype: script\\ndepends_on: [previous_loop_task]\\nscript: |\\n# Debug: Print all available data\\nprint(f\\\" Available inputs: {list(inputs.keys())}\\\")\\nprint(f\\\" Loop state keys: {list(loop_state.keys())}\\\")\\n\\n# Debug: Print specific values\\nfor key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\\nvalue = inputs.get(key, \u0027NOT_FOUND\u0027)\\nprint(f\\\" {key}: {value}\\\")\\n\\n# Now access the flattened data\\ncompany_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\ncompany_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n\\n# Process your data...\\noutputs = {\\\"processed\\\": f\\\"Company {company_name}\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\", \"relevance_score\": 0.5546090302952729, \"distance_score\": 0.8030719757080078, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Debugging Loop Task Data Access\", \"source\": \"content\", \"content_length\": 772}, {\"path\": \"content.chunk_62\", \"content\": \"#### Complete Loop Task Example \\n```yaml\\n- id: financial_processing_loop\\ntype: loop\\nloop_type: for\\nmax_iterations: 3\\niteration_variable: company_index\\ndepends_on: [list_companies] # External dependency\\n\\nstate_variables:\\nprocessed_companies: []\\ntotal_extractions: 0\\nsuccessful_extractions: 0\\n\\nloop_tasks:\\n- id: process_current_company\\ntype: script\\nscript: |\\n# Access external workflow task data\\ncompanies_data = inputs.get(\u0027list_companies\u0027, {})\\ncompany_idx = inputs.get(\u0027company_index\u0027, 0)\\n\\n# Access state variables\\nprocessed = loop_state.get(\u0027processed_companies\u0027, [])\\n\\n# Process company\\ncompany = companies_data[\u0027companies\u0027][company_idx]\\nresult = process_company(company)\\n\\noutputs = {\\\"company_name\\\": company[\u0027name\u0027], \\\"status\\\": \\\"success\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\", \"relevance_score\": 0.5519489927660971, \"distance_score\": 0.8117616176605225, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Complete Loop Task Example\", \"source\": \"content\", \"content_length\": 784}, {\"path\": \"content.chunk_72\", \"content\": \"#### Safe Template Resolution Pattern for Loop States \\nWhen accessing loop final state fields, use this safe parsing pattern: \\n```yaml\\nscript: |\\ntry:\\n# Template resolution with safety checks\\nfinal_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\\nprocessed_count_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_count\\\"]}\u0027\u0027\u0027\\nitems_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_items\\\"]}\u0027\u0027\u0027\\niterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027\", \"relevance_score\": 0.5517749447607281, \"distance_score\": 0.8123331069946289, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Safe Template Resolution Pattern for Loop States\", \"source\": \"content\", \"content_length\": 432}, {\"path\": \"content.chunk_56\", \"content\": \"#### Loop Task Internal Data Access \\nWithin loop tasks, data access works through injected context files rather than template resolution: \\n```yaml\\n# Loop task with state variables and internal task dependencies\\n- id: data_processing_loop\\ntype: loop\\nloop_type: for\\nmax_iterations: 3\\nstate_variables:\\nprocessed_items: []\\ntotal_processed: 0\\nerror_count: 0\\nloop_tasks:\\n# First task in loop\\n- id: fetch_data\\ntype: script\\nscript: |\\n# Access loop state variables through injected context\\ncurrent_total = loop_state.get(\u0027total_processed\u0027, 0)\\nprocessed_items = loop_state.get(\u0027processed_items\u0027, [])\\n\\n# Access workflow inputs and external task outputs\\nexternal_data = inputs.get(\u0027previous_workflow_task\u0027, {})\\niteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable\", \"relevance_score\": 0.551580913054056, \"distance_score\": 0.8129706382751465, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Internal Data Access\", \"source\": \"content\", \"content_length\": 770}, {\"path\": \"content.chunk_81\", \"content\": \"# Loop through arrays from previous tasks\\n- id: process_items\\ntype: script\\ndepends_on: [fetch_items]\\nprevious_node: fetch_items\\nscript: |\\nitems = ${fetch_items.data}\\nprocessed_items = [] \\nfor item in items:\\nprocessed_item = {\\n\\\"id\\\": item[\\\"id\\\"],\\n\\\"processed_at\\\": \\\"${current_timestamp}\\\",\\n\\\"status\\\": \\\"processed\\\"\\n}\\nprocessed_items.append(processed_item) \\nprint(f\\\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\\\")\", \"relevance_score\": 0.5490929478973238, \"distance_score\": 0.821185290813446, \"headers\": \"Header 1: Loop through arrays from previous tasks\", \"source\": \"content\", \"content_length\": 422}], \"search_query\": \"how to create a loop node and use it?\", \"total_results\": 12, \"total_chunks\": 246, \"status\": \"success\"}\n",
"return_code": 0,
"status": "completed",
"stderr": ""
},
"duration_seconds": 10.630208,
"end_time": "2025-08-14T11:23:18.363877",
"message_sent": true,
"start_time": "2025-08-14T11:23:07.733669",
"timestamp": "2025-08-14T11:23:18.363877",
"worker_executed": true,
"workers_notified": true
},
"output": "Processing vector search for query: \u0027how to create a loop node and use it?\u0027\nExtracted 1 content sections from documentation\nSplit into 246 semantic chunks\nInitializing embeddings...\nEmbedding 246 text chunks...\nPerforming semantic similarity search...\nFound 12 semantic matches\nReturning 8 most relevant results:\n 1. content... [Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Context Injection and Output Flattening] (score: 0.580)\n 2. content... [Header 1: For-Loop (Fixed Iterations)] (score: 0.560)\n 3. content... [Header 1: Loop configuration] (score: 0.555)\n__OUTPUTS__ {\"search_results\": [{\"path\": \"content.chunk_67\", \"content\": \"#### Loop Task Context Injection and Output Flattening \\nThe loop executor injects context through temporary files and flattens task outputs for easier access: \\n```python\\n# Auto-injected at the start of each loop task script:\\nimport json\\nimport os\\n\\n# Load context from temporary files (auto-generated)\\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\\ninputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\\n\\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\\nloop_state = json.load(f) # Contains current state variables\\n\\noutputs = {}\\nstate_updates = {}\\n\\n# Your script code follows...\\n```\", \"relevance_score\": 0.5802012712597091, \"distance_score\": 0.7235398292541504, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Context Injection and Output Flattening\", \"source\": \"content\", \"content_length\": 662}, {\"path\": \"content.chunk_177\", \"content\": \"# For-Loop (Fixed Iterations)\\n- id: for_loop_task\\nname: \\\"For Loop Processor\\\"\\ntype: loop\\ndescription: \\\"Process data in fixed batches\\\"\", \"relevance_score\": 0.5599206689699311, \"distance_score\": 0.7859672904014587, \"headers\": \"Header 1: For-Loop (Fixed Iterations)\", \"source\": \"content\", \"content_length\": 132}, {\"path\": \"content.chunk_178\", \"content\": \"# Loop configuration\\nloop_type: for # REQUIRED: for|while\\nmax_iterations: 10 # REQUIRED for \u0027for\u0027 loops\\niteration_variable: batch_index # Default: iteration\", \"relevance_score\": 0.5549294896027557, \"distance_score\": 0.8020307421684265, \"headers\": \"Header 1: Loop configuration\", \"source\": \"content\", \"content_length\": 198}, {\"path\": \"content.chunk_68\", \"content\": \"#### Debugging Loop Task Data Access \\nWhen debugging loop tasks, always print available keys to understand what data is accessible: \\n```yaml\\n- id: debug_loop_task\\ntype: script\\ndepends_on: [previous_loop_task]\\nscript: |\\n# Debug: Print all available data\\nprint(f\\\" Available inputs: {list(inputs.keys())}\\\")\\nprint(f\\\" Loop state keys: {list(loop_state.keys())}\\\")\\n\\n# Debug: Print specific values\\nfor key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\\nvalue = inputs.get(key, \u0027NOT_FOUND\u0027)\\nprint(f\\\" {key}: {value}\\\")\\n\\n# Now access the flattened data\\ncompany_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\ncompany_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n\\n# Process your data...\\noutputs = {\\\"processed\\\": f\\\"Company {company_name}\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\", \"relevance_score\": 0.5546090302952729, \"distance_score\": 0.8030719757080078, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Debugging Loop Task Data Access\", \"source\": \"content\", \"content_length\": 772}, {\"path\": \"content.chunk_62\", \"content\": \"#### Complete Loop Task Example \\n```yaml\\n- id: financial_processing_loop\\ntype: loop\\nloop_type: for\\nmax_iterations: 3\\niteration_variable: company_index\\ndepends_on: [list_companies] # External dependency\\n\\nstate_variables:\\nprocessed_companies: []\\ntotal_extractions: 0\\nsuccessful_extractions: 0\\n\\nloop_tasks:\\n- id: process_current_company\\ntype: script\\nscript: |\\n# Access external workflow task data\\ncompanies_data = inputs.get(\u0027list_companies\u0027, {})\\ncompany_idx = inputs.get(\u0027company_index\u0027, 0)\\n\\n# Access state variables\\nprocessed = loop_state.get(\u0027processed_companies\u0027, [])\\n\\n# Process company\\ncompany = companies_data[\u0027companies\u0027][company_idx]\\nresult = process_company(company)\\n\\noutputs = {\\\"company_name\\\": company[\u0027name\u0027], \\\"status\\\": \\\"success\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\", \"relevance_score\": 0.5519489927660971, \"distance_score\": 0.8117616176605225, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Complete Loop Task Example\", \"source\": \"content\", \"content_length\": 784}, {\"path\": \"content.chunk_72\", \"content\": \"#### Safe Template Resolution Pattern for Loop States \\nWhen accessing loop final state fields, use this safe parsing pattern: \\n```yaml\\nscript: |\\ntry:\\n# Template resolution with safety checks\\nfinal_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\\nprocessed_count_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_count\\\"]}\u0027\u0027\u0027\\nitems_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_items\\\"]}\u0027\u0027\u0027\\niterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027\", \"relevance_score\": 0.5517749447607281, \"distance_score\": 0.8123331069946289, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Safe Template Resolution Pattern for Loop States\", \"source\": \"content\", \"content_length\": 432}, {\"path\": \"content.chunk_56\", \"content\": \"#### Loop Task Internal Data Access \\nWithin loop tasks, data access works through injected context files rather than template resolution: \\n```yaml\\n# Loop task with state variables and internal task dependencies\\n- id: data_processing_loop\\ntype: loop\\nloop_type: for\\nmax_iterations: 3\\nstate_variables:\\nprocessed_items: []\\ntotal_processed: 0\\nerror_count: 0\\nloop_tasks:\\n# First task in loop\\n- id: fetch_data\\ntype: script\\nscript: |\\n# Access loop state variables through injected context\\ncurrent_total = loop_state.get(\u0027total_processed\u0027, 0)\\nprocessed_items = loop_state.get(\u0027processed_items\u0027, [])\\n\\n# Access workflow inputs and external task outputs\\nexternal_data = inputs.get(\u0027previous_workflow_task\u0027, {})\\niteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable\", \"relevance_score\": 0.551580913054056, \"distance_score\": 0.8129706382751465, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Internal Data Access\", \"source\": \"content\", \"content_length\": 770}, {\"path\": \"content.chunk_81\", \"content\": \"# Loop through arrays from previous tasks\\n- id: process_items\\ntype: script\\ndepends_on: [fetch_items]\\nprevious_node: fetch_items\\nscript: |\\nitems = ${fetch_items.data}\\nprocessed_items = [] \\nfor item in items:\\nprocessed_item = {\\n\\\"id\\\": item[\\\"id\\\"],\\n\\\"processed_at\\\": \\\"${current_timestamp}\\\",\\n\\\"status\\\": \\\"processed\\\"\\n}\\nprocessed_items.append(processed_item) \\nprint(f\\\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\\\")\", \"relevance_score\": 0.5490929478973238, \"distance_score\": 0.821185290813446, \"headers\": \"Header 1: Loop through arrays from previous tasks\", \"source\": \"content\", \"content_length\": 422}], \"search_query\": \"how to create a loop node and use it?\", \"total_results\": 12, \"total_chunks\": 246, \"status\": \"success\"}\n",
"return_code": 0,
"search_query": "how to create a loop node and use it?",
"search_results": [
{
"content": "#### Loop Task Context Injection and Output Flattening \nThe loop executor injects context through temporary files and flattens task outputs for easier access: \n```python\n# Auto-injected at the start of each loop task script:\nimport json\nimport os\n\n# Load context from temporary files (auto-generated)\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\ninputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\n\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\nloop_state = json.load(f) # Contains current state variables\n\noutputs = {}\nstate_updates = {}\n\n# Your script code follows...\n```",
"content_length": 662,
"distance_score": 0.7235398292541504,
"headers": "Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Context Injection and Output Flattening",
"path": "content.chunk_67",
"relevance_score": 0.5802012712597091,
"source": "content"
},
{
"content": "# For-Loop (Fixed Iterations)\n- id: for_loop_task\nname: \"For Loop Processor\"\ntype: loop\ndescription: \"Process data in fixed batches\"",
"content_length": 132,
"distance_score": 0.7859672904014587,
"headers": "Header 1: For-Loop (Fixed Iterations)",
"path": "content.chunk_177",
"relevance_score": 0.5599206689699311,
"source": "content"
},
{
"content": "# Loop configuration\nloop_type: for # REQUIRED: for|while\nmax_iterations: 10 # REQUIRED for \u0027for\u0027 loops\niteration_variable: batch_index # Default: iteration",
"content_length": 198,
"distance_score": 0.8020307421684265,
"headers": "Header 1: Loop configuration",
"path": "content.chunk_178",
"relevance_score": 0.5549294896027557,
"source": "content"
},
{
"content": "#### Debugging Loop Task Data Access \nWhen debugging loop tasks, always print available keys to understand what data is accessible: \n```yaml\n- id: debug_loop_task\ntype: script\ndepends_on: [previous_loop_task]\nscript: |\n# Debug: Print all available data\nprint(f\" Available inputs: {list(inputs.keys())}\")\nprint(f\" Loop state keys: {list(loop_state.keys())}\")\n\n# Debug: Print specific values\nfor key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\nvalue = inputs.get(key, \u0027NOT_FOUND\u0027)\nprint(f\" {key}: {value}\")\n\n# Now access the flattened data\ncompany_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\ncompany_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\n\n# Process your data...\noutputs = {\"processed\": f\"Company {company_name}\"}\nprint(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```",
"content_length": 772,
"distance_score": 0.8030719757080078,
"headers": "Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Debugging Loop Task Data Access",
"path": "content.chunk_68",
"relevance_score": 0.5546090302952729,
"source": "content"
},
{
"content": "#### Complete Loop Task Example \n```yaml\n- id: financial_processing_loop\ntype: loop\nloop_type: for\nmax_iterations: 3\niteration_variable: company_index\ndepends_on: [list_companies] # External dependency\n\nstate_variables:\nprocessed_companies: []\ntotal_extractions: 0\nsuccessful_extractions: 0\n\nloop_tasks:\n- id: process_current_company\ntype: script\nscript: |\n# Access external workflow task data\ncompanies_data = inputs.get(\u0027list_companies\u0027, {})\ncompany_idx = inputs.get(\u0027company_index\u0027, 0)\n\n# Access state variables\nprocessed = loop_state.get(\u0027processed_companies\u0027, [])\n\n# Process company\ncompany = companies_data[\u0027companies\u0027][company_idx]\nresult = process_company(company)\n\noutputs = {\"company_name\": company[\u0027name\u0027], \"status\": \"success\"}\nprint(f\"__OUTPUTS__ {json.dumps(outputs)}\")",
"content_length": 784,
"distance_score": 0.8117616176605225,
"headers": "Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Complete Loop Task Example",
"path": "content.chunk_62",
"relevance_score": 0.5519489927660971,
"source": "content"
},
{
"content": "#### Safe Template Resolution Pattern for Loop States \nWhen accessing loop final state fields, use this safe parsing pattern: \n```yaml\nscript: |\ntry:\n# Template resolution with safety checks\nfinal_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\nprocessed_count_str = \u0027\u0027\u0027${loop_task.final_state[\"processed_count\"]}\u0027\u0027\u0027\nitems_str = \u0027\u0027\u0027${loop_task.final_state[\"processed_items\"]}\u0027\u0027\u0027\niterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027",
"content_length": 432,
"distance_score": 0.8123331069946289,
"headers": "Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Safe Template Resolution Pattern for Loop States",
"path": "content.chunk_72",
"relevance_score": 0.5517749447607281,
"source": "content"
},
{
"content": "#### Loop Task Internal Data Access \nWithin loop tasks, data access works through injected context files rather than template resolution: \n```yaml\n# Loop task with state variables and internal task dependencies\n- id: data_processing_loop\ntype: loop\nloop_type: for\nmax_iterations: 3\nstate_variables:\nprocessed_items: []\ntotal_processed: 0\nerror_count: 0\nloop_tasks:\n# First task in loop\n- id: fetch_data\ntype: script\nscript: |\n# Access loop state variables through injected context\ncurrent_total = loop_state.get(\u0027total_processed\u0027, 0)\nprocessed_items = loop_state.get(\u0027processed_items\u0027, [])\n\n# Access workflow inputs and external task outputs\nexternal_data = inputs.get(\u0027previous_workflow_task\u0027, {})\niteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable",
"content_length": 770,
"distance_score": 0.8129706382751465,
"headers": "Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Internal Data Access",
"path": "content.chunk_56",
"relevance_score": 0.551580913054056,
"source": "content"
},
{
"content": "# Loop through arrays from previous tasks\n- id: process_items\ntype: script\ndepends_on: [fetch_items]\nprevious_node: fetch_items\nscript: |\nitems = ${fetch_items.data}\nprocessed_items = [] \nfor item in items:\nprocessed_item = {\n\"id\": item[\"id\"],\n\"processed_at\": \"${current_timestamp}\",\n\"status\": \"processed\"\n}\nprocessed_items.append(processed_item) \nprint(f\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\")",
"content_length": 422,
"distance_score": 0.821185290813446,
"headers": "Header 1: Loop through arrays from previous tasks",
"path": "content.chunk_81",
"relevance_score": 0.5490929478973238,
"source": "content"
}
],
"status": "completed",
"stderr": "",
"task_id": "vector_search",
"total_chunks": 246,
"total_results": 12
}
},
"vector_search": {
"execution_details": {
"actual_result": {
"output": "Processing vector search for query: \u0027how to create a loop node and use it?\u0027\nExtracted 1 content sections from documentation\nSplit into 246 semantic chunks\nInitializing embeddings...\nEmbedding 246 text chunks...\nPerforming semantic similarity search...\nFound 12 semantic matches\nReturning 8 most relevant results:\n 1. content... [Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Context Injection and Output Flattening] (score: 0.580)\n 2. content... [Header 1: For-Loop (Fixed Iterations)] (score: 0.560)\n 3. content... [Header 1: Loop configuration] (score: 0.555)\n__OUTPUTS__ {\"search_results\": [{\"path\": \"content.chunk_67\", \"content\": \"#### Loop Task Context Injection and Output Flattening \\nThe loop executor injects context through temporary files and flattens task outputs for easier access: \\n```python\\n# Auto-injected at the start of each loop task script:\\nimport json\\nimport os\\n\\n# Load context from temporary files (auto-generated)\\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\\ninputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\\n\\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\\nloop_state = json.load(f) # Contains current state variables\\n\\noutputs = {}\\nstate_updates = {}\\n\\n# Your script code follows...\\n```\", \"relevance_score\": 0.5802012712597091, \"distance_score\": 0.7235398292541504, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Context Injection and Output Flattening\", \"source\": \"content\", \"content_length\": 662}, {\"path\": \"content.chunk_177\", \"content\": \"# For-Loop (Fixed Iterations)\\n- id: for_loop_task\\nname: \\\"For Loop Processor\\\"\\ntype: loop\\ndescription: \\\"Process data in fixed batches\\\"\", \"relevance_score\": 0.5599206689699311, \"distance_score\": 0.7859672904014587, \"headers\": \"Header 1: For-Loop (Fixed Iterations)\", \"source\": \"content\", \"content_length\": 132}, {\"path\": \"content.chunk_178\", \"content\": \"# Loop configuration\\nloop_type: for # REQUIRED: for|while\\nmax_iterations: 10 # REQUIRED for \u0027for\u0027 loops\\niteration_variable: batch_index # Default: iteration\", \"relevance_score\": 0.5549294896027557, \"distance_score\": 0.8020307421684265, \"headers\": \"Header 1: Loop configuration\", \"source\": \"content\", \"content_length\": 198}, {\"path\": \"content.chunk_68\", \"content\": \"#### Debugging Loop Task Data Access \\nWhen debugging loop tasks, always print available keys to understand what data is accessible: \\n```yaml\\n- id: debug_loop_task\\ntype: script\\ndepends_on: [previous_loop_task]\\nscript: |\\n# Debug: Print all available data\\nprint(f\\\" Available inputs: {list(inputs.keys())}\\\")\\nprint(f\\\" Loop state keys: {list(loop_state.keys())}\\\")\\n\\n# Debug: Print specific values\\nfor key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\\nvalue = inputs.get(key, \u0027NOT_FOUND\u0027)\\nprint(f\\\" {key}: {value}\\\")\\n\\n# Now access the flattened data\\ncompany_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\ncompany_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n\\n# Process your data...\\noutputs = {\\\"processed\\\": f\\\"Company {company_name}\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\", \"relevance_score\": 0.5546090302952729, \"distance_score\": 0.8030719757080078, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Debugging Loop Task Data Access\", \"source\": \"content\", \"content_length\": 772}, {\"path\": \"content.chunk_62\", \"content\": \"#### Complete Loop Task Example \\n```yaml\\n- id: financial_processing_loop\\ntype: loop\\nloop_type: for\\nmax_iterations: 3\\niteration_variable: company_index\\ndepends_on: [list_companies] # External dependency\\n\\nstate_variables:\\nprocessed_companies: []\\ntotal_extractions: 0\\nsuccessful_extractions: 0\\n\\nloop_tasks:\\n- id: process_current_company\\ntype: script\\nscript: |\\n# Access external workflow task data\\ncompanies_data = inputs.get(\u0027list_companies\u0027, {})\\ncompany_idx = inputs.get(\u0027company_index\u0027, 0)\\n\\n# Access state variables\\nprocessed = loop_state.get(\u0027processed_companies\u0027, [])\\n\\n# Process company\\ncompany = companies_data[\u0027companies\u0027][company_idx]\\nresult = process_company(company)\\n\\noutputs = {\\\"company_name\\\": company[\u0027name\u0027], \\\"status\\\": \\\"success\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\", \"relevance_score\": 0.5519489927660971, \"distance_score\": 0.8117616176605225, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Complete Loop Task Example\", \"source\": \"content\", \"content_length\": 784}, {\"path\": \"content.chunk_72\", \"content\": \"#### Safe Template Resolution Pattern for Loop States \\nWhen accessing loop final state fields, use this safe parsing pattern: \\n```yaml\\nscript: |\\ntry:\\n# Template resolution with safety checks\\nfinal_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\\nprocessed_count_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_count\\\"]}\u0027\u0027\u0027\\nitems_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_items\\\"]}\u0027\u0027\u0027\\niterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027\", \"relevance_score\": 0.5517749447607281, \"distance_score\": 0.8123331069946289, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Safe Template Resolution Pattern for Loop States\", \"source\": \"content\", \"content_length\": 432}, {\"path\": \"content.chunk_56\", \"content\": \"#### Loop Task Internal Data Access \\nWithin loop tasks, data access works through injected context files rather than template resolution: \\n```yaml\\n# Loop task with state variables and internal task dependencies\\n- id: data_processing_loop\\ntype: loop\\nloop_type: for\\nmax_iterations: 3\\nstate_variables:\\nprocessed_items: []\\ntotal_processed: 0\\nerror_count: 0\\nloop_tasks:\\n# First task in loop\\n- id: fetch_data\\ntype: script\\nscript: |\\n# Access loop state variables through injected context\\ncurrent_total = loop_state.get(\u0027total_processed\u0027, 0)\\nprocessed_items = loop_state.get(\u0027processed_items\u0027, [])\\n\\n# Access workflow inputs and external task outputs\\nexternal_data = inputs.get(\u0027previous_workflow_task\u0027, {})\\niteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable\", \"relevance_score\": 0.551580913054056, \"distance_score\": 0.8129706382751465, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Internal Data Access\", \"source\": \"content\", \"content_length\": 770}, {\"path\": \"content.chunk_81\", \"content\": \"# Loop through arrays from previous tasks\\n- id: process_items\\ntype: script\\ndepends_on: [fetch_items]\\nprevious_node: fetch_items\\nscript: |\\nitems = ${fetch_items.data}\\nprocessed_items = [] \\nfor item in items:\\nprocessed_item = {\\n\\\"id\\\": item[\\\"id\\\"],\\n\\\"processed_at\\\": \\\"${current_timestamp}\\\",\\n\\\"status\\\": \\\"processed\\\"\\n}\\nprocessed_items.append(processed_item) \\nprint(f\\\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\\\")\", \"relevance_score\": 0.5490929478973238, \"distance_score\": 0.821185290813446, \"headers\": \"Header 1: Loop through arrays from previous tasks\", \"source\": \"content\", \"content_length\": 422}], \"search_query\": \"how to create a loop node and use it?\", \"total_results\": 12, \"total_chunks\": 246, \"status\": \"success\"}\n",
"return_code": 0,
"status": "completed",
"stderr": ""
},
"duration_seconds": 10.630208,
"end_time": "2025-08-14T11:23:18.363877",
"message_sent": true,
"start_time": "2025-08-14T11:23:07.733669",
"timestamp": "2025-08-14T11:23:18.363877",
"worker_executed": true,
"workers_notified": true
},
"output": "Processing vector search for query: \u0027how to create a loop node and use it?\u0027\nExtracted 1 content sections from documentation\nSplit into 246 semantic chunks\nInitializing embeddings...\nEmbedding 246 text chunks...\nPerforming semantic similarity search...\nFound 12 semantic matches\nReturning 8 most relevant results:\n 1. content... [Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Context Injection and Output Flattening] (score: 0.580)\n 2. content... [Header 1: For-Loop (Fixed Iterations)] (score: 0.560)\n 3. content... [Header 1: Loop configuration] (score: 0.555)\n__OUTPUTS__ {\"search_results\": [{\"path\": \"content.chunk_67\", \"content\": \"#### Loop Task Context Injection and Output Flattening \\nThe loop executor injects context through temporary files and flattens task outputs for easier access: \\n```python\\n# Auto-injected at the start of each loop task script:\\nimport json\\nimport os\\n\\n# Load context from temporary files (auto-generated)\\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\\ninputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\\n\\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\\nloop_state = json.load(f) # Contains current state variables\\n\\noutputs = {}\\nstate_updates = {}\\n\\n# Your script code follows...\\n```\", \"relevance_score\": 0.5802012712597091, \"distance_score\": 0.7235398292541504, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Context Injection and Output Flattening\", \"source\": \"content\", \"content_length\": 662}, {\"path\": \"content.chunk_177\", \"content\": \"# For-Loop (Fixed Iterations)\\n- id: for_loop_task\\nname: \\\"For Loop Processor\\\"\\ntype: loop\\ndescription: \\\"Process data in fixed batches\\\"\", \"relevance_score\": 0.5599206689699311, \"distance_score\": 0.7859672904014587, \"headers\": \"Header 1: For-Loop (Fixed Iterations)\", \"source\": \"content\", \"content_length\": 132}, {\"path\": \"content.chunk_178\", \"content\": \"# Loop configuration\\nloop_type: for # REQUIRED: for|while\\nmax_iterations: 10 # REQUIRED for \u0027for\u0027 loops\\niteration_variable: batch_index # Default: iteration\", \"relevance_score\": 0.5549294896027557, \"distance_score\": 0.8020307421684265, \"headers\": \"Header 1: Loop configuration\", \"source\": \"content\", \"content_length\": 198}, {\"path\": \"content.chunk_68\", \"content\": \"#### Debugging Loop Task Data Access \\nWhen debugging loop tasks, always print available keys to understand what data is accessible: \\n```yaml\\n- id: debug_loop_task\\ntype: script\\ndepends_on: [previous_loop_task]\\nscript: |\\n# Debug: Print all available data\\nprint(f\\\" Available inputs: {list(inputs.keys())}\\\")\\nprint(f\\\" Loop state keys: {list(loop_state.keys())}\\\")\\n\\n# Debug: Print specific values\\nfor key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\\nvalue = inputs.get(key, \u0027NOT_FOUND\u0027)\\nprint(f\\\" {key}: {value}\\\")\\n\\n# Now access the flattened data\\ncompany_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\\ncompany_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\\n\\n# Process your data...\\noutputs = {\\\"processed\\\": f\\\"Company {company_name}\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\\n```\", \"relevance_score\": 0.5546090302952729, \"distance_score\": 0.8030719757080078, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Debugging Loop Task Data Access\", \"source\": \"content\", \"content_length\": 772}, {\"path\": \"content.chunk_62\", \"content\": \"#### Complete Loop Task Example \\n```yaml\\n- id: financial_processing_loop\\ntype: loop\\nloop_type: for\\nmax_iterations: 3\\niteration_variable: company_index\\ndepends_on: [list_companies] # External dependency\\n\\nstate_variables:\\nprocessed_companies: []\\ntotal_extractions: 0\\nsuccessful_extractions: 0\\n\\nloop_tasks:\\n- id: process_current_company\\ntype: script\\nscript: |\\n# Access external workflow task data\\ncompanies_data = inputs.get(\u0027list_companies\u0027, {})\\ncompany_idx = inputs.get(\u0027company_index\u0027, 0)\\n\\n# Access state variables\\nprocessed = loop_state.get(\u0027processed_companies\u0027, [])\\n\\n# Process company\\ncompany = companies_data[\u0027companies\u0027][company_idx]\\nresult = process_company(company)\\n\\noutputs = {\\\"company_name\\\": company[\u0027name\u0027], \\\"status\\\": \\\"success\\\"}\\nprint(f\\\"__OUTPUTS__ {json.dumps(outputs)}\\\")\", \"relevance_score\": 0.5519489927660971, \"distance_score\": 0.8117616176605225, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Complete Loop Task Example\", \"source\": \"content\", \"content_length\": 784}, {\"path\": \"content.chunk_72\", \"content\": \"#### Safe Template Resolution Pattern for Loop States \\nWhen accessing loop final state fields, use this safe parsing pattern: \\n```yaml\\nscript: |\\ntry:\\n# Template resolution with safety checks\\nfinal_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\\nprocessed_count_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_count\\\"]}\u0027\u0027\u0027\\nitems_str = \u0027\u0027\u0027${loop_task.final_state[\\\"processed_items\\\"]}\u0027\u0027\u0027\\niterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027\", \"relevance_score\": 0.5517749447607281, \"distance_score\": 0.8123331069946289, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Safe Template Resolution Pattern for Loop States\", \"source\": \"content\", \"content_length\": 432}, {\"path\": \"content.chunk_56\", \"content\": \"#### Loop Task Internal Data Access \\nWithin loop tasks, data access works through injected context files rather than template resolution: \\n```yaml\\n# Loop task with state variables and internal task dependencies\\n- id: data_processing_loop\\ntype: loop\\nloop_type: for\\nmax_iterations: 3\\nstate_variables:\\nprocessed_items: []\\ntotal_processed: 0\\nerror_count: 0\\nloop_tasks:\\n# First task in loop\\n- id: fetch_data\\ntype: script\\nscript: |\\n# Access loop state variables through injected context\\ncurrent_total = loop_state.get(\u0027total_processed\u0027, 0)\\nprocessed_items = loop_state.get(\u0027processed_items\u0027, [])\\n\\n# Access workflow inputs and external task outputs\\nexternal_data = inputs.get(\u0027previous_workflow_task\u0027, {})\\niteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable\", \"relevance_score\": 0.551580913054056, \"distance_score\": 0.8129706382751465, \"headers\": \"Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Internal Data Access\", \"source\": \"content\", \"content_length\": 770}, {\"path\": \"content.chunk_81\", \"content\": \"# Loop through arrays from previous tasks\\n- id: process_items\\ntype: script\\ndepends_on: [fetch_items]\\nprevious_node: fetch_items\\nscript: |\\nitems = ${fetch_items.data}\\nprocessed_items = [] \\nfor item in items:\\nprocessed_item = {\\n\\\"id\\\": item[\\\"id\\\"],\\n\\\"processed_at\\\": \\\"${current_timestamp}\\\",\\n\\\"status\\\": \\\"processed\\\"\\n}\\nprocessed_items.append(processed_item) \\nprint(f\\\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\\\")\", \"relevance_score\": 0.5490929478973238, \"distance_score\": 0.821185290813446, \"headers\": \"Header 1: Loop through arrays from previous tasks\", \"source\": \"content\", \"content_length\": 422}], \"search_query\": \"how to create a loop node and use it?\", \"total_results\": 12, \"total_chunks\": 246, \"status\": \"success\"}\n",
"return_code": 0,
"search_query": "how to create a loop node and use it?",
"search_results": [
{
"content": "#### Loop Task Context Injection and Output Flattening \nThe loop executor injects context through temporary files and flattens task outputs for easier access: \n```python\n# Auto-injected at the start of each loop task script:\nimport json\nimport os\n\n# Load context from temporary files (auto-generated)\nwith open(\u0027/tmp/loop_context_task_iter_N/inputs.json\u0027, \u0027r\u0027) as f:\ninputs = json.load(f) # Contains all workflow data + previous loop task outputs (FLATTENED)\n\nwith open(\u0027/tmp/loop_context_task_iter_N/loop_state.json\u0027, \u0027r\u0027) as f:\nloop_state = json.load(f) # Contains current state variables\n\noutputs = {}\nstate_updates = {}\n\n# Your script code follows...\n```",
"content_length": 662,
"distance_score": 0.7235398292541504,
"headers": "Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Context Injection and Output Flattening",
"path": "content.chunk_67",
"relevance_score": 0.5802012712597091,
"source": "content"
},
{
"content": "# For-Loop (Fixed Iterations)\n- id: for_loop_task\nname: \"For Loop Processor\"\ntype: loop\ndescription: \"Process data in fixed batches\"",
"content_length": 132,
"distance_score": 0.7859672904014587,
"headers": "Header 1: For-Loop (Fixed Iterations)",
"path": "content.chunk_177",
"relevance_score": 0.5599206689699311,
"source": "content"
},
{
"content": "# Loop configuration\nloop_type: for # REQUIRED: for|while\nmax_iterations: 10 # REQUIRED for \u0027for\u0027 loops\niteration_variable: batch_index # Default: iteration",
"content_length": 198,
"distance_score": 0.8020307421684265,
"headers": "Header 1: Loop configuration",
"path": "content.chunk_178",
"relevance_score": 0.5549294896027557,
"source": "content"
},
{
"content": "#### Debugging Loop Task Data Access \nWhen debugging loop tasks, always print available keys to understand what data is accessible: \n```yaml\n- id: debug_loop_task\ntype: script\ndepends_on: [previous_loop_task]\nscript: |\n# Debug: Print all available data\nprint(f\" Available inputs: {list(inputs.keys())}\")\nprint(f\" Loop state keys: {list(loop_state.keys())}\")\n\n# Debug: Print specific values\nfor key in [\u0027company_name\u0027, \u0027company_id\u0027, \u0027status\u0027, \u0027fy_folders\u0027]:\nvalue = inputs.get(key, \u0027NOT_FOUND\u0027)\nprint(f\" {key}: {value}\")\n\n# Now access the flattened data\ncompany_name = inputs.get(\u0027company_name\u0027, \u0027Unknown\u0027)\ncompany_id = inputs.get(\u0027company_id\u0027, \u0027\u0027)\n\n# Process your data...\noutputs = {\"processed\": f\"Company {company_name}\"}\nprint(f\"__OUTPUTS__ {json.dumps(outputs)}\")\n```",
"content_length": 772,
"distance_score": 0.8030719757080078,
"headers": "Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Debugging Loop Task Data Access",
"path": "content.chunk_68",
"relevance_score": 0.5546090302952729,
"source": "content"
},
{
"content": "#### Complete Loop Task Example \n```yaml\n- id: financial_processing_loop\ntype: loop\nloop_type: for\nmax_iterations: 3\niteration_variable: company_index\ndepends_on: [list_companies] # External dependency\n\nstate_variables:\nprocessed_companies: []\ntotal_extractions: 0\nsuccessful_extractions: 0\n\nloop_tasks:\n- id: process_current_company\ntype: script\nscript: |\n# Access external workflow task data\ncompanies_data = inputs.get(\u0027list_companies\u0027, {})\ncompany_idx = inputs.get(\u0027company_index\u0027, 0)\n\n# Access state variables\nprocessed = loop_state.get(\u0027processed_companies\u0027, [])\n\n# Process company\ncompany = companies_data[\u0027companies\u0027][company_idx]\nresult = process_company(company)\n\noutputs = {\"company_name\": company[\u0027name\u0027], \"status\": \"success\"}\nprint(f\"__OUTPUTS__ {json.dumps(outputs)}\")",
"content_length": 784,
"distance_score": 0.8117616176605225,
"headers": "Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Complete Loop Task Example",
"path": "content.chunk_62",
"relevance_score": 0.5519489927660971,
"source": "content"
},
{
"content": "#### Safe Template Resolution Pattern for Loop States \nWhen accessing loop final state fields, use this safe parsing pattern: \n```yaml\nscript: |\ntry:\n# Template resolution with safety checks\nfinal_state_str = \u0027\u0027\u0027${loop_task.final_state}\u0027\u0027\u0027\nprocessed_count_str = \u0027\u0027\u0027${loop_task.final_state[\"processed_count\"]}\u0027\u0027\u0027\nitems_str = \u0027\u0027\u0027${loop_task.final_state[\"processed_items\"]}\u0027\u0027\u0027\niterations_str = \u0027\u0027\u0027${loop_task.iterations_completed}\u0027\u0027\u0027",
"content_length": 432,
"distance_score": 0.8123331069946289,
"headers": "Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Safe Template Resolution Pattern for Loop States",
"path": "content.chunk_72",
"relevance_score": 0.5517749447607281,
"source": "content"
},
{
"content": "#### Loop Task Internal Data Access \nWithin loop tasks, data access works through injected context files rather than template resolution: \n```yaml\n# Loop task with state variables and internal task dependencies\n- id: data_processing_loop\ntype: loop\nloop_type: for\nmax_iterations: 3\nstate_variables:\nprocessed_items: []\ntotal_processed: 0\nerror_count: 0\nloop_tasks:\n# First task in loop\n- id: fetch_data\ntype: script\nscript: |\n# Access loop state variables through injected context\ncurrent_total = loop_state.get(\u0027total_processed\u0027, 0)\nprocessed_items = loop_state.get(\u0027processed_items\u0027, [])\n\n# Access workflow inputs and external task outputs\nexternal_data = inputs.get(\u0027previous_workflow_task\u0027, {})\niteration_num = inputs.get(\u0027iteration\u0027, 0) # Loop iteration variable",
"content_length": 770,
"distance_score": 0.8129706382751465,
"headers": "Header 1: This runs in parallel with validation/processing \u003e Header 2: Template Resolution and Data Flow \u003e Header 3: Loop Task Data Access and Dependencies \u003e Header 4: Loop Task Internal Data Access",
"path": "content.chunk_56",
"relevance_score": 0.551580913054056,
"source": "content"
},
{
"content": "# Loop through arrays from previous tasks\n- id: process_items\ntype: script\ndepends_on: [fetch_items]\nprevious_node: fetch_items\nscript: |\nitems = ${fetch_items.data}\nprocessed_items = [] \nfor item in items:\nprocessed_item = {\n\"id\": item[\"id\"],\n\"processed_at\": \"${current_timestamp}\",\n\"status\": \"processed\"\n}\nprocessed_items.append(processed_item) \nprint(f\"__OUTPUTS__ {json.dumps({\u0027processed_items\u0027: processed_items})}\")",
"content_length": 422,
"distance_score": 0.821185290813446,
"headers": "Header 1: Loop through arrays from previous tasks",
"path": "content.chunk_81",
"relevance_score": 0.5490929478973238,
"source": "content"
}
],
"status": "completed",
"stderr": "",
"task_id": "vector_search",
"total_chunks": 246,
"total_results": 12
}
}
Task Executions
fetch_documentation
COMPLETED
Started: 2025-08-14 11:23:05
Duration: Calculating...
Task Output
Loading...
Loading task output...
vector_search
COMPLETED
Started: 2025-08-14 11:23:05
Duration: Calculating...
Task Output
Loading...
Loading task output...
ai_assistant
COMPLETED
Started: 2025-08-14 11:23:05
Duration: N/A
Task Output
Loading...
Loading task output...
Generated Files
Loading...
Loading generated files...
Total Files: 0
Last Updated: Never
No files were generated during this execution.
Execution Logs
Loading...
Loading execution logs...
Total Logs: 0
Last Updated: Never
No logs available for this execution.