Comprehensive Loop Examples V2 - Fixed Dataset References

Demonstrates both for-loop and while-loop functionality with proper state variable management

Back
Workflow Information

ID: comprehensive_loop_examples_v2_fixed

Namespace: examples

Version: 1.0

Created: 2025-07-22

Updated: 2025-07-22

Tasks: 4

Quick Actions
Manage Secrets
Inputs
Name Type Required Default
dataset_size integer Optional 25
batch_size integer Optional 5
processing_threshold integer Optional 80
max_while_iterations integer Optional 20
Outputs
Name Type Source
for_loop_summary object Summary of for-loop processing results
comparison_report object Comprehensive comparison of both loop types
while_loop_summary object Summary of while-loop processing results
total_items_processed integer Total items processed by for-loop
quality_threshold_achieved boolean Whether while-loop achieved quality threshold
Tasks
initialize_datasets
script

Create test datasets for for-loop and while-loop examples

for_loop_processor
loop

Process data in fixed batches using for-loop pattern

Loop Configuration
Type: for
Max Iterations: ${initialize_datasets.for_loop_batches}
Iterator Variable: batch_index
State Variables: batch_size, average_value, category_counts, total_processed, total_value_sum, for_loop_dataset, batches_completed
Loop Flow (3 steps)
Extract For-Loop Batch script
Process For-Loop Batch script
Update For-Loop State script
while_loop_processor
loop

Process items until quality threshold met using while-loop pattern

Loop Configuration
Type: while
Max Iterations: ${max_while_iterations}
Exit Condition: ${pending_items} <= 0 || ${average_score} >= ${processing_threshold}
Iterator Variable: iteration_count
State Variables: batch_size, total_items, total_score, average_score, pending_items, processed_items, should_continue, working_dataset, reprocessing_count, processing_threshold, completion_percentage
Loop Flow (4 steps)
Initialize While-Loop State script
Select Items for While-Loop Processing script
Improve While-Loop Items script
Update While-Loop State script
generate_comparison_report
script

Compare results from for-loop and while-loop processing using proper template variable access

YAML Source
id: comprehensive_loop_examples_v2_fixed
name: Comprehensive Loop Examples V2 - Fixed Dataset References
retry:
  retryOn:
  - TEMPORARY_FAILURE
  maxDelay: 10s
  maxAttempts: 2
  initialDelay: 1s
  backoffMultiplier: 1.5
tasks:
- id: initialize_datasets
  name: Initialize Test Datasets
  type: script
  script: "import json\nimport random\n\ndataset_size = int(\"${dataset_size}\")\n\
    batch_size = int(\"${batch_size}\")\n\n# Generate dataset for for-loop (fixed\
    \ size array)\nfor_loop_data = []\nfor i in range(dataset_size):\n    item = {\n\
    \        \"id\": i + 1,\n        \"name\": f\"Item_{i+1:03d}\",\n        \"value\"\
    : random.randint(1, 100),\n        \"category\": random.choice([\"Alpha\", \"\
    Beta\", \"Gamma\", \"Delta\"]),\n        \"priority\": random.choice([\"High\"\
    , \"Medium\", \"Low\"])\n    }\n    for_loop_data.append(item)\n\n# Generate dataset\
    \ for while-loop (variable processing needs)\nwhile_loop_data = []\nfor i in range(dataset_size):\n\
    \    item = {\n        \"id\": i + 1,\n        \"name\": f\"Record_{i+1:03d}\"\
    ,\n        \"score\": random.randint(10, 100),\n        \"status\": \"pending\"\
    ,\n        \"attempts\": 0,\n        \"needs_reprocessing\": random.choice([True,\
    \ False])\n    }\n    while_loop_data.append(item)\n\nresult = {\n    \"for_loop_dataset\"\
    : for_loop_data,\n    \"while_loop_dataset\": while_loop_data,\n    \"total_items\"\
    : dataset_size,\n    \"batch_size\": batch_size,\n    \"for_loop_batches\": (dataset_size\
    \ + batch_size - 1) // batch_size  # Ceiling division\n}\n\nprint(f\"Generated\
    \ {dataset_size} items for processing\")\nprint(f\"For-loop will process {result['for_loop_batches']}\
    \ batches\")\nprint(f\"__OUTPUTS__ {json.dumps(result)}\")\n"
  description: Create test datasets for for-loop and while-loop examples
  timeout_seconds: 60
- id: for_loop_processor
  name: For-Loop Batch Processor
  type: loop
  loop_type: for
  depends_on:
  - initialize_datasets
  loop_tasks:
  - id: extract_for_batch
    name: Extract For-Loop Batch
    type: script
    script: '# Loop executor provides: inputs, loop_state, outputs, state_updates


      # Get data from loop_state (now properly resolved via state_variables)

      dataset = loop_state.get(''for_loop_dataset'', [])

      batch_size = loop_state.get(''batch_size'', 3)

      batch_index = inputs.get(''batch_index'', 0)


      print(f"DEBUG: dataset length: {len(dataset)}")

      print(f"DEBUG: batch_size: {batch_size}")

      print(f"DEBUG: batch_index: {batch_index}")


      # Calculate batch boundaries

      start_idx = batch_index * batch_size

      end_idx = min(start_idx + batch_size, len(dataset))

      current_batch = dataset[start_idx:end_idx]


      print(f"For-Loop: Processing batch {batch_index + 1}")

      print(f"Batch range: items {start_idx + 1} to {end_idx}")


      # Populate outputs dictionary (NOT print directly)

      outputs["current_batch"] = current_batch

      outputs["batch_number"] = batch_index + 1

      outputs["batch_size"] = len(current_batch)

      outputs["start_index"] = start_idx

      outputs["end_index"] = end_idx

      '
    description: Extract the current batch using for-loop index
  - id: process_for_batch
    name: Process For-Loop Batch
    type: script
    script: "# Get current batch from previous task output (stored in inputs by loop\
      \ executor)\ncurrent_batch = inputs.get('current_batch', [])\nbatch_number =\
      \ inputs.get('batch_number', 1)\n\n# Process each item in the batch\nprocessed_items\
      \ = []\nbatch_value_sum = 0\ncategory_counts = {}\n\nfor item in current_batch:\n\
      \    # Simulate processing logic\n    processed_item = {\n        \"id\": item[\"\
      id\"],\n        \"name\": item[\"name\"],\n        \"original_value\": item[\"\
      value\"],\n        \"processed_value\": item[\"value\"] * 1.1,  # 10% boost\n\
      \        \"category\": item[\"category\"],\n        \"priority\": item[\"priority\"\
      ],\n        \"processed_at_batch\": batch_number,\n        \"quality_score\"\
      : min(100, item[\"value\"] + 10)\n    }\n    \n    processed_items.append(processed_item)\n\
      \    batch_value_sum += processed_item[\"processed_value\"]\n    \n    # Count\
      \ categories\n    category = item[\"category\"]\n    category_counts[category]\
      \ = category_counts.get(category, 0) + 1\n\n# Populate outputs dictionary\n\
      outputs[\"processed_items\"] = processed_items\noutputs[\"batch_stats\"] = {\n\
      \    \"items_processed\": len(processed_items),\n    \"total_value\": batch_value_sum,\n\
      \    \"average_value\": batch_value_sum / len(processed_items) if processed_items\
      \ else 0,\n    \"category_distribution\": category_counts\n}\n\nprint(f\"Processed\
      \ {len(processed_items)} items in batch {batch_number}\")\nprint(f\"Batch total\
      \ value: {batch_value_sum:.2f}\")\n"
    depends_on:
    - extract_for_batch
    description: Process items in the current for-loop batch
  - id: update_for_state
    name: Update For-Loop State
    type: script
    script: "# Get current state variables from loop_state\ncurrent_total = loop_state.get('total_processed',\
      \ 0)\ncurrent_sum = loop_state.get('total_value_sum', 0)\ncurrent_categories\
      \ = loop_state.get('category_counts', {})\n\n# Get batch results from inputs\
      \ (previous task output)\nbatch_stats = inputs.get('batch_stats', {})\nitems_processed\
      \ = batch_stats.get('items_processed', 0)\nbatch_value = batch_stats.get('total_value',\
      \ 0)\nbatch_categories = batch_stats.get('category_distribution', {})\n\n# Update\
      \ totals\nnew_total = current_total + items_processed\nnew_sum = current_sum\
      \ + batch_value\n\n# Merge category counts\nupdated_categories = current_categories.copy()\
      \ if isinstance(current_categories, dict) else {}\nfor category, count in batch_categories.items():\n\
      \    updated_categories[category] = updated_categories.get(category, 0) + count\n\
      \n# Calculate running average\nrunning_avg = new_sum / new_total if new_total\
      \ > 0 else 0\n\n# Get batch index from inputs\nbatch_index = inputs.get('batch_index',\
      \ 0)\n\n# Populate state_updates to modify loop state variables\nstate_updates[\"\
      total_processed\"] = new_total\nstate_updates[\"total_value_sum\"] = new_sum\n\
      state_updates[\"category_counts\"] = updated_categories\nstate_updates[\"average_value\"\
      ] = running_avg\nstate_updates[\"batches_completed\"] = batch_index + 1\n\n\
      # Populate outputs for this task\noutputs[\"batch_progress\"] = {\n    \"items_this_batch\"\
      : items_processed,\n    \"total_items_so_far\": new_total,\n    \"running_average\"\
      : running_avg,\n    \"batch_number\": batch_index + 1\n}\n\nprint(f\"For-Loop\
      \ Progress: {new_total} items processed across {batch_index + 1} batches\")\n\
      print(f\"Running average value: {running_avg:.2f}\")\n"
    depends_on:
    - process_for_batch
    description: Update cumulative statistics for for-loop
  description: Process data in fixed batches using for-loop pattern
  previous_node: initialize_datasets
  max_iterations: ${initialize_datasets.for_loop_batches}
  state_variables:
    batch_size: ${initialize_datasets.batch_size}
    average_value: 0
    category_counts: {}
    total_processed: 0
    total_value_sum: 0
    for_loop_dataset: ${initialize_datasets.for_loop_dataset}
    batches_completed: 0
  iteration_variable: batch_index
- id: while_loop_processor
  name: While-Loop Quality Processor
  type: loop
  loop_type: while
  depends_on:
  - for_loop_processor
  - initialize_datasets
  loop_tasks:
  - id: initialize_while_state
    name: Initialize While-Loop State
    type: script
    script: "# Initialize state variables for first iteration\nworking_data = loop_state.get('working_dataset',\
      \ [])\ntotal_items = loop_state.get('total_items', len(working_data))\niteration\
      \ = inputs.get('iteration_count', 0)\n\n# On first iteration, initialize pending_items\
      \ if needed\ncurrent_pending = loop_state.get('pending_items', 0)\nif iteration\
      \ == 0 and current_pending == 0:\n    # Count items that are actually pending\n\
      \    pending_count = sum(1 for item in working_data if item.get(\"status\",\
      \ \"pending\") == \"pending\")\n    state_updates[\"pending_items\"] = pending_count\n\
      \    print(f\"Initialized pending_items to {pending_count} from dataset of {len(working_data)}\
      \ items\")\n\noutputs[\"initialization_complete\"] = True\n"
    description: Initialize while-loop state variables on first iteration
  - id: select_while_items
    name: Select Items for While-Loop Processing
    type: script
    script: "# Get data from loop_state (now properly resolved)\nworking_data = loop_state.get('working_dataset',\
      \ [])\niteration = inputs.get('iteration_count', 0)\nthreshold = int(loop_state.get('processing_threshold',\
      \ 80))\nbatch_size = int(loop_state.get('batch_size', 5))\n\n# Find items that\
      \ need processing\nitems_to_process = []\nfor item in working_data:\n    if\
      \ (item[\"status\"] == \"pending\" or \n        (item[\"needs_reprocessing\"\
      ] and item[\"score\"] < threshold)):\n        items_to_process.append(item)\n\
      \n# Limit to batch size for this iteration\nbatch_size = min(batch_size, len(items_to_process))\n\
      current_selection = items_to_process[:batch_size]\n\nprint(f\"While-Loop Iteration\
      \ {iteration + 1}: Found {len(items_to_process)} items needing processing\"\
      )\nprint(f\"Processing {len(current_selection)} items in this iteration\")\n\
      \n# Populate outputs\noutputs[\"selected_items\"] = current_selection\noutputs[\"\
      items_selected\"] = len(current_selection)\noutputs[\"remaining_candidates\"\
      ] = len(items_to_process) - len(current_selection)\noutputs[\"iteration\"] =\
      \ iteration + 1\n"
    depends_on:
    - initialize_while_state
    description: Select items that need processing in this iteration
  - id: improve_while_items
    name: Improve While-Loop Items
    type: script
    script: "import random\n\n# Get data from inputs (previous task output)\nselected_items\
      \ = inputs.get('selected_items', [])\nthreshold = int(loop_state.get('processing_threshold',\
      \ 80))\niteration = inputs.get('iteration', 1)\n\nimproved_items = []\nprocessing_stats\
      \ = {\n    \"items_improved\": 0,\n    \"items_completed\": 0,\n    \"score_increase\"\
      : 0,\n    \"reprocessing_needed\": 0\n}\n\nfor item in selected_items:\n   \
      \ # Simulate quality improvement process\n    old_score = item[\"score\"]\n\
      \    \n    # Apply improvement (random boost between 5-25 points)\n    improvement\
      \ = random.randint(5, 25)\n    new_score = min(100, old_score + improvement)\n\
      \    \n    # Determine if item meets threshold\n    if new_score >= threshold:\n\
      \        status = \"completed\"\n        needs_reprocessing = False\n      \
      \  processing_stats[\"items_completed\"] += 1\n    else:\n        status = \"\
      pending\"\n        needs_reprocessing = random.choice([True, False])  # 50%\
      \ chance needs more work\n        if needs_reprocessing:\n            processing_stats[\"\
      reprocessing_needed\"] += 1\n    \n    improved_item = {\n        \"id\": item[\"\
      id\"],\n        \"name\": item[\"name\"],\n        \"score\": new_score,\n \
      \       \"status\": status,\n        \"attempts\": item[\"attempts\"] + 1,\n\
      \        \"needs_reprocessing\": needs_reprocessing,\n        \"improvement_applied\"\
      : improvement,\n        \"processed_in_iteration\": iteration\n    }\n    \n\
      \    improved_items.append(improved_item)\n    processing_stats[\"items_improved\"\
      ] += 1\n    processing_stats[\"score_increase\"] += improvement\n\navg_improvement\
      \ = processing_stats[\"score_increase\"] / len(improved_items) if improved_items\
      \ else 0\n\n# Populate outputs\noutputs[\"improved_items\"] = improved_items\n\
      outputs[\"processing_stats\"] = processing_stats\noutputs[\"average_improvement\"\
      ] = avg_improvement\n\nprint(f\"Improved {len(improved_items)} items with average\
      \ score increase of {avg_improvement:.1f}\")\nprint(f\"Completed: {processing_stats['items_completed']},\
      \ Still need reprocessing: {processing_stats['reprocessing_needed']}\")\n"
    depends_on:
    - select_while_items
    description: Apply quality improvements to selected items
  - id: update_while_state
    name: Update While-Loop State
    type: script
    script: "# Get current state and improved items\ncurrent_dataset = loop_state.get('working_dataset',\
      \ [])\nimproved_items = inputs.get('improved_items', [])\nthreshold = int(loop_state.get('processing_threshold',\
      \ 80))\n\n# Update the working dataset with improved items\nupdated_dataset\
      \ = []\nimproved_ids = {item[\"id\"]: item for item in improved_items}\n\nfor\
      \ item in current_dataset:\n    if item[\"id\"] in improved_ids:\n        updated_dataset.append(improved_ids[item[\"\
      id\"]])\n    else:\n        updated_dataset.append(item)\n\n# Calculate new\
      \ statistics\npending_count = sum(1 for item in updated_dataset if item[\"status\"\
      ] == \"pending\")\ncompleted_count = sum(1 for item in updated_dataset if item[\"\
      status\"] == \"completed\")\ntotal_score = sum(item[\"score\"] for item in updated_dataset)\n\
      avg_score = total_score / len(updated_dataset) if updated_dataset else 0\n\n\
      reprocessing_needed = sum(1 for item in updated_dataset \n                 \
      \        if item[\"needs_reprocessing\"] and item[\"status\"] == \"pending\"\
      )\n\n# Check exit conditions\nshould_continue = pending_count > 0 and avg_score\
      \ < threshold\ncompletion_pct = (completed_count / len(updated_dataset)) * 100\
      \ if updated_dataset else 0\n\n# Populate state_updates to modify loop state\
      \ variables\nstate_updates[\"working_dataset\"] = updated_dataset\nstate_updates[\"\
      pending_items\"] = pending_count\nstate_updates[\"processed_items\"] = completed_count\n\
      state_updates[\"total_score\"] = total_score\nstate_updates[\"average_score\"\
      ] = avg_score\nstate_updates[\"reprocessing_count\"] = reprocessing_needed\n\
      state_updates[\"should_continue\"] = should_continue\nstate_updates[\"completion_percentage\"\
      ] = completion_pct\n\n# Populate outputs for this task\noutputs[\"iteration_summary\"\
      ] = {\n    \"pending\": pending_count,\n    \"completed\": completed_count,\n\
      \    \"avg_score\": avg_score,\n    \"completion_pct\": completion_pct,\n  \
      \  \"should_continue\": should_continue\n}\n\nprint(f\"While-Loop State Update:\"\
      )\nprint(f\"  Pending: {pending_count}, Completed: {completed_count}\")\nprint(f\"\
      \  Average Score: {avg_score:.1f} (Threshold: {threshold})\")\nprint(f\"  Completion:\
      \ {completion_pct:.1f}%\")\nprint(f\"  Should Continue: {should_continue}\"\
      )\n"
    depends_on:
    - improve_while_items
    description: Update state variables and working dataset for while-loop
  description: Process items until quality threshold met using while-loop pattern
  previous_node: for_loop_processor
  exit_condition: ${pending_items} <= 0 || ${average_score} >= ${processing_threshold}
  max_iterations: ${max_while_iterations}
  state_variables:
    batch_size: ${batch_size}
    total_items: ${initialize_datasets.total_items}
    total_score: 0
    average_score: 0
    pending_items: 0
    processed_items: 0
    should_continue: true
    working_dataset: ${initialize_datasets.while_loop_dataset}
    reprocessing_count: 0
    processing_threshold: ${processing_threshold}
    completion_percentage: 0
  iteration_variable: iteration_count
- id: generate_comparison_report
  name: Generate Loop Comparison Report
  type: script
  script: "import json\n\n# FIXED: Use proper template variable syntax to access loop\
    \ final state\n# Access for-loop results using template variables (not environment\
    \ variables)\nfor_total = ${for_loop_processor.final_state.total_processed}\n\
    for_value_sum = ${for_loop_processor.final_state.total_value_sum}\nfor_avg_value\
    \ = ${for_loop_processor.final_state.average_value}\nfor_categories = ${for_loop_processor.final_state.category_counts}\n\
    for_batches = ${for_loop_processor.final_state.batches_completed}\n\n# Access\
    \ while-loop results using template variables\nwhile_pending = ${while_loop_processor.final_state.pending_items}\n\
    while_completed = ${while_loop_processor.final_state.processed_items}\nwhile_avg_score\
    \ = ${while_loop_processor.final_state.average_score}\nwhile_completion_pct =\
    \ ${while_loop_processor.final_state.completion_percentage}\nwhile_reprocessing\
    \ = ${while_loop_processor.final_state.reprocessing_count}\nwhile_iterations =\
    \ ${while_loop_processor.iterations_completed}\n\n# Calculate comparison metrics\n\
    total_items = int(\"${dataset_size}\")\nthreshold = int(\"${processing_threshold}\"\
    )\n\nreport = {\n    \"execution_summary\": {\n        \"total_items_generated\"\
    : total_items,\n        \"batch_size_used\": int(\"${batch_size}\"),\n       \
    \ \"quality_threshold\": threshold\n    },\n    \"for_loop_results\": {\n    \
    \    \"type\": \"Fixed iteration processing\",\n        \"total_processed\": for_total,\n\
    \        \"batches_completed\": for_batches,\n        \"total_value_generated\"\
    : for_value_sum,\n        \"average_value\": for_avg_value,\n        \"category_distribution\"\
    : for_categories,\n        \"processing_efficiency\": \"100% - All items processed\
    \ in predetermined batches\"\n    },\n    \"while_loop_results\": {\n        \"\
    type\": \"Conditional processing until threshold met\",\n        \"items_completed\"\
    : while_completed,\n        \"items_still_pending\": while_pending,\n        \"\
    average_quality_score\": while_avg_score,\n        \"completion_percentage\":\
    \ while_completion_pct,\n        \"items_needing_reprocessing\": while_reprocessing,\n\
    \        \"threshold_achieved\": while_avg_score >= threshold,\n        \"iterations_completed\"\
    : while_iterations\n    },\n    \"comparison\": {\n        \"for_loop_use_case\"\
    : \"Best for processing fixed datasets with known size\",\n        \"while_loop_use_case\"\
    : \"Best for iterative improvement until quality criteria met\",\n        \"for_loop_predictability\"\
    : \"High - Fixed number of iterations\",\n        \"while_loop_adaptability\"\
    : \"High - Continues until conditions satisfied\",\n        \"recommendation\"\
    : \"Use for-loop for batch processing, while-loop for quality-driven iterative\
    \ processing\"\n    },\n    \"performance_metrics\": {\n        \"for_loop_iterations\"\
    : for_batches,\n        \"while_loop_iterations\": while_iterations,\n       \
    \ \"for_loop_completion_rate\": \"100%\",\n        \"while_loop_completion_rate\"\
    : f\"{while_completion_pct:.1f}%\"\n    }\n}\n\nprint(\"=== LOOP COMPARISON REPORT\
    \ ===\")\nprint(f\"For-Loop: Processed {for_total} items in {for_batches} fixed\
    \ batches\")\nprint(f\"While-Loop: Achieved {while_completion_pct:.1f}% completion\
    \ with {while_avg_score:.1f} average score\")\nprint(f\"Threshold target: {threshold}\
    \ (Achieved: {while_avg_score >= threshold})\")\nprint()\nprint(\"FOR-LOOP SUMMARY:\"\
    )\nprint(f\"  - Processed all {for_total} items systematically\")\nprint(f\" \
    \ - Generated total value of {for_value_sum:.2f}\")\nprint(f\"  - Average value\
    \ per item: {for_avg_value:.2f}\")\nprint()\nprint(\"WHILE-LOOP SUMMARY:\")\n\
    print(f\"  - Completed {while_completed} items to quality standard\")\nprint(f\"\
    \  - {while_pending} items still pending improvement\")\nprint(f\"  - Average\
    \ quality score: {while_avg_score:.1f}/{threshold}\")\nprint(f\"  - {while_reprocessing}\
    \ items need additional reprocessing\")\nprint(f\"  - Completed in {while_iterations}\
    \ iterations\")\n\nprint(f\"__OUTPUTS__ {json.dumps(report)}\")\n"
  depends_on:
  - while_loop_processor
  - for_loop_processor
  description: Compare results from for-loop and while-loop processing using proper
    template variable access
  previous_node: while_loop_processor
inputs:
- name: dataset_size
  type: integer
  default: 25
  required: false
  description: Size of the dataset to generate and process
- name: batch_size
  type: integer
  default: 5
  required: false
  description: Number of items to process per batch
- name: processing_threshold
  type: integer
  default: 80
  required: false
  description: Quality threshold for processing (0-100)
- name: max_while_iterations
  type: integer
  default: 20
  required: false
  description: Maximum iterations for while loop
outputs:
  for_loop_summary:
    type: object
    source: for_loop_processor.final_state
    description: Summary of for-loop processing results
  comparison_report:
    type: object
    source: generate_comparison_report
    description: Comprehensive comparison of both loop types
  while_loop_summary:
    type: object
    source: while_loop_processor.final_state
    description: Summary of while-loop processing results
  total_items_processed:
    type: integer
    source: for_loop_processor.final_state.total_processed
    description: Total items processed by for-loop
  quality_threshold_achieved:
    type: boolean
    source: while_loop_processor.final_state.average_score
    description: Whether while-loop achieved quality threshold
version: '1.0'
namespace: examples
description: Demonstrates both for-loop and while-loop functionality with proper state
  variable management
timeout_seconds: 2400
Execution ID Status Started Duration Actions
59932fb4... COMPLETED 2025-07-28
10:04:13
N/A View
bd39e506... COMPLETED 2025-07-22
11:18:22
N/A View