Automation Workflow Flow Documentation

This document explains the complete flow of what happens when an automation workflow is activated, from user action to email delivery.

📚 For detailed information about trigger events, rate limiting, cadence rules, and how the 5-minute cron job works, see: TRIGGER_EVENTS_AND_CADENCE_RULES.md

Overview

There are three types of automation workflows: 1. Scheduled - Runs on a recurring schedule (daily/weekly/monthly) 2. Trigger - Activates when specific events occur (morale drop, kiosk offline, etc.) 3. Persona - Similar to scheduled but persona-based

Flow Diagram

User Activates Workflow
    ↓
Controller: create_deliveries (DeliveriesConcern)
    ↓
DeliveryScheduleService.create_deliveries
    ↓
Creates CommsInstance + CommsDelivery records
    ↓
CommsDelivery.after_create callback
    ↓
schedule_variable_replacement_job
    ↓
TemplateVariableReplacementJob (scheduled for send_at time)
    ↓
TemplateVariableResolverService.resolve_and_render
    ↓
[If email + has dynamic components]
    CompiledTemplateImageService.generate_compiled_image_url
    ↓
CommsDeliverySendJob
    ↓
ApplicationMailer.delivery_email
    ↓
Email Sent

Detailed Step-by-Step Flow

1. User Activation (Entry Point)

File: app/controllers/concerns/communication_center/deliveries_concern.rb Method: create_deliveries

What happens: - User clicks "Activate" button in the UI - POST request to /communication_center/workflows/:id/create_deliveries - Controller finds the workflow and determines send_at time

Special handling for trigger workflows: - If workflow_type == 'trigger', workflow is just activated (status set to 'active') - No deliveries created immediately - they're created later when triggers fire via cron job - Returns early with success message

For scheduled/persona workflows: - Calculates send_at time based on: - params[:send_at] if provided - workflow.calculate_next_run_at for scheduled workflows - workflow.next_run_at if already set

2. Delivery Creation Service

File: app/services/delivery_schedule_service.rb Method: create_deliveries

What happens:

2.1 Validation

• Validates workflow exists and has a template
• Validates workflow status is 'active' or 'paused'
• Validates send_at time (for trigger workflows, must be within 5 seconds of now)

2.2 Audience Resolution

• Calls resolve_audience to get matching stakeholders
• Filters by:
  • Persona (if configured)
  • Project inclusion/exclusion (if configured)
• Returns array of Stakeholder objects

2.3 Channel Detection

• Determines available channels from template's channel_variants
• Checks for emailhtml, smstext, teams_text content

2.4 Create CommsInstance

• Creates a CommsInstance record that groups all deliveries for this workflow run
• Stores audience snapshot (stakeholder IDs, count, personas)

2.5 Create Deliveries for Each Stakeholder

• For each stakeholder:
  • Checks cadence rules (cooldown, daily/weekly/monthly limits, type limits)
  • If cadence allows: creates delivery with original send_at
  • If cadence blocks: queues delivery with calculated next_available_at
  • Determines project contexts (if workflow filters by projects)
  • Creates one delivery per project context

2.6 Cadence Checking

File: app/services/delivery_schedule_service.rb Method: can_send_to_stakeholder?

Checks multiple limits in priority order: 1. Cooldown hours - Minimum time between any communications 2. Type daily limit - Max per day for specific template types (report, alert, etc.) 3. Daily limit - Max communications per day 4. Weekly limit - Max communications per week 5. Monthly limit - Max communications per month

If any limit is exceeded, calculates next_available_at time and queues the delivery.

2.7 Generate Initial Payload

• Calls generate_merged_payload for each stakeholder
• Extracts template content for the stakeholder's preferred channel
• Creates initial JSON payload with:
  • subject (for email)
  • body (channel-specific content)
  • recipient info (name, email, stakeholderid, projectid)

2.8 Update Workflow

• Updates workflow with:
  • last_run_at = Time.current
  • next_run_at (calculated for scheduled workflows)
  • status = 'active'

2.9 Create Next Scheduled Delivery (if applicable)

• For scheduled workflows, calls workflow.create_next_scheduled_delivery
• This creates deliveries for the next scheduled run time
• Prevents duplicate deliveries by checking existing ones

3. Delivery Record Creation

File: app/models/comms_delivery.rb

What happens: - CommsDelivery.create! is called with: - comms_instance_id - stakeholder_id - project_id - channel (email/sms/teams) - send_at (original or queued time) - status ('scheduled' or 'pending') - merged_payload (initial JSON payload with unresolved variables)

Status determination: - 'pending' if trigger workflow and send_at <= now + 5 seconds - 'scheduled' otherwise

4. After Create Callback

File: app/models/comms_delivery.rb Method: schedule_variable_replacement_job

What happens: - Automatically called after CommsDelivery.create! - Schedules TemplateVariableReplacementJob to run at send_at time - Uses Delayed::Job (not ActiveJob) for precise timing - If send_at <= now + 5 seconds, schedules immediately - Otherwise schedules for future send_at time

Important: This callback is what triggers the variable resolution process!

5. Variable Replacement Job (Runs at send_at time)

File: app/jobs/template_variable_replacement_job.rb Method: perform

What happens:

5.1 Validation

• Checks delivery exists and isn't already processed
• Skips if status is 'paused', 'sent', 'delivered', or 'cancelled'

5.2 Check if Resolution Needed

• Checks if payload contains {{variable}} placeholders
• If no placeholders, skips to email send

5.3 Resolve Variables

File: app/services/template_variable_resolver_service.rb Method: resolve_and_render

Process: 1. Extracts all variables from payload (subject and body) 2. Resolves each variable using resolve_used_variables - Looks up variable in database - Fetches data based on variable type - Returns resolved value 3. Replaces {{variable}} placeholders with actual values

5.4 Handle Dynamic Components (Email Only)

File: app/services/template_variable_resolver_service.rb Method: replace_with_compiled_image

If email channel + has dynamic components (GAUGE, CHART, LIST, THEME, etc.):

1. Generate Compiled Image

   • Calls CompiledTemplateImageService.generate_compiled_image_url
   • Extracts all dynamic component tokens from body
   • Categorizes components (gauges, charts, lists, themes, spotchecks)
   • Renders HTML using _compiled_template.html.erb partial
   • Converts HTML to PNG image using HtmlToImageService
   • Stores image and returns URL

2. Replace in Body

   • Removes all dynamic component tokens from body
   • Inserts <img src="[image_url]"> tag
   • Updates delivery's merged_payload with new body

If no dynamic components: - Just replaces regular variables normally

5.5 Update Delivery Payload

• Updates delivery.merged_payload with resolved JSON
• All {{variables}} are now replaced with actual values

5.6 Trigger Email Send

• Calls CommsDeliverySendJob.perform_later(delivery.id)
• This queues the actual email sending

6. Email Send Job

File: app/jobs/comms_delivery_send_job.rb Method: perform

What happens:

6.1 Validation

• Checks delivery exists and isn't processed
• Validates channel is 'email'
• If send_at is in future, reschedules job

6.2 Send Email

• Updates status to 'pending'
• Calls ApplicationMailer.delivery_email(delivery).deliver_now
• Mailer uses resolved payload (subject + body with compiled image)

6.3 Update Status

• On success: updates status to 'sent'
• Creates CommsEvent record with 'sent' event
• On failure: updates status to 'failed' with error reason

7. Scheduled Workflow Automation

File: lib/tasks/automation_workflows.rake Task: automation_workflows:process_due

What happens: - Cron job runs periodically (e.g., every 5 minutes) - Finds workflows where next_run_at <= Time.current - For each due workflow: - Calls DeliveryScheduleService.create_deliveries - Creates deliveries for all matching stakeholders - Calculates and sets next next_run_at

8. Trigger Workflow Automation

File: lib/tasks/automation_workflows.rake Task: automation_workflows:process_triggers

What happens: - Cron job runs periodically (typically every 5 minutes) - Finds active trigger workflows - Calls TriggerEvaluatorService.evaluate_all - For each workflow: - Checks if trigger conditions are met - Rate Limit Check (Layer 1): Checks if enough time has passed since last trigger send - Uses cooldown_hours from persona cadence rules (or defaults to 1 hour) - If within rate limit → Skips creating deliveries (prevents spam) - If rate limit allows → Continues to create deliveries - If triggered AND rate limit allows: - Calls DeliveryScheduleService.create_deliveries - Creates deliveries immediately (sendat = now) - Updates `lasttriggersentat` for this trigger event type - Note: Rate limiting is workflow-level, not kiosk-specific or issue-specific

For detailed explanation of rate limiting, cadence rules, and how they work together, see: TRIGGER_EVENTS_AND_CADENCE_RULES.md

Key Files and Their Roles

Controllers

• app/controllers/concerns/communication_center/deliveries_concern.rb
  • Entry point for workflow activation
  • Handles preview and download of compiled templates

Services

• app/services/delivery_schedule_service.rb

  • Core service for creating deliveries
  • Handles audience resolution, cadence checking, delivery creation

• app/services/template_variable_resolver_service.rb

  • Resolves template variables
  • Handles compiled image generation for dynamic components

• app/services/compiled_template_image_service.rb

  • Generates single compiled image from all dynamic components
  • Renders HTML using _compiled_template.html.erb

• app/services/trigger_evaluator_service.rb

  • Evaluates trigger conditions for trigger workflows
  • Creates deliveries when triggers fire

Models

• app/models/automation_workflow.rb

  • Workflow configuration and scheduling logic
  • Callbacks for managing scheduled deliveries

• app/models/comms_delivery.rb

  • Individual delivery record
  • After_create callback schedules variable replacement

• app/models/comms_instance.rb

  • Groups deliveries from a single workflow run

Jobs

• app/jobs/template_variable_replacement_job.rb

  • Resolves variables and generates compiled images
  • Runs at send_at time

• app/jobs/comms_delivery_send_job.rb

  • Sends the actual email
  • Runs after variable replacement

• app/jobs/compiled_template_image_job.rb

  • (Not currently used in main flow - may be legacy)

Views

• app/views/communication_center/dynamic_components/_compiled_template.html.erb

  • Template for rendering compiled template HTML
  • Used by CompiledTemplateImageService to generate image

• app/views/communication_center/delivery_schedule/_preview_compiled_template.html.erb

  • Preview view for compiled templates in UI

Rake Tasks

• lib/tasks/automation_workflows.rake
  • Cron tasks for processing scheduled and trigger workflows

Important Callbacks and Hooks

1. CommsDelivery.after_create

   • Automatically schedules TemplateVariableReplacementJob
   • This is critical - without this, variables won't be resolved!

2. AutomationWorkflow.after_update

   • ensure_next_delivery_for_scheduled_workflow - Creates next scheduled delivery
   • cancel_future_deliveries_on_schedule_change - Cancels deliveries when schedule changes
   • cancel_deliveries_outside_active_period - Cancels deliveries outside start/end dates

Data Flow Summary

1. Workflow → Contains template, schedule, audience config
2. CommsInstance → Groups deliveries from one workflow run
3. CommsDelivery → Individual delivery to one stakeholder
4. merged_payload → JSON with subject/body (starts with unresolved variables)
5. TemplateVariableReplacementJob → Resolves variables, generates compiled image
6. merged_payload (updated) → Now has resolved content + image tag
7. CommsDeliverySendJob → Sends email using resolved payload

Common Issues and Debugging

Deliveries not being created

• Check workflow status is 'active'
• Check audience config matches stakeholders
• Check template has channel content configured
• Check cadence rules aren't blocking all stakeholders

Variables not being resolved

• Check TemplateVariableReplacementJob is scheduled (check delayed_jobs table)
• Check delivery has send_at set
• Check delivery status isn't 'paused' or 'cancelled'
• Check payload contains {{variable}} placeholders

Compiled images not generating

• Check body contains dynamic component tokens (GAUGE, CHART, etc.)
• Check CompiledTemplateImageService.has_dynamic_components? returns true
• Check HtmlToImageService is working (may need external service)
• Check logs for image generation errors

Emails not sending

• Check CommsDeliverySendJob is queued after variable replacement
• Check delivery status is 'scheduled' or 'pending'
• Check channel is 'email'
• Check mailer configuration
• Check delivery has resolved payload with subject/body