Background Jobs

Learn how background jobs work in the Shifts platform, including how to monitor job status, troubleshoot issues, configure workers, and optimize performance for system administrators.

Overview

Background jobs in the Shifts platform handle resource-intensive tasks asynchronously to maintain system performance and user experience. These jobs process operations like report generation, data imports/exports, notifications, email delivery, and scheduled maintenance without blocking user interactions. This article explains how background jobs work and how system administrators can monitor, manage, and troubleshoot them.

Understanding Background Jobs

How Background Jobs Work

Background jobs operate on a queuing system:

1. The application creates a job with specific parameters
2. The job is added to a queue
3. Worker processes pick up jobs from the queue
4. Workers execute the jobs asynchronously
5. Results are stored or actions are taken based on job outcomes

Job Types in Shifts

The platform uses several job categories:

• Immediate Jobs: Time-sensitive operations that run as soon as possible
• Scheduled Jobs: Tasks that run at predetermined times
• Recurring Jobs: Regular maintenance operations that repeat on a schedule
• Priority Jobs: Critical operations that take precedence in the queue
• Batch Jobs: Large operations broken into smaller chunks for processing

Accessing Background Job Management

To monitor and manage background jobs:

1. Log in with system administrator credentials
2. Navigate to System Admin > System Tools > Background Jobs
3. The job management dashboard will display
4. Use the tabs and controls to view and manage different aspects of the job system

Job Dashboard Overview

Main Dashboard

The job dashboard displays:

1. Active Workers: Currently running worker processes
2. Queue Depths: Number of jobs waiting in each queue
3. Processing Rate: Jobs processed per minute
4. Recent Jobs: Recently completed or failed jobs
5. Scheduled Jobs: Upcoming tasks with execution times
6. System Health: Worker and queue health indicators

Queue Monitoring

View detailed queue information:

1. Click the Queues tab
2. For each queue, see:
   • Queue name and purpose
   • Current job count
   • Average processing time
   • Oldest job age
   • Queue latency (delay before processing)
3. Set up alerts for queue thresholds

Worker Monitoring

Track worker processes:

1. Navigate to the Workers tab
2. View:
   • Worker ID and host
   • Current status (idle, busy, dead)
   • Currently processing job (if any)
   • Uptime and job count
   • Resource utilization (memory, CPU)
3. Start, stop, or restart workers as needed

Common Background Jobs

AI Assistant Jobs

Jobs related to AI features:

• AI Thread Cleanup: Removes old conversation threads
• Vector Store Upload: Updates knowledge base vector stores
• Assistant Setup: Configures and initializes AI assistants
• Assistant Scheduled: Performs regular maintenance on assistants

Analytics Jobs

Data processing operations:

• Analytics Processor: Calculates and caches analytics data
• Report Generation: Creates scheduled reports
• Dashboard Refresh: Updates dashboard metrics and charts
• Data Aggregation: Compiles data for business intelligence

User Communication Jobs

Notifications and messaging:

• Email Delivery: Sends system emails and notifications
• SMS Delivery: Processes text message notifications
• In-App Notification: Delivers in-platform notifications
• Digest Creation: Compiles periodic notification digests

Data Management Jobs

Data processing and maintenance:

• Data Import: Processes uploaded CSV/Excel files
• Data Export: Generates downloadable data exports
• Bulk Operations: Processes mass updates to records
• Schema Sync: Updates database knowledge repositories

System Maintenance Jobs

Administrative tasks:

• Orphaned Business Cleanup: Removes abandoned business data
• Session Cleanup: Purges expired user sessions
• Temporary File Cleanup: Removes old temporary files
• Database Optimization: Performs database maintenance

Managing Scheduled Jobs

Viewing Scheduled Jobs

See upcoming jobs:

1. Navigate to the Scheduled tab
2. View all pending scheduled jobs
3. For each job, see:
   • Job type and description
   • Scheduled execution time
   • Recurrence pattern (if recurring)
   • Last execution result
   • Next execution time

Creating Scheduled Jobs

Set up new scheduled tasks:

1. Click Add Scheduled Job
2. Select the job type from available options
3. Configure job parameters:
   • Schedule (one-time or recurring)
   • Timing options (daily, weekly, monthly)
   • Priority level
   • Notification settings
4. Click Create Job

Modifying Job Schedules

Adjust existing job schedules:

1. Find the job in the scheduled jobs list
2. Click Edit
3. Modify schedule parameters:
   • Change timing or frequency
   • Adjust parameters
   • Enable/disable the job
4. Click Save Changes

Job Configuration and Performance

Queue Configuration

Optimize queue settings:

1. Navigate to System Admin > System Configuration > Background Jobs
2. Click the Queue Settings tab
3. Configure:
   • Queue Names: Add or modify queue definitions
   • Job Routing: Define which jobs go to which queues
   • Queue Priorities: Set processing priority
   • Concurrency Limits: Maximum jobs processed simultaneously
4. Click Save Configuration

Worker Configuration

Manage worker processes:

1. Go to the Worker Configuration tab
2. Configure:
   • Worker Count: Number of worker processes
   • Memory Limits: Maximum memory per worker
   • Timeout Settings: How long before jobs are considered stuck
   • Retry Settings: When and how to retry failed jobs
   • Resource Allocation: CPU and memory allocation
3. Click Save Worker Configuration

Performance Tuning

Optimize job processing:

1. Navigate to Performance Settings
2. Configure performance parameters:
   • Batch Size: How many jobs to process in batches
   • Rate Limiting: Control processing speed
   • Throttling Rules: Prevent queue overloading
   • Resource Allocation: Balance system resources
3. Apply and monitor performance changes

Monitoring and Troubleshooting

Job Logs

View detailed job history:

1. Go to the Job History tab
2. Search and filter job logs:
   • By job type
   • By status (completed, failed)
   • By date range
   • By queue
3. Click on any job for detailed information:
   • Complete parameters
   • Execution time and duration
   • Error messages (if failed)
   • Stack trace (for debugging)
   • Retry history

Failed Jobs

Manage failed operations:

1. Navigate to the Failed tab
2. View all failed jobs with:
   • Error details
   • Failure time
   • Retry count
   • Job parameters
3. For any failed job, choose to:
   • Retry: Attempt the job again
   • Delete: Remove from the failed queue
   • Edit & Retry: Modify parameters and retry
   • Move to Dead: Archive permanently failed jobs

Job Statistics

Analyze job performance:

1. Go to the Statistics tab
2. View metrics like:
   • Jobs processed per hour/day
   • Average processing time
   • Failure rate by job type
   • Queue throughput
   • Worker efficiency
3. Use graphs and charts to identify:
   • Peak processing times
   • Recurring failures
   • Performance bottlenecks
   • Resource constraints

Advanced Job Management

Job Prioritization

Set processing order:

1. Navigate to Priority Settings
2. Define priority levels (Critical, High, Normal, Low)
3. Assign job types to priority levels
4. Configure queue behavior for different priorities
5. Set up priority bumping for aging jobs

Job Dependencies

Configure job sequences:

1. Go to Job Dependencies
2. Create job chains where:
   • One job must complete before another starts
   • Success/failure conditions trigger different follow-up jobs
   • Complex workflows can be automated
3. Set up error handling for dependency chains

Web Interface Access Control

Manage dashboard access:

1. Navigate to Security Settings
2. Configure:
   • Access Permissions: Who can view/manage jobs
   • IP Restrictions: Limit access by IP address
   • Authentication: Additional security layers
   • Action Logging: Track who performs which actions
3. Save security settings

System Integration

API Access

Access job functionality programmatically:

1. Navigate to API Integration
2. View documentation for:
   • Creating jobs via API
   • Checking job status
   • Managing workers and queues
   • Retrieving job results
3. Generate API tokens for secure access
4. Test API endpoints from the interface

Notification Integration

Configure job notifications:

1. Go to Notification Settings
2. Set up alerts for:
   • Failed critical jobs
   • Queue backlog thresholds
   • Worker failures
   • Job completion
3. Configure delivery channels (email, SMS, webhook)
4. Set notification thresholds and schedules

Disaster Recovery

Backup and Restore

Protect job data:

1. Navigate to Backup Settings
2. Configure:
   • Queue data backups
   • Job history archiving
   • Scheduled backup jobs
   • Retention policies
3. Test restore procedures for job data

Failover Configuration

Prepare for system issues:

1. Go to Failover Settings
2. Configure:
   • Redundant worker processes
   • Automatic worker restarts
   • Queue rebalancing during outages
   • Recovery procedures

Best Practices

For optimal background job management:

1. Regular Monitoring: Check job queues and failures daily
2. Scale Appropriately: Adjust worker count based on system load
3. Prioritize Critical Jobs: Ensure essential operations take precedence
4. Set Job Timeouts: Prevent stuck jobs from blocking queues
5. Configure Retries: Automatically retry transient failures
6. Balance Resources: Don’t let background jobs impact user experience
7. Archive Old Data: Regularly clear old job history to maintain performance
8. Monitor Trends: Track job volume and performance over time

Related Resources

• System Performance
• System Health
• Scheduled Tasks
• System Monitoring

This article should be updated when:

1. New background job types are added
2. Job processing infrastructure changes
3. Configuration options are expanded
4. Management interface is updated
5. Integration capabilities are enhanced