Dynamic Data Storage and Customization

Learn how the Shifts platform implements flexible data storage through JSON columns and dynamic attributes, allowing you to extend standard entities with custom data while maintaining data integrity.

Overview

The Shifts platform uses a flexible approach to data storage that enables entities like users, shifts, and businesses to store additional custom data beyond their standard attributes. This article explains how this dynamic data storage system works, how various entities can be extended, and best practices for using these features.

Understanding Dynamic Data Storage

Rather than a traditional custom fields framework, Shifts uses JSON column storage, which offers several advantages:

• Schema Flexibility: Store complex data structures without altering database tables
• Performance: Efficiently query and index JSON data
• Type Safety: Maintain data types within JSON structures
• Evolution: Easily add new attributes without migrations

Key Dynamic Data Storage Locations

The platform provides several primary locations for storing dynamic data:

User Preferences

Users can store personalized settings and preferences:

1. Storage Location: users.preferences JSON column
2. Access Method: Through virtual attributes via store_accessor
3. Common Uses:
   • UI customization preferences
   • Notification settings
   • Default view configurations
   • Personal display options

Shift Requirements

Shifts can store role-specific requirements and counts:

1. Storage Location: shift_requirements.role_counts JSON column
2. Access Method: Through specialized accessors and helper methods
3. Common Uses:
   • Staff count requirements by role
   • Skill requirements
   • Certification requirements
   • Special role combinations

Notification Metadata

Notifications can include additional context and data:

1. Storage Location: notifications.metadata JSON column
2. Access Method: Direct JSON access or through metadata helpers
3. Common Uses:
   • Event-specific details
   • Action links
   • Status information
   • Related entity references

Business Configuration

Business-level settings and configurations:

1. Storage Location: Various JSON columns in the businesses table
2. Access Method: Through specialized setters and getters
3. Common Uses:
   • Customization settings
   • Feature flags
   • Integration configurations
   • Business-specific rules

Accessing Dynamic Data

User Preferences Example

# Setting a user preference
current_user.update_preferences(
  dark_mode: true,
  notification_sound: 'chime',
  dashboard_layout: 'compact'
)

# Accessing user preferences
if current_user.preferences[:dark_mode]
  # Apply dark theme
end

API Access

You can access and update preferences via API:

PATCH /api/v1/users/preferences
{
  "dashboard_layout": "expanded",
  "notification_sound": "bell"
}

Adding New Dynamic Attributes

While you don’t need database migrations to add new dynamic attributes, you should follow these steps for better structure:

1. Document the Attribute: Add it to your data dictionary or schema documentation
2. Add Type Validation: Ensure the attribute has proper validation when set
3. Create Accessors: Add helper methods if needed for complex logic
4. Update UI Components: Add form fields or display elements for the attribute
5. Consider Defaults: Determine appropriate default values for existing records

Best Practices for Dynamic Data Storage

Data Organization

• Group Related Fields: Keep related dynamic attributes together
• Consistent Naming: Use consistent naming conventions for all dynamic attributes
• Avoid Deeply Nested Structures: Limit nesting to 2-3 levels for maintainability
• Document Schema: Keep up-to-date documentation of your JSON structures

Performance Considerations

• Indexing: Consider indexing frequently queried JSON paths
• Size Limits: Keep JSON documents reasonably sized (under 10KB)
• Querying: Use JSON path queries rather than loading entire documents when possible
• Caching: Cache frequently accessed JSON data to improve performance

Data Integrity

• Validation: Implement validation for dynamic attributes
• Default Values: Provide sensible defaults for missing attributes
• Error Handling: Handle gracefully when expected attributes are missing
• Type Checking: Validate data types for JSON values

Common Use Cases

Extending User Profiles

Add additional user information without altering the core schema:

1. Go to Admin Settings > User Preferences
2. Under the Custom Data section, define additional fields:
   • Display name
   • Data type (string, number, boolean, etc.)
   • Default value
   • Validation rules
3. These fields will appear on user profiles and be stored in the preferences JSON column

Custom Shift Attributes

Add business-specific data to shifts:

1. Navigate to Admin Settings > Shift Configuration
2. In the Custom Attributes section, define additional shift fields
3. These will appear in the shift creation and editing forms

Extended Business Settings

Configure business-specific features and rules:

1. Go to Admin Settings > Business Configuration
2. Define custom business settings in the Advanced Settings section
3. These settings will be stored in the business’s JSON configuration columns

Limitations and Considerations

While dynamic data storage offers flexibility, be aware of these limitations:

• Schema Evolution: No built-in versioning for JSON schema changes
• Validation: Custom validation logic needed for complex rules
• Reporting: Custom reporting may be needed for JSON field data
• Migration: Moving data between JSON fields and standard columns requires custom code

Advanced JSON Storage Techniques

Partial Updates

Update only specific attributes within a JSON document:

# Update just one preference without changing others
user.update_column(
  :preferences,
  user.preferences.merge(dashboard_layout: 'compact')
)

JSON Path Queries

Query records based on JSON content:

# Find users with specific preferences
User.where("preferences->>'dark_mode' = ?", 'true')

JSON Transformations

Apply transformations to JSON data:

# Transform all preferences to a new format
user.preferences.transform_keys { |key| key.to_s.underscore }

Troubleshooting Dynamic Data Issues

Common problems and solutions:

• Missing Attributes: Use Hash#fetch with defaults to handle missing attributes
• Type Mismatches: Add explicit type conversion when accessing JSON values
• Invalid JSON: Validate JSON structure before saving to prevent corruption
• Performance Issues: Add appropriate indexes for frequently queried JSON paths

Related Resources

• Data Management
• System Customization
• JSON Schema

This article should be updated when:

1. JSON storage capabilities change
2. New dynamic attribute locations are added
3. Performance best practices evolve
4. Schema validation tools are enhanced