Introduction

Alertmanager allows flexible routing of alerts to multiple receivers based on labels, severity, or any other alert attribute. This is critical for sending the right alerts to the right teams.

Basic Routing Concepts

The route section in alertmanager.yml defines how alerts are grouped and routed. Child routes can match specific labels and override receivers.

Example Configuration

Here’s a sample alertmanager.yml snippet illustrating multi-receiver routing:

route:
  receiver: 'default-receiver'
  group_by: ['alertname']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 3h

  routes:
    - match:
        severity: 'critical'
      receiver: 'pagerduty-receiver'

    - match_re:
        team: 'devops|backend'
      receiver: 'slack-devops'

    - match:
        team: 'frontend'
      receiver: 'slack-frontend'

How This Works

• Alerts with severity=critical go to PagerDuty.
• Alerts where team matches “devops” or “backend” go to the DevOps Slack channel.
• Alerts for the “frontend” team go to their Slack channel.
• All other alerts go to the default-receiver.

Receiver Definitions

Define the receivers corresponding to each route with appropriate configurations (email, Slack, PagerDuty, etc.):

receivers:
- name: 'default-receiver'
  email_configs:
    - to: '[email protected]'

- name: 'pagerduty-receiver'
  pagerduty_configs:
    - service_key: 'your-pagerduty-service-key'

- name: 'slack-devops'
  slack_configs:
    - api_url: 'https://hooks.slack.com/services/xxx/yyy/zzz'
      channel: '#devops-alerts'

- name: 'slack-frontend'
  slack_configs:
    - api_url: 'https://hooks.slack.com/services/aaa/bbb/ccc'
      channel: '#frontend-alerts'

Tips for Effective Routing

• Use meaningful alert labels to simplify routing.
• Test configurations with the alertmanager --config.file=alertmanager.yml --dry-run command.
• Keep routing hierarchy clear to avoid unexpected alert duplication.

Conclusion

Multi-receiver routing in Alertmanager provides granular control over alert notifications, ensuring your teams get timely and relevant alerts to improve incident response.