Skip to main content
Gå til innhold

Prometheus Rules & Argus

For teams that want alerting configuration as code - version-controlled, reviewed, and deployed alongside your application - you can use PrometheusRules and the Argus alert aggregation system.

When to Use This Approach​

Consider PrometheusRules if:

  • You want alerting config version-controlled in your repository
  • You want alerts deployed with your application via CI/CD
  • You need Probe-based monitoring (health checks, SSL expiry)
  • You prefer configuration files over UI-based setup
  • You want the service center to act on certain alerts

For simpler use cases, Grafana Alerting may be easier.

PrometheusRules​

PrometheusRules evaluate metrics and generate alerts based on predefined conditions.

Example PrometheusRule​

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: my-app-alerts
  labels:
    prometheus: system
    role: alert-rules
spec:
  groups:
    - name: my-app.rules
      rules:
        - alert: HighErrorRate
          expr: |
            sum(rate(http_requests_total{status=~"5..", namespace="my-namespace"}[5m]))
            / sum(rate(http_requests_total{namespace="my-namespace"}[5m])) > 0.05
          for: 5m
          labels:
            severity: warning
            sikt_alert: argus
          annotations:
            summary: "High error rate in my-app"
            description: "Error rate is {{ $value | humanizePercentage }}"

Required Labels​

For Prometheus to pick up your rules:

metadata:
  labels:
    prometheus: system
    role: alert-rules

For alerts to reach Argus:

labels:
  sikt_alert: argus

Best Practices​

  1. Create PrometheusRules only for production environments to avoid noise
  2. Use group names matching *.rules pattern
  3. Verify your expr is valid before deploying
  4. Test in staging before production

Probes (Black-box Monitoring)​

Probes actively check endpoint health from outside your application.

When to Use Probes​

  • Monitor external endpoints
  • Check SSL certificate expiration
  • Verify DNS resolution
  • Test HTTP response codes and latency

Example Probe​

apiVersion: monitoring.coreos.com/v1
kind: Probe
metadata:
  name: my-app-probe
  labels:
    platon-monitor: "true"
spec:
  interval: 30s
  module: http_2xx
  targets:
    staticConfig:
      static:
        - https://my-app.example.com

Label with platon-monitor: "true" for automatic detection.

Default Probe Rules​

Platon provides default PrometheusRules for all Probes that check:

  • Endpoint is responding
  • SSL certificate is valid

For custom probe alerts, create your own PrometheusRule referencing probe metrics.

Argus​

Argus is an alert aggregation system that consolidates alerts from Prometheus into a single dashboard.

Viewing Alerts​

All configured alerts can be viewed at argus.uninett.no.

Configuring Notifications​

  1. Log in to Argus with Feide
  2. Click your profile icon → Destinations

Navigate to destinations

  1. Add a destination (email, Slack channel email, or Teams webhook)

Create destination

  1. Create a filter for your alerts (by tags, severity, source)

Create filter

  1. Create a notification profile linking your filter to your destination

Create notification profile

Severity Levels​

Tag your alerts with severity for filtering:

SeverityArgus Level
critical1
high, error2
medium, moderate3
low, warning4
info, informational5

Alert Flow​

Alert flow from applications to Argus

Learn More​