volume_threshold

Generate your anomaly test with Elementary AI

Let our Slack chatbot create the anomaly test you need.

elementary.volume_threshold

Monitors row count changes between time buckets using configurable percentage thresholds with multiple severity levels.

Unlike volume_anomalies (which uses z-score based anomaly detection as a dbt test, or ML-based detection in Elementary Cloud), this test lets you define explicit percentage thresholds for warnings and errors, giving you precise control over when to be alerted. It uses Elementary’s metric caching infrastructure to avoid recalculating row counts for buckets that have already been computed.

Use Case

“Alert me if my table’s row count drops or spikes by more than 10% compared to the previous period.”

Test Logic

1. Collect row count metrics per time bucket (using Elementary’s incremental metric caching)
2. Compare the most recent completed bucket against the previous bucket
3. Calculate the percentage change between the two
4. If the previous bucket has fewer rows than min_row_count → PASS (insufficient baseline)
5. If the absolute change exceeds error_threshold_percent → ERROR
6. If the absolute change exceeds warn_threshold_percent → WARN
7. Otherwise → PASS

Test configuration

Required configuration: timestamp_column

data_tests:
  — elementary.volume_threshold:
    arguments:
      timestamp_column: column name # Required
      warn_threshold_percent: int # Optional - default: 5
      error_threshold_percent: int # Optional - default: 10
      direction: [both | spike | drop] # Optional - default: both
      time_bucket: # Optional
        period: [hour | day | week | month]
        count: int
      where_expression: sql expression # Optional
      days_back: int # Optional - default: 14
      backfill_days: int # Optional - default: 2
      min_row_count: int # Optional - default: 100

Models

models:
  - name: < model name >
    data_tests:
      - elementary.volume_threshold:
          arguments:
            timestamp_column: < column name > # Required
            warn_threshold_percent: < int > # Optional - default: 5
            error_threshold_percent: < int > # Optional - default: 10
            direction: < both | spike | drop > # Optional - default: both

Features

• Dual severity levels: Separate thresholds for warnings and errors, giving you graduated alerting
• Directional monitoring: Choose to monitor both directions, only spike (increases), or only drop (decreases)
• Incremental metric caching: Uses Elementary’s data_monitoring_metrics table to avoid recalculating row counts for previously computed time buckets
• Minimum baseline protection: The min_row_count parameter prevents false alerts when the baseline is too small
• Configurable time buckets: Works with hourly, daily, weekly, or monthly buckets

Parameters

Parameter	Required	Default	Description
timestamp_column	Yes	-	Column to determine time periods
warn_threshold_percent	No	5	Percentage change that triggers a warning
error_threshold_percent	No	10	Percentage change that triggers an error
direction	No	both	Direction to monitor: both, spike, or drop
time_bucket	No	{period: day, count: 1}	Time bucket configuration
where_expression	No	-	SQL expression to filter the data
days_back	No	14	Days of metric history to retain
backfill_days	No	2	Days to recalculate on each run
min_row_count	No	100	Minimum rows in the previous bucket required to trigger the check

Comparison with volume_anomalies

Feature	volume_threshold	volume_anomalies
Detection method	Fixed percentage thresholds	Z-score (dbt test) / ML (Cloud)
Severity levels	Dual (warn + error)	Single (pass/fail)
Best for	Known acceptable ranges	Unknown/variable patterns
Configuration	Explicit thresholds	Sensitivity tuning
Baseline	Previous bucket	Training period average

How severity levels work

This test has built-in dual severity using dbt’s warn_if / error_if config. You do not need to set config.severity yourself. The behavior is:

• Change exceeds warn_threshold_percent but not error_threshold_percent → dbt warning
• Change exceeds error_threshold_percent → dbt error (test fails)
• Change is below warn_threshold_percent → pass

For example, with warn_threshold_percent: 3 and error_threshold_percent: 8:

• A 2% drop → pass
• A 5% drop → warning
• A 10% drop → error

Do not set config.severity: error on this test. That would override the built-in dual severity and turn all warnings into errors, defeating the purpose of having separate thresholds.

Notes

• The warn_threshold_percent must be less than or equal to error_threshold_percent
• The test uses Elementary’s metric caching infrastructure. Row counts for previously computed time buckets are reused across runs
• If the previous bucket has fewer rows than min_row_count, the test passes (insufficient data for a meaningful comparison)
• The test only evaluates completed time buckets