The CJA Alert Builder: configuring a metric alert that fires on signal

When you need this

Once you've decided a metric deserves monitoring (see the alerts use-cases guide for that decision), the Alert Builder is where the decision becomes a working configuration. This is also the step where a well-intentioned alert most often goes wrong - a threshold set from a gut-feel number instead of the metric's actual historical range, or a dimension breakout so granular that legitimate small-sample noise triggers it constantly.

It's the right tool whether you're setting up a first alert on a new data view or tightening an existing one that's been firing too often (or not often enough) since it went live.

How to work with it correctly

1. Set the threshold from the metric's actual historical distribution, not a round number. Pull the metric's trend in Analysis Workspace over a representative period first, and set a static threshold (or anomaly sensitivity) based on where a real, meaningful deviation actually starts - not an arbitrary number that sounds reasonable.

Data view: <production data view>
Metric: Orders (or a calculated metric)
Dimension breakout: none, or one low-cardinality dimension (e.g. Channel)
Segment: optional - scope to the population that matters
Condition: Static threshold (below / above a value)
       or: Anomaly detection (sensitivity: low / medium / high)
Evaluation frequency: daily / hourly, matched to how fast action is expected
Recipients: specific people or a distribution list, not "everyone"

2. Choose evaluation frequency to match how fast a response is actually possible. An hourly alert on a metric nobody can act on faster than once a day just generates more notifications for the same eventual response - match frequency to the real operational cadence around the metric.

3. Use dimension breakouts sparingly, and only on dimensions with enough volume per value. Breaking an alert out by a high-cardinality dimension (individual product SKU, for instance) multiplies the chance of a small-sample false positive; reserve breakouts for dimensions with a handful of meaningfully-sized values.

4. Name the alert for what it monitors and why, not a generic label. "Checkout completion rate - US, weekday baseline" tells the next person reviewing the Alert Manager what to expect; "Alert 3" does not.

How to verify it worked

• Backtest the configured threshold or sensitivity against the same metric's trend in Analysis Workspace for the past several weeks, confirming it would have fired on real known events and stayed quiet on normal variation.
• Confirm the recipient list actually receives the notification (test with a deliberately tripped condition where the environment allows it) rather than assuming the configuration saved correctly means delivery works.
• Check the alert's segment and data view scope against the underlying data view's own definitions in Workspace, to confirm it's actually reading the population you intended, not a stale or mismatched segment.

Illustrative, not a measured result: retuning an alert's threshold from a guessed number to one derived from twelve weeks of actual metric history might cut a noisy alert's false-positive rate enough that its recipients start trusting it again.

Anomaly-detection sensitivity options and evaluation-frequency choices vary by CJA release - verify current settings against Adobe's own documentation.

FAQ

Can an alert be scoped to a calculated metric instead of a standard one?

Yes - any metric available in the data view, including calculated metrics (including ones scoped with a segment, covered separately), can be the basis of an alert. This is useful for alerting on a ratio or derived KPI rather than only raw counts.

What's the difference between a static threshold and anomaly detection?

A static threshold fires whenever the metric crosses a fixed number you set, regardless of normal pattern - simple and predictable, but blind to seasonality. Anomaly detection compares current behavior against the metric's own historical pattern, which suits metrics with real weekly or seasonal rhythm but requires enough history to establish what "normal" looks like before it's reliable.