Per-Rule Latency: The Measurement Rule Engines Leave to You

A quota check runs in front of every request a SaaS product serves. One Tuesday its p99 goes from 9ms to 26ms, and the graph that catches it is the graph every team has: an APM trace with a span named policy evaluation, sitting between the auth span and the first database call, three times wider than it was the week before.

That span is where the investigation stops. It covers a ruleset that reached fourteen rules over two years, and nothing underneath it is broken out. No rule shipped that week. Traffic shape moved, one of the fourteen is now doing more work per call than it used to, and the trace has no opinion about which one.

The interesting part is not that this team was missing a dashboard. It is that the dashboard they wanted is rare across the entire category, and it is rare for a structural reason that comes from the same property that made extracting the rules from code worth doing.

Rules are data, so the profiler samples the wrong thing

Point a CPU profiler at the service under load and the frames it collects belong to the evaluator: a condition tree walk, an operator dispatch, a map lookup for a fact value. Those frames are identical for every rule in the group. Which rule the evaluator was working on when the sample landed is a value in a field, not a name on the stack, so the flame graph shows you the engine's shape summed over the whole ruleset.

No better profiler closes that gap. A rule engine exists so policy stops being code, and code is the only unit a profiler knows how to attribute time to. The component that could attribute time to a rule has to sit inside the engine, where rule identity is a first-class value rather than an argument that vanished three frames ago.

The obvious objection is to keep the rules in code and give each one a function of its own, so the profiler has a name to attribute time to. That works halfway, and the half that fails is the half being asked about. A small predicate called on every request is the first thing a JIT inlines, and an inlined frame is not on the stack when the sampler wakes up. Resolution is the next wall: a rule costing four microseconds is effectively invisible to a profiler sampling a hundred times a second, so what accumulates over hours is a share of CPU rather than the latency of any single call. A flame graph holds no percentiles at all, so a rule that is quick on most inputs and slow on a few shows up as a narrow frame either way. The largest cost comes last. Rules moved into data so they could change without a deploy, and moving them back into functions to satisfy a profiler gives that up: a threshold change becomes a PR again.

Averaging closes off whatever the profiler left open. The rules worth finding are cheap on the common shape of input and expensive on an uncommon one, which is another way of saying their cost lives in the tail. A mean taken across a busy group moves by a rounding error when that tail thickens, and a rounding error is not something anyone acts on.

So teams reach for timers around the branches. In an if-else policy, that looks roughly like this:

long t0 = System.nanoTime();
boolean tierOk = checkPlanTier(ctx);
metrics.timer("policy.tier").record(System.nanoTime() - t0, NANOSECONDS);

long t1 = System.nanoTime();
boolean quotaOk = tierOk && checkMonthlyQuota(ctx);
metrics.timer("policy.quota").record(System.nanoTime() - t1, NANOSECONDS);

Two problems come with that code, and neither shows up in the numbers it produces. The metric name is a string somebody typed next to a branch, so the day the branch is renamed or moved, the series keeps reporting under its old name against different work. And the tierOk && short-circuit means policy.quota is timed only on calls that cleared the tier check, so the two series have different denominators and nothing on the dashboard admits it.

What other engines actually give you

On 2026-08-19 I read the public documentation of seven rule engines to find out what any of them reports at rule granularity. That was a documentation read and not a product evaluation — I ran none of them for this.

Across those seven, none documents a rule-level latency percentile that the product computes and ranks rules by. Several come close, and where they stop is more useful than the headline.

Percentiles do turn up for four of these seven, and none of the four is per rule. The GoRules one is a mock-up dashboard on a marketing page, and the DecisionRules.io one is a load-test figure in the FAQ. Both are the time one request took. The percentiles Kogito publishes for Drools sit on request elapsed time and on DMN output values. cedar-benchmarking is a tool the Cedar project runs to catch its own regressions, and it times policies a set at a time rather than one by one.

Two of the seven are worth more than a row.

• Drools, on KIE Server, ships the best raw data of the seven. That histogram is genuinely bucketed, so one histogram_quantile line of PromQL returns a per-rule p99. Two things sit between that and an answer. The Grafana dashboard Kogito generates for a ruleset plots a one-minute average of this metric and never calls histogram_quantile on it, so the ten calls in a thousand that ran a hundred times longer than the rest are averaged away in a figure whose buckets still hold them. And the timer wraps consequence execution, which leaves condition evaluation unmeasured, the half that grows as a ruleset accumulates clauses. A query recovers the first. Nothing recovers the second.
• Decisions.com will name a slow rule for you. The vendor's own walkthrough uses that Profiler to point at a slow rule. There is no percentile anywhere in it, the data is held in memory and rolls over every fifteen minutes so nothing accumulates that a percentile could be taken of, and the vendor describes it as a tool for gathering information once an issue has already been identified. Champion Challenger has a comparison labeled performance, but that one weighs a rule in production against a changed copy of it on outcome, not on time.

So the axis is not whether anything times rules. Several of these do, and Drools emits numbers raw enough that you can compute whatever you like from them. The axis is who computes the percentile and who ranks one rule against its siblings, and none of the seven does either. Where the raw timings are exported at all, that work is yours, in Grafana or Datadog or a spreadsheet. That is a standing maintenance cost with an owner, which is the category of cost that goes unpriced when a team compares running the engine itself against letting someone else operate it.

The half the Drools timer never sees is why LexQ splits the condition stage (CONDITION) from the action stage (ACTION). The two do not share a sample size: conditions are evaluated on every call, actions only when a rule matches.

What this measurement has to hold to be worth reading

LexQ has computed it since 2026-07-14, on a screen it calls Rule Performance. Three of the decisions in that build turned out to matter more than the timing itself, because each one is a place where a rule-level number quietly becomes misleading.

The threshold lives inside the group

A policy group is the unit rules are filed under, and this piece shortens it to group. A rule is flagged when its p50 reaches ten times the median of the per-rule p50s in its own policy group. There is no absolute threshold, and leaving it out was deliberate. A fraud check and a shipping-cost calculation have no reason to share a normal range, so an absolute figure means whoever typed it has judged every domain at once, including the ones it was never chosen for. A comparison against siblings makes no such judgment, and it needs no upkeep when the fleet changes under it.

The median inside that comparison is not a stylistic preference either. An average is pulled by the outlier being hunted, so a rule an order of magnitude above its siblings lifts the very bar it is measured against and shrinks its own multiple.

HIT and MISS never share a distribution

A cache miss is paid by whichever calls arrive first at a version, and what they buy with it is loading and compiling that version rather than evaluating anything in it. Merged into one distribution, those calls land in precisely the region an investigation is reading, and an artifact of deployment gets promoted to a finding. Splitting them costs nothing and retires an entire class of wrong conclusion, which is a good trade for numbers people read with an incident open.

The same figure means two different things depending on which side it is counted on. Under HIT it is a rule that is slow. Under MISS it is a cache being filled for the first time.

A sample too small is withheld rather than estimated

Saying p99 at all asks that a few calls actually landed in the slow tail. Three is where that line is drawn, which is what n × (1 − q) ≥ 3 says, and it lands on n ≥ 6 for p50, n ≥ 60 for p95 and n ≥ 300 for p99. Below the bar the field is null. Calling a rule slow asks more of the data than putting its number on screen does, so the flag carries a gate of its own at n ≥ 100, and a group with fewer than three qualifying rules has nothing to compare against and says so with INSUFFICIENT_COHORT.

So a table can carry a p95 with no flag beside it. That is not a statement that the rule is fine. It is a statement that not enough calls have accumulated to judge it, and the two have to be told apart. An estimate would have been the friendlier choice here, and that is the objection to it: a p99 assembled from four observations looks on screen exactly like a p99 assembled from four thousand.

One constraint sits under all three. A profiler that runs on every call taxes the thing it is measuring, so the rule breakdown is built from a deterministic one percent of calls, the same window returning the same numbers however often it is reopened, while the group total keeps every call. The asymmetry is the point. Magnitude is a question the total answers cheaply, and ranking is the question worth spending a sample on. It is also why the display gates above are load-bearing rather than fussy, since a hundredth of the traffic is what has to clear them.

Nothing in the time series charts is smoothed, for a related reason. A gap in the graph is not missing data, it is an interval when no calls arrived, and an interpolated point would be a number the engine never measured.

When you don't need this

Per-rule percentiles earn their cost in one situation: a decision on a hot path, in a ruleset too large for anyone to hold its cost in their head. Four cases fall outside that, and in all of them this measurement is the wrong thing to go get.

• The decision is not on a hot path. A nightly batch or a back-office approval screen absorbs a few hundred milliseconds without anyone noticing. Total runtime answers every question anyone will actually ask of it, and the per-rule breakdown becomes a screen nobody opens twice.
• The ruleset is three or four rules. Hand-rolled timers drift from the logic they wrap, but drift takes edits and time to happen. Four branches that have not changed in a year cost less to instrument by hand than to move anywhere else.
• The volume is too low to sample. A few hundred calls a day leaves a per-rule n in single digits under a one percent sample, and the display gates will hold most of the table at null. That is an honest answer for that traffic, and it is not one you can act on.
• The p99 problem is I/O. If facts arrive from another service, or the slow part is the query assembling them, every rule in the profile will read flat and identical, and the day goes to proving where the time is not. Take the span breakdown first and open the rule profile when the evaluation span is the wide one.

Where the blueprint lives

The implementation side of this — a hot-path quota gate, the rule that quietly grew eight OR branches, the profile that ranked it against its siblings, and the simulation run to prove that restructuring it changed no decisions — is written up separately as Isolating a Slow Rule on a Hot Path with Per-Rule Profiling. The console screens and the actual numbers live there.

The argument here is smaller than the feature. A decision that runs on every request is production code by any definition that matters, and every other piece of production code on that path has a latency distribution attached to it as a matter of course. Rules were left out, not for lack of demand, but because attributing time to data requires whatever evaluates the data to keep the books. That is an engine's job. It is worth asking about before the Tuesday when the graph moves.

→ Find out what each rule costs per call — start free at lexq.io