TL;DR: False alerts in transaction monitoring come down one scenario at a time, because a library-wide rate hides the rule that is noisy and the rule that has gone quiet. EY's November 2024 survey found that almost 65 percent of institutions cited data quality as an execution challenge, while optimization is still predominantly threshold tuning or segmentation. A lower false-positive rate is a useful reading only when alert volume, written closure reasons, and SARs from that same scenario move with it.
Look at one scenario at a time
A false-positive rate is a property of one scenario, measured over a stated window, against a stated disposition code. Transaction monitoring produces alerts from cash activity, rapid movement of funds, velocity against a peer group, and patterns tied to a product or a corridor. Averaging those alerts into one percentage tells a manager that the program feels busy. The average does not say which threshold to change. The unit of work is the scenario that fired, the customer segment it fired on, and the count of alerts that scenario contributed in the window.
EY's November 2024 transaction monitoring survey reported that 43 percent of respondents use machine learning in detection, while rules remain dominant, and that optimization is predominantly threshold tuning or segmentation. A February 2025 survey from SAS, KPMG, and ACAMS found that reducing false positives in existing surveillance systems was the AI and machine learning deployment priority for 38 percent of respondents, an 8 point increase since 2021. Both figures describe work on the installed library. Replacing that library is a separate project from the tuning cycle.
How transaction monitoring generates an alert is the premise under the rate. A scenario compares customer activity with a condition, and the alert is the record of that comparison. A structuring scenario and a rapid-movement scenario can both touch one customer in one week. Closing both in one case note, then charging the closure to the whole program, erases the evidence each rule needs. Split the closure back to the scenario before anyone computes a percentage.
Calculate the rate on a single scenario before any threshold moves. Count alerts closed as not suspicious, using the written disposition reasons, and divide by that scenario's alert count. Hold every other scenario out of both numbers. Repeat by segment when retail and commercial customers use different thresholds under one scenario name, because those are different rules. Open the closures under the rate. A percentage with no alert list behind it is not a tuning input.
A lower false-positive rate can be the wrong result
The rate falls when the numerator shrinks or the denominator shrinks, and either move can leave detection weaker. Auto-closing alerts before they enter the queue removes them from the denominator. Raising a threshold until a scenario barely fires does the same. Recoding "closed, no suspicious activity" so that it no longer counts as a false positive shrinks the numerator without changing a customer outcome. A dashboard that celebrates the new percentage has not shown that the scenario still catches the activity named in the risk assessment.
McKinsey's October 7, 2022 analysis reported that one institution improved suspicious activity identification by up to 40 percent and efficiency by up to 30 percent after replacing rule tools with machine learning. Identification and efficiency moved together in that case. A program that tracks only the false-positive rate can show a lower rate while identification falls, if the scenario was muted. The useful result is suspicious activity still found, at a cost the team can sustain.
EY also found that over 80 percent of respondents cited regulatory-instigated changes in the prior two years, so one annual percentage cannot stand in for performance. A threshold that was honest in January can be the wrong cut by October after a new product, a new corridor, or a new regulatory focus. Read four signals on one scenario, one segment, and one window before accepting a lower rate.
A healthy alert count is reviewable, and a shift in it should trace to a product launch or a documented threshold change. A collapse after a tuning, or a spike from a broken feed, means the new rate describes the pipeline. Closures should name the evidence, such as a payroll pattern or a known merchant. A blank reason or an unreviewed suppression makes the rate look decisive and teaches the next tuner nothing. A small SAR count can fit a rare typology. A long run of zero SARs is a coverage question, answered against the risk assessment, and answered poorly by deleting the scenario. Reducing false positives means changing the scenario whose signals are weak and leaving the one whose rate, SAR count, and written reasons still agree.
Fix the data before you blame the rule
Almost 65 percent of respondents in the EY survey cited data quality as an execution challenge. A scenario compares activity with fields. A stale occupation code puts the customer in the wrong peer group, so a velocity scenario measures the customer against strangers. A counterparty country missing on a subset of wires makes a geographic scenario fire on blanks, or stay silent on the payments that mattered. Tuning on that data picks a number that fits the defect. The next clean month makes the number look arbitrary, and the queue returns.
The fourth requirement on FinCEN's CDD rule page is ongoing monitoring. Monitoring assumes the customer record at scoring time still describes the customer. A risk rating updated in onboarding and unread by the monitoring feed segments the account under the old rating. Investigators then close alerts the segment produced. The written reason can be accurate about the alert and wrong about the customer. Counting that closure as a false positive trains the next threshold on a labeling error.
Name the fields the scenario reads, and measure how often they are missing or stale when the alert is created. Repair the feed before the threshold moves. Segmentation, which EY paired with threshold tuning as the usual form of optimization, fails the same way when the segment key is dirty. A personal account coded as commercial looks like residual noise after the cut moves. Fix the key, rerun the count, and only then decide whether the cut is wrong. The 43 percent of EY respondents who use machine learning in detection inherit those fields. A model trained on closures caused by missing data will reproduce that pattern.
Keep the version of the threshold that fired
Every alert should store the scenario version that created it: the threshold, the segment, the lookback window, and the timestamp at which that version became effective. Investigators write the closure against that version. A later change, made because the rate was high, does not rewrite the condition that was true on the day the alert opened. Without the stamp, a March sample mixes January's threshold with February's. The rate then describes two rules, and neither tuner can tell which cut produced which outcome.
The change record stays with the scenario and is referenced by the alert. It names the old value, the new value, the approver, the date and time the new value became effective, and the evidence: alert counts before and after, a sample of written closures, and the typology the scenario covers. Transaction monitoring rule tuning is where that sample gets chosen, including activity below the line that the current threshold never raised. The version record is what makes the sample readable a year later, when an examiner asks why the number moved. A log that says only that the threshold was updated cannot answer.
Regulatory-instigated changes, cited by over 80 percent of EY's respondents across two years, are why versions accumulate. Deleting a superseded version deletes the explanation for open alerts that still carry the old condition.
Real-time monitoring creates the alert while the payment is in flight, before a later batch reconciles reference data. The version in force at that moment is the version the disposition cites. Rebuilding the decision from today's configuration explains a different rule if the threshold changed between the alert and the review. Store the version on the alert at creation, and read the four signals on that version alone.
Where Sphinx fits
Sphinx agents disposition the alerts a tuned scenario produces and write why each one closed, so the team can see which rule is noise. They do not replace the scenario library.
Frequently Asked Questions
How do I calculate a false-positive rate for one monitoring scenario?
Count the alerts that scenario produced in a defined period. Divide the number closed as not suspicious, using the institution's written disposition reasons, by that alert count, and keep both numbers limited to that scenario. A library-wide rate mixes a noisy scenario with a quiet one and hides which threshold deserves the change.
Should a bank retire a rule that never produces a SAR?
A rule with zero SARs over a long window is a candidate for review, and retirement is one outcome of that review. The review has to ask whether the scenario still matches a risk in the assessment, whether data gaps kept it from firing on real activity, and whether investigators closed alerts that should have been filed. Retiring a scenario only because the SAR count is zero can remove coverage the risk assessment still requires.
How should a threshold change be documented?
Record the scenario name, the old threshold, the new threshold, the effective date and time, the customer segment, and the person who approved the change. Attach the before-and-after alert counts and the sample of closed alerts that justified the move. The alert itself should keep the version that fired, so a later change does not rewrite the reason the case opened.
Why can a very low false-positive rate be a problem?
A rate near zero can mean the scenario almost never fires, including on activity the risk assessment treats as in scope. Raising a threshold, dropping a segment, or auto-closing alerts before they are counted will shrink the rate without showing that suspicious activity is still found. Read the rate next to alert volume, SAR yield from that scenario, and the fields that were missing when the alert was created.
.png)
