Reducing False Positives in Amount Tolerance Rules #

At 06:40 the ACH exception desk lights up: 900 items that auto-posted cleanly yesterday are now breaks, every one off by exactly $15.00. Nothing changed in your engine. A correspondent flipped a handful of USD wires from OUR to SHA charge handling overnight, so a $10,000.00 instruction now settles at $9,985.00 after an intermediary deduction — a perfectly traceable, perfectly legitimate payment that your flat ±$0.10 amount band cannot distinguish from fraud. This is a false positive: a correct match the tolerance rule rejects because it has no way to explain the variance. Within the broader transaction matching and reconciliation algorithms framework, this page drills into one surgical problem inside tolerance threshold configuration — the amount dimension — and shows how to move from a static scalar to a context-aware band without opening a fraud gap.

The failure is structural, not a tuning mistake. A single ±0.1% or ±$0.05 rule applied across ACH credits, Fedwire settlements, and cross-border pacs.008 legs has to be loose enough for the widest legitimate variance on any rail, which makes it far too loose for the tightest — so it simultaneously buries fraud on low-value ACH and flags fee-stripped wires as breaks. The fix is to make the band a function of the transaction's metadata, evaluated after the deterministic keying pass has already cleared the exact-match majority.

Concept Spec: A False Positive Is Explainable Variance the Band Rejects #

Let a candidate pair carry a sent amount $a_s$ and a received amount $a_r$ in the same currency's minor units. The observed delta is $\delta = |a_s - a_r|$. A static rule accepts the pair when $\delta \le \tau$ for one global constant $\tau$. The problem is that the legitimate delta is not a constant — it is the sum of several rail-specific components:

where $f_{\text{corr}}$ is documented correspondent/intermediary fee stripping (SWIFT field 71G / ISO 20022 ChrgBr), $r_{\text{fx}}$ is multi-hop FX conversion rounding, and $t_{\text{dec}}$ is decimal-precision truncation where one system stores scaled integers and another stores two-place decimals. A false positive is any pair where $\delta \le \delta_{\text{legit}}$ but $\delta > \tau$ — the variance is real and explainable, yet the band is blind to why.

The effective band must therefore be computed per pair from its network, charge bearer, and currency pair, not read from a single config cell. Each evaluation is still constant work, so scoring a batch of $n$ pairs remains $O(n)$ time and — with a streaming generator — $O(1)$ memory; the only thing that changes is that $\tau$ becomes $\tau_{\text{eff}}(\text{context})$, a lookup plus a max, not a literal.

Full Annotated Python Implementation #

The evaluator below replaces the flat rule with a context-aware band. It holds every amount as decimal.Decimal (binary float cannot represent exact cents and will, at volume, nudge a pair across the boundary), streams pairs through a generator so a multi-million-row return file never materializes in memory, and — critically — consults a documented correspondent fee schedule so a $15.00 SHA deduction is explained rather than merely tolerated. Explaining a delta is stronger than widening a band: it accepts the specific expected amount and still rejects a $15.01 break.

from __future__ import annotations

from dataclasses import dataclass
from decimal import Decimal, ROUND_HALF_UP, InvalidOperation
from typing import Iterator, Optional

CENTS = Decimal("0.01")


def parse_amount(raw: str) -> Decimal:
    """Parse an amount string to a 2-place Decimal, rejecting float inputs.

    Passing a float here is a bug, not a convenience: Decimal(0.1) preserves
    the binary rounding error we are trying to eliminate. Force strings/ints.
    """
    if isinstance(raw, float):  # guard the most common ingestion mistake
        raise TypeError("amounts must arrive as str/int, never float")
    try:
        return Decimal(str(raw)).quantize(CENTS, rounding=ROUND_HALF_UP)
    except InvalidOperation as exc:
        raise ValueError(f"invalid amount format: {raw!r}") from exc


@dataclass(frozen=True, slots=True)
class AmountRule:
    """Per-context amount tolerance profile, keyed upstream by
    (network, charge_bearer, currency_pair)."""
    absolute_cap: Decimal          # fixed floor, e.g. Decimal("0.05")
    relative_bps: Decimal          # relative ceiling in basis points
    fx_extra: Decimal              # extra allowance for cross-currency rounding
    fee_schedule: tuple[Decimal, ...] = ()  # documented correspondent deductions


@dataclass(frozen=True, slots=True)
class Pair:
    txn_id: str
    network: str                   # "ACH" | "FEDWIRE" | "SWIFT"
    amount_sent: Decimal
    amount_received: Decimal
    charge_bearer: Optional[str] = None    # "OUR" | "SHA" | "BEN"
    currency_sent: str = "USD"
    currency_received: str = "USD"


def effective_band(rule: AmountRule, pair: Pair) -> Decimal:
    """Larger of the absolute floor and the relative ceiling, plus an FX
    allowance only when the legs are genuinely cross-currency."""
    relative = (pair.amount_sent.copy_abs() * rule.relative_bps) / Decimal("10000")
    band = max(rule.absolute_cap, relative)
    if pair.currency_sent != pair.currency_received:
        band += rule.fx_extra
    return band


def evaluate_amount(pair: Pair, rule: AmountRule) -> tuple[str, Decimal, str]:
    """Return (status, delta, reason). Status is STP, EXPLAINED, or EXCEPTION.

    Precedence is deliberate: an exact match posts first; a delta that equals a
    *documented* correspondent fee is EXPLAINED (auto-posts with an audit tag);
    only then does the numeric band apply. Anything past the band is a break.
    """
    delta = (pair.amount_received - pair.amount_sent).copy_abs()

    if delta == Decimal("0.00"):
        return "STP", delta, "exact_match"

    # A fee deduction is only explainable when the bearer permits it.
    if pair.charge_bearer in ("SHA", "BEN") and delta in rule.fee_schedule:
        return "EXPLAINED", delta, f"documented_fee:{delta}"

    band = effective_band(rule, pair)
    if delta <= band:
        return "STP", delta, f"within_band:{band}"

    return "EXCEPTION", delta, f"delta_{delta}_over_band_{band}"


def process_batch(
    pairs: Iterator[Pair],
    rules: dict[str, AmountRule],
) -> Iterator[dict[str, object]]:
    """Stream pairs through the context-aware amount gate in O(1) memory.

    A missing rule is an ERROR, never a silent global default — an unmapped
    context is exactly how a too-loose fallback band gets applied by accident.
    """
    for pair in pairs:
        rule = rules.get(pair.network)
        if rule is None:
            yield {"txn_id": pair.txn_id, "status": "ERROR",
                   "reason": f"no amount rule for network {pair.network!r}"}
            continue
        status, delta, reason = evaluate_amount(pair, rule)
        yield {
            "txn_id": pair.txn_id,
            "network": pair.network,
            "status": status,
            "delta": str(delta),
            "reason": reason,
            "requires_review": status == "EXCEPTION",
        }

Calibration & Configuration #

The band is only as good as the profile behind each network key. Tune them to the rail's real variance, and store them externally (a version-controlled table keyed by (network, charge_bearer, currency_pair)) so a change hot-reloads without a deploy:

• ACH credits (domestic, USD). Set absolute_cap = Decimal("0.05"), relative_bps = 0, fx_extra = 0. Low-value ACH has no fee stripping and no FX; the only legitimate variance is sub-cent core-system rounding. A percentage band here is worse than useless — on a $40.00 credit, 10 bps is $0.004, so rounding noise reads as fraud. Leave the fee schedule empty.
• Fedwire (domestic, high-value USD). Wire finality is absolute, so the base band is tight: absolute_cap = Decimal("0.00"), a small relative_bps (e.g. 2) only to absorb documented aggregation. The lever that kills false positives here is not a wider band but the fee_schedule — load the correspondent's published intermediary tiers (Decimal("15.00"), Decimal("25.00"), …) so SHA/BEN deductions post as EXPLAINED while an undocumented $15.01 still breaks.
• SWIFT / ISO 20022 (cross-border). Use a relative band (relative_bps = 10) because fees and spreads scale with the ticket, and set fx_extra = Decimal("0.03") to cover the accumulated truncation of a multi-hop conversion. Parse ChrgBr and IntrBkSttlmAmt from the pacs.008 before evaluating, or the engine cannot know which bearer regime applies.

The single highest-leverage calibration move is populating fee_schedule from real correspondent data rather than reaching for a bigger relative_bps. Fee schedules keep the band tight everywhere else; a wider percentage band tolerates a six-figure delta on a large wire.

Validation Example: Before and After #

Take four residue pairs that a flat ±$0.05 rule dumps onto the desk, evaluated against the profiles above:

The flat rule produces three false positives on rows 2–3 and still can't tell the real break (row 4) apart from them. The context-aware gate posts the two explainable wires, absorbs the 3-cent FX truncation on the cross-border leg, and — because $15.01 is not in the fee schedule and exceeds the Fedwire band — correctly holds the one item that a human should actually see. Feeding the four pairs through process_batch yields the audit record for the genuine break as:

{
  "txn_id": "FW-CHASUS33-4472",
  "network": "FEDWIRE",
  "status": "EXCEPTION",
  "delta": "15.01",
  "reason": "delta_15.01_over_band_0.00",
  "requires_review": true
}

Failure Modes & Guardrails #

Three edge cases silently reintroduce false positives — or worse, false negatives — even after the gate is live:

1. Float leaks past ingestion. If any upstream service hands the parser a float, Decimal(str(99.99999999999999)) quantizes to 100.00 and a pair straddles the band edge nondeterministically. The parse_amount guard raises TypeError on float on purpose — keep amounts as strings or scaled integers from the file boundary through the evaluator, and never Decimal(some_float).
2. Unbounded relative band on a large wire. A relative_bps with no absolute ceiling turns into a blank cheque: 10 bps of a $50,000,000 wire is $50,000 of silent auto-post. Always pair a relative band with a hard per-transaction and daily-aggregate cap, and alert whenever an accepted delta exceeds the absolute_cap even though it stayed inside the percentage band — that is your early-warning line for a mis-set profile.
3. A missing profile falling back to a global default. The most dangerous configuration is a catch-all rule applied when the (network, bearer, currency) key is absent, because a too-loose default auto-posts unrelated items. process_batch emits ERROR on an unmapped network rather than defaulting; block the deploy on any unmapped context and route ERROR rows to a dead-letter queue, not to STP. Every EXPLAINED post must also carry its documented_fee tag into an append-only log so a NACHA or Reg E error-resolution review can reconstruct exactly why the engine treated a short settlement as a match.

Frequently Asked Questions #

Related #

• Tolerance Threshold Configuration — the parent guide that sets the amount, date, and reference bands this page's amount gate plugs into.
• Implementing Levenshtein Distance for Payment References — the reference-tolerance counterpart to this amount-tolerance work.
• Configuring Rolling 3-Day Reconciliation Windows — the date-tolerance dimension, sized to rail-specific settlement drift.
• Multi-Field Fallback Chains — the substitution order used when a primary identifier is missing before tolerance runs.
• NACHA Record Layouts Explained — the byte offsets the ACH amounts and trace numbers are parsed from upstream.