Lessons from the penny

Most Sentinel cost cuts fail for the same reason: nobody can show which records a detection or an investigation still depends on. Without that evidence, you either keep paying for everything or cut and hope.

This post, adapted from my BSides NYC talk October 17th 2026, lays out a third path: use AI, see the data first, split it only where the evidence supports it, keep a human accountable for the decision and build a decision record/ledger.

The problem: two bad options

When a Sentinel bill climbs, teams usually land in one of two places.

  • Keep everything. You pay Analytics-tier prices without knowing which records actually need that tier.
  • Cut blindly. You move records that a detection or an investigation still depends on, and you find out during an incident.

Both share a root cause. Without dependency evidence, a team cannot defend what to keep and what to move. The goal is narrower than “cut cost”: preserve detection utility while reducing unnecessary spend.

The penny: when cost exceeds face value

In 2024, a U.S. penny cost 3.69 cents to produce against a face value of 1 cent. The Mint struck its last circulating penny on November 12, 2025, after the cost had exceeded face value for years (U.S. Mint release; the 3.69-cent figure is the 2024 per-coin cost reported by the Mint via CoinNews).

Some of your log tables are pennies. They cost more to keep at a premium tier than the value they return. The hard part is telling which ones.

The analogy has a limit, and it matters. A penny has no hidden second use. A log can keep its investigation value even when no active detection reads it. A table with no working rule may still support hunting, incident response or retention. So “nothing alerts on this” is a reason to look closer, never a reason to move it.

See before you split

A quiet connector, an expensive table and a weak rule each need a different response. Telling them apart takes three steps, in this order:

  1. Measure ingestion. Volume, cost and connector health.
  2. Review dependencies. Detection and rule quality, investigation use and retention.
  3. Test the change. Routing, approval and rollback.

Start with Table Insights

Table Insights (see Microsoft’s Tech Community walkthrough) will not make the decision for you. It tells you where to look first. Its costs are list-price estimates, so compare them with your actual pricing and billing.

Ask these questions: Where does the volume sit?

Are there day-over-week fluctuations?

What changed?

What are the top 5 tables by daily cost?

Which table should we review first?

What was the last data received?

Which source may have gone quiet?

Which change needs an explanation?

Then check the rules that read the table

In general, 30 to 45% of the rules I review have an entity mapping issue. Entity mappings support investigation and correlation, so review identifier strength as well as entity coverage.

This range does not measure detection effectiveness. A failed-login rule can still alert without a mapped Account entity, and missing user context does not prove a missed detection. What it does tell you is to review the rule and its dependencies before moving the data it reads.

An event label never justifies a routing decision

Routine-looking event labels can hide detection or investigation value. These three examples are illustrative review scenarios, not routing instructions.

Firewall Action: Allow An allowed connection to an unexpected destination, so that means test detections that use allowed flows

Windows Event ID 4624: A successful logon after repeated failed attempts

GitHub: Test failed-to-success logon correlation GitHub audit Action: git.clone, eg sensitive repository cloned by an unexpected actor.

The pattern is the same each time. The label looks like volume, and the value sits in a specific combination of fields. That is why the evidence comes before any approval, and why the same event can be a keep in one environment and a split candidate in another.

The log adviser skill: it recommends, a human decides

To make this repeatable I designed an adviser skill.

Three rules bound what it does:

  • Evidence first. It uses observed tables, fields and values, and keeps cost separate from detection value.
  • Uncertainty stays visible. It names missing facts. Ambiguous values or weak evidence send a table to manual review.
  • Humans approve. It proposes a narrow rule that preserves critical signals, ready for sign-off and a change ticket.

How it decides: keep, split or manual review

First it establishes the facts: whether the table’s value is high, routine or uncertain; its tier, volume, cost and the rules that read it; and whether an unambiguous routing field exists whose values have been reviewed. Then it selects one of three outcomes.

  1. Keep. Retain the data in Analytics because something depends on it.
  2. Split. Propose only reviewed values for the data lake, subject to testing and human approval.
  3. Manual review. Resolve weak evidence or ambiguous values before changing any routing.

Every recommendation carries risk and rollback notes.

A table reaches keep or split only once its facts are established. Weak evidence or ambiguous values loop back through manual review rather than forcing a split.

Rule quality is one input, not the answer

A table with no working rule may still support investigation, hunting or retention. This framework is illustrative, not measured data. Finding Question Proposed response Strong KQL, poor metadata Can an analyst interpret it? Repair metadata Bad KQL Is a needed detection broken? Fix and test before routing Not ingested Is there a justified coverage gap? Assess benefit and cost No working rule reads it Any investigation or retention use? Review before reducing tier Noisy Can tuning preserve useful signal? Tune and reassess

Who owns what

The skill assembles the 30-day volume and trend, known and unknown dependencies, rule quality and other data uses, and the observed fields with candidate values. The human owns the exact split and its approval, the governance model and the operational risk trade-off. No rule is deployed autonomously.

The sign-off record

For each table, the output is one reviewable record:

  • Recommendation and rationale: keep, split or manual review, with the detection-value assessment.
  • Supporting evidence: table facts, consuming rules, the routing field and its value distribution, and the reviewed keep and route values.
  • Evidence gaps: each missing fact or unresolved dependency, so the approver sees the uncertainty.
  • Risk and rollback: the risk if split, the rollback plan, and the proposed rule attached for sign-off.

A lab design: GitHub audit log routing

Start with the native capability. Microsoft Sentinel’s Defender portal supports ingestion-time filter and split transformations. Use them where you can, and confirm the exact destination, table and supported deployment method before attempting deployment as code.

The five steps

  1. Hypothesis. Some GitHub audit event groups may suit a different tier after dependency review. The evidence needed is the exact connector and table, the observed event mix, and the consumers of the data.
  2. Baseline. For one selected table, record the connector, table, measurement window and billable GB by tier, plus working rules, investigation uses, retention needs and unknown consumers. Do this before changing anything.
  3. Proposed split. Profile the volume and event mix, then confirm the deployed action field. A human then reviews dependencies, tests candidate values and agrees the rollback.
  4. Approval and rollback. Version the configuration with its exact fields, destinations and the prior known-good version. Test with sample records, dependent rules, destination queries and a rollback rehearsal.
  5. Acceptance checks. Covered in the next section.

What the split could look like

Destination Event values: Condition Analytics org.add_member, team.add_member, repo.add_member, permission changes

Retain required detection dependencies, eg data lake candidates git.clone, git.fetch, git.push Only after review. No automatic “else”

The git.clone row connects back to the earlier example. Cloning a sensitive repository by an unexpected actor is a signal.

If a detection, hunt or investigation query depends on it, it stays in Analytics, and the dependency review is how you find out. If the owner is uncertain, keep the current routing.

Gating it in GitHub Actions

The proposed pattern has four moves:

1. Load the table facts from a config file or workflow input.

2. Run the routing skill against the payload, evaluate the decision and its supporting evidence.

3. Require pull request approval before applying any DCR or routing change.

4. Production changes stay blocked until a human approves.

What to measure after the split

Accept the change only if cost and security checks both meet criteria you agreed in advance.Use comparable measurement windows and the same pricing assumptions before and after.

  • Cost and volume. Compare billable GB by tier and estimated cost on the same basis.
  • Security and recoverability. Test that dependent detections still fire, query the routed records in their new tier, and rehearse restoring the prior configuration.

A cost number alone is not an acceptance test. It may not be a penny after all. A split that saves money and breaks a detection has failed.

Try it: one table, one owner, one tested decision

Start small enough that you can finish.

  1. Select one costly table and document its dependencies.
  2. Approve exact routing values only after a test.
  3. Measure the result and keep a rollback ready.

Here is the question I’d like you to answer, in the comments or on LinkedIn: which dependency would make you keep a candidate event group in Analytics? If you can’t answer in one sentence for your own git.clone, that is your first finding.

If you try the pilot, I’d like to hear what the dependency review turned up. You can find me on LinkedIn.

I am not the first to write about Sentinel cost and tiering. Here is what I found, and where this approach fits.

  • Microsoft SOC optimization flags billable tables that analytics rules did not use over the past 30 days and suggests a cheaper plan or stopping ingestion. It works at table level and exempts tables used for UEBA or threat intelligence matching (Microsoft Learn).
  • Practitioner guides from Decryption Digest, Adam the Automator and eGroup (all June 2026) cover tiering by detection need. The first two also describe row-level filtering or routing in data collection rules. Decryption Digest checks rule references before a tier change, and Adam the Automator shows high-severity records going to Analytics with the rest going to the data lake.
  • Where this post differs. It treats a rule reference as one input alongside investigation, hunting and retention use. It routes on exact reviewed event values, not on a table or a severity. And it ends in a signed record with evidence gaps and a rollback plan.

Want a copy of the skill? Reach out directly, otherwise REA all you’d like.

Sources

Leave a Reply