Data Analysis

Anomaly Diagnosis

Retention Analysis Data Verification

Help analysts verify the results of the retention analysis model, locate the statistical difference between retention and event analysis, and explain the definition logic of both models.

IndustriesGamesEntertainmentE-commerce

Agent · Skills Live

skill · loadedindustry+100

Run this Skill on my data

Retention heat-grid

cohort × day

D1D3D7D14D30

Wk1

Wk2

Wk3

Analysis readyevidence attached

Retention Analysis Data Verification is a data-validation capability in ThinkingAI's industry Skill library. When retention report data does not match expectations, it helps data analysts cross-verify by replicating the retention definition into the event analysis model, locating the root cause of the statistical-logic difference layer by layer. This is ThinkingAI's decade of industry experience, packaged into ready-to-use Skills.

Live walkthrough

AgentSave as AgentExport session

retention-analysis-data-verification

The retention report says D7 LTV is $28.5, but when I query the payment amount for the same batch of users in event analysis, it's only $15.2 per user, nearly double the gap. Which number is right? Is there a data problem?

Get the true definition of the retention report for a sample RPG game DEMO

Initial event register, return-visit metric is payment amount shown as a simultaneous metric, daily output by A103

Parse your event analysis config, pinpoint the metric-definition differences

No cohort filter + directly taking the period per-user average, two deviations

Create a conditional cohort, reproducing the "register same-day" cohort

Accumulate D0–D7 from the A103 daily raw values, then divide by the initial user count

MCP pulls the real data, compares the two metric-definition results

Reproduced value $28.47, 0.1% off the retention report

Of the two numbers, the retention report's $28.5 is correct: your $15.2 is off because of how the metric was reproduced, not because the data is broken. Once the retention metric is correctly reproduced in event analysis, the result is $28.47, only 0.1% off the report.

1. The three metric definitions side by side

Metric definition	D7 LTV	Deviation from retention report
Retention analysis report (baseline)	$28.50	N/A
Your event analysis (definition not reproduced)	$15.20	−46.7%
Event analysis after reproducing the definition	$28.47	−0.1%

2. Where the $15.2 goes wrong: three metric-definition mismatches

No cohort filter: the retention definition counts only the 12,384 users in the register same-day cohort, while your event analysis folded in every paying user from the period, so the populations aren't even the same batch

Wrong metric type: you took the period cumulative per-user average directly (A114 logic), but validating LTV requires the A103 daily raw values, accumulated first and then divided by the initial user count

Incomplete time range: you took only the D7 single-day slice, without the daily D0–D7 eight-day accumulation; LTV is a cumulative metric, not a single-day one

3. Reproduction validation result

Reproduction config result: the conditional cohort matched 12,384 users, D0–D7 daily cumulative payment of $352,600, $28.47 per user; differences in statistical logic explain the entire gap, and the data itself has no quality issue. The daily accumulation curve essentially overlaps the retention report from D3 on, with deviation stable within 0.1%.

Conclusion

Conclusion: the retention report is safe to use, and $15.2 is an artifact of a metric-definition mismatch. The reproduction config is saved, so next time you can just change parameters and reuse it. One more hard rule to note: using a result-based cohort to filter event analysis silently returns wrong counts: to reproduce a retention cohort you must use a conditional cohort.

On your data

That was a simulated run

Leave your work email and we will run a live walkthrough on your real business data.

The problem

Retention analysis and event analysis are the two most-used models in AE, but their statistical logic is completely different. Retention analysis restricts to the initial-event user cohort and tracks return behavior, while event analysis applies no cohort restriction and only looks at event triggers. The same metric can differ by more than 50% between the two models, and many analysts don't understand the difference, so when the numbers don't match they spend 2 to 3 days comparing field by field, or even change the configuration at random and make it worse.

What it does

Five-stage standard verification flow: explain the model statistical difference, compare configuration differences, set the correct event-analysis replication config, verify and compare the data, and conclude

Clearly flag six common pitfalls: no cohort filter, wrong metric type, incomplete time range, no day-by-day accumulation, result-cohort filtering, and using a user_property filter in place of a cohort restriction

Mandatorily use real report definitions and MCP query data for verification, never guessing or fabricating numbers, and clearly mark data as unavailable when it is not accessible

A configuration-difference comparison table maps each dimension precisely so the replication config is actionable

When to use it

LTV data in a retention report does not match the payment amount from event analysis

A newly built retention report's data looks suspiciously low and needs cross-verification by replicating the definition in event analysis

A cross-report comparison of retention rate and event-analysis triggering users reveals a difference

A retention analysis uses simultaneously-displayed metrics and you need to understand the underlying calculation logic

Retention data definitions are inconsistent across projects and need a standardized verification method

In the field

Case

A game project · D7 LTV data verification

The retention report showed D7 LTV of $28.5, while event analysis showed only $15.2 in payment amount for the same period. The Skill found event analysis had no cohort filter and used a simple Sum. Following the correct config, it added a conditional cohort to restrict the initial-event cohort and used a day-by-day cumulative sum divided by the initial user count, replicating a result of $28.47, essentially matching the retention analysis.

FAQ

Why do retention and event analysis show different data for the same metric?

The statistical logic differs. Retention restricts to the initial-event cohort and tracks returns, while event analysis applies no cohort restriction and only looks at triggers; you must use a cohort filter to replicate the retention definition.

Should I use A103 or A114 for verification?

A103 is the day-by-day raw Sum and A114 is the period-cumulative per-user average. To verify LTV-type metrics you must use A103 day-by-day data, accumulate it, and divide by the initial user count.