Skip to content

scripts: read the install/deletion cohort, the report that made v1.49 not ship pings - #441

Merged
JasonYeYuhe merged 1 commit into
mainfrom
asc-retention-curve
Aug 18, 2026
Merged

scripts: read the install/deletion cohort, the report that made v1.49 not ship pings#441
JasonYeYuhe merged 1 commit into
mainfrom
asc-retention-curve

Conversation

@JasonYeYuhe

Copy link
Copy Markdown
Collaborator

Why

App Store Installation and Deletion Standard carries an App Download Date on every row, making it the only Apple feed that yields a real cohort: how long after downloading did this install get removed.

That report is the stated reason v1.49 shipped a first-run window instead of D1/D7/D30 retention pings — the pings would have been conditioned on the user having already opened the popover and accepted the disclosure card, i.e. on having cleared the very hurdle under investigation, so they could only ever have described the survivors.

But acquisition_reconcile.py didn't read it. The justification was one session away from evaporating. Now it does.

Also adds App Store Purchases Standard, short enough to print in full: one purchase, 2026-06-24, iPhone / CN / Alipay, $1.03 proceeds.

The coverage line caught its own bug, which is the point

The first version compared total Install events against first-time downloads and printed iPad coverage of 114 % — impossible for a subset, and visible on the first run. Install spans every Download Type (first-time, redownload, auto-download, manual update); first-time downloads are one of them.

Fixed to compare like with like, which moves the sample fraction from a comfortable-looking 18–64 % to the real figure:

device installs (first-time) deletes first-time coverage
Desktop 19 5 6 5/108 = 5 %
iPhone 129 25 20 25/203 = 12 %
iPad 8 1 2 1/7 = 14 %

That is not a cosmetic fix. This feed is a consenting-user sample, not a census, and a far smaller one than the first version implied. The output now says so in the place someone would actually read it:

the SHAPE of the latency distribution is usable;
deletes / installs is NOT a deletion rate and must not be quoted as one — it divides one population by another.

Current shape

n=27 — median 0 days, 63 % same-day, 74 % within one day, 89 % within seven, 100 % within thirty.

A median at or near zero is the signature of "opened it, didn't get it" — abandonment before the app ever proved itself, which no in-app counter can observe because the counter only fires after the menu is opened and the disclosure accepted.

Track this series across releases: it is the readout for the v1.49 first-run window, and the honest one, because it is measured outside the app rather than by the app.

Verification

  • --self-test still passes (the anti-threshold guard is applied to the new feed too).
  • Run end-to-end against live ASC; output above is real.
  • ruff clean — and as of ci: make ruff actually lint the Python it claimed to lint #437 that now actually runs on scripts/.
  • The plan's own caveat has been corrected in cli-pulse-internal (41d4965), since it quoted the same inflated 18 % figure.

🤖 Generated with Claude Code

… not ship pings

`App Store Installation and Deletion Standard` carries an `App Download Date`
on every row, which makes it the only Apple feed that yields a real COHORT:
how long after downloading did this install get removed.

That report is the stated reason v1.49 shipped a first-run window instead of
D1/D7/D30 retention pings — the pings would have been conditioned on the user
having already opened the popover and accepted the disclosure card, i.e. on
having cleared the very hurdle under investigation, so they could only ever
have described the survivors. But the reconcile tool did not read it, so the
justification was one session away from evaporating. Now it does.

Also adds `App Store Purchases Standard`, which is short enough to print in
full: one purchase, 2026-06-24, iPhone/CN/Alipay, $1.03 proceeds.

THE COVERAGE LINE CAUGHT ITS OWN BUG, WHICH IS THE POINT

First version compared total `Install` events against first-time downloads and
printed **iPad coverage of 114%** — impossible for a subset, and visible
immediately. `Install` spans every Download Type (first-time, redownload,
auto-download, manual update); first-time downloads are one of them. Fixed to
compare first-time against first-time, which moves the sample fraction from a
comfortable-looking 18-64% to the real 5-14%:

    Desktop    5/108 =  5%
    iPhone    25/203 = 12%
    iPad       1/7   = 14%

That matters more than a cosmetic fix. This feed is a CONSENTING-USER SAMPLE,
not a census, and it is a far smaller sample than the first version implied.
The SHAPE of the latency distribution is usable; deletes/installs is not a
deletion rate and the output now says so in the place someone would read it.

Current shape, n=27: median 0 days, 63% same-day, 74% within one day, 89%
within seven. Track it across releases — it is the readout for the v1.49
first-run window, and the honest one, because it is measured outside the app
rather than by the app.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 18, 2026 02:42

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@JasonYeYuhe
JasonYeYuhe merged commit a285224 into main Aug 18, 2026
5 checks passed
@JasonYeYuhe
JasonYeYuhe deleted the asc-retention-curve branch August 18, 2026 02:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants