ADFA-5094: ship the topic tree as a gzipped APK floor asset - #428
Merged
Conversation
The committed kolibri_tree.jsonl.gz floor would otherwise fall under the repo's default "* text=auto eol=lf" and risk line-ending normalization corrupting the gzip on checkout (Windows especially). Add *.gz to the explicit binary list, next to *.zip / *.tar.gz.
The topic tree is ~16 MB raw; ship and serve it gzipped (~3 MB) instead. BundledTreeSource now reads kolibri_tree.jsonl.gz — both the APK-bundled floor and the pulled R2 overlay are gzip — decompressing on read with GZIPInputStream (streaming: the 16 MB is never written out, only the in-memory index is kept). A truncated or corrupt .gz trips the gzip CRC and falls through to an empty tree, same as a missing asset, so the live sources still answer. Pairs with: the workflow gzipping what it publishes to R2, and a committed src/main/assets/kolibri_tree.jsonl.gz floor asset (~3 MB).
…eat/ADFA-5094-apk-tree-floor-update
…/ADFA-5094-apk-tree-floor-update
…ore build .gitignore keeps the raw tree out of git but has no say over packaging: mergeAssets bundles every file physically present in src/main/assets/, so a stray kolibri_tree.jsonl left by a local `--tree` generator run would ship in the APK alongside the .gz. Add a preBuild-hooked task that deletes it, so only the gzipped bundle (what BundledTreeSource reads) can ship. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ally ships AGP auto-decompresses any *.gz asset at merge time and drops the extension, so a source kolibri_tree.jsonl.gz ships inside the APK as an already-decompressed kolibri_tree.jsonl. BundledTreeSource opened kolibri_tree.jsonl.gz, which is never present in the APK, so the offline tree loaded nothing and browse was silently empty. Open the APK asset under APK_ASSET = "kolibri_tree.jsonl" (the name AGP produces), keeping ASSET = "kolibri_tree.jsonl.gz" for the R2 overlay + refresh worker, and read both through readGzipAware(): it sniffs the 2-byte gzip signature and decompresses only the real-gzip overlay, reading the AGP-decompressed asset as-is. The byte signature decides, so the two paths cannot drift or fail silently again. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The task (and its premise) were based on a wrong diagnosis: the APK never shipped a stray raw from src/main/assets — AGP itself decompresses the committed kolibri_tree.jsonl.gz into kolibri_tree.jsonl at asset merge. Guarding the source dir never touched that. The real fix is in BundledTreeSource (read the AGP-produced name, gzip-aware). Removing the task also removes its misleading comment. The .gitignore of kolibri_tree.jsonl stays: it still keeps the generator's transient raw out of git. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the offline topic tree work from first launch, without a prior R2 pull. The tree is the
offline floor the ticket calls for, but ~16 MB raw is too much to bundle — so ship it gzipped
(~3.3 MB) and decompress on read.
BundledTreeSource now reads kolibri_tree.jsonl.gz — both the APK-bundled floor and the pulled
R2 overlay are gzip — via GZIPInputStream, streaming: the 16 MB is never written out, only the
in-memory index is kept. A truncated or corrupt .gz trips the gzip CRC and falls through to an
empty tree, same as a missing asset, so the live sources still answer.
Adds the committed floor asset src/main/assets/kolibri_tree.jsonl.gz (topics variant, 78,169
folders), and marks *.gz binary in .gitattributes so the repo's default "text=auto eol=lf"
never line-ending-mangles the gzip on checkout.
Pairs with the workflow change that publishes the .gz to R2 (application/gzip), so the periodic
refresh is a ~3 MB download, not ~16 MB. Merge that first so any overlay the app pulls is
already gzip.
Inert-safe: with no asset and no overlay the tree source is empty and the chain behaves as
before (Local → Studio).