Skip to content

fix: daily snapshots fail every night on fresh-schema installs (+ self-contained tests, master.key gitignore)#4

Open
FrancoMuzzio wants to merge 3 commits into
kmanus88:mainfrom
FrancoMuzzio:fix/fresh-schema-snapshots-and-test-env
Open

fix: daily snapshots fail every night on fresh-schema installs (+ self-contained tests, master.key gitignore)#4
FrancoMuzzio wants to merge 3 commits into
kmanus88:mainfrom
FrancoMuzzio:fix/fresh-schema-snapshots-and-test-env

Conversation

@FrancoMuzzio

Copy link
Copy Markdown

Summary

Three fixes found while self-hosting the bot (Docker Compose, fresh install from 2026-06-03):

1. fix(db): daily snapshots fail every night on fresh-schema installs

createDailySnapshot's INSERT names legacy columns (timestamp, balance_usdt, equity_usdt, *_usdt mirrors) that only exist on pre-monorepo DBs migrated via ALTER TABLE. Fresh installs create daily_snapshots with the new schema only, so the nightly snapshot job dies with:

SQLITE_ERROR: table daily_snapshots has no column named timestamp

and the equity curve / portfolio history stays empty forever. Observed live on my install: 0 snapshot rows after 8 days, the error fires at every midnight UTC. This presumably affects every grvtbot.com user whose DB was created with the current schema.

Fix: probe the table once per process (PRAGMA table_info) and write the column set it actually has — new-schema-only on fresh installs, the existing dual write on legacy DBs whose old columns are NOT NULL. Regression tests cover both schema generations end to end (insert + readback), not just column presence.

2. test(bot): make the suite self-contained — no .env required

Six test files fail to even load on a machine without a populated .env: client.ts instantiates GRVTClient at import time, but tests/setup.ts only set the env vars inside beforeAll, which runs after the test file's imports resolve. Moved them to module level (vitest evaluates setup files before importing test files) and added JWT_SECRET — without it signToken() throws and the three H-5 ADMIN_EMAIL signup tests answer 500 instead of 200.

npm test now passes from a clean clone: 210 tests across the three packages.

3. security: gitignore data/master.key

The .gitignore covers every DB artifact under data/ but not the AES-256-GCM master key, which docker-compose mounts from ./data/ instead of /etc/grvt-grid/. On Docker setups it shows up as untracked — one careless git add -A away from publishing the key that decrypts every user's GRVT credentials.

Test plan

  • Full suite green in a clean Node 22 container with no .env and no env vars: 126 (bot) + 45 (dashboard) + 39 (notifier)
  • npm run typecheck passes
  • New regression test reproduces the snapshot failure on the old code (fails with the exact production error) and passes with the fix

🤖 Generated with Claude Code

FrancoMuzzio and others added 3 commits June 11, 2026 04:34
createDailySnapshot's INSERT named legacy columns (timestamp,
balance_usdt, equity_usdt, *_usdt mirrors) that only exist on
pre-monorepo DBs migrated via ALTER TABLE. Fresh installs create
daily_snapshots with the new schema only, so the nightly snapshot job
died with "SQLITE_ERROR: table daily_snapshots has no column named
timestamp" and the equity curve / portfolio history stayed empty
forever. Observed live on a 2026-06-03 install (0 rows after 8 days).

Probe the table once per process (PRAGMA table_info) and write the
column set it actually has: new-schema-only on fresh installs, the
existing dual write on legacy DBs whose old columns are NOT NULL.

Regression tests cover both schema generations end to end (insert +
readback), not just column presence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Six test files failed to even load without a populated .env:
client.ts instantiates GRVTClient at import time and throws when
GRVT_TRADING_ACCOUNT_ID is missing, but setup.ts only set the env vars
inside beforeAll, which runs after the test file's imports resolve.
Move them to module level in the setup file, which vitest evaluates
before importing any test file.

Also set JWT_SECRET: signToken() throws when it is missing or shorter
than 32 chars, so the three H-5 ADMIN_EMAIL signup tests answered 500
instead of 200 on any machine without the developer's .env.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… tree

The .gitignore covered every DB artifact under data/ but not the
AES-256-GCM master key, which docker-compose mounts from ./data/
instead of /etc/grvt-grid/. It showed up as untracked, one careless
`git add -A` away from publishing the key that decrypts every user's
GRVT credentials.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant