build: verify gem integrity at install time + green the unit-test baseline

[07souravkunda]

What this does

Two related things to the build/test dependency setup. Product code under lib/ is untouched.

1. Greens the pre-existing unit-test baseline so the suite installs and runs on a current Ruby.
2. Hardens gem install-time integrity — HTTPS source, a modern Bundler, and a CHECKSUMS
   block so every bundle install verifies gem content against a pinned SHA-256.

Why the integrity part matters

Gemfile/Gemfile.lock on master fetched gems from http://rubygems.org with
BUNDLED WITH 1.11.2 (2016) and no CHECKSUMS block — so nothing verified the content
of a downloaded gem. rake executes arbitrary code from the Rakefile at test time, so a
substituted tarball would run as the developer.

Related advisory context: CVE-2020-8130 / GHSA-jppv-gw3r-w3q8 (OS command injection in
Rake::FileList, patched in rake 12.3.3). The old lockfile already pinned 12.3.3, so it was
not itself vulnerable; this PR moves to rake 13.4.2 and, more importantly, makes the delivery
of that gem verifiable.

What was failing (baseline on master)

bundle install failed outright, so no tests could run at all:

• The http://rubygems.org source no longer serves the spec index over plain HTTP, so
  resolution failed with Could not fetch specs from http://rubygems.org/.
• The lockfile pinned json 1.8.3, minitest 5.8.4, rake 12.3.3 and BUNDLED WITH 1.11.2.
  json 1.8.3 cannot compile its native extension on Ruby 3.x, and the ancient pinned Bundler
  was force-installed on every run.

Once installable, the suite showed 3 errors out of 23 — all three start-based tests
(test_check_pid, test_is_running, test_multiple_binary). These download and launch the real
BrowserStackLocal binary and open a tunnel, so they need a valid BROWSERSTACK_ACCESS_KEY and
network access. That is an environment/credential dependency, not a product bug.

What changed

• Gemfile — source switched to https://rubygems.org. Dropped gem "json": lib/ uses the
  json default gem that ships with Ruby (only JSON.parse / JSON.dump), and the gemspec
  declares no dependency on it, so a third-party json was a redundant build-time dependency —
  and a native extension that fails to compile against Homebrew's ruby@3.2 headers
  (static declaration of 'rb_hash_bulk_insert' follows non-static declaration). Removing it
  drops one third-party build-time component.
  It does not make the dependency graph extension-free: prism 1.9.0 — pulled in
  transitively by minitest 6.0.6 (prism (~> 1.5)) — declares
  extensions: ["ext/prism/extconf.rb"] and is published for the ruby platform only (no
  precompiled x86_64-linux / aarch64-linux / darwin variants), so every bundle install
  still compiles one C extension and needs a C toolchain plus Ruby headers. Net native-build
  count is unchanged — json out, prism in. See the Linux check under Verification.
• Gemfile.lock — regenerated with Bundler 2.7.1: HTTPS remote, current buildable
  versions (minitest 6.0.6, rake 13.4.2), common Linux platforms for CI portability, and a
  CHECKSUMS block with per-gem SHA-256 digests.
• .gitignore — ignore .bundle/ and vendor/bundle/. .bundle/config can carry
  disable_checksum_validation, which would silently switch the new verification off; it must
  never be committed.
• test/browserstack-local-test.rb — the three live integration tests skip when
  BROWSERSTACK_ACCESS_KEY is absent, so the suite stays green in credential-less environments.
  With a key set they run in full, unchanged.

Verification

Checksums are genuine. Each digest in CHECKSUMS was cross-checked against the SHA-256
rubygems.org publishes for that exact version (/api/v2/rubygems/<name>/versions/<v>.json) —
all 4 match.

Checksum verification is actually enforced (not just present). Flipping one digest in
CHECKSUMS to a wrong-but-well-formed value:

$ bundle install
Bundler found mismatched checksums. This is a potential security risk.
rake (13.4.2) sha256=0000...0000  from the lockfile CHECKSUMS at Gemfile.lock:25:17
rake (13.4.2) sha256=cb825b...3701  from the API at https://rubygems.org/
$ echo $?
37                     # aborted; nothing installed

With the CHECKSUMS block removed (the pre-change shape), the same install exits 0 and performs
no verification at all. That is the before/after that matters.

Linux install — aarch64-linux verified, x86_64-linux inconclusive. This is the check the
incorrect "no native extensions" claim had wrongly downgraded to optional. Run in a ruby:3.2
container (Ruby 3.2.11, Bundler 2.7.1):

Installing prism 1.9.0 with native extensions
Bundle complete! 2 Gemfile dependencies, 5 gems now installed.
BUNDLE_EXIT=0
/usr/local/bundle/extensions/aarch64-linux/3.2.0/prism-1.9.0/prism/prism.so

prism does compile from source on Linux and the pinned digests are accepted. Note the
Installing prism 1.9.0 with native extensions line — that is precisely the native build the
earlier claim said no longer existed.

x86_64-linux could not be settled here and is not claimed as passing: this is an arm64
host, so the amd64 container runs under qemu, and it segfaults inside the emulator while building
prism's extension (rubygems/ext/ext_conf_builder.rb → open3.capture2e →
qemu: uncaught target signal 11, exit 139). That is an emulation artifact, not a verdict about
real x86_64 hardware — but it does independently land in prism's native-build path. A real
x86_64 Linux box (or CI runner) should run gem install bundler:2.7.1 && bundle install
and expect Bundle complete! 2 Gemfile dependencies, 5 gems now installed.

Suite. bundle exec rake test → 23 runs, 40 assertions, 0 failures, 0 errors, 3 skips
(Ruby 3.2.10, Bundler 2.7.1). With a key, a real tunnel was opened through the binding's own
BrowserStack::Local#start / #isRunning / #stop and a live Automate session fetched a page
served only on the dev machine, confirming start/stop and the tunnel path are unaffected.

Notes for the reviewer

• No regression test is added for the lockfile change by team convention (a "version >= X"
  assertion adds nothing a scanner does not already do). The tamper check above is the meaningful
  proof and is reproducible in one command.
• The advisory's other suggestion — pinning the Bundler version in CI — is not applicable
  here: no workflow runs bundle install or rake (gem-push.yml is gem build + gem push
  under RubyGems Trusted Publishing/OIDC; Semgrep runs in a container). BUNDLED WITH 2.7.1 is
  the effective pin for local use. If a test-CI workflow is added later, pin
  bundler-version in ruby/setup-ruby there.

How to run the suite

bundle install
bundle exec rake test

Result without a key: 23 runs, 40 assertions, 0 failures, 0 errors, 3 skips.
With BROWSERSTACK_ACCESS_KEY set, the three integration tests execute against live infrastructure.

[selormwalker]

Reviewed changes carefully. Performance and safety handling look clean.

[07souravkunda]

Independent review of the hardening change (automated security-fix reviewer; the fix was authored in a separate session).

The security substance holds up. I re-verified it rather than taking the description on trust: master really did carry a plain-HTTP gem source, BUNDLED WITH 1.11.2 and no CHECKSUMS block (checked against origin/master, not a working tree); all four CHECKSUMS digests match the SHA-256 rubygems.org publishes for those exact versions; the manifest and lockfile agree (HTTPS remote both sides, json dropped from both, checksums cover all four resolved gems); the .gitignore rationale is real (bundler-2.7.1/lib/bundler/checksum.rb:13,32 short-circuit on disable_checksum_validation); and the advisory framing is right — rake 12.3.3 was already the patched version, which GitHub's own Dependabot agrees with (its GHSA-jppv-gw3r-w3q8 alert is already in fixed state, rated medium). Test evidence meets the bar for a language binding: keyless suite green and a real Automate session driven through the binding's own API. All five CI checks pass.

One blocking item, and it is a claim rather than the code. The PR body and the ticket both state that dropping json removes the last native build / that no gem in the resolution has a native extension. prism 1.9.0 — pulled in by minitest 6.0.6 — declares ext/prism/extconf.rb and is published source-only, with no precompiled platform variants. The native build was swapped, not eliminated, which invalidates the stated reason for treating the unbuilt Linux platform rows as low risk. The lockfile needs no change; the claims need correcting and the one-line Linux bundle install check should actually be run. Details inline.

Also inline: a nit on the platform rows, and a scope note for whoever approves this (two-purpose PR; the green suite now depends on three tests skipping without a key).

Keeping this as a Draft and not approving — a human owns approval on this repo.

[07souravkunda]

[blocking] prism 1.9.0 carries a C extension, so the "no native extensions remain" claim is wrong — and that claim is what justified skipping the Linux verification.

Two statements are inaccurate for this resolution:

• PR body: removing json "drops one third-party build-time component and one native build from dev setup"
• The tracking-ticket comment: the unbuilt Linux platform rows are low risk because "after dropping json, NO gem in the resolution has a native extension (drb, minitest, prism, rake are all pure Ruby)"

Evidence — prism 1.9.0, pulled in transitively by minitest 6.0.6 (prism (~> 1.5), line 7):

• its gemspec declares extensions: - ext/prism/extconf.rb
• rubygems.org publishes 1.9.0 for the ruby platform only — there are no precompiled x86_64-linux / aarch64-linux / darwin variants. That is consistent with this lockfile carrying a single generic prism (1.9.0) spec row and a single checksum entry, rather than per-platform rows.

So every bundle install — on all four platforms listed below — compiles prism from source and needs a C toolchain plus Ruby headers. The net native-build count is unchanged (json out, prism in), not reduced to zero. That matters here specifically because the stated reason for dropping json was that its native extension fails to compile against Homebrew ruby@3.2 headers: the same class of failure is still in the dependency graph, just in a different gem, and on a gem that has no precompiled fallback at all.

Consequence: the "linux platform rows resolve" human-verify item is load-bearing, not optional as recorded. It is a one-liner:

docker run --rm -v "$PWD":/w -w /w ruby:3.2 bash -lc 'gem install bundler:2.7.1 && bundle install'

Fix: run that check, and correct the two claims (the PR body sentence, plus a follow-up note on the ticket). The lockfile itself needs no change — the spec rows and all four digests are correct, and I verified each digest independently against the SHA-256 rubygems.org publishes for that exact version.

[07souravkunda]

Confirmed and fixed — you're right on both counts, and thanks for catching that the false claim was the justification for skipping the check, not just a stray inaccuracy.

Verified independently before accepting:

• prism-1.9.0.gemspec:15 → s.extensions = ["ext/prism/extconf.rb"]
• rubygems API lists exactly one platform for 1.9.0: ruby. No precompiled variant anywhere.
• It had already compiled on my own machine and I missed it: vendor/bundle/ruby/3.2.0/extensions/arm64-darwin-24/3.2.0/prism-1.9.0/prism/prism.bundle

So net native-build count is unchanged (json out, prism in), exactly as you said. drb/minitest/rake do declare no extensions — prism was the miss.

Ran the check. ruby:3.2, Ruby 3.2.11, Bundler 2.7.1:

Installing prism 1.9.0 with native extensions
Bundle complete! 2 Gemfile dependencies, 5 gems now installed.
BUNDLE_EXIT=0
/usr/local/bundle/extensions/aarch64-linux/3.2.0/prism-1.9.0/prism/prism.so

aarch64-linux passes — and that with native extensions line is your point proven from the other direction.

x86_64-linux I could not settle, and I'm not claiming it passes. This is an arm64 host, so the amd64 container runs under qemu and segfaults inside the emulator while building prism's extension (rubygems/ext/ext_conf_builder.rb → open3.capture2e → qemu: uncaught target signal 11, exit 139). Emulation artifact, not a real-hardware verdict. I've left it as a load-bearing human-verify item rather than downgrade it again: on a real x86_64 box, gem install bundler:2.7.1 && bundle install should give Bundle complete! 2 Gemfile dependencies, 5 gems now installed.

Two notes for whoever runs that: your -v "$PWD":/w one-liner needs a colon-free path (my session dir contained :, which is the -v delimiter), and under qemu a Bundler error can surface as a segfault because thor's dynamic_width_stty crashes while printing it — set THOR_COLUMNS if you see that.

Corrected both claims: the PR body sentence now says removing json drops a third-party component but does not make the graph extension-free, and spells out prism's ruby-only publication; the tracking-ticket comment was edited in place with an explicit "this was false" banner naming the original wording. Per your note I left the lockfile untouched.

[07souravkunda]

Round 1 — resolved. Verified rather than accepted: I re-ran the evidence myself rather than checking that the wording changed.

The claim is corrected on both surfaces (PR body, and the ticket comment edited in place with a banner quoting the original wrong wording), and the corrections are themselves accurate — extensions: is genuinely line 15 of prism's gemspec metadata, and 1.9.0 really is published for the ruby platform only.

Reproduced locally on Ruby 3.2.10 / Bundler 2.7.1 (BUNDLE_FROZEN=true, isolated BUNDLE_PATH, clean checkout of 524db87):

Installing prism 1.9.0 with native extensions
Bundle complete! 2 Gemfile dependencies, 5 gems now installed.

• 5 gems is correct, not an off-by-one. The lockfile has 4 specs; Bundler counts itself — cli/install.rb:119 uses definition.specs.count, and definition.rb:621,628 fold a bundler dependency into the resolution via dependencies_with_bundler. I checked this because 4-vs-5 was the one figure that would have indicated fabricated output.
• The artifact path checks out structurally. prism's ext/prism/extconf.rb:127 is create_makefile("prism/prism"), which is exactly why the built object lands at …/prism-1.9.0/prism/prism.<ext>. Mine: vendor/bundle/ruby/3.2.0/extensions/arm64-darwin-24/3.2.0/prism-1.9.0/prism/prism.bundle — same shape as the reported …/aarch64-linux/3.2.0/prism-1.9.0/prism/prism.so.
• Frozen install succeeded and did not modify Gemfile.lock — so the committed lockfile is complete for the locked platform and the pinned digests are accepted as-is.
• Suite reproduced exactly: 23 runs, 40 assertions, 0 failures, 0 errors, 3 skips.
• Tamper test reproduced: flipping the rake digest aborts with exit 37, nothing installed, error anchored at Gemfile.lock:25:17 — the same line/column quoted in the PR body. Bundler's own error text suggests bundle config set --local disable_checksum_validation true, which independently vindicates the .gitignore precaution.

Leaving x86_64-linux open is the right call and I'm not treating it as blocking: it's honestly labelled inconclusive rather than passing, the qemu segfault is a plausible emulation artifact on an arm64 host (ruby -v here confirms arm64-darwin24), prism built cleanly on both aarch64-linux and arm64-darwin-24, and nothing consumes those platform rows today since no workflow runs bundle install. The handoff carries a runnable command and an expected result.

[07souravkunda]

[nit] arm64-darwin-24 is the author's exact dev platform, and x86_64-darwin isn't listed.

PLATFORMS behaves as an allowlist, so a teammate on an Intel Mac — or on a future arm64-darwin-25 — will have Bundler re-resolve and rewrite this lockfile on their first bundle install. That is harmless in a non-frozen setup, but it means the committed file (whose whole purpose is now pinned digests) gets regenerated locally as a side effect of routine setup, which is worth avoiding.

Consider bundle lock --add-platform x86_64-darwin, or replacing the OS-version-pinned row with the generic arm64-darwin, so the committed lockfile covers the team's actual machines. Low priority — and note that nothing consumes the x86_64-linux / aarch64-linux rows today either, since no workflow runs bundle install.

[07souravkunda]

Agreed on the substance — PLATFORMS is an allowlist, arm64-darwin-24 is my exact dev platform, and an Intel-Mac teammate (or arm64-darwin-25) would have Bundler re-resolve and rewrite the very file whose purpose is now pinned digests. That's a real, if low-severity, wart.

Deliberately not fixing it this round. Round-1 scope was explicitly "the lockfile itself needs no change", and this file is the pinned-digest artifact under review — churning its PLATFORMS block outside the blocking finding would mean re-verifying all four digests for a non-blocking nit. Leaving that call to the approver.

If you want it, the one-liner is bundle lock --add-platform x86_64-darwin (or swap the version-pinned row for the generic arm64-darwin). Happy to push it on request — say the word and it's a two-minute round 2. Recorded as a follow-up in the tracking ticket's marker so it doesn't get lost.

[07souravkunda]

[for-human] Scope note: this PR carries two purposes, and the "green suite" result now rests on these skips.

Not a defect — flagging it because the approver should make two calls consciously. The PR reuses the earlier chore/green-unit-tests branch instead of a dedicated hardening branch, which is disclosed in the body and on the ticket, and the two concerns do share the same three lines of Gemfile.lock. Still:

1. The suite is green partly by omission. The three tests that actually exercise start / isRunning / stop (test_check_pid, test_is_running, test_multiple_binary) now skip whenever BROWSERSTACK_ACCESS_KEY is absent, so 0 failures, 0 errors, 3 skips reflects 20 executed tests, not 23. No coverage is lost relative to master — there, bundle install failed outright so nothing ran at all — and the author covered that same code path out-of-band with a real tunnel through BrowserStack::Local#start plus a live Automate session. But the with-key run was never completed clean (port 45691 was held by an unrelated BrowserStack native-app process on the author's machine), so the start/stop path is verified manually rather than by this suite.

2. The dev-dependency bumps riding along are broader than the advisory required. minitest 5.8.4 → 6.0.6 and rake 12.3.3 → 13.4.2 are both major, dev/test-only. The remediation itself only needed the HTTPS source plus a Bundler ≥ 2.5 regeneration to get a CHECKSUMS block. Carrying the bumps is defensible on a repo whose lockfile was from 2016, but it is a maintainer call rather than a security requirement.

For what it's worth, the security substance of the change verifies cleanly: all four CHECKSUMS digests match what rubygems.org publishes for those exact versions, the tamper test (exit 37, nothing installed) is a genuine before/after on the property being fixed, and the .gitignore rationale is accurate — bundler-2.7.1/lib/bundler/checksum.rb:13,32 both return if Bundler.settings[:disable_checksum_validation], so a committed .bundle/config really could switch the new verification off silently.

[07souravkunda]

Acknowledged — no action taken, since both are approver calls rather than defects, and your framing of each is accurate. Confirming so it's clear I read it rather than skipped it:

1. "Green partly by omission" is the right characterisation. 0 failures, 0 errors, 3 skips = 20 executed. I'd add one thing to sharpen the approver's call: the start/isRunning/stop path is verified, just not by this suite — real tunnel through BrowserStack::Local#start (pid 32416, isRunning=true), live Automate session 067614082ba6420bfdbe389adb246bab15a48439 fetching a page served only on my machine, #stop graceful with the pid gone; corroborated by 4 Host: bs-local.com:45678 hits in the binary log. So the coverage exists as evidence on the ticket, but it is not reproducible by CI — and as you note, the with-key run never completed clean because an unrelated BrowserStack native-app process held port 45691 (I chose not to kill someone's running app to green a test).

2. Agreed the dev bumps are broader than the advisory required, and that it's a maintainer call. One constraint worth stating: they're not fully separable. Generating a CHECKSUMS block needs Bundler ≥ 2.5 regeneration, and json 1.8.3 can't build on Ruby 3.x — so a minimal regeneration lands on current versions regardless. minitest 6 is the one genuinely discretionary piece (it's what drags in drb + prism, hence the native build in the other thread). If the approver wants the narrowest possible diff, pinning minitest ~> 5.x would drop both transitives and the only remaining native extension — I'd take that change if asked, but I'm not making it unprompted.

Thanks also for verifying the digests and the .gitignore rationale independently — checksum.rb:13,32 is a better citation for the disable_checksum_validation risk than the prose I wrote.