Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Tapioca makes it easy to work with [Sorbet](https://sorbet.org) in your codebase
* [Writing custom DSL extensions](#writing-custom-dsl-extensions)
* [Rewriting RBS comments to Sorbet signatures](#rewriting-rbs-comments-to-sorbet-signatures)
* [Caching rewrites with Bootsnap](#caching-rewrites-with-bootsnap)
* [Priming the cache from CI](#priming-the-cache-from-ci)
* [Priming the cache](#priming-the-cache)
* [RBI files for missing constants and methods](#rbi-files-for-missing-constants-and-methods)
* [Configuration](#configuration)
* [Editor Integration](#editor-integration)
Expand Down Expand Up @@ -855,7 +855,16 @@ The rewriting is automatic on every `tapioca` invocation: [`require-hooks`](http
$ TAPIOCA_RBS_CACHE=1 bin/tapioca dsl
```

Tapioca configures Bootsnap's iseq cache against a dedicated directory (`tmp/cache/bootsnap-tapioca-rbs` by default; override with `TAPIOCA_BOOTSNAP_CACHE_DIR`). The first run is slower because every file is rewritten and the result is baked into the iseq cache; subsequent runs against the same directory skip the rewrite entirely.
Tapioca configures Bootsnap's iseq cache against a dedicated directory (`tmp/cache/bootsnap-tapioca-rbs` by
default; override with `TAPIOCA_BOOTSNAP_CACHE_DIR`).

Tapioca writes the current `Gemfile.lock` digest to `.gemfile-lock-digest` inside that cache directory. When the
lockfile changes, Tapioca sees the digest mismatch and resets Bootsnap's cache payload before configuring Bootsnap.
This lets gem bumps that affect rewriting, such as `tapioca`, start from a fresh cache without accumulating old cache
directories.

The first run is slower because every file is rewritten and the result is baked into the iseq cache; subsequent runs
against the same lockfile skip the rewrite entirely.

`Bootsnap.setup` mutates a process-wide singleton, and a second call would overwrite Tapioca's dedicated cache directory and start writing rewritten iseqs into the host's normal cache. Tapioca enforces this under `TAPIOCA_RBS_CACHE=1`: after its own setup runs, any subsequent `Bootsnap.setup` raises a clear error pointing at the fix. Gate your host's `Bootsnap.setup` on the same env var. Rails apps do this in `config/boot.rb`:

Expand All @@ -864,16 +873,17 @@ Tapioca configures Bootsnap's iseq cache against a dedicated directory (`tmp/cac
require "bootsnap/setup" unless ENV["TAPIOCA_RBS_CACHE"] == "1"
```

#### Priming the cache from CI
#### Priming the cache

For CI pipelines that want to populate the cache once and have downstream jobs read from a warm copy, use `--only-bootsnap-rbs-cache`. This pattern lets you scope cache writes to a single job (the prime) so PR-side jobs read from it without uploading on every successful build:
For workflows that want to populate the cache once and read from a warm copy later, use `--only-bootsnap-rbs-cache`.
This pattern lets you scope cache writes to a single prime step:

```shell
# Prime: populate the cache.
$ TAPIOCA_RBS_CACHE=1 bin/tapioca dsl --only-bootsnap-rbs-cache

# Consumer: read from the populated cache.
# BOOTSNAP_READONLY=1 prevents bootsnap from writing back to a read-only mount.
# BOOTSNAP_READONLY=1 prevents bootsnap from writing back to a read-only cache.
$ TAPIOCA_RBS_CACHE=1 BOOTSNAP_READONLY=1 bin/tapioca dsl
```

Expand Down
69 changes: 69 additions & 0 deletions lib/tapioca/rbs/bootsnap_cache.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# typed: strict
# frozen_string_literal: true

require "digest"
require "fileutils"

module Tapioca
module RBS
# Prepares the Bootsnap iseq cache used for RBS rewrite output.
#
# RBS rewrite output can change when the lockfile changes, even if the
# source files are unchanged.
# To account for this, we store the current Gemfile.lock SHA256 in a
# `.gemfile-lock-digest` file.
# On writable runs, a digest mismatch deletes Bootsnap's cache payload and
# records the new digest, so this run rebuilds the cache from scratch. On
# read-only runs, a digest mismatch means the cache is stale and must not be
# used.
module BootsnapCache
PrepareResult = Struct.new(:setup_bootsnap, keyword_init: true)

DIGEST_FILE = ".gemfile-lock-digest" #: String

class << self
extend T::Sig

sig { params(cache_dir: String, readonly: T::Boolean).returns(PrepareResult) }
def prepare_for_setup(cache_dir, readonly:)
digest = gemfile_lock_digest

if readonly
return PrepareResult.new(setup_bootsnap: digest_matches?(cache_dir, digest))
end

reset!(cache_dir) unless digest_matches?(cache_dir, digest)
FileUtils.mkdir_p(cache_dir)
File.write(digest_path(cache_dir), digest)
PrepareResult.new(setup_bootsnap: true)
end

private

sig { returns(String) }
def gemfile_lock_digest
lockfile = File.join(Dir.pwd, "Gemfile.lock")
return "missing" unless File.file?(lockfile)

Digest::SHA256.file(lockfile).hexdigest
end

sig { params(cache_dir: String, digest: String).returns(T::Boolean) }
def digest_matches?(cache_dir, digest)
File.file?(digest_path(cache_dir)) && File.read(digest_path(cache_dir)).chomp == digest
end

sig { params(cache_dir: String).void }
def reset!(cache_dir)
FileUtils.rm_rf(File.join(cache_dir, "bootsnap"))
FileUtils.rm_f(digest_path(cache_dir))
end

sig { params(cache_dir: String).returns(String) }
def digest_path(cache_dir)
File.join(cache_dir, DIGEST_FILE)
end
end
end
end
end
67 changes: 44 additions & 23 deletions lib/tapioca/rbs/rewriter.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# typed: strict
# frozen_string_literal: true

require "tapioca/rbs/bootsnap_cache"

# This code rewrites RBS comments back into Sorbet's signatures as the files are being loaded.
# This will allow `sorbet-runtime` to wrap the methods as if they were originally written with the `sig{}` blocks.
# This will in turn allow Tapioca to use this signatures to generate typed RBI files.
Expand All @@ -19,42 +21,61 @@ module BootsnapGuard
def setup(**_kwargs)
Kernel.raise HostBootsnapSetupError, <<~MSG
Bootsnap.setup was called while TAPIOCA_RBS_CACHE=1 is set. Tapioca already
configured bootsnap with a dedicated cache directory; re-running setup
would overwrite that config and start writing rewritten iseqs into your
host's cache.
configured bootsnap for RBS rewriting; re-running setup would overwrite
that config and start writing rewritten iseqs into your host's cache.

Gate your host's Bootsnap.setup on the env var, e.g. in config/boot.rb:

require "bootsnap/setup" unless ENV["TAPIOCA_RBS_CACHE"] == "1"
MSG
end
end

module BootsnapSetup
class << self
extend T::Sig

sig { void }
def setup
require "bootsnap"

# Respect BOOTSNAP_READONLY for consumers reading a pre-populated cache.
readonly = !["0", "false", false].include?(ENV.fetch("BOOTSNAP_READONLY") { false })
cache_dir = ENV.fetch("TAPIOCA_BOOTSNAP_CACHE_DIR", File.join(Dir.pwd, "tmp/cache/bootsnap-tapioca-rbs"))
# A read-only cache with a mismatched lockfile digest may contain stale rewritten iseqs,
# and this process cannot reset it.
return unless Tapioca::RBS::BootsnapCache.prepare_for_setup(
cache_dir,
readonly: readonly,
).setup_bootsnap

Bootsnap.setup(
cache_dir: cache_dir,
development_mode: true,
load_path_cache: true,
compile_cache_iseq: true,
compile_cache_yaml: true,
readonly: readonly,
revalidation: true,
)
Bootsnap.log_stats!
ensure
Bootsnap.singleton_class.prepend(Tapioca::RBS::BootsnapGuard) if defined?(Bootsnap)
end
end
end
end
end

# When TAPIOCA_RBS_CACHE=1, set up bootsnap with a dedicated cache directory
# and load require-hooks so the RBS-rewritten iseqs get cached. Subsequent
# runs read the rewritten iseq directly and skip the rewrite.
# When TAPIOCA_RBS_CACHE=1, use a dedicated Bootsnap cache directory for
# RBS-rewritten iseqs. Stale read-only caches are skipped because this process
# cannot reset them.
#
# After our setup, BootsnapGuard is prepended so the host application can't
# replace our cache directory.
# BootsnapGuard is prepended so the host application can't replace our cache
# configuration.
if ENV["TAPIOCA_RBS_CACHE"] == "1"
begin
require "bootsnap"
# Respect BOOTSNAP_READONLY for consumers reading a pre-populated cache
# (e.g. a CI prime step).
readonly = !["0", "false", false].include?(ENV.fetch("BOOTSNAP_READONLY") { false })
Bootsnap.setup(
cache_dir: ENV.fetch("TAPIOCA_BOOTSNAP_CACHE_DIR", File.join(Dir.pwd, "tmp/cache/bootsnap-tapioca-rbs")),
development_mode: true,
load_path_cache: true,
compile_cache_iseq: true,
compile_cache_yaml: true,
readonly: readonly,
revalidation: true,
)
Bootsnap.log_stats!
Bootsnap.singleton_class.prepend(Tapioca::RBS::BootsnapGuard)
Tapioca::RBS::BootsnapSetup.setup
rescue LoadError
# Bootsnap is not in the bundle, skip iseq caching.
end
Expand Down
57 changes: 57 additions & 0 deletions spec/tapioca/cli/dsl_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -680,6 +680,63 @@ class Post
assert_success_status(result)
end

it "resets the bootsnap cache when Gemfile.lock changes" do
@project.write!("lib/post.rb", <<~RB)
require "smart_properties"

class Post
include SmartProperties
property :title, accepts: String
end
RB

env = {
"TAPIOCA_RBS_CACHE" => "1",
"TAPIOCA_BOOTSNAP_CACHE_DIR" => "tmp/cache/test-bootsnap-tapioca-rbs",
}

result = @project.tapioca("dsl --only-bootsnap-rbs-cache Post", env: env)

assert_success_status(result)
@project.write!("tmp/cache/test-bootsnap-tapioca-rbs/bootsnap/stale-cache-entry", "stale")

@project.write!("Gemfile.lock", "#{@gemfile_lock}\n")
result = @project.tapioca("dsl --only-bootsnap-rbs-cache Post", env: env)

assert_success_status(result)
refute_project_file_exist("tmp/cache/test-bootsnap-tapioca-rbs/bootsnap/stale-cache-entry")
assert_project_file_exist("tmp/cache/test-bootsnap-tapioca-rbs/.gemfile-lock-digest")
end

it "skips a stale read-only bootsnap cache when Gemfile.lock changes" do
@project.write!("lib/post.rb", <<~RB)
require "smart_properties"

class Post
include SmartProperties
property :title, accepts: String
end
RB

env = {
"TAPIOCA_RBS_CACHE" => "1",
"TAPIOCA_BOOTSNAP_CACHE_DIR" => "tmp/cache/test-readonly-bootsnap-tapioca-rbs",
}

result = @project.tapioca("dsl --only-bootsnap-rbs-cache Post", env: env)

assert_success_status(result)
original_digest = @project.read("tmp/cache/test-readonly-bootsnap-tapioca-rbs/.gemfile-lock-digest")
@project.write!("Gemfile.lock", "#{@gemfile_lock}\n")

result = @project.tapioca("dsl Post", env: env.merge("BOOTSNAP_READONLY" => "1"))

assert_success_status(result)
assert_empty_stderr(result)
assert_project_file_exist("sorbet/rbi/dsl/post.rbi")
assert_project_file_equal("tmp/cache/test-readonly-bootsnap-tapioca-rbs/.gemfile-lock-digest", original_digest)
end

it "warns when --only-bootsnap-rbs-cache is set without TAPIOCA_RBS_CACHE=1" do
@project.write!("lib/post.rb", <<~RB)
require "smart_properties"
Expand Down
Loading