Skip to content

Process inline RBS comments natively without the require-hook rewriter #2639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
paracycle wants to merge 11 commits into
base: main
Choose a base branch
from
Draft

Process inline RBS comments natively without the require-hook rewriter #2639

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
e6407d9
Process inline RBS comments natively without the require-hook rewriter
paracycle May 28, 2026
9d4d3d8
Refresh the DSL Rubydex graph when new source files get loaded
paracycle May 28, 2026
2452b61
Use Rubydex's lexical_nesting API and index gem RBI stubs
paracycle May 28, 2026
074de6b
Point Rubydex at main now that the lexical-nesting branch is merged
paracycle May 28, 2026
1896741
Drop runtime T::Generic boilerplate from Dsl::Compiler
paracycle May 28, 2026
4fb2ba4
Wrap method signatures in a polymorphic Tapioca::Runtime::Signature
paracycle May 28, 2026
5216bd9
Wrap inline RBS comments in a Tapioca::Runtime::RbsSignature
paracycle May 28, 2026
958ede2
Unify gem and DSL RBS lookups behind Tapioca::Runtime::Signature
paracycle May 28, 2026
1e332ce
Require Rubydex 0.2.5 or later
paracycle May 29, 2026
4bddb05
Update README.md
paracycle May 29, 2026
437081a
Regenerate Rubydex RBI for 0.2.5
paracycle May 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 11 additions & 13 deletions Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ PATH
netrc (>= 0.11.0)
parallel (>= 1.21.0)
rbi (>= 0.3.7)
require-hooks (>= 0.2.2)
rubydex (>= 0.1.0.beta10)
rubydex (>= 0.2.5)
sorbet-static-and-runtime (>= 0.6.12698)
spoom (>= 1.7.16)
thor (>= 1.2.0)
Expand Down Expand Up @@ -325,7 +324,6 @@ GEM
regexp_parser (2.11.3)
reline (0.6.3)
io-console (~> 0.5)
require-hooks (0.4.0)
rexml (3.4.4)
rubocop (1.84.1)
json (~> 2.3)
Expand Down Expand Up @@ -356,10 +354,11 @@ GEM
ruby-lsp-rails (0.4.8)
ruby-lsp (>= 0.26.0, < 0.27.0)
ruby-progressbar (1.13.0)
rubydex (0.2.3-aarch64-linux)
rubydex (0.2.3-arm64-darwin)
rubydex (0.2.3-x86_64-darwin)
rubydex (0.2.3-x86_64-linux)
rubydex (0.2.5)
rubydex (0.2.5-aarch64-linux)
rubydex (0.2.5-arm64-darwin)
rubydex (0.2.5-x86_64-darwin)
rubydex (0.2.5-x86_64-linux)
securerandom (0.4.1)
shopify-money (4.1.1)
bigdecimal (>= 3.0)
Expand Down Expand Up @@ -502,7 +501,6 @@ CHECKSUMS
benchmark (0.5.0) sha256=465df122341aedcb81a2a24b4d3bd19b6c67c1530713fd533f3ff034e419236c
bigdecimal (4.1.2) sha256=53d217666027eab4280346fba98e7d5b66baaae1b9c3c1c0ffe89d48188a3fbd
builder (3.3.0) sha256=497918d2f9dca528fdca4b88d84e4ef4387256d984b8154e9d5d3fe5a9c8835f
bundler (4.0.12) sha256=7f8b757d28dfb636e7b24fba2344ac6dd13b5b24f4b46d62573d483f211825ac
cityhash (0.9.0) sha256=1c20843d286524de21d0ecf5d43c7e7f18f5fb0c5866294a717f0be13dc1962d
concurrent-ruby (1.3.6) sha256=6b56837e1e7e5292f9864f34b69c5a2cbc75c0cf5338f1ce9903d10fa762d5ab
config (5.6.1) sha256=a9f0f0f9ffa6d12d43147a3fa1ab8486fe484c3098a350c6a2e0f32430e0d1cc
Expand Down Expand Up @@ -589,7 +587,6 @@ CHECKSUMS
redis-client (0.29.0) sha256=0c65bf1f8f6dca22063ddb085c0bb2054feef6f03a84869f4161b18a9a15bea3
regexp_parser (2.11.3) sha256=ca13f381a173b7a93450e53459075c9b76a10433caadcb2f1180f2c741fc55a4
reline (0.6.3) sha256=1198b04973565b36ec0f11542ab3f5cfeeec34823f4e54cebde90968092b1835
require-hooks (0.4.0) sha256=005f4c6435b4edae73e358cdbaba48370a4121f9ce893d5d2a3c66fce855677d
rexml (3.4.4) sha256=19e0a2c3425dfbf2d4fc1189747bdb2f849b6c5e74180401b15734bc97b5d142
rubocop (1.84.1) sha256=14cc626f355141f5a2ef53c10a68d66b13bb30639b26370a76559096cc6bcc1a
rubocop-ast (1.49.0) sha256=49c3676d3123a0923d333e20c6c2dbaaae2d2287b475273fddee0c61da9f71fd
Expand All @@ -599,10 +596,11 @@ CHECKSUMS
ruby-lsp (0.26.9) sha256=33a01c001c00a76b4e821efc04ed7572983430f31ca5d6f3e343d0b6ccab4129
ruby-lsp-rails (0.4.8) sha256=f09d1f926d4063deeb2f3049311925c20dfe6c912371e3bcd04a265a865c44ae
ruby-progressbar (1.13.0) sha256=80fc9c47a9b640d6834e0dc7b3c94c9df37f08cb072b7761e4a71e22cff29b33
rubydex (0.2.3-aarch64-linux) sha256=f666ff383430cc800cb0889d52c77da7457e99165b5eef7c0d45491a5fafea87
rubydex (0.2.3-arm64-darwin) sha256=997d7895a0208ec3d7ef922c9d29243b62e10c67bc3c2396a9f978d7df117390
rubydex (0.2.3-x86_64-darwin) sha256=de4890f91bedb59bfefb90c528939e52475da3afa105feaf855594ad527b0fb9
rubydex (0.2.3-x86_64-linux) sha256=796a54c1af9f8868c87bf92fee5f9ccc39ffd40eb8386e0875058a18beb76b09
rubydex (0.2.5) sha256=0a2112d62603f6f0ae4dfa9b963cf7617331fffae35740f7856a831cbc7f1580
rubydex (0.2.5-aarch64-linux) sha256=7cbc1cd8f926bd3138c7cdd5434f5d3140a287394ca7c4f4e9d93b4b900f20ce
rubydex (0.2.5-arm64-darwin) sha256=487cab3df687ccb2f3360c23236300320d402b1042643054cd4d09221e13915a
rubydex (0.2.5-x86_64-darwin) sha256=1888750fd1369353af67e5fe0d3f1b7fc0fbb864f99be2660dddfc729395a55e
rubydex (0.2.5-x86_64-linux) sha256=4b05f0424b867071c41a1b2ac7d91e965829a422c806a532d43ffdb45ec33604
securerandom (0.4.1) sha256=cc5193d414a4341b6e225f0cb4446aceca8e50d5e1888743fac16987638ea0b1
shopify-money (4.1.1) sha256=523078e44bfde1920f8b3487ddf9144e0fb6af8cdf67e212bed02025c5c5f423
sidekiq (8.1.5) sha256=19821ff6031100c2317f72a5b8ab32304ca84f5acb5a2ef846ed1ec14144ab02
Expand Down
97 changes: 30 additions & 67 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,7 @@ Tapioca makes it easy to work with [Sorbet](https://sorbet.org) in your codebase
* [Using DSL compiler options](#using-dsl-compiler-options)
* [Writing custom DSL compilers](#writing-custom-dsl-compilers)
* [Writing custom DSL extensions](#writing-custom-dsl-extensions)
* [Rewriting RBS comments to Sorbet signatures](#rewriting-rbs-comments-to-sorbet-signatures)
* [Caching rewrites with Bootsnap](#caching-rewrites-with-bootsnap)
* [Priming the cache from CI](#priming-the-cache-from-ci)
* [Inline RBS comments](#inline-rbs-comments)
* [RBI files for missing constants and methods](#rbi-files-for-missing-constants-and-methods)
* [Configuration](#configuration)
* [Editor Integration](#editor-integration)
Expand Down Expand Up @@ -492,35 +490,33 @@ Usage:
tapioca dsl [constant...]

Options:
--out, -o, [--outdir=directory] # The output directory for generated DSL RBI files
# Default: sorbet/rbi/dsl
[--file-header], [--no-file-header], [--skip-file-header] # Add a "This file is generated" header on top of each generated RBI file
# Default: true
[--only=compiler [compiler ...]] # Only run supplied DSL compiler(s)
[--exclude=compiler [compiler ...]] # Exclude supplied DSL compiler(s)
[--verify], [--no-verify], [--skip-verify] # Verifies RBIs are up-to-date
# Default: false
[--only-bootsnap-rbs-cache], [--no-only-bootsnap-rbs-cache], [--skip-only-bootsnap-rbs-cache] # Only boot the application and load DSL extensions/compilers to populate the bootsnap iseq cache, then exit. Skips compiler execution and RBI generation. Mutually exclusive with --verify and --list-compilers.
# Default: false
-q, [--quiet], [--no-quiet], [--skip-quiet] # Suppresses file creation output
# Default: false
-w, [--workers=N] # Number of parallel workers to use when generating RBIs (default: auto)
[--rbi-max-line-length=N] # Set the max line length of generated RBIs. Signatures longer than the max line length will be wrapped
# Default: 120
-e, [--environment=ENVIRONMENT] # The Rack/Rails environment to use when generating RBIs
# Default: development
-l, [--list-compilers], [--no-list-compilers], [--skip-list-compilers] # List all loaded compilers
# Default: false
[--app-root=APP_ROOT] # The path to the Rails application
# Default: .
[--halt-upon-load-error], [--no-halt-upon-load-error], [--skip-halt-upon-load-error] # Halt upon a load error while loading the Rails application
# Default: true
[--skip-constant=constant [constant ...]] # Do not generate RBI definitions for the given application constant(s)
[--compiler-options=key:value] # Options to pass to the DSL compilers
-c, [--config=<config file path>] # Path to the Tapioca configuration file
# Default: sorbet/tapioca/config.yml
-V, [--verbose], [--no-verbose], [--skip-verbose] # Verbose output for debugging purposes
# Default: false
--out, -o, [--outdir=directory] # The output directory for generated DSL RBI files
# Default: sorbet/rbi/dsl
[--file-header], [--no-file-header], [--skip-file-header] # Add a "This file is generated" header on top of each generated RBI file
# Default: true
[--only=compiler [compiler ...]] # Only run supplied DSL compiler(s)
[--exclude=compiler [compiler ...]] # Exclude supplied DSL compiler(s)
[--verify], [--no-verify], [--skip-verify] # Verifies RBIs are up-to-date
# Default: false
-q, [--quiet], [--no-quiet], [--skip-quiet] # Suppresses file creation output
# Default: false
-w, [--workers=N] # Number of parallel workers to use when generating RBIs (default: auto)
[--rbi-max-line-length=N] # Set the max line length of generated RBIs. Signatures longer than the max line length will be wrapped
# Default: 120
-e, [--environment=ENVIRONMENT] # The Rack/Rails environment to use when generating RBIs
# Default: development
-l, [--list-compilers], [--no-list-compilers], [--skip-list-compilers] # List all loaded compilers
# Default: false
[--app-root=APP_ROOT] # The path to the Rails application
# Default: .
[--halt-upon-load-error], [--no-halt-upon-load-error], [--skip-halt-upon-load-error] # Halt upon a load error while loading the Rails application
# Default: true
[--skip-constant=constant [constant ...]] # Do not generate RBI definitions for the given application constant(s)
[--compiler-options=key:value] # Options to pass to the DSL compilers
-c, [--config=<config file path>] # Path to the Tapioca configuration file
# Default: sorbet/tapioca/config.yml
-V, [--verbose], [--no-verbose], [--skip-verbose] # Verbose output for debugging purposes
# Default: false

Generate RBIs for dynamic methods
```
Expand Down Expand Up @@ -841,41 +837,9 @@ In order for DSL extensions to be discovered by Tapioca, they either needs to be

For more concrete and advanced examples, take a look at [Tapioca's default DSL extensions](https://github.com/Shopify/tapioca/tree/main/lib/tapioca/dsl/extensions).

### Rewriting RBS comments to Sorbet signatures
### Inline RBS comments

Tapioca translates [RBS comments](https://sorbet.org/docs/rbs-comments) into Sorbet `sig {}` blocks at file load time, so `sorbet-runtime` wraps the methods as if they had been written with native sigs. This is what lets the DSL command introspect signatures that were originally documented as RBS comments.

The rewriting is automatic on every `tapioca` invocation: [`require-hooks`](https://github.com/Shopify/require-hooks) intercepts `.rb` loads and `Spoom::Sorbet::Translate.rbs_comments_to_sorbet_sigs` translates the source before Ruby compiles it to bytecode.

#### Caching rewrites with Bootsnap

`tapioca dsl` boots the app and eager-loads source files for introspection, so the rewrite runs across the whole codebase. On large applications this adds noticeable overhead. To cache the rewrite output across runs using [bootsnap](https://github.com/Shopify/bootsnap)'s iseq cache, you can set `TAPIOCA_RBS_CACHE=1`:

```shell
$ TAPIOCA_RBS_CACHE=1 bin/tapioca dsl
```

Tapioca configures Bootsnap's iseq cache against a dedicated directory (`tmp/cache/bootsnap-tapioca-rbs` by default; override with `TAPIOCA_BOOTSNAP_CACHE_DIR`). The first run is slower because every file is rewritten and the result is baked into the iseq cache; subsequent runs against the same directory skip the rewrite entirely.

`Bootsnap.setup` mutates a process-wide singleton, and a second call would overwrite Tapioca's dedicated cache directory and start writing rewritten iseqs into the host's normal cache. Tapioca enforces this under `TAPIOCA_RBS_CACHE=1`: after its own setup runs, any subsequent `Bootsnap.setup` raises a clear error pointing at the fix. Gate your host's `Bootsnap.setup` on the same env var. Rails apps do this in `config/boot.rb`:

```ruby
# e.g. config/boot.rb
require "bootsnap/setup" unless ENV["TAPIOCA_RBS_CACHE"] == "1"
```

#### Priming the cache from CI

For CI pipelines that want to populate the cache once and have downstream jobs read from a warm copy, use `--only-bootsnap-rbs-cache`. This pattern lets you scope cache writes to a single job (the prime) so PR-side jobs read from it without uploading on every successful build:

```shell
# Prime: populate the cache.
$ TAPIOCA_RBS_CACHE=1 bin/tapioca dsl --only-bootsnap-rbs-cache

# Consumer: read from the populated cache.
# BOOTSNAP_READONLY=1 prevents bootsnap from writing back to a read-only mount.
$ TAPIOCA_RBS_CACHE=1 BOOTSNAP_READONLY=1 bin/tapioca dsl
```
Tapioca understands [inline RBS comments](https://sorbet.org/docs/rbs-comments) natively. While compiling a gem, signatures and class-level annotations written as `#: ...` / `# @abstract` / `# @requires_ancestor: ...` are read directly from the source via a [Rubydex](https://github.com/Shopify/rubydex)-built graph and translated into RBI alongside the runtime reflection that powers Sorbet `sig {}` blocks. There is no require-hook or load-time rewriter: Tapioca parses the source itself, so adding RBS comments to a gem doesn't change how the host application loads.

### RBI files for missing constants and methods

Expand Down Expand Up @@ -998,7 +962,6 @@ dsl:
only: []
exclude: []
verify: false
only_bootsnap_rbs_cache: false
quiet: false
workers: 1
rbi_max_line_length: 120
Expand Down
12 changes: 1 addition & 11 deletions lib/tapioca/cli.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,10 +103,6 @@ def todo
type: :boolean,
default: false,
desc: "Verifies RBIs are up-to-date"
option :only_bootsnap_rbs_cache,
type: :boolean,
default: false,
desc: "Only boot the application and load DSL extensions/compilers to populate the bootsnap iseq cache, then exit. Skips compiler execution and RBI generation. Mutually exclusive with --verify and --list-compilers."
option :quiet,
aliases: ["-q"],
type: :boolean,
Expand Down Expand Up @@ -150,12 +146,6 @@ def todo
def dsl(*constant_or_paths)
set_environment(options)

if options[:only_bootsnap_rbs_cache] && (options[:verify] || options[:list_compilers])
conflicting = options[:verify] ? "--verify" : "--list-compilers"
raise MalformattedArgumentError,
"Options '--only-bootsnap-rbs-cache' and '#{conflicting}' are mutually exclusive"
end

# Assume anything starting with a capital letter or colon is a class, otherwise a path
constants, paths = constant_or_paths.partition { |c| c =~ /\A[A-Z:]/ }

Expand Down Expand Up @@ -183,7 +173,7 @@ def dsl(*constant_or_paths)
elsif options[:list_compilers]
Commands::DslCompilerList.new(**command_args)
else
Commands::DslGenerate.new(**command_args, only_bootsnap_rbs_cache: options[:only_bootsnap_rbs_cache])
Commands::DslGenerate.new(**command_args)
end

command.run
Expand Down
11 changes: 0 additions & 11 deletions lib/tapioca/commands/dsl_generate.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,13 @@
module Tapioca
module Commands
class DslGenerate < AbstractDsl
#: (?only_bootsnap_rbs_cache: bool, **untyped) -> void
def initialize(only_bootsnap_rbs_cache: false, **kwargs)
@only_bootsnap_rbs_cache = only_bootsnap_rbs_cache
super(**T.unsafe(kwargs))
end

private

# @override
#: -> void
def execute
load_application

if @only_bootsnap_rbs_cache
say("Bootsnap RBS cache populated, exiting before RBI generation.", :green)
return
end

say("Compiling DSL RBI files...")
say("")

Expand Down
28 changes: 3 additions & 25 deletions lib/tapioca/dsl/compiler.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,32 +111,11 @@ def add_error(error)
private

# Get the types of each parameter from a method signature
#: ((Method | UnboundMethod) method_def, untyped signature) -> Array[String]
#: ((Method | UnboundMethod) method_def, Tapioca::Runtime::Signature? signature) -> Array[String]
def parameters_types_from_signature(method_def, signature)
params = [] #: Array[String]

return method_def.parameters.map { "T.untyped" } unless signature

# parameters types
signature.arg_types.each { |arg_type| params << arg_type[1].to_s }

# keyword parameters types
signature.kwarg_types.each { |_, kwarg_type| params << kwarg_type.to_s }

# rest parameter type
rest_type = signature.rest_type
params << rest_type.to_s if rest_type

# keyrest parameter type
keyrest_type = signature.keyrest_type
params << keyrest_type.to_s if keyrest_type

# special case `.void` in a proc
unless signature.block_name.nil?
params << signature.block_type.to_s.gsub("returns(<VOID>)", "void")
end

params
signature.parameter_type_strings
end

#: (RBI::Scope scope, (Method | UnboundMethod) method_def, ?class_method: bool) -> void
Expand Down Expand Up @@ -191,8 +170,7 @@ def compile_method_parameters_to_rbi(method_def)
#: ((Method | UnboundMethod) method_def) -> String
def compile_method_return_type_to_rbi(method_def)
signature = signature_of(method_def)
return_type = signature.nil? ? "T.untyped" : name_of_type(signature.return_type)
sanitize_signature_types(return_type)
signature&.return_type_string || "T.untyped"
end
end
end
Expand Down
Loading
Loading