Repository navigation
Conversation
Gem::SpecificationPolicy#validate_extension_require_relative read each file with File.foreach, which tags lines with Encoding.default_external. Under an unset or ASCII locale (build sandboxes, minimal containers) a valid UTF-8 non-ASCII line raised ArgumentError: invalid byte sequence in US-ASCII, and a non-UTF-8 source did the same under a UTF-8 locale. The error escaped validation and aborted gem build. The pattern is ASCII-only, so read the file as binary. Fixes ruby#9966
codespell read "caf" in the fixture as a misspelling of "calf". The test only needs bytes that are not valid UTF-8, so use \xFF\xFE. Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
paracycle
marked this pull request as ready for review
October 23, 2026 18:46
…reach JRuby raises "TypeError: no implicit conversion of Hash into Integer" when File.foreach is called with keyword arguments and no block, as it is when chained with with_index. That broke every gem build in the JRuby Bundler specs. Opening the file with "rb" and iterating each_line reads the same binary lines on CRuby and JRuby. Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
paracycle
commented
Oct 23, 2026
|
|
||
| File.foreach(rb_file).with_index(1) do |line, lineno| | ||
| next unless line =~ /^\s*require_relative\s+["']([^"']+)["']/ | ||
| File.open(rb_file, "rb") do |file| |
Contributor
Author
There was a problem hiding this comment.
JRuby doesn't seem to handle options passed to File.foreach, so I had to change this to File.open followed with each_line.
/cc @headius
Contributor
Author
There was a problem hiding this comment.
Failing action with the form that passes the encoding: https://github.com/ruby/rubygems/actions/runs/40005835119/job/120594010717?pr=9968#step:15:190
Contributor
There was a problem hiding this comment.
Thank you for the ping, but where is the call passing options? I don't see it in this diff.
Contributor
Author
There was a problem hiding this comment.
It is here: e158b75#diff-e7827558b551a4ab93ed20ca153ef8a1eb957bbbb8683267f52c00008f8e6fa7R509 (in the first commit in this series)
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was the end-user or developer problem that led to this PR?
gem buildaborts withArgumentError: invalid byte sequence in US-ASCIIfor a gem with native extensions whose.rbfiles under its require paths contain non-ASCII characters, whenever the locale is unset or ASCII (build sandboxes such as Nix, minimal containers). We hit it buildingextlz40.3.5. Reported in #9966.What is your fix for the problem, implemented in this PR?
Gem::SpecificationPolicy#validate_extension_require_relative(added in #9390) read each file withFile.foreach(rb_file), which tags every line withEncoding.default_external. Therequire_relativeregexp then raised on any line containing a non-ASCII byte, and the error escaped validation instead of producing a warning. A source that is not valid UTF-8 fails the same way under a UTF-8 locale.The pattern is ASCII-only, so the file is now opened in binary mode (
File.open(rb_file, "rb")witheach_line). Passingencoding:toFile.foreachwould also work on CRuby, but JRuby raisesTypeError: no implicit conversion of Hash into IntegerforFile.foreachwith keyword arguments and no block. Two tests cover both cases: a non-ASCII UTF-8 source under a US-ASCII default external encoding, and a non-UTF-8 source under a UTF-8 one. Both raise without the fix, and the existingextension_require_relativetests still pass.Fixes #9966
Make sure the following tasks are checked