bundler: Fix Bundler::Fetcher for PQC support, adding integration connection tests

[junaruga]

Summary

This PR is related to #9543. The first commit is the same with #9615. I want to see the #9615 is reviewed and merged. After that, I will rebase this PR on the latest master branch. The second commit is this PR's commit.

Created spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb adding non-PQC and PQC server/client connection integration tests.

As "Bundler::Fetcher local SSL server #connection PQC connects with client cert auth" failed with the following error due to hardcoded OpenSSL::PKey::RSA.new in Bundler::Fetcher#connection, fixed it to support ML-DSA ssl_client_cert.

$ bin/rspec spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb
...
Failures:

  1) Bundler::Fetcher local SSL server #connection PQC connects with client cert auth
     Failure/Error: fetcher = Bundler::Fetcher.new(remote)

     OpenSSL::PKey::PKeyError:
       incorrect pkey type: UNDEF
     # /home/jaruga/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/openssl/pkey.rb:394:in 'OpenSSL::PKey::RSA#initialize'
     # /home/jaruga/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/openssl/pkey.rb:394:in 'Class#new'
     # /home/jaruga/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/openssl/pkey.rb:394:in 'OpenSSL::PKey::RSA.new'
     # ./bundler/lib/bundler/fetcher.rb:321:in 'Bundler::Fetcher#connection'
     # ./bundler/lib/bundler/fetcher.rb:140:in 'Bundler::Fetcher#initialize'
     # ./spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb:69:in 'RSpec::ExampleGroups::BundlerFetcherLocalSSLServer#fetch_path'
     # ./spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb:60:in 'block (4 levels) in <top (required)>'
...

Created test/rubygems/local_ssl_server_utilities.rb to manage utility methods
called by RubyGems test-unit and Bundler rspec tests.

Co-Authored-By: Claude noreply@anthropic.com

Files

spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb

I aligned the file structure with test/rubygems/test_gem_remote_fetcher_local_ssl_server.rb. I also referred to spec/bundler/fetcher/gem_remote_fetcher_spec.rb.

test/rubygems/local_ssl_server_utilities.rb

For the file's naming, I referred to test/rubygems/multifactor_auth_utilities.rb, there are following files as support files, I think test/rubygems/*_utilities.rb file is suitable in this case in my opinion.

test/rubygems/helper.rb
test/rubygems/utilities.rb
test/rubygems/multifactor_auth_utilities.rb
test/rubygems/mock_gem_ui.rb

What was the end-user or developer problem that led to this PR?

Bundler::Fetcher#connection doesn't work with ML-DSA ssl_client_cert due to hardcoded OpenSSL::PKey::RSA.new.

What is your fix for the problem, implemented in this PR?

Fixed the Bundler::Fetcher#connection to support ML-DSA ssl_client_cert.
Added integration HTTPS server/client connection tests.

Make sure the following tasks are checked

• Describe the problem / feature
• Write tests for features and bug fixes
• Write code to solve the problem
• Make sure you follow the current code style and write meaningful commit messages without tags

[junaruga]

CI ruby-core cases failed.

https://github.com/ruby/rubygems/actions/runs/28041447077/job/83008544582?pr=9637

[junaruga]

CI ruby-core cases failed.

I am testing by adding the 3rd commit.

[junaruga]

CI ruby-core cases failed.

I fixed the ruby-core cases.

In CI Bundler on macOS ruby-4.0, ruby-3.2 cases, the following non-PQC/PQC tests failed. Investigating.

https://github.com/ruby/rubygems/actions/runs/28050419021/job/83039282611?pr=9637#step:12:109

Failures:

  1) Bundler::Fetcher local SSL server #connection PQC connects
     Failure/Error: expect(response.code).to eq("200")

       expected: #<Encoding:UTF-8> "200"
            got: #<Encoding:ASCII-8BIT> "403"

       (compared using ==)
     # ./spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb:49:in 'block (4 levels) in <top (required)>'
     # ./spec/spec_helper.rb:164:in 'block (4 levels) in <top (required)>'
     # ./spec/spec_helper.rb:164:in 'block (3 levels) in <top (required)>'
     # ./spec/support/helpers.rb:414:in 'block in Spec::Helpers#with_gem_path_as'
     # ./spec/support/helpers.rb:428:in 'Spec::Helpers#without_env_side_effects'
     # ./spec/support/helpers.rb:409:in 'Spec::Helpers#with_gem_path_as'
     # ./spec/spec_helper.rb:163:in 'block (2 levels) in <top (required)>'
     # ./lib/rubygems.rb:306:in 'Kernel#load'
     # ./lib/rubygems.rb:306:in 'Gem.activate_and_load_bin_path'

  2) Bundler::Fetcher local SSL server #connection PQC connects with client cert auth
     Failure/Error: expect(response.code).to eq("200")

       expected: #<Encoding:UTF-8> "200"
            got: #<Encoding:ASCII-8BIT> "403"

       (compared using ==)
     # ./spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb:61:in 'block (4 levels) in <top (required)>'
     # ./spec/spec_helper.rb:164:in 'block (4 levels) in <top (required)>'
     # ./spec/spec_helper.rb:164:in 'block (3 levels) in <top (required)>'
     # ./spec/support/helpers.rb:414:in 'block in Spec::Helpers#with_gem_path_as'
     # ./spec/support/helpers.rb:428:in 'Spec::Helpers#without_env_side_effects'
     # ./spec/support/helpers.rb:409:in 'Spec::Helpers#with_gem_path_as'
     # ./spec/spec_helper.rb:163:in 'block (2 levels) in <top (required)>'
     # ./lib/rubygems.rb:306:in 'Kernel#load'
     # ./lib/rubygems.rb:306:in 'Gem.activate_and_load_bin_path'

  3) Bundler::Fetcher local SSL server #connection non-PQC connects with client cert auth
     Failure/Error: expect(response.code).to eq("200")

       expected: #<Encoding:UTF-8> "200"
            got: #<Encoding:ASCII-8BIT> "403"

       (compared using ==)
     # ./spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb:35:in 'block (4 levels) in <top (required)>'
     # ./spec/spec_helper.rb:164:in 'block (4 levels) in <top (required)>'
     # ./spec/spec_helper.rb:164:in 'block (3 levels) in <top (required)>'
     # ./spec/support/helpers.rb:414:in 'block in Spec::Helpers#with_gem_path_as'
     # ./spec/support/helpers.rb:428:in 'Spec::Helpers#without_env_side_effects'
     # ./spec/support/helpers.rb:409:in 'Spec::Helpers#with_gem_path_as'
     # ./spec/spec_helper.rb:163:in 'block (2 levels) in <top (required)>'
     # ./lib/rubygems.rb:306:in 'Kernel#load'
     # ./lib/rubygems.rb:306:in 'Gem.activate_and_load_bin_path'

  4) Bundler::Fetcher local SSL server #connection non-PQC connects
     Failure/Error: expect(response.code).to eq("200")

       expected: #<Encoding:UTF-8> "200"
            got: #<Encoding:ASCII-8BIT> "403"

       (compared using ==)
     # ./spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb:24:in 'block (4 levels) in <top (required)>'
     # ./spec/spec_helper.rb:164:in 'block (4 levels) in <top (required)>'
     # ./spec/spec_helper.rb:164:in 'block (3 levels) in <top (required)>'
     # ./spec/support/helpers.rb:414:in 'block in Spec::Helpers#with_gem_path_as'
     # ./spec/support/helpers.rb:428:in 'Spec::Helpers#without_env_side_effects'
     # ./spec/support/helpers.rb:409:in 'Spec::Helpers#with_gem_path_as'
     # ./spec/spec_helper.rb:163:in 'block (2 levels) in <top (required)>'
     # ./lib/rubygems.rb:306:in 'Kernel#load'
     # ./lib/rubygems.rb:306:in 'Gem.activate_and_load_bin_path'

Finished in 18 minutes 55 seconds (files took 0.49493 seconds to load)
3859 examples, 4 failures, 23 pending

Failed examples:

rspec ./spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb:44 # Bundler::Fetcher local SSL server #connection PQC connects
rspec ./spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb:52 # Bundler::Fetcher local SSL server #connection PQC connects with client cert auth
rspec ./spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb:27 # Bundler::Fetcher local SSL server #connection non-PQC connects with client cert auth
rspec ./spec/bundler/fetcher/gem_remote_fetcher_local_ssl_server_spec.rb:19 # Bundler::Fetcher local SSL server #connection non-PQC connects

Error: Process completed with exit code 1.