Fix the address resolution for hostnames

[zeroSteiner]

Rex::Socket.getaddress can return either IPv4 or IPv6 addresses so when it's a hostname, use the socket type hint (#v6) to resolve the correct family of address.

This fixes an issue when the hostname resolves to both IPv4 and IPv6 addresses. In that case we should select the type of address based on the socket type we have lest, we get an error stating the it's incompatible.

Related to: rapid7/metasploit-framework#21512
Fixes: rapid7/metasploit-framework#21513

Demo

In this case localhost maps to both 127.0.0.1 and ::1 via the /etc/hosts file. A service is listening only on 0.0.0.0:80, not on an IPv6 socket. The bug surfaces because we resolve localhost to ::1 and try to connect to it with the IPv4 socket.

Old and broken:

[2] pry(main)> require 'rex/socket'
=> true
[3] pry(main)> Rex::Socket::Tcp.create('PeerHost' => 'localhost', 'PeerPort' => 80)
Errno::EAFNOSUPPORT: Address family not supported by protocol - connect(2) for [::1]:80 (Errno::EAFNOSUPPORT)
from /home/smcintyre/Projects/rex-socket/lib/rex/socket/comm/local.rb:267:in `connect'
[4] pry(main)> 

New and fixed:

[2] pry(main)> require 'rex/socket'
=> true
[3] pry(main)> Rex::Socket::Tcp.create('PeerHost' => 'localhost', 'PeerPort' => 80)
=> #<Socket:fd 7>
[4] pry(main)> 

[Copilot]

Pull request overview

This PR updates Rex::Socket::Comm::Local.create_by_type to resolve hostnames to an address family (IPv4 vs IPv6) that matches the socket being created, avoiding IPv6 resolutions being used with IPv4 sockets (and vice versa) when a hostname has both A and AAAA records.

Changes:

• Resolve param.localhost via Rex::Socket.getaddresses and select an address matching param.v6 before bind.
• Resolve param.peerhost via Rex::Socket.getaddresses and select an address matching param.v6 before connect.
• Skip DNS resolution when PeerHost/LocalHost is already an IP literal.

Impact Analysis:

• Blast radius: high (all call sites that create TCP/UDP/SCTP sockets through Rex::Socket::Comm::Local.create_by_type); downstream consumers unknown.
• Data and contract effects: no schema/payload changes; behavioral change in hostname resolution order/selection and failure modes.
• Rollback and test focus: rollback is straightforward (single-file logic change); validate hostname resolving to both A/AAAA for IPv4 sockets and IPv6 sockets, plus single-family-only hostnames (A-only and AAAA-only) to ensure errors are clear and no unexpected exceptions occur.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[smcintyre-r7]

.getaddresses will resolve the address which will raise Socket::ResolutionError when the name can't be resolved which seems like the correct behavior. ip will not be nil.

[3] pry(#<Msf::Modules::Auxiliary__Gather__Kerberoast::MetasploitModule>)> ::Addrinfo.getaddrinfo('doesnotexist.lan', 0, ::Socket::AF_UNSPEC, ::Socket::SOCK_STREAM)
Socket::ResolutionError: getaddrinfo: Name or service not known (Socket::ResolutionError)

[smcintyre-r7]

Problem: choosing a resolved address via .sample makes socket binding/connecting non-deterministic and can cause intermittent failures when some returned addresses are unreachable/unusable.

This is by design and replicates real-world behavior. When there are multiple addresses, we should be choosing one at random.

[smcintyre-r7]

This is incorrect, see #84 (comment)