Skip to content

feat(iroh): Avoid allocations in AddrFilter#4018

Merged
matheus23 merged 9 commits intomainfrom
matheus23/addr-filter-btreeset
Mar 19, 2026
Merged

feat(iroh): Avoid allocations in AddrFilter#4018
matheus23 merged 9 commits intomainfrom
matheus23/addr-filter-btreeset

Conversation

@matheus23
Copy link
Copy Markdown
Member

@matheus23 matheus23 commented Mar 12, 2026

Description

  • Makes AddrFilter essentially a wrapper around &Vec -> Cow<Vec> instead of &BTreeSet -> Vec.
  • Reworks EndpointInfo and EndpointData APIs. Some docs were outdated (e.g. claiming with_relay_url(None) would remove relays), etc. Generally also removed Deref and DerefMut implementations from EndpointInfo -> EndpointData.

Breaking Changes

iroh::address_lookup::EndpointInfo:

  • EndpointInfo::with_relay_url now takes RelayUrl instead of Option<RelayUrl>
  • EndpointInfo::with_ip_addrs now takes Vec<SocketAddr> instead of BTreeSet<SocketAddr>.
  • Removed Deref and DerefMut with Target = EndpointData from EndpointInfo.
  • Added EndpointData::{addrs, relay_urls, user_data, ip_addrs}.

iroh::address_lookup::EndpointData:

  • EndpointData::new now takes a Vec<TransportAddr> instead of impl IntoIterator<Item = TransportAddr> and keeps the ordering of addresses. Use EndpointAddr::from_iter instead (although less performant).
  • EndpointData::with_user_data now takes UserData instead of Option<UserData>.
  • EndpointData::filtered_addrs now returns Cow<Vec<TransportAddr>> instead of Vec<TransportAddr>.
  • Added EndpointData::{add_relay_url, add_ip_addrs, apply_filter}.
  • Added impl From<BTreeSet<{TransportAddr, SocketAddr}>> for EndpointData.
  • Added impl FromIterator<TransportAddr> for EndpointData.

iroh::address_lookup::AddrFilterFn:

  • Changed from being BTreeSet-based to Vec-based.
  • Return type changed to Cow<Vec<TransportAddr>> fromVec<TransportAddr>.

Notes

There's some work required to keep the EndpointData transport addresses deduplicated, and deduplicating a Vec is unfortunately somewhat expensive and requires allocations, but I think it's better to have this than not to.
I already saw some code in some publishing services that just keep adding new information to a single EndpointData and keep publishing. Without deduplication, this would go wrong. Maybe that should change nonetheless, but that'd be a PR for another day.

Change checklist

  • Self-review.
  • Documentation updates following the style guide, if relevant.
  • Tests if relevant.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 12, 2026

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh/pr/4018/docs/iroh/

Last updated: 2026-03-19T10:53:44Z

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 12, 2026

Netsim report & logs for this PR have been generated and is available at: LOGS
This report will remain available for 3 days.

Last updated for commit: 010db72

@n0bot n0bot Bot added this to iroh Mar 12, 2026
@github-project-automation github-project-automation Bot moved this to 🚑 Needs Triage in iroh Mar 12, 2026
@dignifiedquire
Copy link
Copy Markdown
Contributor

yeah, having just a vec might make more sense

@rklaehn
Copy link
Copy Markdown
Contributor

rklaehn commented Mar 16, 2026

I had something similar, but one of the goals of the filter functionality was to be able to not just filter but also reorder addrs. Hence the vec.

See the PR from ramfox: #3960

/// A filter and/or reordering function applied to transport addresses before publishing.

and/or reordering

Now, do we need reordering? I am not sure. But just wanted to mention that this design is making reordering impossible.

I would argue that ordering can be important in some cases. E.g. you can order from "good" to "bad" or "new" to "old", and dialing the addrs in a reasonable order will have much more benefit than saving an allocation.

Copy link
Copy Markdown
Contributor

@rklaehn rklaehn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine if we decide that reordering is not needed. But we should at least briefly discuss why the reordering of the original design is no longer needed.

@github-project-automation github-project-automation Bot moved this from 🚑 Needs Triage to 🏗 In progress in iroh Mar 16, 2026
@matheus23
Copy link
Copy Markdown
Member Author

Yeah - thanks for raising this @rklaehn

I briefly discussed this with @Frando. Address reordering became impossible with #3987, but bringing it back makes sense.

My response in discord was: Let's switch everything to Vec, including the addresses in EndpointData. Does that make sense?

@rklaehn
Copy link
Copy Markdown
Contributor

rklaehn commented Mar 18, 2026

Yeah - thanks for raising this @rklaehn

I briefly discussed this with @Frando. Address reordering became impossible with #3987, but bringing it back makes sense.

My response in discord was: Let's switch everything to Vec, including the addresses in EndpointData. Does that make sense?

Yes, if the order is interesting we should also allow it there. However, I would prefer if we don't allow duplicates, since I really don't see a good reason to have those.

addrs are not pub in EndpointData, so I guess having this a vec but preventing dupes is viable.

@matheus23
Copy link
Copy Markdown
Member Author

Yes, if the order is interesting we should also allow it there. However, I would prefer if we don't allow duplicates, since I really don't see a good reason to have those.

addrs are not pub in EndpointData, so I guess having this a vec but preventing dupes is viable.

The only problem is that this is somewhat ugly to enforce. You need to enforce everywhere you can now add ordered elements to EndpointData. But now that I looked into that struct and its methods, I realize there's a lot of weird APIs that need cleanup. I think I'll try to work on this.

@matheus23 matheus23 force-pushed the matheus23/addr-filter-btreeset branch from a1f280b to 20edb8e Compare March 18, 2026 10:48
@matheus23 matheus23 requested a review from rklaehn March 19, 2026 09:51
Comment thread iroh-relay/src/endpoint_info.rs Outdated
Comment thread iroh-relay/src/endpoint_info.rs
Comment thread iroh-relay/src/endpoint_info.rs Outdated
Comment thread iroh-relay/src/endpoint_info.rs Outdated
Comment thread iroh-relay/src/endpoint_info.rs
@matheus23 matheus23 enabled auto-merge March 19, 2026 10:53
@matheus23 matheus23 added this pull request to the merge queue Mar 19, 2026
Merged via the queue into main with commit d26cbd2 Mar 19, 2026
29 of 30 checks passed
@github-project-automation github-project-automation Bot moved this from 🏗 In progress to ✅ Done in iroh Mar 19, 2026
@matheus23 matheus23 deleted the matheus23/addr-filter-btreeset branch March 19, 2026 12:09
eleboucher pushed a commit to eleboucher/towonel that referenced this pull request Apr 18, 2026
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [iroh](https://github.com/n0-computer/iroh) | workspace.dependencies | minor | `0.97.0` → `0.98.0` |

---

### Release Notes

<details>
<summary>n0-computer/iroh (iroh)</summary>

### [`v0.98.0`](https://github.com/n0-computer/iroh/blob/HEAD/CHANGELOG.md#0980---2026-04-17)

[Compare Source](n0-computer/iroh@v0.97.0...v0.98.0)

##### ⛰️  Features

- *(iroh)* Avoid allocations in `AddrFilter` ([#&#8203;4018](n0-computer/iroh#4018)) - ([d26cbd2](n0-computer/iroh@d26cbd2))
- *(iroh)* Pluggable crypto backends ([#&#8203;3992](n0-computer/iroh#3992)) - ([387c2e4](n0-computer/iroh@387c2e4))
- *(iroh)* Add ability to configure external addrs ([#&#8203;4098](n0-computer/iroh#4098)) - ([372aab9](n0-computer/iroh@372aab9))
- *(iroh)* Do not advertise deprecated IPv6 addrs ([#&#8203;4106](n0-computer/iroh#4106)) - ([b2b1d46](n0-computer/iroh@b2b1d46))
- *(iroh-relay)* Expose more metrics on the relay server ([#&#8203;4085](n0-computer/iroh#4085)) - ([ee844a6](n0-computer/iroh@ee844a6))
- Expose the new fn decrypt on noq. ([#&#8203;4002](n0-computer/iroh#4002)) - ([197d8db](n0-computer/iroh@197d8db))
- Update noq and net-tools to latest ([#&#8203;4088](n0-computer/iroh#4088)) - ([159a5cf](n0-computer/iroh@159a5cf))
- Rate limiting in router ([#&#8203;3951](n0-computer/iroh#3951)) - ([ea7a634](n0-computer/iroh@ea7a634))
- Update relay protocol to iroh-relay-v2, add new Health frame, test protocol update mechanism ([#&#8203;3955](n0-computer/iroh#3955)) - ([0a22d76](n0-computer/iroh@0a22d76))
- Update to noq\@&#8203;0.18 ([#&#8203;4121](n0-computer/iroh#4121)) - ([72e4538](n0-computer/iroh@72e4538))
- Vendor pkarr implementation ([#&#8203;4026](n0-computer/iroh#4026)) - ([3ab6222](n0-computer/iroh@3ab6222))

##### 🐛 Bug Fixes

- *(ci)* Install cmake and nasm for aws-lc-sys on Windows ([#&#8203;4032](n0-computer/iroh#4032)) - ([405de1d](n0-computer/iroh@405de1d))
- *(iroh)* \[**breaking**] Faster relay health check after network change ([#&#8203;4041](n0-computer/iroh#4041)) - ([b11b0eb](n0-computer/iroh@b11b0eb))
- *(iroh)* `Endpoint::online` should only return once connected to a relay ([#&#8203;4115](n0-computer/iroh#4115)) - ([3424c6d](n0-computer/iroh@3424c6d))
- *(iroh)* \[**breaking**] Don't stop address lookup when one service errors ([#&#8203;4126](n0-computer/iroh#4126)) - ([321443d](n0-computer/iroh@321443d))
- *(iroh)* Don't falsely report home relay as connected on change ([#&#8203;4136](n0-computer/iroh#4136)) - ([1b04d34](n0-computer/iroh@1b04d34))
- *(iroh-relay)* Send EndpointGone message to the right peers ([#&#8203;4079](n0-computer/iroh#4079)) - ([6be4b3c](n0-computer/iroh@6be4b3c))
- *(iroh-relay)* Treat frames not allowed in a protocol version as error ([#&#8203;4127](n0-computer/iroh#4127)) - ([1c92a39](n0-computer/iroh@1c92a39))
- Improve holepunching after network changes ([#&#8203;3928](n0-computer/iroh#3928)) - ([cc21f51](n0-computer/iroh@cc21f51))
- Poll for default route with exponential backoff after network change ([#&#8203;4039](n0-computer/iroh#4039)) - ([dc01a77](n0-computer/iroh@dc01a77))
- Modify test\_ip\_wins\_over\_custom so it is a noop if no ip transport is… ([#&#8203;4048](n0-computer/iroh#4048)) - ([b141e02](n0-computer/iroh@b141e02))
- Increase path idle timeouts and configure relay paths ([#&#8203;4038](n0-computer/iroh#4038)) - ([eba6afc](n0-computer/iroh@eba6afc))
- Handle wine properly ([#&#8203;3902](n0-computer/iroh#3902)) - ([8bf7002](n0-computer/iroh@8bf7002))
- Set default crypto provider for iroh-relay binary ([#&#8203;4087](n0-computer/iroh#4087)) - ([2433645](n0-computer/iroh@2433645))

##### 🚜 Refactor

- *(iroh)* \[**breaking**] Improve address lookup registry ([#&#8203;4130](n0-computer/iroh#4130)) - ([481c870](n0-computer/iroh@481c870))
- *(iroh, iroh-relay)* \[**breaking**] Mark public types as non\_exhaustive ([#&#8203;4107](n0-computer/iroh#4107)) - ([86d3ac6](n0-computer/iroh@86d3ac6))
- *(iroh-base)* \[**breaking**] Don't expose third-party error types in iroh-base ([#&#8203;4073](n0-computer/iroh#4073)) - ([d803dfe](n0-computer/iroh@d803dfe))
- *(iroh-base)* \[**breaking**] Rename `CustomAddr::as_vec`  to `CustomAddr::to_vec` and improve docs ([#&#8203;4074](n0-computer/iroh#4074)) - ([3077f17](n0-computer/iroh@3077f17))
- *(iroh-base)* \[**breaking**] Change `SecretKey::generate` to not take an `Rng` arg ([#&#8203;4075](n0-computer/iroh#4075)) - ([36781ad](n0-computer/iroh@36781ad))
- *(iroh-relay)* \[**breaking**] Proper timeouts on relay connections ([#&#8203;4083](n0-computer/iroh#4083)) - ([79bb0e4](n0-computer/iroh@79bb0e4))
- Use latest noq api ([#&#8203;4057](n0-computer/iroh#4057)) - ([8af8370](n0-computer/iroh@8af8370))
- Remove needless alloc in relay server ([#&#8203;4084](n0-computer/iroh#4084)) - ([40b50a1](n0-computer/iroh@40b50a1))
- Move to\_z32/from\_z32 into iroh-base and remove EndpointIdExt trait ([#&#8203;4133](n0-computer/iroh#4133)) - ([382bf36](n0-computer/iroh@382bf36))

##### 🧪 Testing

- *(iroh)* Add patchbay test matrix for switching uplinks ([#&#8203;4095](n0-computer/iroh#4095)) - ([d72f1cb](n0-computer/iroh@d72f1cb))
- *(iroh)* Add more patchbay tests and improve existing tests ([#&#8203;4118](n0-computer/iroh#4118)) - ([f5ec24a](n0-computer/iroh@f5ec24a))
- *(iroh)* Improve patchbay NAT matrix ([#&#8203;4135](n0-computer/iroh#4135)) - ([2ad657a](n0-computer/iroh@2ad657a))
- Fix ip\_wins\_over\_custom flakiness ([#&#8203;4047](n0-computer/iroh#4047)) - ([174b58b](n0-computer/iroh@174b58b))
- Add patchbay tests ([#&#8203;3986](n0-computer/iroh#3986)) - ([2ab1240](n0-computer/iroh@2ab1240))
- Improve patchbay test setup ([#&#8203;4078](n0-computer/iroh#4078)) - ([9b01751](n0-computer/iroh@9b01751))
- Minor patchbay test improvements ([#&#8203;4091](n0-computer/iroh#4091)) - ([0cc2441](n0-computer/iroh@0cc2441))

##### ⚙️ Miscellaneous Tasks

- *(ci)* Move more CI jobs to self hosted runners ([#&#8203;4072](n0-computer/iroh#4072)) - ([54442c3](n0-computer/iroh@54442c3))
- *(iroh)* Update hickory to 0.26.0-beta.4 and use exact-version deps for prereleased crates ([#&#8203;4117](n0-computer/iroh#4117)) - ([f829593](n0-computer/iroh@f829593))
- *(iroh-bench)* Allow configuring the number of worker threads for each endpoint in iroh-bench ([#&#8203;4063](n0-computer/iroh#4063)) - ([065b448](n0-computer/iroh@065b448))
- Update to netwatch\@&#8203;0.16 and portmapper\@&#8203;0.16 ([#&#8203;4128](n0-computer/iroh#4128)) - ([24efedf](n0-computer/iroh@24efedf))
- Fix changelog generation - ([08c6454](n0-computer/iroh@08c6454))

##### Deps

- *(iroh)* Update patchbay to 0.5 ([#&#8203;4097](n0-computer/iroh#4097)) - ([811a062](n0-computer/iroh@811a062))
- Bump noq and net-tools ([#&#8203;4113](n0-computer/iroh#4113)) - ([184e378](n0-computer/iroh@184e378))

##### Examples

- *(iroh)* Allow to configure the receive window in the transfer example ([#&#8203;4082](n0-computer/iroh#4082)) - ([c865251](n0-computer/iroh@c865251))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJ0eXBlL21pbm9yIl19-->

Reviewed-on: https://git.erwanleboucher.dev/eleboucher/towonel/pulls/19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

3 participants