build(afxdp): add USE_AF_XDP=1 Makefile conditional — Phase 2 PR C (fork)#3
Merged
Merged
Conversation
The AF_XDP TX (PR A) and RX (PR B) source files are already in-tree but
were not yet pulled into the build because the Makefile didn't know about
the flag. With this change, `make USE_AF_XDP=1`:
- defines `-DUSE_AF_XDP` so the io_engine_af_xdp vtable registers in
engine.c and the AF_XDP source files (which are wrapped in
`#ifdef USE_AF_XDP`) compile in,
- prepends `pkg-config --cflags/--libs libxdp libbpf` so the right
include and link paths are picked up on Debian/Ubuntu (libxdp.pc
and libbpf.pc are shipped by the -dev packages); falls back to
explicit `-lxdp -lbpf` when pkg-config is unavailable,
- adds `-lelf -lz` unconditionally because libxdp transitively
needs both for ELF section walking and zlib decompression,
- adds src/send-afxdp.c and src/recv-afxdp.c to SRCS so they get
compiled and linked.
The default build (USE_AF_XDP unset) is unchanged: the AF_XDP source
files compile to nothing under their `#ifdef` guards, the io_engine_af_xdp
vtable is not registered, and `--io-engine=af_xdp` at runtime exits
with the existing "rebuild with USE_AF_XDP=1" message in engine.c /
conf.c.
Verified locally on Debian with libxdp 1.5.4, libbpf 1.5.0, libelf 0.192:
$ make clean && make USE_AF_XDP=1
[...build succeeds...]
$ make USE_AF_XDP=1 test
=== Results: 11 passed, 0 failed ===
$ ldd scanner | grep -E 'libxdp|libbpf|libelf'
libxdp.so.1 => /lib/x86_64-linux-gnu/libxdp.so.1
libbpf.so.1 => /lib/x86_64-linux-gnu/libbpf.so.1
libelf.so.1 => /lib/x86_64-linux-gnu/libelf.so.1
Default build still passes the dispatch smoke tests and does not link
libxdp/libbpf/libelf:
$ make clean && make && make test
=== Results: 11 passed, 0 failed ===
Plan: AnyVM-Tech/AnyScan plans/2026-04-27-portscan-afxdp-plan-v1.md §3.6
skullcrushercmd
added a commit
that referenced
this pull request
Apr 28, 2026
#4) Adds a fourth I/O engine, --io-engine=dpdk, alongside af_packet, pfring_zc and af_xdp. Mirrors the AF_XDP wire-up shape introduced by PRs #1-#3 but swaps the kernel XSK socket path for a DPDK PMD running directly in userspace (rte_eth_tx_burst / rte_eth_rx_burst against vfio-pci-bound NICs). Why: AWS ENA on kernel ≤6.12.74 forces AF_XDP into drv+copy mode, capping c6in.metal at ~22M pps aggregate (memory: anyscan_afxdp_ena_constraint). DPDK bypasses the kernel ENA driver entirely via vfio-pci and removes both the syscall kick and the lower-half-channels ZC constraint, opening up the 50-100M pps ceiling identified in plans/2026-04-28-portscan-dpdk-impl-v1.md. What lands here: - include/dpdk-defs.h: opaque struct dpdk_state forward-decl + sizing constants (mbuf pool 8192/sender, 1024-deep TX/RX rings, 256-mbuf per-lcore cache). - src/dpdk-eal.c: process-wide EAL bring-up + teardown — rte_eal_init, port probe, mempool create, port_configure with RSS-on-IP/TCP/UDP, TX and RX queue setup, dev_start, promiscuous-on. Runs once on the main thread before run_scan; teardown drains ports and rte_eal_cleanup. - src/send-dpdk.c: TX-burst loop. Same BlackRock walk + blacklist / alive-queue filter as send-afxdp.c; only the I/O layer changes (rte_pktmbuf_alloc_bulk → packet patch in mbuf data area → rte_eth_tx_burst). Same partial-send rollback so sustained backpressure doesn't silently drop scan targets. - src/recv-dpdk.c: RX-burst loop. process_packet is reused unchanged (Ethernet→IP→TCP/UDP/ICMP filter + scoreboard). One receiver per RX queue; rejects mismatched receivers/queue counts loudly (rte_eth_rx_burst is MT-unsafe per queue). - src/engine.c: io_engine_dpdk vtable + IO_ENGINE_DPDK case in pick_io_engine. - include/scanner.h, include/scanner_defs.h: forward decls + new config / per-thread fields under #ifdef USE_DPDK. - src/conf.c: --io-engine=dpdk recognized; new --dpdk-port, --dpdk-num-{tx,rx}q, --dpdk-eal-args flags. Mandatory --gateway-mac when io_engine=dpdk (no kernel ARP in DPDK). - src/main.c: argv pre-scan splits scanner argv vs raw EAL argv on the conventional `--` separator; calls dpdk_eal_bringup before setup_scan and dpdk_eal_teardown after run_scan. - Makefile: USE_DPDK=1 conditional that adds -DUSE_DPDK, pkg-config libdpdk cflags+libs, and the new TUs. Default build is bit-for-bit identical to upstream when USE_DPDK is unset. - tests/dpdk_dispatch.sh: smoke test for --io-engine=dpdk parse-time behaviour. Mirrors tests/io_engine_dispatch.sh (CLI parse + build-flag rejection / acceptance assertions, no root or NIC required). Build verification (all in this branch on a Debian bookworm host with libdpdk-dev 24.11 + libxdp 1.5.4 / libbpf 1.5.0 installed): - `make` → 11/11 io_engine_dispatch + 11/11 dpdk_dispatch pass. - `make USE_AF_XDP=1` → 11/11 + 11/11 pass. - `make USE_DPDK=1` → 11/11 + 8/8 pass (the DPDK-only assertions activate on this build). - `make USE_AF_XDP=1 USE_DPDK=1` → 11/11 + 8/8 pass. No new compiler warnings (the strncpy ones in receiver.c pre-date this PR). Note on parsing.c reference in the task brief: io_engine_from_string and io_engine_name actually live in src/conf.c (this PR extends them there); src/parsing.c is the IP/port range parser and is untouched. Out of scope (Phase 2 next steps in the AnyScan repo, not here): - install-external-deps.sh / package-worker-bundle.sh / deploy.sh ANYSCAN_USE_DPDK build-flag plumbing (mirrors PR #71's AF_XDP wire-up). - tools/setup-dpdk.sh hugepages + vfio-pci bind/unbind script. - install-worker-bundle.sh probe_dpdk_runtime_available. - vulnscanner-zmap-adapter.py SUPPORTED_IO_ENGINES + EAL argv emit. - Live c6in.metal bench (separate worker per the plan's §5.3). Refs: AnyVM-Tech/AnyScan plans/2026-04-28-portscan-dpdk-impl-v1.md anygpt-50 Co-authored-by: skullcmd <skullcmd@anyvm.tech> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add the
USE_AF_XDP=1Makefile block that pulls the in-treesrc/send-afxdp.c(PR #1, 4dd3a2a) andsrc/recv-afxdp.c(PR #2, c69872f) into the build and links libxdp + libbpf + libelf. WithUSE_AF_XDPunset (the default), the build is unchanged — the AF_XDP TUs compile to nothing under their#ifdef USE_AF_XDPguards, no XDP libs are linked, and--io-engine=af_xdpat runtime keeps emitting the existing "rebuild with USE_AF_XDP=1" message fromengine.c/conf.c.This is Phase 2 PR C of the AF_XDP integration plan (AnyVM-Tech/AnyScan §3.6 of
plans/2026-04-27-portscan-afxdp-plan-v1.md). The companion AnyScan PR C adds the apt-get deps, systemdCAP_BPF, and the runtime probe so worker bundles can detect when the AF_XDP code path is available; that one will follow shortly.What changed
Makefile— one new conditional block, mirroring the shape of the existingUSE_PFRING_ZCblock:-DUSE_AF_XDPtoCFLAGSso theio_engine_af_xdpvtable insrc/engine.c:146is registered and the AF_XDP TUs compile in.pkg-config --cflags/--libs libxdp libbpffor include + link flags. Debian/Ubuntu shiplibxdp.pc/libbpf.pcin the-devpackages, so pkg-config is the supported source. Falls back to plain-lxdp -lbpfwhen pkg-config returns empty (hosts where libxdp/libbpf were source-built without.pcfiles).-lelf -lzunconditionally — libxdp transitively walks ELF sections and decompresses BTF, and pkg-config sometimes does not pull these through depending on howlibxdp.pcwas generated.src/send-afxdp.c src/recv-afxdp.ctoSRCS.Test status
Verified on Debian (libxdp 1.5.4, libbpf 1.5.0, libelf 0.192):
make USE_AF_XDP=1:Default build (USE_AF_XDP unset):
Smoke tests cover both build flag states — see
tests/io_engine_dispatch.shbinary_has_af_xdpbranch — so this PR exercises both modes of the dispatch path that PR #1 + PR #2 added.Out of scope (deferred)
libxdp-dev libbpf-dev libelf-dev), worker systemd unitCAP_BPF, and the install-time runtime probe — these land in the companion AnyScan PR C.ANYSCAN_SCANNER_IO_ENGINEenv, adapter--io-engineplumbing) — Phase 2 PR D, after PR C lands.c6in.metalbench — Phase 2 PR E, owned out of session.Plan reference
plans/2026-04-27-portscan-afxdp-plan-v1.md§3.6 ("Build-system integration").