Vendor binary ABI incompatibility with musl: systematic audit and shim roadmap

[widgetii]

Problem

OpenIPC runs vendor-prebuilt .so libraries (from HiSilicon, Sigmastar, Ingenic, Goke, Rockchip, etc) that were compiled against uclibc or old glibc, but the firmware uses musl libc. Key ABI differences cause:

1. Link failures — vendor binary imports a symbol musl doesn't export (__ctype_b, __fgetc_unlocked, __assert, memcpy_s, etc.)
2. Silent data corruption — same function exists in both but struct layouts differ:
   • struct stat: uclibc=88 bytes vs musl=152 bytes (field offsets completely different)
   • off_t: uclibc=4 bytes vs musl=8 bytes (breaks mmap calling convention on ARM — offset argument shifts registers)
   • struct timespec: uclibc=8 bytes vs musl=16 bytes (breaks nanosleep, clock_gettime)

The existing libhisicompat for hi3516cv100 was a narrow hack discovered by chance from a crash (only wrapped st_mode via a custom sTaT symbol). We need a systematic approach.

Audit tool

A new general/scripts/audit-vendor-abi.py performs two definitive checks (no guessing):

1. Symbol check: extracts every imported symbol from vendor .so files and checks whether the platform's actual musl libc.so exports it
2. Struct probe: compiles a C program with the platform's musl cross-compiler, runs it under qemu, and gets exact sizeof/offsetof for every struct that crosses the vendor-musl boundary

The tool auto-downloads the correct per-platform toolchain from OpenIPC firmware releases.

Audit results

Scanned 492 binaries across 15 packages (platforms with available toolchains):

Package	Arch	Source libc	Missing symbols	Struct mismatches
fullhan-osdrv-fh8852v100	ARM	uclibc	8 (__assert, __fgetc_unlocked, __stdin, ...)	off_t, time_t
fullhan-osdrv-fh8852v200	ARM	uclibc	6 (__assert, __fgetc_unlocked, __stdin, ...)	off_t
goke-osdrv-gk7205v200	ARM	uclibc	8 (__ctype_b, memcpy_s, memset_s, ...)	off_t, timespec
goke-osdrv-gk7205v500	ARM	uclibc	8 (memcpy_s, memset_s, snprintf_s, ...)	off_t
hisilicon-osdrv-hi3516av100	ARM	uclibc	0	off_t, timespec
hisilicon-osdrv-hi3516cv100	ARM	uclibc	1 (__aeabi_d2iz)	struct stat, off_t
hisilicon-osdrv-hi3516cv200	ARM	uclibc	7 (__aeabi_f2d, __aeabi_fmul, ...)	off_t, timespec
hisilicon-osdrv-hi3516cv300	ARM	uclibc	0	off_t, timespec
hisilicon-osdrv-hi3516ev200	ARM	uclibc	11 (__ctype_b, memcpy_s, snprintf_s, ...)	off_t, timespec
hisilicon-osdrv-hi3519v101	ARM	uclibc	0	off_t, timespec, time_t
ingenic-osdrv-t20	MIPS	unknown	11 (__assert, __ctype_b, __fgetc_unlocked, ...)	(no qemu-mips)
ingenic-osdrv-t21	MIPS	unknown	11 (same)	(no qemu-mips)
ingenic-osdrv-t31	MIPS	unknown	5 (__assert, __fgetc_unlocked, ...)	(no qemu-mips)
ingenic-osdrv-t40	MIPS	unknown	5 (same)	(no qemu-mips)
xiongmai-osdrv-xm530	ARM	uclibc	0	off_t

Platforms with stat-family struct mismatch

15 vendor binaries across 8 packages import stat/fstat/stat64 — uclibc struct stat is 88 bytes, musl's is 152, with completely different field offsets. This causes stack corruption when musl writes 152 bytes into an 88-byte buffer:

allwinner-osdrv-v85x:     libion.so, libmedia_mpp.so, libISP.so (stat), libisp_ini.so (fstat)
hisilicon-osdrv-hi3516cv100: libmpi.so (stat) — fixed in this PR with shim
hisilicon-osdrv-hi3536dv100: libmpi.so (stat)
ingenic-osdrv-t20..t40:   libimp.so (stat)
novatek-osdrv-nt9856x:    libvos.so (stat, fstat)
rockchip-osdrv-rv11xx:    librockit.so (stat), librkaiq.so (stat64)

Reference: majestic's existing workaround

majestic/src/libc/uclibc.c already handles mmap (raw SYS_mmap2 syscall), fcntl64, __ctype_b, __isnan/__isinf, __fgetc_unlocked/__fputc_unlocked, __assert, and _stdlib_mb_cur_max. The approach proven in production.

Per-platform roadmap

Struct layout mismatches (silent corruption)

struct stat (uclibc=88B, musl=152B):

• hisilicon-osdrv-hi3516cv100: libmpi.so — shim in this PR
• allwinner-osdrv-v85x: libion.so, libmedia_mpp.so, libISP.so, libisp_ini.so
• hisilicon-osdrv-hi3536dv100: libmpi.so
• ingenic-osdrv-t20: libimp.so
• ingenic-osdrv-t21: libimp.so
• ingenic-osdrv-t23: libimp.so
• ingenic-osdrv-t30: libimp.so
• ingenic-osdrv-t31: libimp.so
• ingenic-osdrv-t40: libimp.so
• novatek-osdrv-nt9856x: libvos.so
• rockchip-osdrv-rv11xx: librockit.so, librkaiq.so

off_t (uclibc=4B, musl=8B) — breaks mmap on ARM:

• hisilicon-osdrv-hi3516cv100 — shim in this PR
• hisilicon-osdrv-hi3516ev200 (16 binaries)
• hisilicon-osdrv-hi3516av100
• hisilicon-osdrv-hi3516cv200
• hisilicon-osdrv-hi3516cv300
• hisilicon-osdrv-hi3519v101
• fullhan-osdrv-fh8852v100
• fullhan-osdrv-fh8852v200
• goke-osdrv-gk7205v200
• goke-osdrv-gk7205v500
• xiongmai-osdrv-xm530

Missing symbols (link failures)

• hisilicon-osdrv-hi3516ev200: __ctype_b, __fgetc_unlocked, _stdlib_mb_cur_max, memcpy_s/memset_s/snprintf_s family
• goke-osdrv-gk7205v200: __ctype_b, __fgetc_unlocked, _stdlib_mb_cur_max, memcpy_s family
• fullhan-osdrv-fh8852v100: __assert, __fgetc_unlocked, __stdin, __pthread_*
• fullhan-osdrv-fh8852v200: __assert, __fgetc_unlocked, __stdin
• ingenic-osdrv-t20..t40: __assert, __ctype_b, __fgetc_unlocked, __pthread_*

Call for testing

Maintainers with access to affected hardware: please run the audit tool on your platform and report results:

python3 general/scripts/audit-vendor-abi.py --package <your-package-name>

The tool will auto-download the correct toolchain (~100MB, cached in ~/.cache/openipc-audit-toolchains/).