Skip to content

Anchor menus on Wayland and load Motif Xft fonts - #73

Open
jserv wants to merge 5 commits into
mainfrom
detached-menu
Open

Anchor menus on Wayland and load Motif Xft fonts#73
jserv wants to merge 5 commits into
mainfrom
detached-menu

Conversation

@jserv

@jserv jserv commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Motif drop-down menus rendered as free-standing windows and FONT_IS_XFT render tables came out as tiny unreadable blocks on the SDL3 backend under a Wayland compositor.

Menus: an override-redirect popup is a top-level, and a Wayland client cannot self-position a top-level, so each menu shell floated wherever the compositor dropped it. Under the SDL3 backend these popups are now created with SDL_CreatePopupWindow anchored to the parent surface whose rectangle contains the menu anchor. findPopupParentToplevel picks the innermost containing realized window (cascade submenus anchor to their parent menu), and the popup is placed at a parent-relative offset. SDL owns the popup as a child window, so parent teardown unrealizes any anchored popup first to avoid a dangling SDL_Window double free. The SDL2 path is unchanged: popupParent stays None and the absolute-position top-level path is preserved.

Fonts: libXm was built --without-xft, so a FONT_IS_XFT rendition took the INVALID_TYPE branch and loaded no font. Build Motif --with-xft against the in-tree Xft, adding Fc{Init,Fini} and XftDrawCreateBitmap the Motif Xft path links. The differential reference builds flip to --with-xft as well, with matching libxft/libfontconfig deps, so both sides render the same font stack; measured baselines are unchanged.

Add a wayland CI job that builds a Wayland-enabled SDL3 from source and runs scripts/validate-menu-rendering.sh under weston headless: a real Motif XmPulldownMenu, a raw override-redirect popup, a FONT_IS_XFT rendition, and a parent-first teardown under AddressSanitizer.

Close #68


Summary by cubic

Fix Wayland menus by anchoring override‑redirect popups as parent‑anchored SDL3 xdg_popups and make Motif Xft fonts render at readable sizes. Also track compositor moves and re-run the popup positioner on resize so menus land and grow correctly.

  • Bug Fixes

    • Wayland menus: realize override‑redirect menus as anchored popups via SDL_CreatePopupWindow; findPopupParentToplevel picks the innermost mapped parent (skips zero‑area transients). SDL2 still uses absolute toplevels.
    • Compositor behavior: accept an anchored popup’s MOVED to record the real origin (no spurious ConfigureNotify), and re-run the xdg positioner after resize via repositionAnchoredPopup; drop invalid always‑on‑top on xdg_popups.
    • Safe teardown: drainAnchoredPopups unmaps/unrealizes anchored popups before parent unmap/destroy/unrealize; tests assert the popup’s UnmapNotify and final IsUnmapped; guard XGetWindowAttributes failure.
    • Fonts: build Motif --with-xft and add FcInit/FcFini and XftDrawCreateBitmap so XmFONT_IS_XFT loads scalable fonts.
  • New Features

    • Add a Wayland CI job that builds Wayland‑enabled SDL3/SDL3_ttf with scripts/build-sdl3.sh and runs scripts/validate-menu-rendering.sh under headless weston.
    • New tests cover Xft font legibility, raw popup anchoring, a real XmPulldownMenu, parent destroy/unmap teardown, popup resize growth, and compositor slide offsets; CI asserts the Wayland driver, runs under AddressSanitizer, and enforces required checks with CI_REQUIRE=1.

Written for commit ad85532. Summary will update on new commits.

Review in cubic

cubic-dev-ai[bot]

This comment was marked as resolved.

cubic-dev-ai[bot]

This comment was marked as resolved.

jserv added a commit that referenced this pull request Jul 24, 2026
The new wayland job failed at SDL3 configure with "could not find X11 or
Wayland development libraries": SDL3's Wayland video driver needs EGL, which
the job did not install, so with no usable X11 set either SDL3 disabled both
backends and aborted. Install libegl-dev and libgles-dev, and build SDL3 with
SDL_X11=OFF since this path only exercises the Wayland driver: the incomplete
system X11 set is irrelevant and turning X11 off keeps the configure honest.

Also reflow scripts/validate-menu-rendering.sh and scripts/build-sdl3.sh
through shfmt so the lint job's `shfmt -d` gate passes, and pick up the
clang-format reflow of the popup unmap comment and the parent-unmap test.
jserv added a commit that referenced this pull request Jul 24, 2026
Address the PR #73 review: popup-parent-unmap.c printed PARENT_UNMAP_OK after
the map/unmap/remap cycle regardless of what happened to the popup, so the gate
passed on mere absence of an ASan crash. It now reads the popup's map_state
after XUnmapWindow(main) and fails unless the parent-unmap sweep carried it to
IsUnmapped, so the check validates the lifecycle behavior rather than only
teardown cleanliness.

While validating that, the teardown check turned out not to be re-runnable: its
ASan rebuild instruments both build/src and the staged upstream libX11 objects
under build/upstream/src, but restore_library wiped only build/src, so it
relinked the leftover instrumented upstream objects and left libX11-compat.so
carrying __asan_* references that break the ordinary test compiles on the next
run. A new wipe_first_party_objs drops the upstream object files too (keeping
their staged .c sources) before both the ASan build and the restore, so each
rebuild is clean and the workspace is usable afterward. Confirmed on node1:
two back-to-back runs both report 4/4 and the library is left uninstrumented.
cubic-dev-ai[bot]

This comment was marked as resolved.

jserv added 4 commits July 25, 2026 07:29
A Wayland client cannot self-position a top-level, so override-redirect
menu created as an ordinary window floats wherever the compositor drops
it, detached from the widget that spawned it. When a mapped parent
contains the popup's anchor, realize it as an SDL3 xdg_popup positioned
relative to that parent surface. findPopupParentToplevel picks the
innermost containing window (smallest area wins, so a cascade submenu
anchors to its parent menu, not the shell); a zero-area transient is
skipped so it cannot misanchor.

Anchored popups are torn down before their parent's SDL surface through
a single drainAnchoredPopups helper shared by the unmap, unrealize, and
destroy paths. It unmaps each child through the full path (grab release,
UnmapNotify, mapState flip) and forces the unrealize on any child left
holding an sdlWindow, so the drain always converges and never leaves a
popup stranded with a surface SDL would free out from under it. New
popupParent field on WindowStruct, initialized in initWindowStruct;
xc_CreatePopupWindow shim in sdl-compat.h (SDL3 only, no SDL2
equivalent).
Motif's FONT_IS_XFT render tables need a working Xft. The bundled
thentenaar/motif was built --without-xft, so those tables fell through
to an invalid rendition type and painted as a tiny crushed block. Build
it --with-xft against the in-tree Xft-over-SDL layer and add the entry
points that path links against: FcInit/FcFini, which Motif's Display.c
calls at display open and close, and XftDrawCreateBitmap, the XmRenderT
fallback that must at least resolve. Bump MOTIF_COMMIT to the matching
source revision.

The app differentials build the same --with-xft Motif, so keep them
green: install lib{fontconfig1,xft}-dev in CI, flip motif/mosaic/violawww
differential drivers to --with-xft with a pkg-config preflight,
recalibrate the demo comparison thresholds now that both sides render
through Xft, and widen the GIMP-Motif startup window-wait. GIMP's
heavier --with-xft cold start, captured concurrently with the system
side, now maps its toolbox past the old five-second bound on a two-core
runner; the wait is a max, so the single-process path is unaffected.
tests/menu-rendering covers the two behaviors from the detached-menu
report. A raw override-redirect popup and a real Motif XmPulldownMenu
must each emit a Wayland xdg_popup anchored to their parent rather than
a detached second toplevel; a FONT_IS_XFT render table must measure a
legible height; and tearing a live popup's parent down, by both destroy
and unmap, must carry the popup through the full unmap path (its own
UnmapNotify, IsUnmapped, a clean remap) with no leak or use-after-free
under AddressSanitizer.

scripts/validate-menu-rendering.sh drives them under weston headless,
building a Wayland-enabled SDL3 via scripts/build-sdl3.sh (pinned source
with published checksums). For the teardown check it rebuilds the
library with the sanitizer and restores an uninstrumented one afterward;
the restore wipes the staged upstream objects the sanitizer instrumented
too, so the harness leaves a clean tree and is re-runnable. Wayland
checks skip with a reason on an X11-only host; CI_REQUIRE=1 turns a skip
into a failure for a required gate.
Add a wayland job: it builds a Wayland-enabled SDL3 and SDL3_ttf from
source (with EGL/GLES dev libs so SDL3's Wayland video driver actually
configures), builds the compat stack and the --with-xft Motif, and runs
scripts/validate-menu-rendering.sh against weston headless with
CI_REQUIRE=1. It is the only job that exercises compositor-dependent
override-redirect xdg_popup path; the SDL2 jobs and the console-only
sdl3 job cannot see it.
@jserv

jserv commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

@hak-foo , Can you please help determine whether this pull request resolves #68 ?

@hak-foo

hak-foo commented Jul 25, 2026

Copy link
Copy Markdown

Significantly better-- the menus are in the right places, and the font is legible. It's not the font specified in the .Xdefaults file, but it's usable.

Some other menu oddities I noticed on this version:

  • If you open one menu, then click on another without dismissing it, the new menu may be weirdly sized, like it inherited the dimensions of the old menu. For example, opening the "Preferences" menu then clicking on "Edit" sometimes produces this.
2026-07-24T20:45:08,441724726-07:00

It also sometimes goes the other way-- a large menu is cropped to a smaller size with parts missing.

If you open a second-level menu and keep the button held down as you enter it (for example, click "Preferences", then click on "Default Settings" and hold the mouse button down until you enter the secondary menu), the highlight is somewhat off from the mouse cursor. This seems to happen when the second-level menu starts "above" the first-level menu on the screen.

I tried to press the accelerator commands for menus (Alt-F, Alt-E, etc) to see if the menu sizing thing responded differently and they don't register.

@hak-foo

hak-foo commented Jul 25, 2026

Copy link
Copy Markdown

Oh, another fun menu behaviour:

Pop up a menu, and then go over to a Firefox window and hover over the tab bar. After a short time, The menu will suddenly move over to be there!
2026-07-24T21:10:37,852842430-07:00

I was also able to get the menu to jump by switching to a Galculator or Qmmp window and pressing that menu bar.

Override-redirect popups on the SDL3/Wayland backend had two defects.

An xdg_popup's geometry is fixed by its positioner at creation, so
SDL_SetWindowSize only grew the surface buffer while the compositor kept
clipping a popup resized after mapping to its creation rectangle: a Motif
menu bar switching the armed pulldown from a short menu to a taller one
rendered the taller menu cropped. configureWindow now re-runs the
positioner once, after the move and resize blocks have settled the
geometry, through repositionAnchoredPopup, so the compositor grows the
popup to the final size.

A constraint-enforcing compositor slides or flips a popup that would fall
off screen and reports the new position only through SDL's MOVED event,
which onSdlEvent dropped. ws->x/y then kept the requested position, so
pointer-to-root translation for a grabbed submenu was offset by the
slide and the highlight drifted from the cursor. onSdlEvent now lets an
anchored popup's MOVED through; convertEvent records the real origin
(parent origin plus the parent-relative offset) and emits no
ConfigureNotify, so the toolkit's own menu tracking is undisturbed.
@jserv

jserv commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Significantly better-- the menus are in the right places, and the font is legible. It's not the font specified in the .Xdefaults file, but it's usable.
Some other menu oddities I noticed on this version:

Can you help validate commit ad85532 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

XNEdit "detached" menus and odd fonts

2 participants