diff --git a/dev/modules/poe.md b/dev/modules/poe.md new file mode 100644 index 000000000..4777db31e --- /dev/null +++ b/dev/modules/poe.md @@ -0,0 +1,616 @@ +# POE Fix Plan + +## Overview + +**Module**: POE 1.370 (Perl Object Environment - event-driven multitasking framework) +**Test command**: `./jcpan -t POE` +**Status**: 38/53 unit+resource tests pass, ses_session.t 37/41, ses_nfa.t 39/39, k_alarms.t 37/37, k_aliases.t 20/20, k_selects.t 17/17, filehandles.t 131/132, 01_sysrw.t 17/17, 15_kernel_internal.t 12/12 + +## Dependency Tree + +``` +POE 1.370 +├── POE::Test::Loops 1.360 PASS (2/2 tests) +├── IO::Pipely 0.006 OK (loads successfully after fixes) +│ └── IO::Socket (>= 1.38) FIXED (exists &sub in require context) +│ └── Symbol (>= 1.08) FIXED ($VERSION set in Java module) +├── Time::HiRes (>= 1.59) STUB (POE has it commented out) +├── IO::Tty 1.08 UNAVAILABLE (XS/native, needs C compiler) +├── IO::Poll (optional) MISSING +├── IO::Pty (optional) MISSING (needs IO::Tty) +├── Curses (optional) MISSING +├── Term::Cap (optional) MISSING +├── Term::ReadKey (optional) MISSING ($VERSION not set) +├── Socket::GetAddrInfo (opt) MISSING +├── POSIX FIXED (added uname, signals, errno consts) +├── Errno OK (pure Perl, complete) +├── Storable OK (XSLoader backend) +└── HTTP::Request/Response PARTIAL (for Filter::HTTPD) +``` + +## Bugs Fixed (Commits 743c26461 through f119640a5) + +### Bug 1: `exists(&Errno::EINVAL)` fails in require context - FIXED + +**Root cause**: `ConstantFoldingVisitor.java` was replacing `IdentifierNode("Errno::EINVAL")` with `NumberNode("22")` inside the `&` sigil operator, so `EmitOperatorDeleteExists.java` couldn't match the pattern. + +**Fix**: Added guard in `ConstantFoldingVisitor.visit(OperatorNode)` to skip folding when operator is `"&"`. + +### Bug 2: Cross-file `use vars` under strict - NOT A BUG + +Investigation confirmed this was a cascading failure from Bug 1. When POE::Kernel.pm crashed during loading (before reaching `use vars qw($poe_kernel)`), the variable was never declared. Once Bug 1 was fixed, POE loads correctly and `$poe_kernel` is visible across files. + +### Bug 3: Symbol.pm `$VERSION` not set - FIXED + +**Fix**: Added `GlobalVariable.getGlobalVariable("Symbol::VERSION").set("1.09")` in `Symbol.java`. + +### Bug 4: Indirect object syntax `import $package ()` - FIXED + +**Root cause**: `Variable.java` line 168 errored when it saw `(` after a `$var` in indirect object context. + +**Fix**: Added `parsingIndirectObject` flag to Parser, set it in SubroutineParser before parsing the class variable in indirect object syntax. + +### Bug 5: POSIX missing functions and constants - FIXED + +**Fix**: Added to `POSIX.java`: signal constants (SIGHUP-SIGTSTP), errno constants (EPERM-ERANGE), `uname()`, `sigprocmask()` stub. Added to `POSIX.pm`: stub classes for SigSet/SigAction. + +### Bug 6: ConcurrentModificationException in hash each() - FIXED + +**Root cause**: Java's HashMap iterator throws when the hash is modified during iteration. Perl's `each` tolerates this (common idiom: `while (each %h) { delete ... }`). + +**Fix**: `RuntimeHash.RuntimeHashIterator` now snapshots entries at creation time via `new ArrayList<>(elements.entrySet()).iterator()`. + +### Bug 7: Socket.pm missing IPPROTO constants - FIXED + +**Fix**: Added `IPPROTO_TCP`, `IPPROTO_UDP`, `IPPROTO_ICMP` to both `Socket.java` and `Socket.pm`. + +### Bug 8: %SIG not pre-populated with signal names - FIXED (commit ba803dc49) + +**Root cause**: `%SIG` was empty in PerlOnJava. Perl pre-populates it with signal names as keys (undef values). POE discovers available signals via `keys %SIG`. + +**Fix**: `RuntimeSigHash.java` constructor now pre-populates with POSIX signals plus platform-specific signals (macOS: EMT, INFO, IOT; Linux: CLD, STKFLT, PWR, IOT). + +### Bug 9: DESTROY not called for blessed objects - ATTEMPTED AND REVERTED + +**Root cause**: PerlOnJava had no DESTROY support. POE and many modules rely on DESTROY for cleanup. + +**Attempted fix**: `DestroyManager.java` using `java.lang.ref.Cleaner` to detect GC-unreachable blessed objects and reconstruct proxy objects for DESTROY calls. + +**Why it was reverted**: The proxy reconstruction approach is fundamentally fragile: +- `close()` inside DESTROY on a proxy hash corrupts subsequent hash access (File::Temp "Not a HASH reference" at line 205) +- Overloaded classes get negative blessIds; `Math.abs()` on cache keys collided with normal class IDs +- Proxy can't fully replicate tied/magic/overloaded behavior of original objects + +**Current state**: DESTROY is not called for regular blessed objects. Tied variable DESTROY still works (uses scope-based cleanup via `TieScalar.tiedDestroy()`). See `dev/design/object_lifecycle.md` for future directions (scope-based ref counting recommended). + +**Impact on POE**: POE's core event loop works without DESTROY. The only affected feature is `POE::Session::AnonEvent` postback cleanup — sessions using postbacks won't get automatic refcount decrement. Workaround: explicit cleanup or patching POE. + +### Bug 10: foreach doesn't see array modifications during iteration - FIXED (commit f79f9f6e8) + +**Root cause**: `RuntimeArrayIterator` cached `elements.size()` at creation time. Perl's foreach sees elements pushed to the array during the loop body. This broke POE's `Kernel->stop()` which uses exactly this pattern to walk the session tree: +```perl +my @children = ($self); +foreach my $session (@children) { + push @children, $self->_data_ses_get_children($session->ID); +} +``` + +**Fix**: Changed `hasNext()` to check `elements.size()` dynamically instead of using a cached value. This was the root cause of ses_session.t hanging after test 31 — nested child sessions were not being found during stop(), leaving orphan sessions keeping the event loop alive. + +### Bug 11: `require File::Spec->catfile(...)` parsed as module name - FIXED (commit 6b9fa2c30) + +**Root cause**: POE's `Resource/Clock.pm` does `require File::Spec->catfile(qw(Time HiRes.pm))`. The parser's `parseRequire` method consumed `File::Spec` as a bareword module name without checking for `->` method call after it. + +**Fix**: In `OperatorParser.java`, save parser position after consuming identifier, peek for `->`, and if found, restore position and fall through to expression parsing. This allows `Time::HiRes` to load correctly, making `monotime()` return float time instead of integer seconds. + +### Bug 12: Non-blocking I/O for pipe handles - FIXED (commit 6b9fa2c30) + +**Root cause**: POE's `_data_handle_condition` calls `IO::Handle::blocking($handle, 0)` on the signal pipe, but PerlOnJava's pipes didn't support non-blocking mode. `sysread` on empty non-blocking pipe blocked forever. + +**Fix**: Added `isBlocking()`/`setBlocking()` to `IOHandle` interface, implemented in `InternalPipeHandle` with EAGAIN (errno 11) return on empty non-blocking read. Fixed `IO::Handle::blocking()` Perl method to use `($fh, @args) = @_` instead of `shift` (which copied the glob and lost connection to the underlying handle). + +### Bug 13: DestroyManager crash with overloaded classes - FIXED (commit cddf4b121) + +**Root cause**: `DestroyManager.registerForDestroy` used `Math.abs(blessId)` as cache keys, but overloaded classes get negative blessIds (-1, -2, ...). `Math.abs(-1) = 1` collided with the first normal class ID, causing `getBlessStr` to return null and NPE in `normalizeVariableName`. + +**Fix**: Used original `blessId` directly as cache key (fixed before DestroyManager was removed). + +### Bug 14: 4-arg select() marks pipes as always ready - FIXED (commit f119640a5) + +**Root cause**: The NIO-based `selectWithNIO()` in `IOOperator.java` treated all non-socket handles (pipes, files) as unconditionally ready (`nonSocketReady++`). This caused `select()` to return immediately when monitoring POE's signal pipe, preventing the event loop from blocking for timer timeouts. + +**Impact**: POE's `ses_session.t` hung at test 7 (before `POE::Kernel->run()`) because the event loop never slept — `select()` always returned immediately with the pipe "ready", POE tried to read (got nothing), and looped back. + +**Fix**: Replaced the "always ready" assumption with proper polling: +- `InternalPipeHandle.hasDataAvailable()` checks if data is actually in the pipe +- Write ends and regular files remain always-ready +- A poll loop with 10ms intervals respects the timeout parameter +- Both pollable fds and NIO selector are checked each iteration + +## Current Test Results (2026-04-05) + +### Unit Tests (t/10_units/) + +| Test File | Result | Notes | +|-----------|--------|-------| +| 03_base/01_poe.t | **PASS** (4/4) | | +| 03_base/03_component.t | **PASS** (1/1) | | +| 03_base/04_driver.t | **PASS** (2/2) | | +| 03_base/05_filter.t | **PASS** (2/2) | | +| 03_base/06_loop.t | **PASS** (1/1) | | +| 03_base/07_queue.t | **PASS** (2/2) | | +| 03_base/08_resource.t | **PASS** (1/1) | | +| 03_base/09_resources.t | FAIL (1/7) | CORE::GLOBAL::require not supported | +| 03_base/10_wheel.t | **PASS** (7/7) | | +| 03_base/11_assert_usage.t | **PASS** (76/76) | | +| 03_base/12_assert_retval.t | **PASS** (22/22) | | +| 03_base/13_assert_data.t | **PASS** (7/7) | | +| 03_base/14_kernel.t | **PASS** (6/6) | | +| 03_base/15_kernel_internal.t | **PASS** (12/12) | Fixed by DupIOHandle (Phase 4.8) | +| 03_base/16_nfa_usage.t | **PASS** (11/11) | | +| 03_base/17_detach_start.t | **PASS** (14/14) | | +| 04_drivers/01_sysrw.t | **PASS** (17/17) | Fixed by DupIOHandle + non-blocking pipe I/O | +| 05_filters/01_block.t | **PASS** (42/42) | | +| 05_filters/02_grep.t | **PASS** (48/48) | | +| 05_filters/03_http.t | PARTIAL (79/137) | HTTP::Message bytes issue | +| 05_filters/04_line.t | **PASS** (50/50) | | +| 05_filters/05_map.t | FAIL | Minor test failure | +| 05_filters/06_recordblock.t | **PASS** (36/36) | | +| 05_filters/07_reference.t | FAIL | Storable not available at test time | +| 05_filters/08_stream.t | **PASS** (24/24) | | +| 05_filters/50_stackable.t | **PASS** (29/29) | | +| 05_filters/51_reference_die.t | FAIL (0/5) | Storable not available at test time | +| 05_filters/99_filterchange.t | FAIL | Filter::Reference compilation | +| 06_queues/01_array.t | **PASS** (2047/2047) | | +| 07_exceptions/01_normal.t | **PASS** (7/7) | | +| 07_exceptions/02_turn_off.t | **PASS** (4/4) | | +| 07_exceptions/03_not_handled.t | **PASS** (8/8) | | +| 08_loops/01_explicit_loop.t | **PASS** (2/2) | | +| 08_loops/02_explicit_loop_fail.t | **PASS** (1/1) | | +| 08_loops/03_explicit_loop_poll.t | FAIL | IO::Poll not available | +| 08_loops/04_explicit_loop_envvar.t | FAIL | IO::Poll not available | +| 08_loops/05_kernel_loop.t | **PASS** (2/2) | | +| 08_loops/06_kernel_loop_poll.t | FAIL | IO::Poll not available | +| 08_loops/07_kernel_loop_fail.t | **PASS** (1/1) | | +| 08_loops/08_kernel_loop_search_poll.t | FAIL | IO::Poll not available | +| 08_loops/09_naive_loop_load.t | TODO | Feature not implemented yet | +| 08_loops/10_naive_loop_load_poll.t | TODO | Feature not implemented yet | +| 08_loops/11_double_loop.t | TODO | Feature not implemented yet | + +### Resource Tests (t/20_resources/10_perl/) + +| Test File | Result | Notes | +|-----------|--------|-------| +| aliases.t | **PASS** (14/14) | Fixed ConcurrentModificationException | +| caller_state.t | **PASS** (6/6) | | +| events.t | **PASS** (38/38) | | +| extrefs.t | **PASS** (31/31) | | +| extrefs_gc.t | **PASS** (5/5) | | +| filehandles.t | **PASS** (131/132) | Fixed by DupIOHandle; 1 TODO test | +| sessions.t | **PASS** (58/58) | | +| sids.t | **PASS** (7/7) | | +| signals.t | **PASS** (46/46) | 2 TODO skips count as pass | + +### Summary: 38 test files fully pass, 15 fail/partial + +## Remaining Issues + +### Pre-existing PerlOnJava limitations (not POE-specific) + +| Issue | Impact | Category | +|-------|--------|----------| +| CORE::GLOBAL::require override not supported | 09_resources.t | Runtime feature | +| DESTROY not called for blessed objects | wheel_readwrite, wheel_tail, wheel_sf_*, wheel_accept, ses_session (4 tests) | JVM limitation | +| IO::Poll not available | 4 loop tests | Missing module | + +### Issues worth fixing + +| Issue | Impact | Difficulty | +|-------|--------|------------| +| DESTROY workaround (Phase 4.5) | 20-30+ tests across 5+ wheel test files | Medium-Hard | +| Storable not found by POE test runner | 3 filter tests | Low (path issue?) | +| HTTP::Message bytes handling | 03_http.t (58 tests) | Medium | +| TIOCSWINSZ stub (Phase 4.6) | wheel_run, k_signals_rerun | Low | + +### Event Loop Tests (t/30_loops/select/) + +| Test File | Result | Notes | +|-----------|--------|-------| +| 00_info.t | **PASS** (2/2) | | +| all_errors.t | SKIP | | +| k_alarms.t | **PASS** (37/37) | Alarm scheduling works | +| k_aliases.t | **PASS** (20/20) | Session aliases work | +| k_detach.t | **PASS** (9/9) | Session detach works | +| k_run_returns.t | **PASS** (1/1) | | +| k_selects.t | **PASS** (17/17) | File handle watchers | +| k_sig_child.t | PARTIAL (5/15) | Child signal handling | +| k_signals.t | PARTIAL (2/8) | Signal delivery | +| k_signals_rerun.t | FAIL | | +| sbk_signal_init.t | **PASS** (1/1) | | +| ses_nfa.t | **PASS** (39/39) | NFA state machine works | +| ses_session.t | PARTIAL (37/41) | 4 failures from DESTROY count checks | +| comp_tcp.t | FAIL (0/34) | TCP networking | +| wheel_accept.t | PARTIAL (1/2) | Hangs after test 1 | +| wheel_readwrite.t | PARTIAL (16/28) | I/O events don't fire, hangs | +| wheel_run.t | PARTIAL (42/103) | 10 pass, 32 skip (IO::Pty), blocked by TIOCSWINSZ | +| wheel_sf_tcp.t | PARTIAL (4/9) | Hangs after test 4 | +| wheel_sf_udp.t | PARTIAL (4/10) | UDP datagrams never delivered | +| wheel_sf_unix.t | FAIL (0/12) | Socket factory Unix | +| wheel_tail.t | PARTIAL (4/10) | sysseek now works; hangs due to DESTROY | +| z_kogman_sig_order.t | **PASS** (7/7) | | +| z_merijn_sigchld_system.t | **PASS** (4/4) | | +| z_steinert_signal_integrity.t | **PASS** (2/2) | | +| connect_errors.t | **PASS** (3/3) | | +| k_signals_rerun.t | PARTIAL (1/9) | TIOCSWINSZ error in child processes | + +**Event loop summary**: 14/35 fully pass. Core event loop works (alarms, aliases, detach, signals, NFA). + +## Fix Plan - Remaining Phases + +### Completed Phases (1-3, 4.1-4.4, 4.8, 4.11) + +All phases through 4.4, Phase 4.8, and Phase 4.11 are complete. See Progress Tracking below for details. + +### Phase 4.5: Implement DESTROY workaround — HIGHEST REMAINING IMPACT + +**Status**: Not started +**Difficulty**: Medium-Hard +**Expected impact**: 20-30+ additional test passes across 5+ test files + +**Problem**: All POE::Wheel test hangs (wheel_readwrite, wheel_tail, wheel_sf_tcp, wheel_accept, +wheel_sf_udp) are caused by DESTROY never being called when POE::Wheel objects go out of scope. +When wheels are created in eval or deleted via `delete $heap->{wheel}`, DESTROY never fires. +This leaves orphan select() watchers and anonymous event handlers registered in the kernel, +preventing sessions from stopping. + +**Affected tests**: +- wheel_readwrite (28 tests) — constructor tests pass, I/O events work, hangs on cleanup +- wheel_tail (10 tests) — FollowTail watching works, hangs on `delete $heap->{wheel}` +- wheel_sf_tcp (9 tests) — TCP server works, hangs between phases +- wheel_accept (2 tests) — accept works, hangs on cleanup +- wheel_sf_udp (10 tests) — UDP sockets created, hangs on cleanup +- ses_session (4 of 41 tests) — explicitly count DESTROY invocations + +**POE::Wheel DESTROY cleanup pattern** (all wheels follow this): +1. Remove I/O watchers: `$poe_kernel->select_read($handle)`, `select_write($handle)` +2. Cancel timers: `$poe_kernel->delay($state_name)` (FollowTail only) +3. Remove anonymous states: `$poe_kernel->state($state_name)` +4. Free wheel ID: `POE::Wheel::free_wheel_id($id)` + +**Recommended approach**: Option A — trigger DESTROY on hash `delete`/scalar overwrite when +a blessed reference is replaced. POE's pattern is always `$heap->{wheel} = Wheel->new(...)` +with a single reference, so calling DESTROY when the hash value is overwritten or deleted is +correct 99% of the time. For safety, DESTROY should be made idempotent. + +**Implementation plan**: +1. In `RuntimeHash.delete()`: before removing a value, check if it's a blessed reference + whose class defines DESTROY. If so, call DESTROY on it. +2. In `RuntimeScalar.set()`: when overwriting a blessed reference, check for DESTROY. +3. Guard against double-DESTROY: track whether DESTROY has already been called (e.g., + a flag on the RuntimeScalar or the blessed object's internal hash). +4. DESTROY should be called in void context, catching any exceptions (like Perl does). + +**Alternative approaches** (see DESTROY Workaround Options section for full analysis): +- Option B: Scope-based tied proxy (more accurate, more complex) +- Option C: Patch POE::Wheel subclasses (fragile, no Java changes) +- Option D: Full reference counting (correct but very complex) +- Option F: POE::Kernel session GC (targeted but doesn't generalize) + +### Phase 4.6: Add TIOCSWINSZ stub — LOW EFFORT + +**Status**: Not started +**Difficulty**: Low +**Expected impact**: k_signals_rerun (8 tests), some wheel_run child process tests + +**Problem**: Wheel::Run uses `require 'sys/ioctl.ph'` inside an eval to get TIOCSWINSZ for +terminal window size. PerlOnJava doesn't have sys/ioctl.ph, causing bareword errors. + +**Implementation plan**: +1. Create `src/main/perl/lib/sys/ioctl.ph` with TIOCSWINSZ constant (0x5413 on Linux, + 0x40087468 on macOS) +2. OR: make the eval silently fail (already in an eval, just needs the require to not + blow up with a compilation error) + +**Note**: Most wheel_run tests also need fork, so the real impact is limited to +k_signals_rerun (8 of 9 failures are from TIOCSWINSZ in child processes). + +### Phase 4.7: Windows platform support — CI CRITICAL + +**Status**: Not started +**Difficulty**: Low-Medium (mostly lookup tables) + +POE's core event loop uses Java NIO (cross-platform), and POE itself has `$^O eq 'MSWin32'` +guards. But several PerlOnJava subsystems only have macOS/Linux branches. + +| Step | Target | Severity | Difficulty | +|------|--------|----------|------------| +| 4.7.1 | **Fix EAGAIN/errno on Windows** — `FFMPosixWindows.strerror()` is a 10-entry stub. `ErrnoVariable` probes strerror to discover EAGAIN/EINPROGRESS/etc., which all resolve to 0 on Windows. POE's non-blocking I/O depends on these. Fix: add Windows errno constants directly, bypassing strerror probing. | Critical | Medium | +| 4.7.2 | **Add Windows errno table** — `ErrnoHash.java` and `Errno.pm` have macOS/Linux tables only, fall through to Linux on Windows. Fix: add `MSWin32` branch with MSVC CRT errno values. | High | Low | +| 4.7.3 | **Add Windows branch to RuntimeSigHash** — `%SIG` is pre-populated with Unix signals (HUP, USR1, PIPE, etc.) that don't exist on Windows. POE iterates `keys %SIG` and tries to install handlers. Fix: Windows branch with only INT, TERM, ABRT, FPE, ILL, SEGV, BREAK. | High | Low | +| 4.7.4 | **Add Windows branch to POSIX.java** — IS_MAC flag gives macOS vs "everything else" (Linux). Windows gets Linux signal/termios values. Fix: add IS_WINDOWS flag, return Windows-correct signal constants, skip termios. | Medium | Low | +| 4.7.5 | **Fix Socket.java constants** — SOL_SOCKET=1 (Linux) vs 0xFFFF (Windows), SO_REUSEADDR=2 vs 4, etc. These matter if passed to native setsockopt. Java abstracts most of this, so impact depends on implementation path. Fix: platform-detect and use correct values. | Medium | Low | +| 4.7.6 | **Fix sysconf for Windows** — Runs `ulimit -n` which doesn't exist on Windows. Already has catch block defaulting to 1024, but should use a cleaner approach (e.g., `_getmaxstdio()` or just return 2048). | Low | Low | + +**What already works on Windows (no changes needed):** +- `select()` via Java NIO Selector — cross-platform +- Pipe handling via Java `PipedInputStream`/`PipedOutputStream` — cross-platform +- POE::Loop::Select — has `$^O eq 'MSWin32'` guards +- `socketpair` via loopback TCP — the standard Windows approach +- `$^O` correctly set to `MSWin32` on Windows + +### Phase 4.9: Storable path fix — LOW EFFORT + +**Status**: Not started +**Difficulty**: Low +**Expected impact**: 3 filter tests (07_reference.t, 51_reference_die.t, 99_filterchange.t) + +**Problem**: POE's test runner can't find Storable at test time. Storable is available in +PerlOnJava (XSLoader backend) but the test's @INC doesn't include the right path. + +**Implementation plan**: Investigate why `use Storable` fails inside POE's filter tests. +Likely needs adding the correct lib path or fixing Storable's module resolution. + +### Phase 4.10: HTTP::Message bytes handling — MEDIUM EFFORT + +**Status**: Not started +**Difficulty**: Medium +**Expected impact**: 58 additional tests in 03_http.t (79/137 currently) + +**Problem**: HTTP::Message byte-string handling has issues when processing HTTP requests/responses +through POE::Filter::HTTPD. The exact nature of the bytes vs. characters mismatch needs investigation. + +### Phase 4.12: fileno for regular file handles — BLOCKED BY DESTROY + +**Status**: Blocked (needs DESTROY/scope-exit cleanup first) +**Difficulty**: Low (implementation ready) + Hard (DESTROY prerequisite) +**Expected impact**: +3 tests (require_37033.t: tests 2, 5, 7; io/dup.t: tests 22, 24) + +**Problem**: Regular file opens don't call `assignFileno()`, so `fileno($fh)` returns undef +for regular files. The fix is trivial (add `fh.assignFileno()` in RuntimeIO.open()), but it +breaks perlio_leaks.t (12→4) because allocated fds aren't recycled without deterministic +filehandle close (DESTROY) on lexical scope exit. + +**Current workaround**: fileno returns undef for regular files, which makes `is(undef, undef)` +pass in tests that compare fd numbers of handles that were supposed to have been closed. + +**Implementation plan**: +1. Implement DESTROY or scope-exit cleanup for lexical filehandles (Phase 4.5) +2. Then add `fh.assignFileno()` in RuntimeIO.open() for regular files, JAR handles, and scalar-backed handles +3. Verify both perlio_leaks.t (12/12) and require_37033.t (+3) pass + +### Phase 5: JVM limitations (not fixable without major work) + +| Feature | Reason | Tests affected | +|---------|--------|----------------| +| fork() | JVM cannot fork | k_sig_child (10), k_signals (6), wheel_run IO::Pty tests | +| IO::Tty / IO::Pty | XS module, needs C compiler | wheel_run (32 skip), wheel_curses, wheel_readline | +| DESTROY count accuracy | JVM tracing GC, not refcounting | ses_session (4 tests check exact DESTROY counts) | +| CORE::GLOBAL::require | Not implemented | 09_resources.t (6 tests) | + +## Progress Tracking + +### Current Status: Phase 4.11 complete — DESTROY workaround (Phase 4.5) is next highest impact + +### Completed Phases +- [x] Phase 1: Initial analysis (2026-04-04) + - Ran `./jcpan -t POE`, identified 7 root causes + - ~15/97 tests pass, all failures traced to root causes +- [x] Phase 1: Fix blockers (2026-04-04, commit 743c26461) + - Fixed exists(&sub) constant folding bypass + - Added Socket IPPROTO constants + - Set Symbol.pm $VERSION + - Added POSIX errno/signal constants, uname(), sigprocmask() + - Fixed sigprocmask return value for POE +- [x] Phase 2: Core fixes (2026-04-04, commit 76bf09bd9) + - Fixed indirect object syntax with variable class + parenthesized args + - Fixed ConcurrentModificationException in hash each() iteration + - 35/53 unit+resource tests fully pass, 10/35 event loop tests fully pass +- [x] Phase 3.1: Session lifecycle fixes (2026-04-04, commits ba803dc49, 338bd4a90, f79f9f6e8) + - Pre-populated %SIG with OS signal names (Bug 8) + - Implemented DESTROY for blessed objects via java.lang.ref.Cleaner (Bug 9) + - Fixed foreach to see array modifications during iteration (Bug 10) + - ses_session.t: 7/41 → 35/41 (28 new passing tests) + - Event loop restart, session tree walk, postbacks/callbacks all work +- [x] Phase 3.2: I/O and parser fixes (2026-04-04, commits 6b9fa2c30, cddf4b121, 2777d2e46) + - Fixed `require File::Spec->catfile(...)` parser bug (Bug 11) — enables Time::HiRes dynamic loading + - Added non-blocking I/O for pipe handles (Bug 12) — POE signal pipe no longer blocks + - Fixed IO::Handle::blocking() argument passing (shift vs @_ glob copy issue) + - Added 4-arg select() and FileDescriptorTable for I/O multiplexing + - Fixed DestroyManager blessId collision with overloaded classes (Bug 13) + - Removed DestroyManager — proxy reconstruction too fragile (close() corrupts proxy hash) + - Updated dev/design/object_lifecycle.md with findings +- [x] Phase 3.3: select() polling fix (2026-04-04, commit f119640a5) + - Fixed 4-arg select() to poll pipe readiness instead of marking always ready (Bug 14) + - select() now properly blocks when monitoring InternalPipeHandle with timeout + - POE event loop no longer busy-loops; timer-based events fire correctly +- [x] Phase 3.4: Signal pipe and postback fixes (2026-04-04, commit eff2f356d) + - Fixed pipe fd registry mismatch (Bug 15) — pipe() created RuntimeIO objects but never + registered them in RuntimeIO.filenoToIO, making pipes invisible to select(). Added + registerExternalFd() to RuntimeIO and InternalPipeHandle.getFd() getter. + - Fixed platform EAGAIN value (Bug 16) — InternalPipeHandle.sysread() hard-coded errno 11 + (Linux EAGAIN). On macOS EAGAIN=35, causing POE to see "Resource deadlock avoided" instead + of EAGAIN. Fixed to use ErrnoVariable.EAGAIN() for platform-correct values. + - Patched POE::Session postback/callback auto-cleanup (Bug 17) — DESTROY won't fire on + PerlOnJava, so postbacks/callbacks now auto-decrement session refcount when called. + This allows sessions to exit properly without relying on DESTROY. + - ses_session.t: 7/41 → 37/41 (signal delivery + postback cleanup working) + - ses_nfa.t: 39/39 (perfect), k_alarms.t: 37/37 (perfect), k_aliases.t: 20/20 (perfect) +- [x] Phase 3.5: select() and fd allocation fixes (2026-04-04, commit b995a5f81) + - Fixed select() bitvector write-back (Bug 18) — 4-arg select() copied bitvector args + but never wrote results back, causing callers to see unchanged bitvectors + - Fixed fd allocation collision (Bug 19) — FileDescriptorTable and RuntimeIO had + separate fd counters; socket/socketpair advanced one but not the other, causing + pipe() to allocate fds overlapping with existing sockets + - Fixed socketpair() stream init (Bug 20) — SocketIO created without input/output + streams, causing sysread() to fail. Also improved sysread() channel fallback. + - k_selects.t: 5/17 → 17/17 (all pass) +- [x] Phase 4.1: Socket pack_sockaddr_un/unpack_sockaddr_un stubs (2026-04-05, commit a15fbce47) + - Added stub implementations for AF_UNIX sockaddr packing/unpacking + - Registered in Socket.java and Socket.pm @EXPORT + - Unblocked POE::Wheel::SocketFactory loading + - connect_errors: 3/3 PASS, wheel_sf_tcp: 4/9 (hangs), wheel_accept: 1/2 (hangs) +- [x] Phase 4.2: POSIX terminal/file constants (2026-04-05, commit 34934234d) + - Added 80+ POSIX constants: stat permissions, terminal I/O, baud rates, sysconf + - Added S_IS* file type functions as pure Perl (S_ISBLK, S_ISCHR, S_ISDIR, etc.) + - Added setsid() and sysconf(_SC_OPEN_MAX) implementations + - Platform-aware macOS/Linux detection for all platform-dependent constants + - Wheel::FollowTail loads (4/10 pass), Wheel::Run loads (42/103: 6 pass, 36 skip) +- [x] Phase 4.3: fileno fix + sysseek + event loop analysis (2026-04-04, commits 14ea123a9, 5b0ca1383) + - Fixed fileno() returning undef for regular file handles (Bug 21) — open() paths for + regular files, JAR resources, scalar-backed handles, and pipes all created IO handles + without calling assignFileno(), making fileno($fh) return undef. Added assignFileno() + calls in all four open paths in RuntimeIO.java. + - Implemented sysseek operator for JVM backend (Bug 22) — sysseek was only in the + interpreter backend. Added JVM support via CoreOperatorResolver, EmitBinaryOperatorNode, + OperatorHandler, and CompileBinaryOperator. Returns new position (or "0 but true"), + unlike seek which returns 1/0. + - Analyzed event loop I/O hang pattern — root cause is DESTROY (see below) +- [x] Phase 4.8: Refcounted filehandle duplication (2026-04-05, commits 490c53f89, 116d88c7a) + - Fixed non-blocking syswrite for pipe handles — writer checks buffer capacity, returns + EAGAIN when full. Added shared writerClosedFlag for EOF detection. + - Added EBADF errno support — sysread/syswrite on closed/invalid handles now set $! to EBADF + - Created DupIOHandle class — refcounted IOHandle wrapper enabling proper Perl dup semantics. + Each dup'd handle has independent closed state and fd number; underlying resource only + closed when last dup is closed. Original handle preserves its fileno after duplication. + - Fixed findFileHandleByDescriptor() to check RuntimeIO's fileno registry — dup'd handles + registered via registerExternalFd() were invisible to open-by-fd-number (e.g., open($fh, ">&6")) + - Added FileDescriptorTable.registerAt() and nextFdValue() methods + - 01_sysrw.t: 15/17 → 17/17 (dup/close cycle works) + - 15_kernel_internal.t: 7/12 → 12/12 (fd management works) + - filehandles.t: 1/132 → 131/132 (only 1 TODO test fails) + - signals.t: 45/46 → 46/46 +- [x] Phase 4.11: IO infrastructure fixes (2026-04-04) + - Fixed ClosedIOHandle.fileno() to return undef (not false) — matches Perl 5 semantics + - Wired DupIOHandle into duplicateFileHandle() — proper reference-counted dup with distinct + filenos. Handles that were already DupIOHandles use addDup(); first-time dups use createPair(). + Original handle's fd is preserved (queried from IOHandle.fileno() for StandardIO). + - Fixed findFileHandleByDescriptor() to also check RuntimeIO.getByFileno() registry — enables + open-by-fd-number (e.g., `open($fh, ">&5")`) to find dup'd handles registered via registerExternalFd() + - Fixed openFileHandleDup() to use glob table for STDIN/STDOUT/STDERR instead of static + RuntimeIO.stdout/stdin/stderr fields — prevents stale handle references after redirections + - Added BorrowedIOHandle usage in all parsimonious dup paths (3-arg open with &= mode, + 2-arg open with &= mode, and blessed object path) — close on parsimonious dup no longer + kills the original handle + - Fixed double-FETCH on tied $@ in WarnDie.warn() — Perl 5 fetches $@ exactly once when + warn() is called with no arguments; PerlOnJava was fetching twice (getDefinedBoolean + toString) + - Added InternalPipeHandle.hasDataAvailable() for select() readiness checking + - Added fd recycling pool (ConcurrentLinkedQueue) in RuntimeIO for POSIX-like fd reuse + - Files changed: ClosedIOHandle.java, IOOperator.java (duplicateFileHandle, findFileHandleByDescriptor, + openFileHandleDup, parsimonious dup paths), WarnDie.java, InternalPipeHandle.java, + DupIOHandle.java, BorrowedIOHandle.java, RuntimeIO.java + - tie_fetch_count.t: 175/343 → 178/343 (+3 from warn double-FETCH fix) + - perlio_leaks.t: 4/12 → 12/12 (fd recycling) + - All other tests at baseline (no regressions) + +### Key Findings (Phase 3.1-3.4) +- **foreach-push pattern**: Perl's foreach dynamically sees elements pushed during iteration. + PerlOnJava's RuntimeArrayIterator was caching size at creation. This broke POE::Kernel->stop() + which walks the session tree by pushing children during foreach. +- **DESTROY not feasible via GC**: Java's Cleaner/GC-based DESTROY is unreliable across JVM + implementations and cannot guarantee deterministic timing. Perl's DESTROY depends on + reference counting (fires immediately when last reference drops). The JVM's tracing GC + is fundamentally incompatible with this semantic. DESTROY is not implemented. +- **DESTROY workaround for POE**: POE::Session::AnonEvent postbacks/callbacks now auto-cleanup + when called (decrement session refcount on first invocation). This replaces DESTROY-based + cleanup for the common one-shot postback pattern. 4 ses_session.t failures remain for tests + that explicitly count DESTROY invocations. +- **Dual fd registry**: pipe() registered handles in FileDescriptorTable but not RuntimeIO.filenoToIO. + select() only consulted RuntimeIO, making pipes invisible. Fixed by registerExternalFd(). +- **Fd counter collision**: FileDescriptorTable and RuntimeIO maintained separate fd counters. + socketpair/socket advanced RuntimeIO.nextFileno but not FileDescriptorTable.nextFd, causing + pipe() to allocate fds that overlapped with existing socket fds. Fixed by cross-synchronizing + both counters (advancePast/advanceFilenoCounterPast). +- **select() bitvector write-back**: 4-arg select() created snapshot copies of bitvector args + for tied-variable safety, but never wrote the modified bitvectors back to the original + variables. Callers always saw their original bitvectors unchanged. Fixed to write back + after selectWithNIO returns. +- **socketpair() stream init**: socketpair() used SocketIO(channel, family) constructor which + doesn't initialize inputStream/outputStream. sysread() failed with "No input stream + available" on blocking sockets. Fixed to use SocketIO(socket) constructor, and made + sysread() fall back to channel-based I/O when streams are unavailable. +- **Platform errno**: Hard-coded Linux errno values (EAGAIN=11) caused mismatches on macOS + (EAGAIN=35). Fixed to use ErrnoVariable.EAGAIN() which probes the platform. +- **Signal delivery**: Now works end-to-end: kill() → %SIG handler → signal pipe write → + select() detects pipe → POE dispatches signal event. +- **require expression parsing**: `require File::Spec->catfile(...)` was parsed as + `require File::Spec` (module) instead of `require `. This prevented Time::HiRes + from loading, causing monotime() to return integer seconds instead of float. + +### Key Findings (Phase 4.3) — DESTROY Root Cause Analysis + +**The event loop I/O hang is caused by POE::Wheel DESTROY not being called**, not by +I/O subsystem bugs. The underlying I/O works correctly: + +- `fileno()` now works for all handle types (regular files, JARs, scalars, pipes) +- `select()` correctly detects readability on regular file handles +- `sysread()`/`syswrite()` work on tmpfiles, pipes, and sockets +- `sysseek()` works correctly (returns position, "0 but true" at offset 0) +- POE::Wheel::ReadWrite I/O events fire correctly in isolation + +**Verified working in isolation**: A standalone POE session with a ReadWrite wheel on a +tmpfile receives all input events, flushes writes, and completes. The state machine +(read→pause→seek→resume→write→shutdown) works as expected. + +**What fails**: When wheels are created in eval (test_new validation) or deleted via +`delete $heap->{wheel}`, DESTROY never fires. This leaves orphan select() watchers +and anonymous event handlers registered in the kernel, preventing sessions from stopping. + +**POE::Wheel DESTROY cleanup pattern** (all wheels follow this): +1. Remove I/O watchers: `$poe_kernel->select_read($handle)`, `select_write($handle)` +2. Cancel timers: `$poe_kernel->delay($state_name)` (FollowTail only) +3. Remove anonymous states: `$poe_kernel->state($state_name)` +4. Free wheel ID: `POE::Wheel::free_wheel_id($id)` + +**Specific test impacts:** +- **wheel_readwrite.t test 12**: `ReadWrite->new(Handle=>\*DATA, LowMark=>3, HighMark=>8, + LowEvent=>"low")` succeeds when it should die (missing HighEvent validation in POE). + The wheel registers a select watcher on \*DATA, and without DESTROY, it's never removed. + This keeps Part 1's session alive forever, preventing Part 2 from running. +- **wheel_tail.t**: FollowTail file-based watching works (creates file, reads lines, detects + resets), but the session hangs after `delete $heap->{wheel}` because the FollowTail's + delay timer and select watcher aren't cleaned up. +- **wheel_sf_tcp.t**, **wheel_accept.t**: TCP server accepts connections (test 1 passes), + but subsequent wheel lifecycle depends on DESTROY for cleanup between phases. + +### DESTROY Workaround Options (Not Yet Implemented) + +| Option | Approach | Pros | Cons | +|--------|----------|------|------| +| A | **Trigger DESTROY on `delete`/`set`** — when overwriting a blessed reference in a hash/scalar, check if the class defines DESTROY and call it | Simple to implement; covers the `delete $heap->{wheel}` pattern | May call DESTROY too early if other references exist; not refcount-accurate | +| B | **Scope-based cleanup via tied proxy** — wrap wheel references in a tied scalar that calls DESTROY when the scalar is overwritten | More accurate lifecycle tracking | Complex; requires patching POE internals | +| C | **Patch POE::Wheel subclasses** — add explicit `_cleanup()` methods, patch POE::Kernel to call them when sessions stop | Clean, no Java changes needed | Fragile; must patch every Wheel subclass | +| D | **Implement reference counting** — track refcounts for blessed objects at the RuntimeScalar level | Correct Perl 5 semantics | Very complex; massive changes to RuntimeScalar, affects performance | +| E | **GC-based DESTROY with Cleaner** — register cleanup actions with java.lang.ref.Cleaner, run when GC collects | No refcounting needed | Unpredictable timing; previously attempted and reverted | +| F | **POE::Kernel session GC** — when a session has no pending events/timers *except* select watchers, force-remove all watchers | Targeted to POE's specific deadlock pattern | Doesn't generalize; requires understanding POE's invariants | + +**Recommended approach**: Option A (trigger DESTROY on delete/set) is the most pragmatic. +POE's pattern is always `$heap->{wheel} = Wheel->new(...)` with a single reference, so +calling DESTROY when the hash value is overwritten or deleted is correct 99% of the time. +For safety, DESTROY could be made idempotent (track whether it's already been called). + +### Next Steps + +**Priority order:** +1. **Phase 4.5: DESTROY workaround** — highest remaining impact (20-30+ tests). Recommended + approach is Option A (trigger DESTROY on `delete`/`set` when overwriting a blessed reference). + Implementation plan: + - In `RuntimeHash.delete()`: check if removed value is a blessed ref with DESTROY, call it + - In `RuntimeScalar.set()`: when overwriting a blessed ref, check for DESTROY + - Guard against double-DESTROY: track whether DESTROY already called (flag on blessed object) + - DESTROY should be called in void context, catching any exceptions + - This also unblocks Phase 4.12 (fileno for regular files + scope-exit cleanup) +2. **Phase 4.6: TIOCSWINSZ stub** — low effort, unblocks k_signals_rerun (8 tests). + Create `src/main/perl/lib/sys/ioctl.ph` with TIOCSWINSZ constant. +3. **Phase 4.9: Storable path fix** — low effort, 3 filter tests. + Investigate why `use Storable` fails inside POE's filter tests (likely @INC path issue). +4. **Phase 4.10: HTTP::Message bytes** — medium effort, 58 tests in 03_http.t. + Investigate bytes vs characters mismatch in HTTP::Message processing. +5. **Phase 4.12: fileno for regular files** — blocked by Phase 4.5 (DESTROY). + Trivial implementation (add `assignFileno()` in RuntimeIO.open()) but requires DESTROY + for fd recycling. Will gain +3 tests in require_37033.t and io/dup.t. +6. **Phase 4.7: Windows platform support** — CI critical. See detailed sub-steps above. + +## Related Documents +- `dev/modules/smoke_test_investigation.md` - Symbol $VERSION pattern +- `dev/modules/io_stringy.md` - IO module porting patterns +- `dev/design/object_lifecycle.md` - DESTROY and object lifecycle design diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index 5d718bcd5..05f9813a7 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,14 +33,14 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "a7261e446"; + public static final String gitCommitId = "3223df950"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitDate = "2026-04-04"; + public static final String gitCommitDate = "2026-04-05"; // Prevent instantiation private Configuration() { diff --git a/src/main/java/org/perlonjava/runtime/io/BorrowedIOHandle.java b/src/main/java/org/perlonjava/runtime/io/BorrowedIOHandle.java new file mode 100644 index 000000000..0427f4323 --- /dev/null +++ b/src/main/java/org/perlonjava/runtime/io/BorrowedIOHandle.java @@ -0,0 +1,188 @@ +package org.perlonjava.runtime.io; + +import org.perlonjava.runtime.runtimetypes.RuntimeScalar; + +import java.nio.charset.Charset; + +import static org.perlonjava.runtime.runtimetypes.RuntimeIO.handleIOException; +import static org.perlonjava.runtime.runtimetypes.RuntimeScalarCache.scalarTrue; + +/** + * A non-owning IOHandle wrapper for Perl's parsimonious dup semantics ({@code >&=} / {@code <&=}). + * + *

Background: parsimonious dup in Perl

+ *

When Perl executes {@code open(F, ">&=STDOUT")}, it performs an {@code fdopen()} — + * creating a new FILE* that shares the same fd as STDOUT. The key semantic difference + * from a full dup ({@code >&}) is:

+ * + * + *

Implementation

+ *

BorrowedIOHandle delegates all I/O operations to the underlying delegate IOHandle, + * but overrides {@link #close()} to only flush — never closing the delegate. This + * ensures that after {@code close F}, the original handle (e.g. STDOUT) keeps working.

+ * + *

Unlike {@link DupIOHandle}, this wrapper:

+ * + * + * @see DupIOHandle for full dup semantics ({@code >&}) with reference counting + * @see IOOperator#openFileHandleDup(String, String) where this is created + */ +public class BorrowedIOHandle implements IOHandle { + + /** The underlying handle we're borrowing — never closed by us. */ + private final IOHandle delegate; + /** Per-instance closed flag. Once true, all I/O operations on THIS wrapper fail. */ + private boolean closed = false; + + /** + * Creates a BorrowedIOHandle wrapping the given delegate. + * + * @param delegate the underlying IOHandle to borrow (not owned — will not be closed) + */ + public BorrowedIOHandle(IOHandle delegate) { + this.delegate = delegate; + } + + /** + * Returns the underlying delegate IOHandle. + */ + public IOHandle getDelegate() { + return delegate; + } + + // ---- Delegated I/O operations (check closed state first) ---- + + @Override + public RuntimeScalar write(String string) { + if (closed) return handleClosed("write"); + return delegate.write(string); + } + + @Override + public RuntimeScalar flush() { + if (closed) return scalarTrue; + return delegate.flush(); + } + + @Override + public RuntimeScalar sync() { + if (closed) return scalarTrue; + return delegate.sync(); + } + + @Override + public RuntimeScalar doRead(int maxBytes, Charset charset) { + if (closed) return handleClosed("read"); + return delegate.doRead(maxBytes, charset); + } + + @Override + public RuntimeScalar fileno() { + if (closed) return handleClosed("fileno"); + // Return the delegate's fileno — parsimonious dup shares the same fd + return delegate.fileno(); + } + + @Override + public RuntimeScalar eof() { + if (closed) return scalarTrue; + return delegate.eof(); + } + + @Override + public RuntimeScalar tell() { + if (closed) return handleClosed("tell"); + return delegate.tell(); + } + + @Override + public RuntimeScalar seek(long pos, int whence) { + if (closed) return handleClosed("seek"); + return delegate.seek(pos, whence); + } + + @Override + public RuntimeScalar truncate(long length) { + if (closed) return handleClosed("truncate"); + return delegate.truncate(length); + } + + @Override + public RuntimeScalar flock(int operation) { + if (closed) return handleClosed("flock"); + return delegate.flock(operation); + } + + @Override + public RuntimeScalar bind(String address, int port) { + if (closed) return handleClosed("bind"); + return delegate.bind(address, port); + } + + @Override + public RuntimeScalar connect(String address, int port) { + if (closed) return handleClosed("connect"); + return delegate.connect(address, port); + } + + @Override + public RuntimeScalar listen(int backlog) { + if (closed) return handleClosed("listen"); + return delegate.listen(backlog); + } + + @Override + public RuntimeScalar accept() { + if (closed) return handleClosed("accept"); + return delegate.accept(); + } + + @Override + public RuntimeScalar sysread(int length) { + if (closed) return handleClosed("sysread"); + return delegate.sysread(length); + } + + @Override + public RuntimeScalar syswrite(String data) { + if (closed) return handleClosed("syswrite"); + return delegate.syswrite(data); + } + + // ---- Close: flush only, do NOT close the delegate ---- + + /** + * Closes this borrowed handle. + * + *

Only flushes the delegate — does NOT close the underlying resource. + * This matches Perl's fdopen semantics where closing an fdopen'd FILE* + * does not invalidate the original handle.

+ */ + @Override + public RuntimeScalar close() { + if (closed) { + return handleIOException( + new java.io.IOException("Handle is already closed."), + "Handle is already closed."); + } + closed = true; + // Only flush — never close the delegate. The original handle still owns it. + delegate.flush(); + return scalarTrue; + } + + private RuntimeScalar handleClosed(String operation) { + return handleIOException( + new java.io.IOException("Cannot " + operation + " on a closed handle."), + operation + " on closed handle failed"); + } +} diff --git a/src/main/java/org/perlonjava/runtime/io/ClosedIOHandle.java b/src/main/java/org/perlonjava/runtime/io/ClosedIOHandle.java index 71846409f..bc1327510 100644 --- a/src/main/java/org/perlonjava/runtime/io/ClosedIOHandle.java +++ b/src/main/java/org/perlonjava/runtime/io/ClosedIOHandle.java @@ -3,8 +3,7 @@ import org.perlonjava.runtime.runtimetypes.RuntimeIO; import org.perlonjava.runtime.runtimetypes.RuntimeScalar; -import static org.perlonjava.runtime.runtimetypes.RuntimeScalarCache.scalarFalse; -import static org.perlonjava.runtime.runtimetypes.RuntimeScalarCache.scalarTrue; +import static org.perlonjava.runtime.runtimetypes.RuntimeScalarCache.*; public class ClosedIOHandle implements IOHandle { @@ -27,7 +26,8 @@ public RuntimeScalar flush() { @Override public RuntimeScalar fileno() { - return RuntimeIO.handleIOError("Cannot get file number from a closed handle."); + // Perl 5: fileno() on a closed handle returns undef (not false) + return scalarUndef; } @Override diff --git a/src/main/java/org/perlonjava/runtime/io/DupIOHandle.java b/src/main/java/org/perlonjava/runtime/io/DupIOHandle.java new file mode 100644 index 000000000..d37945028 --- /dev/null +++ b/src/main/java/org/perlonjava/runtime/io/DupIOHandle.java @@ -0,0 +1,282 @@ +package org.perlonjava.runtime.io; + +import org.perlonjava.runtime.runtimetypes.RuntimeScalar; + +import java.nio.charset.Charset; +import java.util.concurrent.atomic.AtomicInteger; + +import static org.perlonjava.runtime.runtimetypes.RuntimeIO.handleIOException; +import static org.perlonjava.runtime.runtimetypes.RuntimeScalarCache.*; + +/** + * A reference-counted IOHandle wrapper that enables proper filehandle duplication + * semantics — the Java equivalent of POSIX {@code dup(2)}. + * + *

Background: Perl's filehandle duplication

+ *

When Perl executes {@code open(SAVE, ">&STDERR")}, it creates a new file descriptor + * (via dup()) that shares the same underlying file description. Both fds are independent: + * closing one does not affect the other. The underlying OS resource (file, pipe, socket) + * is only released when ALL duplicates are closed.

+ * + *

Java doesn't expose POSIX file descriptors, so we simulate this behavior with + * reference-counted wrappers around an underlying {@link IOHandle} delegate.

+ * + *

Architecture

+ * 
+ *   ┌──────────────────────┐     ┌──────────────────────┐
+ *   │ DupIOHandle (fd=1)   │     │ DupIOHandle (fd=5)   │
+ *   │ closed=false         │     │ closed=false         │
+ *   │ refCount ─────────────┼─────┼─▶ AtomicInteger(2)  │
+ *   │ delegate ─────────────┼─────┼─▶ StandardIO         │
+ *   └──────────────────────┘     └──────────────────────┘
+ *                                      (shared)
+ *
+ * + *

Lifecycle

+ *
    + *
  1. Creation via {@link #createPair(IOHandle, int)}: Called the first time + * a handle is duplicated. Creates two DupIOHandles sharing the same delegate and + * a new {@code AtomicInteger(2)} refCount. The first wrapper preserves the + * original's fd; the second gets a new fd from {@link FileDescriptorTable}.
    2. + *
  2. Subsequent dups via {@link #addDup(DupIOHandle)}: Increments the shared + * refCount and creates a new DupIOHandle with a new fd. No limit on how many + * dups can be created.
    3. + *
  3. Close: Each DupIOHandle tracks its own {@code closed} flag. On close: + *
      + *
    • Marks itself as closed (further I/O operations will fail)
      • + *
    • Unregisters its fd from {@link FileDescriptorTable}
      • + *
    • Decrements the shared refCount
      • + *
    • If refCount reaches 0 (last dup closed), actually closes the delegate
      • + *
    • If refCount > 0 (other dups still open), just flushes the delegate
      • + *
    + *
    4. + *
+ * + *

Thread safety

+ *

The refCount uses {@link AtomicInteger} for thread-safe decrement. The {@code closed} + * flag is per-instance and not synchronized — this matches the Perl model where each + * filehandle is used by a single thread. If concurrent access is needed in the future, + * the closed flag should be made volatile or synchronized.

+ * + *

fd number management

+ *

Each DupIOHandle holds a synthetic fd number assigned by {@link FileDescriptorTable}. + * The {@link #fileno()} method returns this fd (not the delegate's fd), so each duplicate + * has a unique fileno as Perl expects. This fd is registered in FileDescriptorTable for + * lookup by select() and in RuntimeIO for lookup by {@code findFileHandleByDescriptor()}.

+ * + * @see IOOperator#duplicateFileHandle(RuntimeIO) where DupIOHandles are created + * @see IOOperator#openFileHandleDup(String, String) entry point for Perl's open() dup modes + * @see FileDescriptorTable synthetic fd allocation and lookup + */ +public class DupIOHandle implements IOHandle { + + /** The underlying I/O implementation (StandardIO, FileIOHandle, etc.) — never another DupIOHandle. */ + private final IOHandle delegate; + /** Shared across all dups of the same delegate. Decremented on close; delegate closed at zero. */ + private final AtomicInteger refCount; + /** Per-instance closed flag. Once true, all I/O operations on THIS dup return errors. */ + private boolean closed = false; + /** Synthetic fd number unique to this dup, assigned by FileDescriptorTable. */ + private final int fd; + + /** + * Creates a DupIOHandle wrapping the given delegate with a shared refcount. + * Allocates a new fd from FileDescriptorTable. + */ + DupIOHandle(IOHandle delegate, AtomicInteger refCount) { + this.delegate = delegate; + this.refCount = refCount; + this.fd = FileDescriptorTable.register(this); + } + + /** + * Creates a DupIOHandle wrapping the given delegate with a shared refcount + * and an explicit fd number (used to preserve the original handle's fileno). + */ + DupIOHandle(IOHandle delegate, AtomicInteger refCount, int explicitFd) { + this.delegate = delegate; + this.refCount = refCount; + this.fd = explicitFd; + FileDescriptorTable.registerAt(explicitFd, this); + } + + /** + * Creates a pair of DupIOHandles sharing the same delegate and refcount. + * The first handle preserves the original's fileno; the second gets a new fd. + * + * @param delegate the underlying IOHandle to share + * @param originalFd the fd number to preserve for the original handle + * @return array of two DupIOHandles [forOriginal, forDuplicate] + */ + public static DupIOHandle[] createPair(IOHandle delegate, int originalFd) { + AtomicInteger refCount = new AtomicInteger(2); + DupIOHandle a = new DupIOHandle(delegate, refCount, originalFd); + DupIOHandle b = new DupIOHandle(delegate, refCount); + return new DupIOHandle[]{a, b}; + } + + /** + * Creates an additional duplicate sharing the same delegate and refcount + * as an existing DupIOHandle. + * + * @param existing an existing DupIOHandle to share with + * @return a new DupIOHandle sharing the same delegate + */ + public static DupIOHandle addDup(DupIOHandle existing) { + existing.refCount.incrementAndGet(); + return new DupIOHandle(existing.delegate, existing.refCount); + } + + /** + * Returns the file descriptor number for this duplicate. + */ + public int getFd() { + return fd; + } + + /** + * Returns the underlying delegate IOHandle. + */ + public IOHandle getDelegate() { + return delegate; + } + + // ---- Delegated I/O operations (check closed state first) ---- + + @Override + public RuntimeScalar write(String string) { + if (closed) return handleClosed("write"); + return delegate.write(string); + } + + @Override + public RuntimeScalar flush() { + if (closed) return scalarFalse; + return delegate.flush(); + } + + @Override + public RuntimeScalar sync() { + if (closed) return scalarFalse; + return delegate.sync(); + } + + @Override + public RuntimeScalar doRead(int maxBytes, Charset charset) { + if (closed) return handleClosed("read"); + return delegate.doRead(maxBytes, charset); + } + + @Override + public RuntimeScalar fileno() { + if (closed) return handleClosed("fileno"); + return new RuntimeScalar(fd); + } + + @Override + public RuntimeScalar eof() { + if (closed) return scalarTrue; + return delegate.eof(); + } + + @Override + public RuntimeScalar tell() { + if (closed) return handleClosed("tell"); + return delegate.tell(); + } + + @Override + public RuntimeScalar seek(long pos, int whence) { + if (closed) return handleClosed("seek"); + return delegate.seek(pos, whence); + } + + @Override + public RuntimeScalar truncate(long length) { + if (closed) return handleClosed("truncate"); + return delegate.truncate(length); + } + + @Override + public RuntimeScalar flock(int operation) { + if (closed) return handleClosed("flock"); + return delegate.flock(operation); + } + + @Override + public RuntimeScalar bind(String address, int port) { + if (closed) return handleClosed("bind"); + return delegate.bind(address, port); + } + + @Override + public RuntimeScalar connect(String address, int port) { + if (closed) return handleClosed("connect"); + return delegate.connect(address, port); + } + + @Override + public RuntimeScalar listen(int backlog) { + if (closed) return handleClosed("listen"); + return delegate.listen(backlog); + } + + @Override + public RuntimeScalar accept() { + if (closed) return handleClosed("accept"); + return delegate.accept(); + } + + @Override + public RuntimeScalar sysread(int length) { + if (closed) return handleClosed("sysread"); + return delegate.sysread(length); + } + + @Override + public RuntimeScalar syswrite(String data) { + if (closed) return handleClosed("syswrite"); + return delegate.syswrite(data); + } + + // ---- Close with reference counting ---- + + /** + * Closes this duplicate handle. + * + *

Semantics: + *

    + *
  1. If already closed, returns an error (matches Perl's "close on closed fh").
    2. + *
  2. Marks this instance as closed — subsequent I/O operations will fail.
    3. + *
  3. Unregisters this fd from {@link FileDescriptorTable}.
    4. + *
  4. Decrements the shared refCount atomically.
    5. + *
  5. If this was the last duplicate (refCount → 0), closes the delegate.
    6. + *
  6. Otherwise, just flushes the delegate to ensure buffered data is written.
    7. + *
+ */ + @Override + public RuntimeScalar close() { + if (closed) { + return handleIOException( + new java.io.IOException("Handle is already closed."), + "Handle is already closed."); + } + closed = true; + FileDescriptorTable.unregister(fd); + + int remaining = refCount.decrementAndGet(); + if (remaining <= 0) { + // Last reference — actually close the underlying handle + return delegate.close(); + } + // Other duplicates still open — just flush, don't close the delegate + delegate.flush(); + return scalarTrue; + } + + private RuntimeScalar handleClosed(String operation) { + return handleIOException( + new java.io.IOException("Cannot " + operation + " on a closed handle."), + operation + " on closed handle failed"); + } +} diff --git a/src/main/java/org/perlonjava/runtime/io/FileDescriptorTable.java b/src/main/java/org/perlonjava/runtime/io/FileDescriptorTable.java new file mode 100644 index 000000000..bb96cba9a --- /dev/null +++ b/src/main/java/org/perlonjava/runtime/io/FileDescriptorTable.java @@ -0,0 +1,180 @@ +package org.perlonjava.runtime.io; + +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicInteger; + +import org.perlonjava.runtime.runtimetypes.RuntimeIO; + +/** + * Maps simulated file descriptor numbers to {@link IOHandle} objects. + * + *

Why this exists

+ *

Java doesn't expose real POSIX file descriptors. Perl code, however, relies on + * numeric fd values for operations like {@code fileno()}, {@code select()}, and + * {@code open(FH, ">&=", $fd)}. This table assigns sequential integers starting + * from 3 (0, 1, 2 are reserved for stdin, stdout, stderr) and provides fd→IOHandle + * lookup.

+ * + *

Relationship to other fd registries

+ *

There are three fd-related registries in the system, each serving a + * different purpose:

+ * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + *
RegistryMapsUsed by
{@code FileDescriptorTable} (this class)fd → {@link IOHandle}{@code select()}, {@link DupIOHandle} registration
{@code RuntimeIO.filenoToIO}fd → {@link RuntimeIO}{@link IOOperator#findFileHandleByDescriptor(int)} fallback
{@code IOOperator.fileDescriptorMap}fd → {@link RuntimeIO}{@link IOOperator#findFileHandleByDescriptor(int)} first check
+ * + *

These registries are kept in sync via {@link #advancePast(int)} and + * {@link RuntimeIO#advanceFilenoCounterPast(int)} to prevent fd collisions between + * handles allocated by different subsystems (e.g., DupIOHandle vs. socket/pipe).

+ * + *

Thread safety

+ *

Uses {@link ConcurrentHashMap} and {@link AtomicInteger} for thread-safe access.

+ * + * @see DupIOHandle uses this table for fd allocation and registration + * @see IOOperator uses this table indirectly via DupIOHandle + * @see RuntimeIO parallel fd→RuntimeIO registry for higher-level lookups + */ +public class FileDescriptorTable { + + /** Next fd to allocate. Starts at 3 (0=stdin, 1=stdout, 2=stderr are reserved). */ + private static final AtomicInteger nextFd = new AtomicInteger(3); + + /** Forward map: fd number → IOHandle. Used by select() to find handles from fd bits. */ + private static final ConcurrentHashMap fdToHandle = new ConcurrentHashMap<>(); + + /** + * Reverse map: IOHandle identity hash → fd number. + * Prevents assigning multiple fds to the same IOHandle object (identity, not equality). + */ + private static final ConcurrentHashMap handleToFd = new ConcurrentHashMap<>(); + + /** + * Register an IOHandle and return its FD number. + * If the handle was already registered, returns the existing FD. + * + * @param handle the IOHandle to register + * @return the file descriptor number + */ + public static int register(IOHandle handle) { + int identity = System.identityHashCode(handle); + Integer existing = handleToFd.get(identity); + if (existing != null) { + return existing; + } + int fd = nextFd.getAndIncrement(); + fdToHandle.put(fd, handle); + handleToFd.put(identity, fd); + // Keep RuntimeIO in sync to prevent fd collisions with socket()/socketpair() + RuntimeIO.advanceFilenoCounterPast(fd); + return fd; + } + + /** + * Register an IOHandle at a specific FD number. + * Used when wrapping an existing handle (e.g., for dup) to preserve its fd number. + * Replaces any existing handle at that fd. + * + * @param fd the file descriptor number to register at + * @param handle the IOHandle to register + */ + public static void registerAt(int fd, IOHandle handle) { + // Remove any previous handle at this fd + IOHandle oldHandle = fdToHandle.put(fd, handle); + if (oldHandle != null) { + handleToFd.remove(System.identityHashCode(oldHandle)); + } + handleToFd.put(System.identityHashCode(handle), fd); + // Ensure nextFd is past this fd + nextFd.updateAndGet(current -> Math.max(current, fd + 1)); + RuntimeIO.advanceFilenoCounterPast(fd); + } + + /** + * Advances the nextFd counter past the given fd value. + * Called by RuntimeIO.assignFileno() to keep the two fd allocation + * systems in sync and prevent fd collisions. + * + * @param fd the fd value to advance past + */ + public static void advancePast(int fd) { + nextFd.updateAndGet(current -> Math.max(current, fd + 1)); + } + + /** + * Returns the next fd number that would be allocated, without actually allocating it. + * Useful as a fallback when the original fd is unknown. + */ + public static int nextFdValue() { + return nextFd.get(); + } + + /** + * Look up an IOHandle by its FD number. + * + * @param fd the file descriptor number + * @return the IOHandle, or null if not found + */ + public static IOHandle getHandle(int fd) { + return fdToHandle.get(fd); + } + + /** + * Remove a handle from the table (e.g., on close). + * + * @param fd the file descriptor number to remove + */ + public static void unregister(int fd) { + IOHandle handle = fdToHandle.remove(fd); + if (handle != null) { + handleToFd.remove(System.identityHashCode(handle)); + } + } + + /** + * Check if a read-end handle has data available without blocking. + * Returns true if the handle is "ready for reading". + * + * @param handle the IOHandle to check + * @return true if data is available or handle is at EOF/closed + */ + public static boolean isReadReady(IOHandle handle) { + if (handle instanceof InternalPipeHandle pipeHandle) { + return pipeHandle.hasDataAvailable(); + } + if (handle instanceof StandardIO) { + // stdin: check System.in.available() + try { + return System.in.available() > 0; + } catch (Exception e) { + return false; + } + } + // For unknown handle types, report as ready to avoid blocking + return true; + } + + /** + * Check if a write-end handle can accept writes without blocking. + * + * @param handle the IOHandle to check + * @return true if the handle can accept writes + */ + public static boolean isWriteReady(IOHandle handle) { + // Pipes and most handles can always accept writes (they buffer internally) + return true; + } +} diff --git a/src/main/java/org/perlonjava/runtime/io/InternalPipeHandle.java b/src/main/java/org/perlonjava/runtime/io/InternalPipeHandle.java index d5f4e51c2..a0caae4cd 100644 --- a/src/main/java/org/perlonjava/runtime/io/InternalPipeHandle.java +++ b/src/main/java/org/perlonjava/runtime/io/InternalPipeHandle.java @@ -253,4 +253,20 @@ public RuntimeScalar sysread(int length) { return new RuntimeScalar(); // undef } } + + /** + * Checks if this pipe has data available for reading without blocking. + * Used by {@link FileDescriptorTable#isReadReady(IOHandle)} to implement + * select() readiness checking for pipe handles. + * + * @return true if data is available, the pipe is closed, or at EOF + */ + public boolean hasDataAvailable() { + if (isClosed || isEOF || !isReader) return false; + try { + return inputStream.available() > 0; + } catch (IOException e) { + return false; + } + } } diff --git a/src/main/java/org/perlonjava/runtime/operators/IOOperator.java b/src/main/java/org/perlonjava/runtime/operators/IOOperator.java index 3ad8c7349..536911a69 100644 --- a/src/main/java/org/perlonjava/runtime/operators/IOOperator.java +++ b/src/main/java/org/perlonjava/runtime/operators/IOOperator.java @@ -471,8 +471,9 @@ public static RuntimeScalar open(int ctx, RuntimeBase... args) { RuntimeIO sourceHandle = findFileHandleByDescriptor(fd); if (sourceHandle != null && sourceHandle.ioHandle != null) { if (isParsimonious) { - // &= mode: reuse the same file descriptor (parsimonious) - fh = sourceHandle; + // &= mode: non-owning wrapper — close won't close the original + fh = new RuntimeIO(); + fh.ioHandle = new BorrowedIOHandle(sourceHandle.ioHandle); } else { // & mode: create a new handle that duplicates the original fh = duplicateFileHandle(sourceHandle); @@ -499,8 +500,9 @@ else if (secondArg.type == RuntimeScalarType.GLOB || secondArg.type == RuntimeSc System.err.flush(); } if (isParsimonious) { - // &= mode: reuse the same file descriptor (parsimonious) - fh = sourceHandle; + // &= mode: non-owning wrapper — close won't close the original + fh = new RuntimeIO(); + fh.ioHandle = new BorrowedIOHandle(sourceHandle.ioHandle); } else { // & mode: create a new handle that duplicates the original fh = duplicateFileHandle(sourceHandle); @@ -522,7 +524,8 @@ else if (secondArg.type == RuntimeScalarType.GLOB || secondArg.type == RuntimeSc if (sourceHandle != null && sourceHandle.ioHandle != null) { if (isParsimonious) { - fh = sourceHandle; + fh = new RuntimeIO(); + fh.ioHandle = new BorrowedIOHandle(sourceHandle.ioHandle); } else { fh = duplicateFileHandle(sourceHandle); } @@ -536,8 +539,8 @@ else if (secondArg.type == RuntimeScalarType.GLOB || secondArg.type == RuntimeSc sourceHandle = ((RuntimeGlob) handleRef.value).getIO().getRuntimeIO(); if (sourceHandle != null && sourceHandle.ioHandle != null) { if (isParsimonious) { - // &= mode: reuse the same file descriptor (parsimonious) - fh = sourceHandle; + fh = new RuntimeIO(); + fh.ioHandle = new BorrowedIOHandle(sourceHandle.ioHandle); } else { // & mode: create a new handle that duplicates the original fh = duplicateFileHandle(sourceHandle); @@ -2468,7 +2471,8 @@ private static String pack(String template, int value) { /** * Find a RuntimeIO handle by its file descriptor number. - * This is a simplified implementation that maps standard file descriptors. + * Checks multiple registries: IOOperator's local fileDescriptorMap, standard fds, + * and the RuntimeIO fileno registry (which includes dup'd handles and sockets). */ private static RuntimeIO findFileHandleByDescriptor(int fd) { // Check if we have it in our mapping @@ -2486,6 +2490,11 @@ private static RuntimeIO findFileHandleByDescriptor(int fd) { case 2: // STDERR return RuntimeIO.stderr; default: + // Check the RuntimeIO fileno registry (dup'd handles, sockets, regular files) + RuntimeIO registeredHandle = RuntimeIO.getByFileno(fd); + if (registeredHandle != null) { + return registeredHandle; + } return null; // Unknown file descriptor } } @@ -2515,53 +2524,67 @@ public static RuntimeIO openFileHandleDup(String fileName, String mode) { throw new PerlCompilerException("Bad file descriptor: " + fd); } } else { - // Handle named filehandles like STDERR, STDOUT, STDIN - switch (fileName.toUpperCase()) { - case "STDIN": - sourceHandle = RuntimeIO.stdin; - break; - case "STDOUT": - sourceHandle = RuntimeIO.stdout; - break; - case "STDERR": - sourceHandle = RuntimeIO.stderr; - break; - default: - // Try to look up as a global filehandle - // First, try the current package if no :: qualifier is present - if (!fileName.contains("::")) { - // Use RuntimeCode.getCurrentPackage() which uses caller() to determine - // the current package - this works for both JVM-compiled and interpreter code - String currentPkg = RuntimeCode.getCurrentPackage(); - // Remove trailing "::" for consistent naming - if (currentPkg.endsWith("::")) { - currentPkg = currentPkg.substring(0, currentPkg.length() - 2); - } - if (currentPkg != null && !currentPkg.isEmpty() && !currentPkg.equals("main")) { - String currentPkgName = currentPkg + "::" + fileName; - RuntimeGlob currentGlob = GlobalVariable.getGlobalIO(currentPkgName); - if (currentGlob != null) { - sourceHandle = currentGlob.getRuntimeIO(); - if (sourceHandle != null && sourceHandle.ioHandle != null) { - break; // Found it in current package - } - } + // Handle named filehandles — always use glob table to get the CURRENT handle, + // not the static RuntimeIO.stdout/stdin/stderr fields which may be stale + // after redirections like open(STDOUT, ">file") + open(STDOUT, ">&SAVED"). + String normalizedName; + if (fileName.equalsIgnoreCase("STDIN") || fileName.equalsIgnoreCase("STDOUT") || fileName.equalsIgnoreCase("STDERR")) { + normalizedName = "main::" + fileName.toUpperCase(); + } else if (!fileName.contains("::")) { + // Try current package first, then fall back to main:: + String currentPkg = RuntimeCode.getCurrentPackage(); + if (currentPkg.endsWith("::")) { + currentPkg = currentPkg.substring(0, currentPkg.length() - 2); + } + if (currentPkg != null && !currentPkg.isEmpty() && !currentPkg.equals("main")) { + String currentPkgName = currentPkg + "::" + fileName; + RuntimeGlob currentGlob = GlobalVariable.getGlobalIO(currentPkgName); + if (currentGlob != null) { + sourceHandle = currentGlob.getRuntimeIO(); + if (sourceHandle != null && sourceHandle.ioHandle != null) { + normalizedName = null; // Already found + } else { + normalizedName = "main::" + fileName; } + } else { + normalizedName = "main::" + fileName; } - // Fall back to main:: or fully qualified name - String normalizedName = fileName.contains("::") ? fileName : "main::" + fileName; - RuntimeGlob glob = GlobalVariable.getGlobalIO(normalizedName); - if (glob != null) { - sourceHandle = glob.getRuntimeIO(); + } else { + normalizedName = "main::" + fileName; + } + } else { + normalizedName = fileName; + } + + if (sourceHandle == null) { + RuntimeGlob glob = GlobalVariable.getGlobalIO(normalizedName); + if (glob != null) { + sourceHandle = glob.getRuntimeIO(); + } + if (sourceHandle == null || sourceHandle.ioHandle == null) { + // Last resort: try static fields for standard handles + switch (fileName.toUpperCase()) { + case "STDIN": sourceHandle = RuntimeIO.stdin; break; + case "STDOUT": sourceHandle = RuntimeIO.stdout; break; + case "STDERR": sourceHandle = RuntimeIO.stderr; break; + default: + throw new PerlCompilerException("Unsupported filehandle duplication: " + fileName); } if (sourceHandle == null || sourceHandle.ioHandle == null) { throw new PerlCompilerException("Unsupported filehandle duplication: " + fileName); } + } } } if (isParsimonious) { - return sourceHandle; + // Parsimonious dup (>&=) shares the same underlying IOHandle but must not + // close it when the duplicate is closed. BorrowedIOHandle delegates all I/O + // but only flushes (never closes) the delegate on close(). + RuntimeIO result = new RuntimeIO(); + result.ioHandle = new BorrowedIOHandle(sourceHandle.ioHandle); + result.currentLineNumber = sourceHandle.currentLineNumber; + return result; } else { return duplicateFileHandle(sourceHandle); } @@ -2572,11 +2595,38 @@ private static RuntimeIO duplicateFileHandle(RuntimeIO original) { return null; } - // Create a new RuntimeIO that shares the same IOHandle RuntimeIO duplicate = new RuntimeIO(); - duplicate.ioHandle = original.ioHandle; duplicate.currentLineNumber = original.currentLineNumber; + if (original.ioHandle instanceof DupIOHandle existingDup) { + // Already reference-counted — add another dup sharing the same delegate + duplicate.ioHandle = DupIOHandle.addDup(existingDup); + } else { + // First duplication — wrap both original and duplicate in DupIOHandles + // so they share a refcount and get distinct filenos. + // Get the original's fd from the registry, or from the IOHandle itself + // (e.g. StandardIO.fileno() returns 0/1/2 for stdin/stdout/stderr). + int origFd = original.getAssignedFileno(); + if (origFd < 0) { + // Not in the registry — ask the IOHandle directly + RuntimeScalar fdScalar = original.ioHandle.fileno(); + origFd = fdScalar.getDefinedBoolean() ? fdScalar.getInt() : -1; + } + if (origFd < 0) { + // Still no fd — assign a new one + origFd = original.assignFileno(); + } + DupIOHandle[] pair = DupIOHandle.createPair(original.ioHandle, origFd); + original.ioHandle = pair[0]; // Replace original's handle with refcounted wrapper + duplicate.ioHandle = pair[1]; // New handle with unique fd + } + + // Register the duplicate's fd in RuntimeIO's fileno registry + int dupFd = duplicate.ioHandle.fileno().getInt(); + if (dupFd >= 0) { + duplicate.registerExternalFd(dupFd); + } + if (System.getenv("JPERL_IO_DEBUG") != null) { String origFileno; try { @@ -2584,8 +2634,15 @@ private static RuntimeIO duplicateFileHandle(RuntimeIO original) { } catch (Throwable t) { origFileno = ""; } + String dupFileno; + try { + dupFileno = duplicate.ioHandle.fileno().toString(); + } catch (Throwable t) { + dupFileno = ""; + } System.err.println("[JPERL_IO_DEBUG] duplicateFileHandle: origIoHandle=" + original.ioHandle.getClass().getName() + " origFileno=" + origFileno + + " dupFileno=" + dupFileno + " origIoHandleId=" + System.identityHashCode(original.ioHandle) + " dupIoHandleId=" + System.identityHashCode(duplicate.ioHandle)); System.err.flush(); diff --git a/src/main/java/org/perlonjava/runtime/operators/WarnDie.java b/src/main/java/org/perlonjava/runtime/operators/WarnDie.java index 580a7486b..11c5b3683 100644 --- a/src/main/java/org/perlonjava/runtime/operators/WarnDie.java +++ b/src/main/java/org/perlonjava/runtime/operators/WarnDie.java @@ -128,6 +128,10 @@ public static RuntimeBase warn(RuntimeBase message, RuntimeScalar where, String if (messageStr.isEmpty()) { RuntimeScalar err = getGlobalVariable("main::@"); + // Resolve tied $@ once to avoid double FETCH (Perl 5 fetches $@ exactly once) + if (err.type == RuntimeScalarType.TIED_SCALAR) { + err = err.tiedFetch(); + } if (err.getDefinedBoolean()) { // If $@ is a reference, pass it directly to the signal handler if (RuntimeScalarType.isReference(err)) { diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeIO.java b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeIO.java index 31fb3c625..030fcb78b 100644 --- a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeIO.java +++ b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeIO.java @@ -23,6 +23,7 @@ Handling pipes (e.g., |- or -| modes). import java.nio.file.StandardOpenOption; import java.util.*; import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.ConcurrentLinkedQueue; import java.util.concurrent.atomic.AtomicInteger; import static org.perlonjava.runtime.runtimetypes.GlobalVariable.getGlobalIO; @@ -142,19 +143,30 @@ protected boolean removeEldestEntry(Map.Entry eldest) { private static final AtomicInteger nextFileno = new AtomicInteger(3); private static final ConcurrentHashMap filenoToIO = new ConcurrentHashMap<>(); private static final ConcurrentHashMap ioToFileno = new ConcurrentHashMap<>(); + /** Pool of released fd numbers available for reuse. Perl reuses fds (lowest available), + * so we must do the same to pass tests like io/perlio_leaks.t that verify fd recycling. */ + private static final ConcurrentLinkedQueue recycledFds = new ConcurrentLinkedQueue<>(); /** - * Assigns a small sequential fileno to this RuntimeIO and registers it. - * Returns the assigned fileno. + * Assigns a fileno to this RuntimeIO and registers it in the fd→IO maps. + * Reuses released fd numbers when available (lowest first via the recycle pool), + * otherwise allocates the next sequential fd. This matches POSIX semantics where + * close() releases an fd and the next open() reuses the lowest available fd. + * + * @return the assigned fileno */ public int assignFileno() { Integer existing = ioToFileno.get(this); if (existing != null) { return existing; } - int fd = nextFileno.getAndIncrement(); + // Try to reuse a recycled fd first (POSIX: lowest available) + Integer recycled = recycledFds.poll(); + int fd = (recycled != null) ? recycled : nextFileno.getAndIncrement(); filenoToIO.put(fd, this); ioToFileno.put(this, fd); + // Keep FileDescriptorTable in sync to prevent fd collisions with pipe() + FileDescriptorTable.advancePast(fd); return fd; } @@ -174,15 +186,43 @@ public static RuntimeIO getByFileno(int fd) { } /** - * Unregisters this RuntimeIO from the fileno registry. + * Unregisters this RuntimeIO from the fileno registry and returns + * the fd to the recycle pool for reuse by future {@link #assignFileno()} calls. */ public void unregisterFileno() { Integer fd = ioToFileno.remove(this); if (fd != null) { filenoToIO.remove(fd); + // Return fd to the recycle pool so it can be reused (POSIX: lowest available) + recycledFds.add(fd); } } + /** + * Registers this RuntimeIO at a specific fd number (e.g. one already assigned + * by FileDescriptorTable for pipes). Advances nextFileno past this fd to + * prevent future collisions with assignFileno(). + * + * @param fd the file descriptor number to register at + */ + public void registerExternalFd(int fd) { + filenoToIO.put(fd, this); + ioToFileno.put(this, fd); + // Advance nextFileno past this fd to avoid collisions + nextFileno.updateAndGet(current -> Math.max(current, fd + 1)); + } + + /** + * Advances the nextFileno counter past the given fd value. + * Called by FileDescriptorTable.register() to keep the two fd allocation + * systems in sync and prevent fd collisions. + * + * @param fd the fd value to advance past + */ + public static void advanceFilenoCounterPast(int fd) { + nextFileno.updateAndGet(current -> Math.max(current, fd + 1)); + } + static { // Initialize mode options mapping MODE_OPTIONS.put("<", EnumSet.of(StandardOpenOption.READ)); @@ -1196,6 +1236,9 @@ public RuntimeScalar close() { ioHandle.flush(); RuntimeScalar ret = ioHandle.close(); ioHandle = new ClosedIOHandle(); + // Release our fd back to the recycle pool so it can be reused by future opens. + // This must happen AFTER close so that the fd is still valid during the close. + unregisterFileno(); return ret; } diff --git a/src/main/perl/lib/Sys/ioctl.ph b/src/main/perl/lib/Sys/ioctl.ph new file mode 100644 index 000000000..24d90ebea --- /dev/null +++ b/src/main/perl/lib/Sys/ioctl.ph @@ -0,0 +1,18 @@ +# sys/ioctl.ph - stub for PerlOnJava +# These are standard ioctl constants. Values are for Linux compatibility. +# ioctl() itself is a no-op stub on JVM, so these values are never actually +# used for real system calls. + +unless (defined &TIOCSWINSZ) { + eval 'sub TIOCSWINSZ () { 0x5414; }'; +} +unless (defined &TIOCGWINSZ) { + eval 'sub TIOCGWINSZ () { 0x5413; }'; +} +unless (defined &FIONREAD) { + eval 'sub FIONREAD () { 0x541B; }'; +} +unless (defined &FIONBIO) { + eval 'sub FIONBIO () { 0x5421; }'; +} +1;