Skip to content

fix(mm/oom): staged OOM killer fixes — victim marking, direct reclaim, observability#1995

Open
sparkzky wants to merge 4 commits into
DragonOS-Community:masterfrom
sparkzky:fix/oom-killer-staged-fixes
Open

fix(mm/oom): staged OOM killer fixes — victim marking, direct reclaim, observability#1995
sparkzky wants to merge 4 commits into
DragonOS-Community:masterfrom
sparkzky:fix/oom-killer-staged-fixes

Conversation

@sparkzky

@sparkzky sparkzky commented Jul 3, 2026

Copy link
Copy Markdown
Member

Phase 1 (bug fixes + observability):

  • Fix wait_until_recoverable generation guard dead code (Some(_) => true)
  • Add OOM kill counter (OOM_KILL_COUNT atomic) wired to /proc/vmstat oom_kill
  • Add /proc/[pid]/oom_score read-only file (normalized [0,1000])
  • Fix hardcoded order:1 → order:0 in pagefault OOM context

Phase 2 (victim forward-progress guarantee, prevents OOM livelock):

  • Add ProcessFlags::OOM_VICTIM (equivalent to Linux TIF_MEMDIE)
  • Mark all tasks sharing victim mm with OOM_VICTIM before SIGKILL delivery
  • Clear OOM_VICTIM in exit path via exit_oom_victim()
  • Skip already-marked victims in should_skip_candidate()

Phase 3 (direct reclaim before kill):

  • Insert PageReclaimer::shrink_list(64) before pagefault_out_of_memory() (matching Linux try_to_free_pages before out_of_memory)
  • Use tried_direct_reclaim flag to bound to one reclaim attempt per fault

Test:

  • Add test_proc_oom_score and test_vmstat_oom_kill_counter to user/apps/c_unitest/test_oom_killer.c

Verified: nix develop -c make kernel + make fmt (cargo fmt + clippy) pass.

ref #1976

…, observability

Phase 1 (bug fixes + observability):
- Fix wait_until_recoverable generation guard dead code (Some(_) => true)
- Add OOM kill counter (OOM_KILL_COUNT atomic) wired to /proc/vmstat oom_kill
- Add /proc/[pid]/oom_score read-only file (normalized [0,1000])
- Fix hardcoded order:1 → order:0 in pagefault OOM context

Phase 2 (victim forward-progress guarantee, prevents OOM livelock):
- Add ProcessFlags::OOM_VICTIM (equivalent to Linux TIF_MEMDIE)
- Mark all tasks sharing victim mm with OOM_VICTIM before SIGKILL delivery
- Clear OOM_VICTIM in exit path via exit_oom_victim()
- Skip already-marked victims in should_skip_candidate()

Phase 3 (direct reclaim before kill):
- Insert PageReclaimer::shrink_list(64) before pagefault_out_of_memory()
  (matching Linux try_to_free_pages before out_of_memory)
- Use tried_direct_reclaim flag to bound to one reclaim attempt per fault

Test:
- Add test_proc_oom_score and test_vmstat_oom_kill_counter to
  user/apps/c_unitest/test_oom_killer.c

Verified: nix develop -c make kernel + make fmt (cargo fmt + clippy) pass.
Signed-off-by: sparkzky <sparkhhhhhhhhhh@outlook.com>
@github-actions github-actions Bot added Bug fix A bug is fixed in this pull request test Unitest/User space test labels Jul 3, 2026
@sparkzky

sparkzky commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ff53205ed7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread kernel/src/mm/oom.rs
Comment on lines +343 to +346
pub fn current_is_oom_victim() -> bool {
ProcessManager::current_pcb()
.flags()
.contains(ProcessFlags::OOM_VICTIM)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge 将 OOM_VICTIM 接入页帧分配器

我检查了全仓,current_is_oom_victim() 只有这里定义而没有任何分配路径调用;因此在真实 OOM 下,SIGKILL 后的受害者并不会获得注释中说的内存储备,退出/拆 mm 仍可能在需要页表或内核对象分配时失败,而且 should_skip_candidate() 已经会跳过带该标志的进程,可能导致继续杀其他任务而不是让已选受害者前进。请把这个标志用于页帧/内核分配的 reserve 判断,或不要依赖它作为前进保障。

Useful? React with 👍 / 👎.

Comment thread kernel/src/arch/x86_64/mm/fault.rs Outdated
Comment on lines +522 to +529
if !tried_direct_reclaim {
tried_direct_reclaim = true;
drop(space_guard);
crate::mm::page::PageReclaimer::shrink_list(
<crate::mm::allocator::page_frame::PageFrameCount>::new(64),
);
flags |= FaultFlags::FAULT_FLAG_TRIED;
continue;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge 保留注入 OOM 触发 killer 的路径

/proc/sys/vm/oom_fault_inject 配置为 fail_times=1test_current_is_victim 和新增的 test_vmstat_oom_kill_counter 都这样用)时,第一次缺页会由 should_inject_fault_oom() 直接返回 VM_FAULT_OOM 并消耗唯一一次失败;这里先执行 direct reclaim 后 continue,第二次缺页不再 OOM,于是子进程会走到 _exit(3) 而不是 SIGKILL,oom_kill 也不会增加。需要让注入的 OOM 跳过这次 reclaim,或把测试注入次数调到经过 reclaim 后仍会触发 killer。

Useful? React with 👍 / 👎.

@sparkzky sparkzky changed the title fix(mm/oom): staged OOM killer fixes — victim marking, direct reclaim… fix(mm/oom): staged OOM killer fixes — victim marking, direct reclaim, observability Jul 3, 2026
sparkzky added 3 commits July 4, 2026 02:31
P1: Wire OOM_VICTIM into page-frame allocator (oom.rs:343 / page_frame.rs:349)
- allocate_page_frames now retries up to 1000 times when the current task
  is an OOM victim, preventing livelock when the victim is stuck in an
  uninterruptible allocation during exit.
- Also wakes the page reclaim thread to accelerate memory availability.
- This eliminates the previous dead_code warning on current_is_oom_victim.

P2: Skip direct reclaim for fault-injection OOM (fault.rs:521 / oom.rs:349)
- Add is_fault_inject_target() to check if current task matches the
  oom_fault_inject config.
- When true, skip the shrink_list(64) direct-reclaim step so the injection
  is consumed by the killer path rather than a reclaim-then-retry cycle.
- Without this, fail_times=1 injections were consumed by reclaim, causing
  test_current_is_victim and test_vmstat_oom_kill_counter to fail.

Verified: make kernel + make fmt (cargo fmt + clippy) pass, zero warnings.
Signed-off-by: sparkzky <sparkhhhhhhhhhh@outlook.com>
…GENTS.md

- Add Nix develop/yolo command pointer to '常见命令' section
- Add WHERE TO LOOK table with 3 non-intuitive paths:
  - Running DragonOS -> docs/introduction/develop_nix.md
  - OOM Killer -> kernel/src/mm/oom.rs
  - Post-refactor process management -> state.rs / exit.rs (mod.rs split in DragonOS-Community#1993)

Signed-off-by: sparkzky <sparkhhhhhhhhhh@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug fix A bug is fixed in this pull request test Unitest/User space test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant