Add per-context memory accounting and limits

[aabbdev]

Introduce memory usage tracking at the JSContext level, allowing embedding runtimes to enforce hard memory caps per tenant or sandbox.

• JSContext gains:

  • size_t mem_limit: configurable hard limit (0 = unlimited)
  • size_t mem_used: live allocation counter

• New API:

  • void JS_SetContextMemoryLimit(JSContext *ctx, size_t limit);
  • void JS_ResetContextMemory(JSContext *ctx); size_t JS_GetContextMemoryUsage(JSContext *ctx);

With this change, embedders can harden multi-tenant runtimes by sandboxing memory usage per JSContext, instead of only at the runtime level.

TODO:

• Disable evals
• Disable Atomics/SharedArrayBuffer
• Disable new Function("...")
• Restricted import()
• helpers or an API to measure cpu time?

[saghul]

Can't this be accomplished with the custom memory allocation functions already?

[aabbdev]

@saghul we can set limits at the runtime level, not per context. The runtime loses track of the context after the high-level allocation functions.

[saghul]

What use case do you have which has multiple contexts per runtime?

[aabbdev]

@saghul ultra dense use cases like serverless functions: e.g 128 contexts per runtime per core. Many companies using QuickJS face this challenge; I've seen several discussions on the official channels, and I’m dealing with the same use case. Because they don't have this capability they instantiate one runtime per context.

[aabbdev]

There's a significant inconsistency in the allocation functions: sometimes they take rt, sometimes ctx, and in many cases if not the majority *_rt(...) are called even when ctx is available like *_rt(ctx->rt). We should enforce a clear rule: we must never deconstruct ctx and give rt in param but directly the ctx pointer.
No allocation or manipulations should be made without explicitly precise who is the initiator (the context) especially if 100% of the time it's indirectly linked to a context

[aabbdev]

My planning:

• Rewrite all memory paths for consistent memory accounting and a cleaner implementation.
• Add a build-time flag to enable hardening (and disable certain features), or possibly an hybrid runtime switch needs benchmarking.

This introduces breaking API changes, except If I include compatibility APIs, so it may end up as a fork/divergence. I completely understand if it’s not something maintainers want to integrate into quickjs-ng.

[past-due]

Just a note that upstream may be planning to refactor how JSRuntime / JSContext works (see: bellard/quickjs#421 ). As such, it might be prudent to wait a bit to see what the plans are there (and that may also clear up some of your inconsistency concerns listed above?)

[aabbdev]

@past-due They’re basically just renaming things: since there are no memory limits at the context level, they feel uneasy about instantiating a runtime, so they renamed runtime → context and context → realm.

With proper memory limits on contexts (or realms), there’s no need to create a separate runtime+context for every script. You can run a single runtime with N contexts, each configured with its own memory limit. If you need strict multi-tenancy, you can still isolate by creating one runtime per script.

My proposal is to avoid allocations from the Runtime entirely and always go through the Context APIs (or Realm, depending on naming). Right now the internal API is inconsistent: sometimes you allocate with a context, but free through the runtime. Instead, allocations and frees should always go through context APIs, which can delegate internally to the runtime if needed, never directly through the runtime.

This approach prevents bypassing memory accounting, eliminates confusion from mixed APIs, and ensures that per-context memory limits are applied consistently.

Regarding memory accounting: it can be enabled with a compile-time flag. If the flag is not used, the API still exists but will throw an error indicating that you need to recompile with memory accounting enabled to enforce limits.

[bnoordhuis]

I'm not dead set against the idea but I don't want to add functionality that only has a single user. I'd like to see first if more people want/need this.

As a bit of background, I run a quickjs-based multi-tenant service but I don't have a need for per-context memory tracking, a JSRuntime + JSContext works and performs well enough. I keep some around pre-initialized so there's always one ready to go.

---

The fact JSRuntime's atoms/class/shape tables are shared across contexts undeniably has some performance benefits but they're mostly minor though.

My back-of-the-napkin math says that on a 64 bits system at 128 contexts, there's ~6.6 MiB additional overhead in the JSRuntime + JSContext model vis-a-vis a single shared JSRuntime. Not nothing but pretty close to nothing in this day and age.

I guess if you preload your contexts with a lot of additional stuff (like WinterCG) the overhead will be bigger but we'd still be talking low dozens of MiBs.

[bnoordhuis]

Can't use __builtin_mul_overflow, not universally supported.

[aabbdev]

Yes, you're right, I only put it there as a draft.
Instead of using __builtin_mul_overflow, we can #include <stdckdint.h>.
For portability, I'll add a compile-time fallback with a pure C implementation as backup.

[bnoordhuis]

This is arguably excessive. js_malloc_usable_size_rt (wrapper around malloc_usable_size on linux) is not necessarily very cheap. Probably not worth it to avoid going a few bytes over the limit.

[aabbdev]

You're right, I should only check once before the pre-allocation. Fragmentation and the actual physical size shouldn't be included in the limits. I will make the change

[bnoordhuis]

This is better handled by breaking out js_alloc_string_rt into two separate functions, one that calls js_malloc_rt and then calls the second function to initialize the string. Call said second function from here with memory allocated with js_malloc.

[aabbdev]

@bnoordhuis Glad you took the time to review this and share your experience.
This work is still in its early stages I need to run more benchmarks to properly assess the impact. For context, I’m currently working on full WinterTC compatibility, a preemptive scheduler, and a few other interesting optimizations.

I opened this PR to share the idea and start a discussion. From what I've seen, several companies are running multi-tenant architectures with a single runtime and multiple contexts, but without any per-context memory limits.

The proposal here is to simplify and improve the allocate/free function signatures and introduce optional per-context memory limits, enabled at build time via a flag + some disabled features.

We don't necessarily have to merge this feature into quickjs-ng, it's a proposal

[dannote]

@aabbdev @saghul @bnoordhuis Thanks for opening this. I wanted to add another concrete user for per-context memory limits.

In QuickBEAM we embed QuickJS-NG in the BEAM/Elixir runtime. We have a ContextPool abstraction where each pool thread owns one JSRuntime, and that runtime hosts many lightweight JSContext instances. The native pool worker creates one runtime per pool thread, then creates many contexts inside it via JS_NewContext(rt). Users can create many contexts on the same runtime thread, each representing an isolated script/tenant/task; the Elixir API passes a per-context :memory_limit through ContextPool.create_context, and our local QuickJS patch applies it with JS_SetContextMemoryLimit(ctx, ...). A runtime-level memory limit is still useful as a hard process/thread guard, but it does not let us prevent one context from consuming the whole runtime budget and impacting unrelated contexts. We also expose the per-context accounting in memory usage as context_malloc_size.

We currently carry a local patch for this. It is similar in spirit, but differs in a few details:

• API shape:
  • this PR: JS_SetContextMemoryLimit, JS_ResetContextMemory, JS_GetContextMemoryUsage
  • our patch: JS_SetContextMemoryLimit, JS_GetContextMallocSize
• Accounting:
  • our patch tracks js_malloc_usable_size(...) + MALLOC_OVERHEAD, matching QuickJS's existing runtime malloc accounting style more closely;
  • this PR mostly accounts usable allocation size without the runtime accounting overhead.
• Defensive behavior:
  • our free path clamps accounting to zero on mismatch instead of allowing unsigned underflow;
  • this PR's ctx->mem_used -= actual can underflow if accounting ever gets out of sync.
• Reset:
  • I’m worried about JS_ResetContextMemory(ctx) as a public API. If it is called while the context still owns live allocations, future frees/reallocs will corrupt the counter. I think it would be safer to drop it, or make it an internal initialization-only operation with a very explicit invariant.
• realloc paths:
  • after a successful realloc, if the post-allocation limit check fails and the new pointer is freed, the counter needs very careful adjustment. This is an easy place for memory accounting to diverge.

Our local implementation is not something I’d suggest merging as-is either; it is an embedding patch. But we have been running it in QuickBEAM because we need exactly this feature for pooled multi-context runtimes. With QuickJS-NG 0.15.1 plus our patch, QuickBEAM’s full source-built test suite passes, including coverage, reset, N-API, and context-pool tests.

I think the most upstreamable version would be a small, conservative API:

void JS_SetContextMemoryLimit(JSContext *ctx, size_t limit);
size_t JS_GetContextMallocSize(JSContext *ctx);

and no public reset function. It should also document the scope clearly: this can only account allocations made through context-aware allocation paths; runtime-shared structures such as atoms/shapes/classes and any direct *_rt allocation paths remain runtime-level accounting.

Happy to help test or adapt our local patch if there’s interest in moving this forward.