Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 22 additions & 12 deletions doc/modules/ROOT/pages/4.guide/4c.io-context.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -53,27 +53,33 @@ corosio::io_context ioc;
----

Creates an `io_context` with a concurrency hint of
`std::max(2u, std::thread::hardware_concurrency())` the default constructor
never selects single-threaded mode. Single-threaded (lockless) mode is keyed
on a `concurrency_hint` of exactly `1`; pass `1` explicitly to opt into it.
`std::max(1u, std::thread::hardware_concurrency())` and the default, fully
thread-safe locking tier. Thread safety is governed by the `locking` option
(see xref:4.guide/4c2.configuration.adoc#single-threaded-mode[Locking Tiers]).

=== With Concurrency Hint

[source,cpp]
----
corosio::io_context ioc(1); // Single-threaded, no synchronization
corosio::io_context ioc(4); // Up to 4 threads, thread-safe
corosio::io_context ioc(1); // one thread expected to call run()
corosio::io_context ioc(4); // up to 4 threads expected to call run()
----

The concurrency hint affects:
The concurrency hint is a performance tuning value indicating how many threads
are expected to call `run()`. It tunes:

* Internal synchronization strategy
* On Windows, the hint is passed to `CreateIoCompletionPort` as
* The reactor's inline-completion budget defaults: a multi-thread context favors
posting completions for cross-thread work-stealing, while a single-thread
context keeps the inline fast path.
* On Windows, it is passed to `CreateIoCompletionPort` as
`NumberOfConcurrentThreads`, bounding how many completion threads may run
concurrently. It is not a library-managed thread pool, and it is distinct
from `thread_pool_size`.

Use `1` for single-threaded programs to avoid synchronization overhead.
The hint does not affect thread safety, which is governed by the `locking`
tier. To reduce synchronization overhead in a single-threaded program, select a
lockless `locking` tier (see
xref:4.guide/4c2.configuration.adoc#single-threaded-mode[Locking Tiers]).

== Running the Event Loop

Expand Down Expand Up @@ -244,8 +250,9 @@ int main()

== Thread Safety

The `io_context` can be used from multiple threads when constructed with
a concurrency hint greater than 1:
The `io_context` is thread-safe by default, regardless of the concurrency hint:
multiple threads may call `run()` concurrently, and any thread may `post()` work
into it.

[source,cpp]
----
Expand All @@ -260,7 +267,10 @@ for (auto& t : threads)
----

Multiple threads can call `run()` concurrently. The `io_context` distributes
work across threads.
work across threads. The one exception is a lockless `locking` tier: constructing
with `io_context_options::locking` set to `unsafe_io` or `unsafe` drops these
guarantees (see
xref:4.guide/4c2.configuration.adoc#single-threaded-mode[Locking Tiers]).

WARNING: Individual I/O objects (sockets, timers) are not thread-safe.
Don't access the same socket from multiple threads without synchronization.
Expand Down
79 changes: 61 additions & 18 deletions doc/modules/ROOT/pages/4.guide/4c2.configuration.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -80,12 +80,13 @@ corosio::native_io_context<corosio::epoll> ioc(opts);
blocking file I/O and DNS resolution. Ignored on IOCP where
file I/O uses native overlapped I/O.

| `single_threaded`
| false
| `locking`
| `locking_mode::safe`
| all
| Disable all scheduler mutex and condition variable operations.
Eliminates synchronization overhead when only one thread calls
`run()`. See <<single-threaded-mode>> for restrictions.
| Locking-safety tier: `safe` (default, full thread safety),
`unsafe_io` (per-descriptor I/O locks off), or `unsafe` (all locks
off). Governs the thread-safety contract. See
<<single-threaded-mode>> for the tiers and their restrictions.
|===

Options that do not apply to the active backend are silently ignored.
Expand Down Expand Up @@ -144,31 +145,73 @@ and DNS resolution use a shared thread pool.
* *No file I/O*: leave at 1 (the pool is created lazily).

[#single-threaded-mode]
=== Single-Threaded Mode (`single_threaded`)
=== Locking Tiers (`locking`)

Disables all mutex and condition variable operations inside the
scheduler and per-socket descriptor states. This eliminates
15-25% of overhead on the post-and-dispatch hot path.
The `locking` option selects which internal locks the scheduler and
reactor elide, trading thread-safety guarantees for reduced
synchronization overhead. It governs the thread-safety contract,
independently of the `concurrency_hint` (see
<<concurrency-hint-and-locking>>).

[cols="1,3"]
|===
| Tier | Behavior

| `locking_mode::safe` (default)
| Full thread safety. All locks enabled; any thread may use the
context.

| `locking_mode::unsafe_io`
| Disables only the per-descriptor I/O locks; scheduler locking stays
on. A single thread must run and drive the context, but DNS
resolution and POSIX file I/O remain available (they rely on
scheduler locking).

| `locking_mode::unsafe`
| Disables all locking. Eliminates 15-25% of overhead on the
post-and-dispatch hot path. Imposes the full restrictions below.
|===

[source,cpp]
----
corosio::io_context_options opts;
opts.single_threaded = true;
opts.locking = corosio::locking_mode::unsafe;

corosio::io_context ioc(opts);
ioc.run(); // only one thread may call this
----

WARNING: Single-threaded mode imposes hard restrictions.
WARNING: The `unsafe` and `unsafe_io` tiers impose hard restrictions.
Violating them is undefined behavior.

* Only **one thread** may call `run()` (or any run/poll variant).
* **Posting work from another thread** is undefined behavior.
* **DNS resolution** returns `operation_not_supported`.
* **POSIX file I/O** (`stream_file`, `random_access_file`) returns
`operation_not_supported` on `open()`.
* **Signal sets** should not be shared across contexts.
* **Delay and timeout cancellation via `stop_token`** from another
thread is not permitted: the cancel path posts into the scheduler,
whose synchronization is disabled in single-threaded mode.
Cross-thread cancellation requires a multi-threaded-capable context.
* **Delay and timeout cancellation via `stop_token`** from another thread
is not permitted: the cancel path posts into the scheduler, whose locking
these tiers disable. Cross-thread cancellation requires the `safe` tier.

The `unsafe` tier additionally makes:

* **DNS resolution** return `operation_not_supported`.
* **POSIX file I/O** (`stream_file`, `random_access_file`) return
`operation_not_supported` on `open()`.

These two services stay available under `unsafe_io` because they rely
on scheduler locking, which that tier keeps enabled.

[NOTE]
====
These tiers correspond to Boost.Asio's `SAFE` / `UNSAFE_IO` / `UNSAFE`
concurrency-hint constants, exposed here as a dedicated option separate
from the `concurrency_hint`.
====

[#concurrency-hint-and-locking]
==== Concurrency hint and locking tier

The `concurrency_hint` is a performance tuning value indicating how many
threads are expected to call `run()`; it tunes the reactor inline-completion
budget defaults and, on IOCP, the completion-port concurrency. A lockless
tier is single-threaded, so its effective hint for that tuning is `1`
regardless of the value passed.
4 changes: 3 additions & 1 deletion include/boost/corosio/backend.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -359,7 +359,9 @@ struct iocp_t

@param ctx The execution context that owns the scheduler.
@param concurrency_hint Hint for the number of threads that
will call `run()`. Pass `1` for single-threaded mode.
will call `run()`; a performance tuning knob. The
thread-safety contract is set separately by
@ref io_context_options::locking.

@return Reference to the newly created scheduler.
*/
Expand Down
26 changes: 19 additions & 7 deletions include/boost/corosio/detail/scheduler.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -97,13 +97,25 @@ struct BOOST_COROSIO_DECL scheduler
*/
virtual void register_signal_reader(int read_fd) { (void)read_fd; }

/// True if the scheduler is configured for single-threaded use.
/// Default false; overridden by backends that support the mode.
virtual bool is_single_threaded() const noexcept { return false; }

/// Enable or disable single-threaded mode. Default no-op for
/// backends that don't support the mode.
virtual void configure_single_threaded(bool) noexcept {}
/// Decomposed threading configuration applied via @ref configure_threading.
struct threading_config
{
/// Scheduler mutex/condvar enabled. Off only in the `unsafe` tier.
bool scheduler_locking = true;
/// Per-descriptor (reactor) or ring (io_uring) I/O lock enabled.
/// Off in the `unsafe_io` and `unsafe` tiers.
bool reactor_io_locking = true;
/// A single run thread is guaranteed (a lockless tier): elide
/// inter-run-thread wakeups.
bool one_thread = false;
};

/// True in the fully-lockless (`unsafe`) tier. The resolver and POSIX
/// file services gate their `operation_not_supported` result on this.
virtual bool scheduler_locking_disabled() const noexcept { return false; }

/// Apply @ref threading_config. Default no-op.
virtual void configure_threading(threading_config) noexcept {}
};

} // namespace boost::corosio::detail
Expand Down
103 changes: 70 additions & 33 deletions include/boost/corosio/io_context.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,42 @@

namespace boost::corosio {

/** Locking-safety tier for an @ref io_context.

Selects which internal locks the scheduler and reactor elide, trading
thread-safety guarantees for reduced synchronization overhead. This is
the analog of Boost.Asio's `SAFE` / `UNSAFE_IO` / `UNSAFE` concurrency
hint constants. The tier is chosen explicitly, not derived from the
`concurrency_hint`. (The reverse does apply: a lockless tier reduces the
effective hint used for performance tuning to 1.)

@see io_context_options::locking
*/
enum class locking_mode
{
/** Full thread safety (default). All locks enabled; equivalent to
Boost.Asio's `SAFE`/`DEFAULT`. Any thread may use the context. */
safe,

/** Disable only the per-descriptor I/O locks; keep scheduler locking.
Equivalent to Boost.Asio's `UNSAFE_IO`. The context must be run
and driven by a single thread, but resolver and POSIX file
services remain available (they rely on scheduler locking, which
stays on). */
unsafe_io,

/** Disable all locking (fully lockless). Equivalent to Boost.Asio's
`UNSAFE`.

@par Restrictions
- Only one thread may call `run()` (or any run variant).
- Posting work from another thread is undefined behavior.
- DNS resolution returns `operation_not_supported`.
- POSIX file I/O returns `operation_not_supported`.
- Signal sets should not be shared across contexts. */
unsafe
};

/** Runtime tuning options for @ref io_context.

All fields have defaults that match the library's built-in
Expand Down Expand Up @@ -97,26 +133,10 @@ struct io_context_options
*/
unsigned thread_pool_size = 1;

/** Enable single-threaded mode (disable scheduler locking).

When true, the scheduler skips all mutex lock/unlock and
condition variable operations on the hot path. This
eliminates synchronization overhead when only one thread
calls `run()`.

@par Restrictions
- Only one thread may call `run()` (or any run variant).
- Posting work from another thread is undefined behavior.
- DNS resolution returns `operation_not_supported`.
- POSIX file I/O returns `operation_not_supported`.
- Signal sets should not be shared across contexts.

@note Constructing an `io_context` with `concurrency_hint == 1`
automatically enables single-threaded mode regardless of
this field's value, matching asio's convention. To opt out,
pass `concurrency_hint > 1`.
/** Thread-safety tier. See @ref locking_mode for the tiers and their
restrictions.
*/
bool single_threaded = false;
locking_mode locking = locking_mode::safe;

/** Enable IORING_SETUP_SQPOLL on the io_uring backend.

Expand All @@ -126,7 +146,7 @@ struct io_context_options
path. Most useful for sustained traffic. Idle thread parks
after `sq_thread_idle_ms` of no activity.

Independent of `single_threaded`. Default: off.
Independent of `locking`. Default: off.

Ignored on non-io_uring backends.
*/
Expand Down Expand Up @@ -158,6 +178,16 @@ struct io_context_options

namespace detail {
class timer_service;

/** Return the hint used for performance tuning: the lockless tiers are
single-threaded, so their effective hint is 1 whatever the caller passed.
*/
inline unsigned
effective_concurrency_hint(
io_context_options const& opts, unsigned hint) noexcept
{
return opts.locking == locking_mode::safe ? hint : 1u;
}
} // namespace detail

/** An I/O context for running asynchronous operations.
Expand Down Expand Up @@ -196,8 +226,9 @@ class timer_service;

@par Thread Safety
Distinct objects: Safe.@n
Shared objects: Safe, if using a concurrency hint greater
than 1.
Shared objects: Safe, unless the context was constructed with a
lockless @ref io_context_options::locking tier (`unsafe_io` or
`unsafe`), in which case a single thread must drive it.

@see epoll_t, select_t, kqueue_t, iocp_t
*/
Expand All @@ -211,8 +242,11 @@ class BOOST_COROSIO_DECL io_context : public capy::execution_context
io_context_options const& opts,
unsigned concurrency_hint);

/// Switch the scheduler to single-threaded (lockless) mode.
void configure_single_threaded_();
/** Apply only the decomposed threading configuration (locking tiers).
Used by the plain constructors, which — unlike the options
constructors — deliberately leave the reactor budget at its defaults
rather than engaging the multi-thread post-everything heuristic. */
void apply_threading_(io_context_options const& opts);

protected:
detail::scheduler* sched_;
Expand All @@ -223,11 +257,10 @@ class BOOST_COROSIO_DECL io_context : public capy::execution_context

/** Construct with default concurrency and platform backend.

Uses `std::thread::hardware_concurrency()` clamped to a minimum
of 2 as the concurrency hint, so the default constructor never
silently engages single-threaded mode (see
@ref io_context_options::single_threaded). Pass an explicit
`concurrency_hint == 1` to opt into single-threaded mode.
Uses `std::thread::hardware_concurrency()` (floored to 1, in
case it reports 0) as the concurrency hint, and the default
@ref locking_mode::safe tier. Select a lockless tier via
@ref io_context_options::locking.
*/
io_context();

Expand Down Expand Up @@ -266,8 +299,9 @@ class BOOST_COROSIO_DECL io_context : public capy::execution_context
{
(void)backend;
sched_ = &Backend::construct(*this, concurrency_hint);
if (concurrency_hint == 1)
configure_single_threaded_();
// Apply threading config only (locking tier). Unlike the options
// ctor, the plain path leaves the reactor budget at its defaults.
apply_threading_(io_context_options{});
}

/** Construct with an explicit backend tag and runtime options.
Expand All @@ -290,8 +324,11 @@ class BOOST_COROSIO_DECL io_context : public capy::execution_context
{
(void)backend;
apply_options_pre_(opts);
sched_ = &Backend::construct(*this, concurrency_hint);
apply_options_post_(opts, concurrency_hint);
// Effective hint (1 for lockless tiers); see effective_concurrency_hint.
unsigned const eff =
detail::effective_concurrency_hint(opts, concurrency_hint);
sched_ = &Backend::construct(*this, eff);
apply_options_post_(opts, eff);
}

~io_context();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ epoll_scheduler::register_descriptor(int fd, reactor_descriptor_state* desc) con
desc->registered_events = ev.events;
desc->fd = fd;
desc->scheduler_ = this;
desc->mutex.set_enabled(!single_threaded_);
desc->mutex.set_enabled(reactor_io_locking_);
desc->ready_events_.store(0, std::memory_order_relaxed);

conditionally_enabled_mutex::scoped_lock lock(desc->mutex);
Expand Down
Loading
Loading