Skip to content

feat(facade): Basic RESP support in ioloopv2, part 2#6885

Merged
dranikpg merged 4 commits intodragonflydb:mainfrom
dranikpg:redis-loopv2-2
Mar 29, 2026
Merged

feat(facade): Basic RESP support in ioloopv2, part 2#6885
dranikpg merged 4 commits intodragonflydb:mainfrom
dranikpg:redis-loopv2-2

Conversation

@dranikpg
Copy link
Copy Markdown
Contributor

@dranikpg dranikpg commented Mar 13, 2026

Second part of #6870, testing on CI

Implement very basic admin message processing for ioloopv2

@dranikpg dranikpg changed the title Redis loopv2 2 feat(facade): Basic RESP support in ioloopv2, part 2 Mar 13, 2026
@dranikpg
Copy link
Copy Markdown
Contributor Author

Some tests timed out

@dranikpg
Copy link
Copy Markdown
Contributor Author

Not that are tests have good coverage, but those results are not bad

FAILED 😰  dragonfly/connection_test.py::test_reply_count[df_factory0] - assert 3 <= 2
FAILED 😰  dragonfly/connection_test.py::test_timeout[df_factory0] - AssertionError: assert 2 <= 1
 +  where 2 = len([{'addr': '127.0.0.1:33574', 'age': '52', 'db': '0', 'fd': '15', ...}, {'addr': '127.0.0.1:33560', 'age': '52', 'db': '0', 'fd': '14', ...}])
FAILED 😰  dragonfly/connection_test.py::test_pipeline_cache_only_async_squashed_dispatches[df_factory0] - assert 2520 == 0
FAILED 😰  dragonfly/connection_test.py::test_client_migrate_no_conn_leak[df_factory0] - TimeoutError: Timed out!
FAILED 😰  dragonfly/memory_test.py::test_throttle_on_commands_squashing_replies_bytes[df_factory0] - assert 0 > 0
 +  where 0 = len([])
==== 5 failed, 496 passed, 50 skipped, 75 deselected in 1409.74s (0:23:29) =====

Signed-off-by: Vlad <vlad@dragonflydb.io>
Signed-off-by: Vlad <vlad@dragonflydb.io>
Signed-off-by: Vlad <vlad@dragonflydb.io>
@dranikpg
Copy link
Copy Markdown
Contributor Author

I disabled the v2 loop for redis and kept everything as is, rounding it up here

@dranikpg dranikpg marked this pull request as ready for review March 18, 2026 14:34
Copilot AI review requested due to automatic review settings March 18, 2026 14:34
@augmentcode
Copy link
Copy Markdown

augmentcode Bot commented Mar 18, 2026

🤖 Augment PR Summary

Summary: Extends the experimental IoLoopV2 path toward basic RESP/control-path support.

Changes:

  • Prevents AsyncFiber-based control-path processing when ioloop_v2_ is enabled and instead wakes the main loop via io_event_.
  • Updates IoLoopV2() to wake on pending dispatch_q_ items and drain them inline, flushing replies and notifying pubsub backpressure waiters.
  • Adds !ioloop_v2_ guards to migration/async-fiber related paths to avoid unsupported interactions.
  • Minor parsing-path adjustments to keep V2 compatible with the revised control-path handling.

🤖 Was this summary useful? React with 👍 or 👎

Copy link
Copy Markdown

@augmentcode augmentcode Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 1 suggestion posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.


// If we launch while closing, it won't be awaited. Control messages will be processed on cleanup.
if (!cc_->conn_closing) {
if (!cc_->conn_closing && !ioloop_v2_) {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since ioloop_v2_ no longer launches async_fb_, the early return for cc_->conn_closing can drop CheckpointMessages even after SendCheckpoint() increments the BlockingCounter, potentially causing waits/timeouts. Consider ensuring checkpoints are either always decremented on closing v2 connections or SendCheckpoint only increments the counter if the message is guaranteed to be queued/processed.

Severity: high

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates facade::Connection to better integrate the experimental IoLoopV2 path by avoiding the legacy AsyncFiber wake/launch logic and instead waking IoLoopV2 directly and draining control-path (dispatch_q_) messages inside IoLoopV2.

Changes:

  • Gate AsyncFiber-related behaviors (launch/wake/migration handling) when ioloop_v2_ is enabled.
  • Add dispatch_q_ wake condition to IoLoopV2 and drain/flush control messages from within IoLoopV2.
  • Disable RequestAsyncMigration() when ioloop_v2_ is enabled.

You can also share your feedback on Copilot code review. Take the survey.

}

// TODO: Properly handle backpressure unblocking and flusing
reply_builder_->Flush();
Comment on lines +2163 to 2166
if ((!force && !migration_enabled_) || cc_ == nullptr || ioloop_v2_) {
return;
}

Signed-off-by: Vlad <vlad@dragonflydb.io>
@dranikpg dranikpg requested review from glevkovich and romange March 18, 2026 16:02
auto msg = std::move(dispatch_q_.front());
dispatch_q_.pop_front();
// Temporary; ProcessAdminMessage logic doesn't apply here (migrations for example)
std::visit(AsyncOperations{reply_builder_.get(), this}, msg.handle);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about setting here the async_dispatch flag (with a cleanup)?

if (!dispatch_q_.empty()) {
  cc_->async_dispatch = true;
  absl::Cleanup reset = [this] { cc_->async_dispatch = false; };

  while (!dispatch_q_.empty()) {
    auto msg = std::move(dispatch_q_.front());
    dispatch_q_.pop_front();
    std::visit(AsyncOperations{reply_builder_.get(), this}, msg.handle);
    UpdateDispatchStats(msg, false /* subtract */);
  }
}


// TODO: Properly handle backpressure
GetQueueBackpressure().pubsub_ec.notifyAll();
continue;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we jump back to the beginning of the while loop, giving the dispatch queue strict priority over command parsing and execution. This may cause starvation - especially with a high volume of pubsub messages, the parsed command queue might never be served.

At a minimum, I suggest adding a TODO comment here to acknowledge the issue for a future task, or fixing it on the spot as part of this PR.

@dranikpg dranikpg merged commit 0a47562 into dragonflydb:main Mar 29, 2026
27 of 30 checks passed
@dranikpg dranikpg deleted the redis-loopv2-2 branch March 29, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants