Enhance queue handling with new configurations and tie-breaking logic#1
Draft
twisti-dev wants to merge 7 commits intomasterfrom
Draft
Enhance queue handling with new configurations and tie-breaking logic#1twisti-dev wants to merge 7 commits intomasterfrom
twisti-dev wants to merge 7 commits intomasterfrom
Conversation
…ueueStartHook for server readiness tasks
…er readiness tasks
- set buffer size in QueueEntryCodec to a constant for better readability - improve buffer initialization by specifying min and max capacity
…handling - rename startTransferring to startTicking for clarity - add SurfQueueConfig for managing queue settings - implement TransferAction enum for better transfer state management - enhance PaperQueue with tickSecond method for periodic processing - add QueueMetrics for tracking queue performance metrics
twisti-dev
added
status: in progress
- change command tree name from "squeue" to "squeue-velocity" for clarity
…ated updates - rename AbstractSurfQueue to AbstractQueue for consistency - update references in PaperSurfQueue and related files - introduce QueueTicker for managing queue ticks - implement cleanup logic in PaperQueueCleanup - enhance performance with optimized UUID handling in queue methods
… update references - rename VelocitySurfQueue class to VelocityQueueImpl for consistency - update all references to VelocitySurfQueue in related files - adjust QueueDisplay and QueuePlayerListener to use new class name
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new, more robust queue scoring system for Redis, improving ordering precision and tie-breaking for queue entries. It replaces the previous
RedisQueueScorePackerwith a newRedisQueueScorevalue class, updates all queue logic to use this new system, and refactors several queue-related classes for better reliability and maintainability. Additionally, it includes minor dependency and configuration updates.Queue scoring and ordering improvements:
RedisQueueScore, a value class that packs priority, timestamp, and a 6-bit sequence into a 53-bit value, ensuring total ordering in Redis sorted sets and supporting up to 64 entries per millisecond. This replaces the oldRedisQueueScorePacker[1] [2].RedisQueueScorefor packing/unpacking scores, including tie-breaking logic with a per-instanceenqueueSequencecounter to prevent collisions when multiple entries are enqueued in the same millisecond [1] [2] [3] [4] [5].Queue store and API enhancements:
RedisQueueStoreto use helper methods for atomic Redis batch operations and updated all relevant methods to work withRedisQueueScoreinstead of rawDoublevalues [1] [2] [3].AbstractSurfQueuefor retrieving queue entry metadata, score, last seen time, and retry count.Reliability and logging:
RedisQueueLockManagerby adding error handling and logging for failed lock releases [1] [2] [3].Configuration and dependency updates:
queue:v2:to avoid collisions with previous queue versions.Performance and safety:
QueueEntryCodecfor more efficient serialization.These changes collectively provide more accurate queue ordering, improved concurrency safety, and better maintainability for the queue system.