[server] Add negative cache for non-existent partition IDs in metadata requests#3511
[server] Add negative cache for non-existent partition IDs in metadata requests#3511swuferhong wants to merge 4 commits into
Conversation
loserwang1024
left a comment
loserwang1024
left a comment
There was a problem hiding this comment.
I have left some comment
|
@loserwang1024 comments addressed. |
|
Hi, @loserwang1024 Could you take another look？ |
| // Only cache when the authoritative partition assignment is also gone. A miss | ||
| // from the scoped table-path lookup may simply mean that the request omitted | ||
| // the owning table or the session is not authorized for it. | ||
| if (isPartitionAssignmentMissingFromZk(partitionId)) { |
There was a problem hiding this comment.
Look good to me, only one mirror problem: whether partitionId in partitionIdsNotExistsInCache is included in partitionNegativeCache? If not, this is no need. Maybe this can move blew partitionIdAndPaths = zkClient.getPartitionIdAndPaths(authorizedTables) for each existed partitionIdAndPaths.
There was a problem hiding this comment.
Thanks for the comment. partitionIdsNotExistsInCache indeed does not include IDs already hit by the negative cache.
The extra isPartitionAssignmentMissingFromZk(partitionId) check is still needed before markNonExistent, because getPartitionIdAndPaths(authorizedTables) is scoped by the requested/authorized table paths. A miss there may mean the partition belongs to another table or is not visible in this request scope, rather than globally non-existent.
I addressed the mirror case by invalidating stale negative-cache entries when we confirm the partition exists:
metadata cache hit: call partitionNegativeCache.markExistent(partitionId)
ZK partitionIdAndPaths hit: call partitionNegativeCache.markExistent(partitionId)
This keeps the negative cache effective for real non-existent partitions while avoiding stale negative entries blocking existing partitions.
loserwang1024
left a comment
loserwang1024
left a comment
There was a problem hiding this comment.
LGTM! @wuchong , WDYT?
2 participants
Purpose
Linked issue: close #3510
During hourly partition rotation, clients holding stale partition IDs repeatedly trigger
ZooKeeper lookups that always return "not found". This adds unnecessary pressure on ZK.
The negative cache eliminates these redundant queries by remembering the "not found" result
for a configurable TTL period.
Brief change log
Tests
API and Format
Documentation