Add unit tests for JsonIndexHandler index lifecycle#18962
Add unit tests for JsonIndexHandler index lifecycle#18962Akanksha-kedia wants to merge 4 commits into
Conversation
JsonIndexHandler had no unit test coverage. Adds JsonIndexHandlerTest with 5 tests covering: - Column dropped from config triggers index removal - updateIndices removes index for dropped column - New column with metadata present triggers index creation - Column in config but metadata absent skips index creation - Index already present and in config requires no update Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #18962 +/- ##
============================================
+ Coverage 64.86% 65.12% +0.26%
- Complexity 1347 1403 +56
============================================
Files 3392 3407 +15
Lines 211663 213087 +1424
Branches 33306 33606 +300
============================================
+ Hits 137290 138770 +1480
+ Misses 63296 63127 -169
- Partials 11077 11190 +113
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
The newly added test is failing. Please take a look |
…dler preconditions Stub getTotalDocs()→1 and getAllColumns()→TreeSet in all SegmentMetadataImpl mocks so the BaseIndexHandler constructor precondition check does not throw.
There was a problem hiding this comment.
Pull request overview
Adds focused unit test coverage for JsonIndexHandler in pinot-segment-local, exercising the handler’s “should we rebuild?” decision logic and the index-removal path during updates.
Changes:
- Introduces
JsonIndexHandlerTestcoveringneedUpdateIndices()for: dropped-config columns, newly-configured columns (metadata present/absent), and already-up-to-date indexes. - Adds a unit test for
updateIndices()verifying JSON index removal when the column is no longer configured.
|
@xiangfu0 @shounakmk219 would appreciate a review when you get a chance. This PR adds unit tests for |
gortiz
left a comment
There was a problem hiding this comment.
Thanks for adding coverage here — JsonIndexHandler genuinely had none before this.
On the mocking approach: this follows existing precedent well. VectorIndexHandlerTest (same package) tests the identical needUpdateIndices/updateIndices decision logic with the same Mockito pattern (mock(SegmentDirectory.class), mock(SegmentMetadataImpl.class), mocked Reader/Writer), so this isn't introducing a new style — it's consistent with how this specific kind of lifecycle-dispatch logic is tested elsewhere in the module.
One gap worth flagging: all 5 tests exercise the add/remove decision logic, but none exercise the actual index-creation path (createJsonIndexForColumn, handleDictionaryBasedColumn/handleNonDictionaryBasedColumn — forward-index rebuild, marker-file crash recovery, dictionary vs. non-dictionary handling, v3 single-file writing). That's the riskiest part of this class and it's still untested. Testing it properly would need a real on-disk segment rather than more mocks (see ForwardIndexHandlerTest's approach with SegmentGeneratorConfig + SegmentIndexCreationDriverImpl + SegmentLocalFSDirectory) — mocking the index-creator/reader chain would just replace one set of untested assumptions with another. Would be a great follow-up PR if not in scope here.
Minor nit: testNeedUpdateReturnsTrueWhenColumnRemovedFromConfig passes Map.of() and relies on BaseIndexHandler's constructor auto-filling missing columns with FieldIndexConfigs.EMPTY when the keyset doesn't match segmentMetadata.getAllColumns(). It's correct, but the intent would read more clearly if the config for COLUMN were built explicitly (e.g. a FieldIndexConfigs without json enabled) rather than via that implicit fallback.
…Javadoc; drop section dividers
Summary
JsonIndexHandlerhad no unit test coverage. AddsJsonIndexHandlerTestwith 5 tests:testNeedUpdateReturnsTrueWhenColumnRemovedFromConfigneedUpdateIndicesreturnstruetestUpdateIndicesRemovesIndexWhenColumnDroppedFromConfigwriter.removeIndexcalledtestNeedUpdateReturnsTrueWhenNewColumnAddedtruetestNeedUpdateReturnsFalseWhenColumnMetadataAbsentgetColumnMetadataForreturns null →falsetestNeedUpdateReturnsFalseWhenIndexUpToDatefalseTest plan
JsonIndexHandlerTest— 5 unit tests, all green./mvnw checkstyle:check -pl pinot-segment-local— 0 violations