Skip to content

Return search Metadata and Query Insights#4195

Open
irina-herciu wants to merge 19 commits intoaws:developmentfrom
irina-herciu:features/SearchMetadata
Open

Return search Metadata and Query Insights#4195
irina-herciu wants to merge 19 commits intoaws:developmentfrom
irina-herciu:features/SearchMetadata

Conversation

@irina-herciu
Copy link
Contributor

@irina-herciu irina-herciu commented Dec 4, 2025

Description

Adds support for returning DynamoDB Query/Scan response metadata from DocumentModel Search operations.

Motivation and Context

This enables callers to access pagination and capacity-related information that is already returned by DynamoDB, without switching to low-level request APIs.

Query with metadata

var search = table.Query(new QueryOperationConfig
{
    HashKey = pk,
    ReturnConsumedCapacity = ReturnConsumedCapacity.TOTAL
});

var docs = await search.GetNextSetAsync();

// Access metadata from the last request
var metadata = search.SearchMetadata;

var lastEvaluatedKey = metadata.LastEvaluatedKey;
var capacityUnits = metadata.ConsumedCapacity?.CapacityUnits;

Testing

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project
  • My change requires a change to the documentation
  • I have updated the documentation accordingly
  • I have read the README document
  • I have added tests to cover my changes
  • All new and existing tests passed

License

  • I confirm that this pull request can be released under the Apache 2 license

@irina-herciu irina-herciu marked this pull request as ready for review January 29, 2026 13:11
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for returning DynamoDB Query/Scan response metadata from DocumentModel Search operations, enabling callers to access pagination and capacity-related information without switching to low-level request APIs.

Changes:

  • Introduces a new SearchMetrics class that aggregates capacity consumption, item counts, and scan statistics across paginated operations
  • Adds ReturnConsumedCapacity property to Query/Scan document operation request classes to control whether DynamoDB returns capacity consumption details
  • Updates internal Search methods to collect and expose metrics through a new Metrics property

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
sdk/src/Services/DynamoDBv2/Custom/DocumentModel/Search.cs Adds SearchMetrics class, Metrics property, ReturnConsumedCapacity support, and UpdateMetricsAfterPage logic to track capacity and count metrics across all search operations
sdk/src/Services/DynamoDBv2/Custom/DocumentModel/Table.cs Passes ReturnConsumedCapacity from operation requests to Search objects for both Query and Scan operations
sdk/src/Services/DynamoDBv2/Custom/DocumentModel/DocumentOperationRequest.cs Adds ReturnConsumedCapacity properties to QueryDocumentOperationRequest and ScanDocumentOperationRequest with appropriate defaults and documentation
sdk/src/Services/DynamoDBv2/Custom/DocumentModel/Expression.cs Formatting cleanup - reorders imports and removes blank line
sdk/test/Services/DynamoDBv2/UnitTests/Custom/SearchTests.cs Adds unit tests for metrics aggregation and ReturnConsumedCapacity parameter handling in Scan operations
sdk/test/Services/DynamoDBv2/UnitTests/Custom/DataModel/ContextInternalTests.cs Renames TestEntity to ContextTestEntity to avoid naming conflicts with other test files
sdk/test/Services/DynamoDBv2/UnitTests/Custom/DataModel/ItemStorageConfigCacheTests.cs Updates reference to renamed ContextTestEntity class
generator/.DevConfigs/5694947f-209f-47e9-8c70-3698872df305.json Adds dev config with minor version bump for DynamoDBv2 service

/// <summary>
/// Aggregated per-call and accumulated metrics for this search operation.
/// </summary>
public SearchMetrics Metrics => _metrics;
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description shows an example using search.SearchMetadata, but the actual property name in the code is search.Metrics (line 266 of Search.cs). The PR description should be updated to match the actual implementation, or the code should be updated to match the description if SearchMetadata was the intended name.

Copilot uses AI. Check for mistakes.
/// <summary>
/// Total capacity units accumulated.
/// </summary>
public double? TotalCapacityUnits { get; internal set; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: for each of these can you add the corresponding href to the metric? and then say something like This is the total capacity units across all operations performed during the search.

and similar to the other ones. so users can know easily see what each metric is

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. for the consumed capacity one can add link to https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/metrics-dimensions.html#ConsumedReadCapacityUnits (assuming this is correct)

Copy link
Contributor

@GarrettBeatty GarrettBeatty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code seems fine. just the one comment about updating docs to add links

/// operations that support capacity reporting, such as paginated database queries. All properties reflect the
/// cumulative or most recent values as appropriate, and are updated as additional pages are processed. This type is
/// not intended to be instantiated directly.</remarks>
public sealed class SearchMetrics
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class seems big enough to put its own SearchMetrics.cs file and this Search.cs is already big enough.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

normj
normj previously approved these changes Mar 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants