Skip to content

HGETALL Throw an exception when using on too many elements #1616

@damecole

Description

@damecole

Morning,

During our test of Garnet, we had a use case where we store few millions elements inside the Hash DataStructure.
We tried to use HGETALL to get every values from Garnet in one go but Garnet throw an exception.

03::58::09 crit: Session[0] [100.95.16.223:62071] [010F8C99] ProcessMessages threw an exception: System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter 'sourceBytesToCopy') at Garnet.server.HashObject.HashGetAll(GarnetObjectStoreOutput& output, Byte respProtocolVersion) in //libs/server/Objects/Hash/HashObjectImpl.cs:line 67 at Garnet.server.HashObject.Operate(ObjectInput& input, GarnetObjectStoreOutput& output, Byte respProtocolVersion, Int64& sizeChange) in //libs/server/Objects/Hash/HashObject.cs:line 213 at Garnet.server.ObjectSessionFunctions.SingleReader(Byte[]& key, ObjectInput& input, IGarnetObject& value, GarnetObjectStoreOutput& dst, ReadInfo& readInfo) in //libs/server/Storage/Functions/ObjectStore/ReadMethods.cs:line 18 at Garnet.server.StorageSession.ReadObjectStoreOperationWithOutput[TObjectContext](Byte[] key, ObjectInput& input, TObjectContext& objectStoreContext, GarnetObjectStoreOutput& output) in //libs/server/Storage/Session/ObjectStore/Common.cs:line 85 at Garnet.server.StorageSession.HashGetAll[TObjectContext](Byte[] key, ObjectInput& input, GarnetObjectStoreOutput& output, TObjectContext& objectStoreContext) in //libs/server/Storage/Session/ObjectStore/HashOps.cs:line 410 at Garnet.server.GarnetApi`2.HashGetAll(Byte[] key, ObjectInput& input, GarnetObjectStoreOutput& output) in //libs/server/API/GarnetApiObjectCommands.cs:line 482 at Garnet.server.RespServerSession.HashGetAll[TGarnetApi](RespCommand command, TGarnetApi& storageApi) in //libs/server/Resp/Objects/HashCommands.cs:line 142 at Garnet.server.RespServerSession.ProcessArrayCommands[TGarnetApi](RespCommand cmd, TGarnetApi& storageApi) in //libs/server/Resp/RespServerSession.cs:line 791 at Garnet.server.RespServerSession.ProcessMessages() in //libs/server/Resp/RespServerSession.cs:line 609 at Garnet.server.RespServerSession.TryConsumeMessages(Byte* reqBuffer, Int32 bytesReceived) in //libs/server/Resp/RespServerSession.cs:line 457

While we switch to Hash Get by batch to get every elements we find this crash quite weird.
Is this behavior expected ?

Regards,

Steps to reproduce the bug

  • Start a new Garnet server
  • Load few million elements in the Hash data structure.
  • Try to use HGETALL to get all elements in one go

Expected behavior

No response

Screenshots

No response

Release version

No response

IDE

No response

OS version

Microsoft.Garnet: 1.0.94

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions