Skip to content

Commit c701ef8

Browse files
committed
Convert Packer's buf_size once
buf_size was untyped, so Cython converted it to size_t separately for the PyMem_Malloc call and for pk.buf_size. An object whose __int__ answers differently each call made the packer allocate one size and record another, and pack.h then grew the buffer against the recorded capacity, so a large enough payload was memcpy'd past the allocation. Typing the parameter converts it once during argument unpacking, the same way Unpacker takes read_size and max_buffer_size. Fixes #723
1 parent 809bfcd commit c701ef8

2 files changed

Lines changed: 22 additions & 1 deletion

File tree

msgpack/_packer.pyx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ cdef class Packer:
110110
cdef bint autoreset
111111
cdef bint datetime
112112

113-
def __cinit__(self, buf_size=256*1024, **_kwargs):
113+
def __cinit__(self, size_t buf_size=256*1024, **_kwargs):
114114
self.pk.buf = <char*> PyMem_Malloc(buf_size)
115115
if self.pk.buf == NULL:
116116
raise MemoryError("Unable to allocate internal buffer.")

test/test_pack.py

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,3 +179,24 @@ def test_get_buffer():
179179

180180
expected = packb([1, 2], use_bin_type=True)
181181
assert written == expected
182+
183+
184+
@pytest.mark.skipif(
185+
Packer.__module__ == "msgpack.fallback",
186+
reason="buf_size only allocates in the C extension",
187+
)
188+
def test_buf_size_is_converted_once():
189+
# Asking twice let the allocation and the recorded capacity disagree,
190+
# so the packer overflowed a buffer smaller than the size it recorded.
191+
class Counting:
192+
count = 0
193+
194+
def __int__(self):
195+
self.count += 1
196+
return 600
197+
198+
__index__ = __int__
199+
200+
buf_size = Counting()
201+
Packer(buf_size=buf_size)
202+
assert buf_size.count == 1

0 commit comments

Comments
 (0)