Runaway memory use when repeatedly printing empty string

[pastries-yay]

Bug report

Bug description:

Printing the empty string in a loop seems to cause memory use of Python to grow without bound.

while True:
    print('', end='')

or

import sys

while True:
    sys.stdout.write('')

The latter apparently does not work in interactive mode.

Relatedly,

while True:
    for _ in range(10**8):
        print('', end='')
    print()

causes memory use to grow to about 1.5GB and then reset when the newline is printed.

CPython versions tested on:

3.14, 3.11, 3.13

Operating systems tested on:

Linux

Linked PRs

• gh-151814: Fix unbounded memory growth from repeated empty writes to io.TextIOWrapper #151817

[picnixz]

I think it's simply because there is no flushing so I think it's still being buffered.

cc @cmaloney

[zware]

Agreed that it seems to be a buffering issue, but since there's not actually anything to buffer it seems there may be a real bug here.

On the other hand this is a degenerate case, so I'm not sure how much effort it's worth to track down :)

[StanFromIreland]

Agreed that it seems to be a buffering issue, but since there's not actually anything to buffer it seems there may be a real bug here.

On the other hand this is a degenerate case, so I'm not sure how much effort it's worth to track down :)

+1 for fixing, this is a real bug.

I took a look and it seems straightforward. Each write('') appended an empty bytes object to the TextIOWrapper's pending_bytes list but added 0 to pending_bytes_count (below, bytes_len is 0 with b''). Since the buffer flush is triggered only when pending_bytes_count reaches chunk_size, it never flushed but did accumulate the objects.

cpython/Modules/_io/textio.c

Lines 1848 to 1853 in aec0aed

	self->pending_bytes_count += bytes_len;
	if (self->pending_bytes_count >= self->chunk_size || needflush ||
	text_needflush) {
	if (_textiowrapper_writeflush(self) < 0)
	return NULL;
	}

[cmaloney]

👍 real bug should fix; I think this is missing a path for bytes_len == 0 to avoid appending to the list in that case. Currently in that case in write_through mode we still flush which I think is important to keep.

[StanFromIreland]

Indeed, I already wrote a little patch: #151817 :-)