Reduce the amount of unnecessary heap allocations while parsing string tables and fields - #34
Open
Ovahlord wants to merge 3 commits into
Open
Reduce the amount of unnecessary heap allocations while parsing string tables and fields#34Ovahlord wants to merge 3 commits into
Ovahlord wants to merge 3 commits into
Conversation
…g tables and fields
justMaku
approved these changes
Jul 31, 2026
Fabi
reviewed
Jul 31, 2026
| var curOfs = 0; | ||
| var decoded = Encoding.UTF8.GetString(reader.ReadBytes(stringTableSize)); | ||
| foreach (var str in decoded.Split('\0')) | ||
| Span<byte> stringTableBytes = stackalloc byte[stringTableSize]; |
There was a problem hiding this comment.
Do not stackalloc such possibly big arrays. if you are crazy you can go up to 1MB (and if you are really insane 4MB), but otherwise stay under it. That is not an option here.
Fabi
reviewed
Jul 31, 2026
| var curOfs = 0; | ||
| var decoded = Encoding.UTF8.GetString(reader.ReadBytes(stringTableSize)); | ||
| foreach (var str in decoded.Split('\0')) | ||
| Span<byte> stringTableBytes = stackalloc byte[stringTableSize]; |
Author
|
Will refactor the string table stack alloc to use ArrayPool instead later so we re-use arrays then |
…lloc to prevent possible stack overflows when parsing gigantic amounts of strings at once
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Right now DBCD is quite horrendous when it comes to memory usage as about 100mb DBC data can easily bloat into 400mb+ RAM usage during runtime.
Part of if is because of excessive use of reflection, some because of unnecessary long living objects.
However, my current focus was on the speed part as I have noticed that the garbage collector goes nuts while loading storages as there were lots of unneeded heap allocations which triggered Gen0 quite frequently.
This PR focuses on string tables and field parsing. Using modern .NET features, we now stack allocate temporary buffers to read bytes. Additionally, we no longer double-allocate strings when loading the string tables, which eases up the GC pressure a bit.