Open
Conversation
Ensure that all files are flushed after they've been written. At current, files are not explicitly flushed to disk, which can cause data corruption. In extreme circumstances, it's possible that uploaded files may not ever be persisted at all.
Add flush/fsync to all file write paths including thumbnails, transcoded videos, exif writes, file copies, and backups.
|
Testing performance this morning with small and large JPGs across local SSD and NFS enterprise HDD storage. |
|
Testing this now, with read cache enabled so that we're focusing purely on the difference with thumbnail generation. Have deployed the stack directly on the NAS itself so it's local storage. Doing three runs of each to get firm numbers compared to vanilla v2.5.6. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add flush/fsync to all file write paths including thumbnails, transcoded videos, exif writes, file copies, and backups.
See #26881 for more discussion. These changes have been split out so that the most important changes can be merged without delay (syncing uploaded files).
How Has This Been Tested?
N/A
Checklist:
src/services/uses repositories implementations for database calls, filesystem operations, etc.src/repositories/is pretty basic/simple and does not have any immich specific logic (that belongs insrc/services/)Please describe to which degree, if any, an LLM was used in creating this pull request.
N/A