minor: bump Log4j dep to 2.26 and make JsonTemplateLayout available to operators in their custom log4j configs#19629
Open
capistrant wants to merge 1 commit into
Open
Conversation
…erators to use modern json formatter instead of deprecated JsonLayout
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Using json formatted logs is not the default format shipped with Apache Druid stock log4j files, but many operators choose to customize their logging format. It is likely pretty common for them to use JSON formatted logs. As it stands, operators who do not add
log4j-layout-template-jsonto their own druid bundle would be stuck with JsonLayout which is deprecated in favor of JsonTemplateLayout. In addition to being the supported way to use JSON formatted logs, this layout provides superior performance to the legacy Layout. Also,JsonLayoutis completely removed in log4j 3.x so we will need the new dependency eventually either way.One of the main reasons I found myself down a rabbit hole in log4j was in doing some chaos testing of my configs for a virtual storage enabled data server. I was purposely stress testing with different configs and one such test put artificial load on the s3 connection pool by way oversubscribing segment read requests compared to the s3 client thread pool size. This led to a cascade of s3 related exceptions which end up logged with extended traces by RetryUtils. Sadly, the data server ended up deadlocked because all of the carrier threads for the virtual storage load virtual threads had pinned virtual threads due to native calls in their stacks that I traced back to log4j exception logging. While I don't know that what I was doing was a very realistic scenario for day to day Druid operations, I do think that it raised a real potential problem for both perf and operations in a cluster using JSON formatted logs.
Release note
Bump log4j dependency to 2.26.0. Add
log4j-layout-template-jsonas a dependency to allow operators to switch from deprecatedJsonFormattoJsonTemplateFormatif they are using Json formatted logs. Any operator wishing to migrate should refer to JsonTemplateLayout. The jar ships with default Layouts that are defined in log4j docs, and you can also ship custom layouts if the defaults do not meet your needs.Key changed/added classes in this PR
pom.xmlprocessing/pom.xmlThis PR has: