Skip to content

minor: bump Log4j dep to 2.26 and make JsonTemplateLayout available to operators in their custom log4j configs#19629

Open
capistrant wants to merge 1 commit into
apache:masterfrom
capistrant:log4j-json-template-layout
Open

minor: bump Log4j dep to 2.26 and make JsonTemplateLayout available to operators in their custom log4j configs#19629
capistrant wants to merge 1 commit into
apache:masterfrom
capistrant:log4j-json-template-layout

Conversation

@capistrant

@capistrant capistrant commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Description

Using json formatted logs is not the default format shipped with Apache Druid stock log4j files, but many operators choose to customize their logging format. It is likely pretty common for them to use JSON formatted logs. As it stands, operators who do not add log4j-layout-template-json to their own druid bundle would be stuck with JsonLayout which is deprecated in favor of JsonTemplateLayout. In addition to being the supported way to use JSON formatted logs, this layout provides superior performance to the legacy Layout. Also, JsonLayout is completely removed in log4j 3.x so we will need the new dependency eventually either way.

One of the main reasons I found myself down a rabbit hole in log4j was in doing some chaos testing of my configs for a virtual storage enabled data server. I was purposely stress testing with different configs and one such test put artificial load on the s3 connection pool by way oversubscribing segment read requests compared to the s3 client thread pool size. This led to a cascade of s3 related exceptions which end up logged with extended traces by RetryUtils. Sadly, the data server ended up deadlocked because all of the carrier threads for the virtual storage load virtual threads had pinned virtual threads due to native calls in their stacks that I traced back to log4j exception logging. While I don't know that what I was doing was a very realistic scenario for day to day Druid operations, I do think that it raised a real potential problem for both perf and operations in a cluster using JSON formatted logs.

Release note

Bump log4j dependency to 2.26.0. Add log4j-layout-template-json as a dependency to allow operators to switch from deprecated JsonFormat to JsonTemplateFormat if they are using Json formatted logs. Any operator wishing to migrate should refer to JsonTemplateLayout. The jar ships with default Layouts that are defined in log4j docs, and you can also ship custom layouts if the defaults do not meet your needs.


Key changed/added classes in this PR
  • pom.xml
  • processing/pom.xml

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

…erators to use modern json formatter instead of deprecated JsonLayout
@capistrant capistrant changed the title bump Log4j dep to 2.26 and make JsonTemplateLayout available to operators in their custom log4j configs minor: bump Log4j dep to 2.26 and make JsonTemplateLayout available to operators in their custom log4j configs Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant