Broaden update_versions.py regex to handle custom XML property tags

[jeet1995]

Problem

The external_dependency_version_regex in eng/versioning/utils.py only matches version values inside <version>...</version> XML elements:

external_dependency_version_regex = r'(?<=<version>).+?(?=</version>)'

When a POM uses a custom Maven property tag with a valid {x-version-update} comment, like:

<scala-jackson.version>2.18.6</scala-jackson.version> <!-- {x-version-update;...;external_dependency} -->

The script finds the line (via the {x-version-update} comment), but the re.sub on line 119 of update_versions.py is a silent no-op because the regex lookbehind expects <version> and finds <scala-jackson.version> instead. The version is never updated, no warning is emitted.

This caused the bannedDependencies failure fixed in PR #49263.

Fix

Broaden the regex to match the content of any XML element, not just <version>:

external_dependency_version_regex = r'(?<=>)[^<]+(?=</[a-zA-Z])'

How the regex works

Component	Matches	Purpose
(?<=>)	Lookbehind for >	Anchors after the opening tag's > -- works for <version>, <scala-jackson.version>, or any tag
[^<]+	One or more non-< characters	Captures the version value (e.g., 2.18.6)
(?=</[a-zA-Z])	Lookahead for </ followed by a letter	Ensures we stop at a closing XML tag, not at <!-- --> comments

Why (?=</[a-zA-Z]) and not just (?=</)

Lines with {x-version-update} comments look like:

    <version>2.18.6</version> <!-- {x-version-update;...} -->
                     ^^^^^^^    ^^^^
                     </version> <!-- (comment start)

After </version>, the > is followed by <!-- .... Without the [a-zA-Z] guard, the regex would also match the space between </version> and <!-- as a second match (since re.sub replaces all matches). The [a-zA-Z] ensures we only match content before closing XML tags (</version>, </scala-jackson.version>), never before comment markers (<!--).

Local validation

Ran update_versions.py --sr across all 874 POM files with old vs new regex:

Regex	Files processed	Files changed	Delta
OLD (<version> only)	874	0	--
NEW (any XML element)	874	4	Exactly the 4 stale scala-jackson.version properties

The 4 changed files are the same Spark POMs manually fixed in PR #49263, with identical diffs (2.18.4/2.18.6 -> 2.18.7). No other files in the repo are affected.

Known limitation

If a {x-version-update} line contained multiple XML elements (e.g., <a>1</a><b>2</b>), re.sub would replace both values. This pattern does not exist on any {x-version-update} line in the repo. A future hardening option would be to add count=1 to the re.sub call on line 119.

Related

• PR [INTERNAL]: Fix bannedDependencies failure: bump scala-jackson.version to 2.18.7 #49263 -- immediate fix (bumps the stale property values)
• This PR prevents the same class of silent version drift for any future custom property tags

[alzimmermsft]

Would it be possible to restrict this slightly to either matching <version>...</version> exactly or a common pattern we use for <properties>...</properties> version definitions? Such as all version declarations in <properties>...</properties> suffix with .version.

The regex then can be something like (?<=<((?:[\w-.]+\.)?version)>).+?(?=<\/\1>), where we look for an XML tag that is either <version> itself or <something-custom.morestuff.version>. This also changes the logic to use the first capture group to match the closing tag so we don't need to double define the capture group.

[jeet1995]

@alzimmermsft — PR was auto-closed when I rebased the branch. The updated fix is in #49267 with the tighter regex you requested:

\\python
r'(?<=>)[^<]+(?=</(?:[\w.-]+.)?version>)'
\\

This restricts matches to </version>\ and </something.version>\ closing tags only (rejects </description>, </subversion>, etc.). Python's
e\ module doesn't support variable-length lookbehinds, so the restriction is placed entirely in the lookahead. Full details and validation table in the new PR description.