Skip to content

CHANGELOG.md

Latest commit

 

History

History
495 lines (363 loc) · 22.1 KB

CHANGELOG.md

File metadata and controls

495 lines (363 loc) · 22.1 KB

Changelog

All notable changes to this project will be documented in this file.

[Unreleased]

Changed

  • Document Helm deployed RBAC permissions and remove unnecessary permissions (#770).
  • BREAKING: configOverrides now only accepts the known config files (hdfs-site.xml, core-site.xml, hadoop-policy.xml, ssl-server.xml, ssl-client.xml and security.properties). Previously, arbitrary file names were silently accepted and ignored (#777).
  • Bump stackable-operator to 0.110.1 (#777).

[26.3.0] - 2026-03-16

[26.3.0-rc1] - 2026-03-16

Added

Changed

  • Bump stackable-operator to 0.108.0, and strum to 0.28 (#760, #764).
  • Gracefully shutdown all concurrent tasks by forwarding the SIGTERM signal (#747).
  • Added warning and exit condition to format-namenodes container script to check for corrupted data after formatting (#751).

Fixed

  • Fix "404 page not found" error for the initial object list (#764).
  • Previously, some shell output of init-containers was not logged properly and therefore not aggregated, which is fixed now (#746).

[25.11.0] - 2025-11-07

[25.11.0-rc1] - 2025-11-06

Added

  • Helm: Allow Pod priorityClassName to be configured (#713).
  • Add end-of-support checker (#718).
    • EOS_CHECK_MODE (--eos-check-mode) to set the EoS check mode. Currently, only "offline" is supported.
    • EOS_INTERVAL (--eos-interval) to set the interval in which the operator checks if it is EoS.
    • EOS_DISABLED (--eos-disabled) to disable the EoS checker completely.
  • Add prometheus.io/path|port|scheme annotations to metrics service (#721).

Changed

  • The prometheus.io/scrape label was moved to the metrics service (#721).
  • The headless service now only exposes product / data ports, the metrics service only metrics ports (#721, #726).
  • Bump stackable-operator to 0.100.1 and product-config to 0.8.0 (#722).
  • Bump testing-tools to 0.3.0-stackable0.0.0-dev (#740).

[25.7.0] - 2025-07-23

[25.7.0-rc1] - 2025-07-18

Added

  • Adds new telemetry CLI arguments and environment variables (#672).
    • Use --file-log-max-files (or FILE_LOG_MAX_FILES) to limit the number of log files kept.
    • Use --file-log-rotation-period (or FILE_LOG_ROTATION_PERIOD) to configure the frequency of rotation.
    • Use --console-log-format (or CONSOLE_LOG_FORMAT) to set the format to plain (default) or json.
  • The operator now defaults to AES/CTR/NoPadding for dfs.encrypt.data.transfer.cipher.suite to improve security and performance (#693).
  • The built-in Prometheus servlet is now enabled and metrics are exposed under the /prom path of all UI services (#695).
  • Add several properties to hdfs-site.xml and core-site.xml that improve general performance and reliability (#696).
  • Add RBAC rule to helm template for automatic cluster domain detection (#699).

Changed

  • BREAKING: Replace stackable-operator initialize_logging with stackable-telemetry Tracing (#661, #668, #672).
    • The console log level was set by HDFS_OPERATOR_LOG, and is now set by CONSOLE_LOG_LEVEL.
    • The file log level was set by HDFS_OPERATOR_LOG, and is now set by FILE_LOG_LEVEL.
    • The file log directory was set by HDFS_OPERATOR_LOG_DIRECTORY, and is now set by FILE_LOG_DIRECTORY (or via --file-log-directory <DIRECTORY>).
    • Replace stackable-operator print_startup_string with tracing::info! with fields.
  • BREAKING: Inject the vector aggregator address into the vector config using the env var VECTOR_AGGREGATOR_ADDRESS instead of having the operator write it to the vector config (#671).
  • test: Bump to Vector 0.46.1 (#677).
  • BREAKING: Previously this operator would hardcode the UID and GID of the Pods being created to 1000/0, this has changed now (#683)
    • The runAsUser and runAsGroup fields will not be set anymore by the operator
    • The defaults from the docker images itself will now apply, which will be different from 1000/0 going forward
    • This is marked as breaking because tools and policies might exist, which require these fields to be set
  • Use versioned common structs (#684).
  • BREAKING: remove legacy service account binding for cluster role nodes (#697).
  • BREAKING: Bump stackable-operator to 0.94.0 and update other dependencies (#699).
    • The default Kubernetes cluster domain name is now fetched from the kubelet API unless explicitly configured.
    • This requires operators to have the RBAC permission to get nodes/proxy in the apiGroup "". The helm-chart takes care of this.
    • The CLI argument --kubernetes-node-name or env variable KUBERNETES_NODE_NAME needs to be set. The helm-chart takes care of this.
  • The operator helm-chart now grants RBAC patch permissions on events.k8s.io/events, so events can be aggregated (e.g. "error happened 10 times over the last 5 minutes") (#700).

Fixed

  • Use json file extension for log files (#667).
  • Fix a bug where changes to ConfigMaps that are referenced in the HdfsCluster spec didn't trigger a reconciliation (#671).
  • Allow uppercase characters in domain names (#699).

Removed

  • Remove support for HDFS 3.3.4, 3.3.6, and 3.4.0 (#675).
  • Remove the lastUpdateTime field from the stacklet status (#699).
  • Remove role binding to legacy service accounts (#699).

[25.3.0] - 2025-03-21

Added

  • The lifetime of auto generated TLS certificates is now configurable with the role and roleGroup config property requestedSecretLifetime. This helps reducing frequent Pod restarts (#619).
  • Run a containerdebug process in the background of each HDFS container to collect debugging information (#629).
  • Support configuring JVM arguments (#636).
  • Aggregate emitted Kubernetes events on the CustomResources (#643).
  • Add support for version 3.4.1 (#656).

Changed

  • Bump stackable-operator to 0.87.0 and stackable-versioned to 0.6.0 (#655).
  • Switch the WebUI liveness probe from httpGet to checking the tcp socket. This helps with setups where configOverrides are used to enable security on the HTTP interfaces. As this results in 401 HTTP responses (instead of 200), this previously failed the liveness checks.
  • Set the JVM argument -Xms in addition to -Xmx (with the same value). This ensure consistent JVM configs across our products (#636).
  • Default to OCI for image metadata and product image selection (#640).

[24.11.1] - 2025-01-10

Fixed

  • BREAKING: Use distinct ServiceAccounts for the Stacklets, so that multiple Stacklets can be deployed in one namespace. Existing Stacklets will use the newly created ServiceAccounts after restart (#616).

[24.11.0] - 2024-11-18

Added

  • The operator can now run on Kubernetes clusters using a non-default cluster domain. Use the env var KUBERNETES_CLUSTER_DOMAIN or the operator Helm chart property kubernetesClusterDomain to set a non-default cluster domain (#591).

Changed

  • Reduce CRD size from 1.4MB to 136KB by accepting arbitrary YAML input instead of the underlying schema for the following fields (#574):
    • podOverrides
    • affinity

Fixed

  • An invalid HdfsCluster doesn't cause the operator to stop functioning (#594).

[24.7.0] - 2024-07-24

Added

  • Add experimental support for version 3.4.0 (#545, #557). We do NOT support upgrading from 3.3 to 3.4 yet!

Changed

  • Bump stackable-operator from 0.64.0 to 0.70.0 (#546).
  • Bump product-config from 0.6.0 to 0.7.0 (#546).
  • Bump other dependencies (#549).

Fixed

  • Revert changing the getting started script to use the listener class cluster-internal (#492) (#493).
  • Fix HDFS pods crashing on launch when any port names contain dashes (#517).
  • Add labels to ephemeral (listener) volumes. These allow stackablectl stack list to display datanode endpoints (#534)
  • Processing of corrupted log events fixed; If errors occur, the error messages are added to the log event (#536).

[24.3.0] - 2024-03-20

Added

  • Added rack awareness support via topology provider implementation (#429, #495).
  • More CRD documentation ([#433]).
  • Support for exposing HDFS clusters to clients outside of Kubernetes (#450).
  • Helm: support labels in values.yaml (#460).
  • Add support for OPA authorizer (#474).

Changed

  • Use new label builders (#454).
  • Change the liveness probes to use the web UI port and to fail after one minute (#491).
  • Update the getting started script to use the listener class cluster-internal (#492).

Removed

  • [BREAKING] .spec.clusterConfig.listenerClass has been split to .spec.nameNodes.config.listenerClass and .spec.dataNodes.config.listenerClass, migration will be required when using external-unstable (#450, #462).
  • [BREAKING] Removed legacy node selector on roleGroups (#454).
  • Change default value of dfs.ha.nn.not-become-active-in-safemode from true to false (#458).
  • Removed support for Hadoop 3.2 (#475).

Fixed

  • Include hdfs principals dfs.journalnode.kerberos.principal, dfs.namenode.kerberos.principal and dfs.datanode.kerberos.principal in the discovery ConfigMap in case Kerberos is enabled (#451).
  • User provided env overrides now work as expected (#499).

[23.11.0] - 2023-11-24

Added

  • Default stackableVersion to operator version (#381).
  • Configuration overrides for the JVM security properties, such as DNS caching (#384).
  • Support PodDisruptionBudgets (#394).
  • Support graceful shutdown (#407).
  • Added support for 3.2.4, 3.3.6 (#409).

Changed

  • vector 0.26.0 -> 0.33.0 (#378, #409).
  • Let secret-operator handle certificate conversion (#392).
  • operator-rs 0.44.0 -> 0.55.0 (#381, #394, #404, #405, #409).
  • Consolidate Rust workspace members (#425).

Fixed

  • Don't default roleGroup replicas to zero when not specified (#402).
  • [BREAKING] Removed field autoFormatFs, which was never read (#422).

Removed

  • Removed support for 3.3.1, 3.3.3 (#409).

[23.7.0] - 2023-07-14

Added

  • Add support for enabling secure mode with Kerberos (#334).
  • Generate OLM bundle for Release 23.4.0 (#350).
  • Missing CRD defaults for status.conditions field (#354).
  • Set explicit resources on all containers (#359).
  • Support podOverrides (#368).

Changed

  • Operator-rs: 0.40.2 -> 0.44.0 (#349, #372).
  • Use 0.0.0-dev product images for testing (#351)
  • Use testing-tools 0.2.0 (#351)
  • Run as root group (#353).
  • Added kuttl test suites (#364)
  • Increase the size limit of the log volumes (#372)

[23.4.0] - 2023-04-17

Added

  • Deploy default and support custom affinities (#319).
  • Added OLM bundle files (#328).
  • Extend cluster resources for status and cluster operation (paused, stopped) (#337).
  • Cluster status conditions (#339).

Changed

  • [Breaking] Moved top level config option to clusterConfig (#326).
  • [BREAKING] Support specifying Service type. This enables us to later switch non-breaking to using ListenerClasses for the exposure of Services. This change is breaking, because - for security reasons - we default to the cluster-internal ListenerClass. If you need your cluster to be accessible from outside of Kubernetes you need to set clusterConfig.listenerClass to external-unstable (#340).
  • operator-rs 0.36.0 -> 0.40.2 (#326, #337, #341, #342).
  • Use build_rbac_resources from operator-rs (#342).

Fixed

  • Avoid empty log events dated to 1970-01-01 and improve the precision of the log event timestamps (#341).

Removed

  • Removed the --debug flag for HDFS container start up (#332).

[23.1.0] - 2023-01-23

Added

  • Log aggregation added (#290).
  • Support for multiple storage directories (#296).

Changed

  • [BREAKING] Use Product image selection instead of version. spec.version has been replaced by spec.image (#281).
  • Updated stackable image versions (#271).
  • Fix the previously ignored node selector on role groups (#286).
  • operator-rs 0.25.2 -> 0.30.2 (#276, #286, #290).
  • Replaced thiserror with snafu (#290).

[0.6.0] - 2022-11-07

Added

  • Orphaned resources are deleted (#249)
  • Support Hadoop 3.3.4 (#250)

Changed

  • operator-rs 0.24.0 -> 0.25.2 (#249).

Fixed

  • Set specified resource request and limit on namenode main container (#259).

[0.5.0] - 2022-09-06

Changed

  • Include chart name when installing with a custom release name (#205).
  • Added OpenShift compatibility (#225).
  • Add recommended labels to NodePort services (#240).

[0.4.0] - 2022-06-30

Added

Changed

  • operator-rs 0.10.0 -> 0.15.0 (#130, #134, #148).
  • HADOOP_OPTS for jmx exporter specified to HADOOP_NAMENODE_OPTS, HADOOP_DATANODE_OPTS and HADOOP_JOURNALNODE_OPTS to fix cli tool (#148).
  • [BREAKING] Specifying the product version has been changed to adhere to ADR018 instead of just specifying the product version you will now have to add the Stackable image version as well, so version: 3.5.8 becomes (for example) version: 3.5.8-stackable0.1.0 (#180)

[0.3.0] - 2022-02-14

Added

  • Monitoring scraping label prometheus.io/scrape: true (#104).

Changed

  • Complete rewrite to use StatefulSets, hostPath volumes and the Kubernetes overlay network. (#68)
  • operator-rs 0.9.0 → 0.10.0 (#104).

[0.2.0] - 2021-11-12

  • operator-rs 0.3.0 → 0.4.0 (#20).
  • Adapted pod image and container command to docker image (#20).
  • Adapted documentation to represent new workflow with docker images (#20).

[0.1.0] - 2021-10-27

Changed

  • Switched to operator-rs tag 0.3.0 (#13)