filter_kubernetes: Attach k8s metadata for internal logs

[cosmo0920]

Plus, fixing shutdown lifecycle glitches on in_fluentbit_logs plugin.

Closes #11741.

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• Run local packaging test showing all targets (including any new ones) build.
• Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• Documentation required for this feature

Backporting

• Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

• New Features

  • Added support for loading Kubernetes metadata from local namespace/token files, including namespace metadata.
  • Fluent Bit logs processed via the “local” path now include Kubernetes context.

• Bug Fixes

  • Improved pod/namespace metadata lookup and cache handling for more reliable local metadata retrieval.
  • Strengthened annotation container matching to avoid incorrect metadata application.
  • Improved collector shutdown reliability for the Fluent Bit logs input.

• Tests

  • Added runtime coverage for local Kubernetes metadata with Fluent Bit logs, using new local namespace/token test data.

[coderabbitai]

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (1)

• tests/runtime/data/kubernetes/out/kairosdb-914055854-b63vq.out is excluded by !**/*.out

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b3edf2c9-510e-4816-8251-95c6cad859d6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds a file-based Kubernetes metadata path for the fluentbit_logs input, refactors pod and namespace metadata lookup, guards collector cleanup in the input plugin, and adds runtime coverage and fixture data.

Changes

Local Kubernetes metadata for fluentbit_logs

Layer / File(s)	Summary
Data shape and public API contracts plugins/filter_kubernetes/kube_conf.h, plugins/filter_kubernetes/kube_meta.h	Adds namespace_file to struct flb_kube and declares flb_kube_meta_get_local.
Pod and namespace metadata lookup refactor plugins/filter_kubernetes/kube_meta.c	Reads namespace from ctx->namespace_file, adds local metadata initializers, extracts shared lookup helpers, updates cache/unpack failure handling, adds local getter variants, and exports flb_kube_meta_get_local.
Kubernetes filter wiring and config map plugins/filter_kubernetes/kubernetes.c	Adds the input header include and local input constant, routes cb_kube_filter to the local metadata path for fluentbit_logs, and adds the kube_namespace_file config option.
fluentbit_logs collector cleanup guard plugins/in_fluentbit_logs/fluentbit_logs.c	Initializes coll_fd to -1 and guards collector deletion during plugin exit.
Runtime test for local fluentbit_logs metadata tests/runtime/data/kubernetes/local/*, tests/runtime/filter_kubernetes.c	Adds namespace and token fixture files, the local result struct, the record-check callback, the runtime test, and its TEST_LIST registration.
Container annotation matching guard plugins/filter_kubernetes/kube_property.c	Strengthens container matching with null and length checks before comparing container names.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested reviewers

• edsiper

Poem

🐇 I hopped through logs with nimble feet,
Found namespace files and tokens neat.
The kube filter now knows the trail,
And local metadata won’t fail.
With floppy ears and twinkling eyes,
I cheer these tidy log-time ties.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 4.55% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title concisely reflects the main change: enabling Kubernetes metadata on Fluent Bit internal logs.
Linked Issues check	✅ Passed	The changes implement Kubernetes metadata enrichment for the internal logs input, matching issue #11741's goal.
Out of Scope Changes check	✅ Passed	The ancillary cleanup and test-support changes still align with the internal-logs metadata feature and don't look unrelated.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch cosmo0920-attach-k8s-metadata-for-internal-logs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8e6f2ca107

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

plugins/filter_kubernetes/kube_meta.c (1)

2426-2438: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Handle cache insertion failures before unpacking.

If flb_hash_table_add() returns < 0, hash_meta_buf is still uninitialized in the cache-miss path, but both helpers continue to msgpack_unpack_next(). Bail out or use a safely owned buffer before continuing.

Proposed fix

         id = flb_hash_table_add(ctx->hash_table,
                                 meta->cache_key, meta->cache_key_len,
                                 tmp_hash_meta_buf, hash_meta_size);
-        if (id >= 0) {
-            /*
-             * Release the original buffer created on extract_pod_meta() as a new
-             * copy has been generated into the hash table, then re-set
-             * the outgoing buffer and size.
-             */
-            flb_free(tmp_hash_meta_buf);
-            flb_hash_table_get_by_id(ctx->hash_table, id, meta->cache_key,
-                                     &hash_meta_buf, &hash_meta_size);
+        if (id < 0) {
+            flb_free(tmp_hash_meta_buf);
+            *out_buf = NULL;
+            *out_size = 0;
+            return 0;
         }
+
+        flb_free(tmp_hash_meta_buf);
+        flb_hash_table_get_by_id(ctx->hash_table, id, meta->cache_key,
+                                 &hash_meta_buf, &hash_meta_size);
@@
         id = flb_hash_table_add(ctx->namespace_hash_table,
                                 meta->cache_key, meta->cache_key_len,
                                 tmp_hash_meta_buf, hash_meta_size);
-        if (id >= 0) {
-            /*
-             * Release the original buffer created on extract_namespace_meta()
-             * as a new copy has been generated into the hash table, then reset
-             * the outgoing buffer and size.
-             */
-            flb_free(tmp_hash_meta_buf);
-            flb_hash_table_get_by_id(ctx->namespace_hash_table, id, meta->cache_key,
-                                     &hash_meta_buf, &hash_meta_size);
+        if (id < 0) {
+            flb_free(tmp_hash_meta_buf);
+            *out_buf = NULL;
+            *out_size = 0;
+            return 0;
         }
+
+        flb_free(tmp_hash_meta_buf);
+        flb_hash_table_get_by_id(ctx->namespace_hash_table, id, meta->cache_key,
+                                 &hash_meta_buf, &hash_meta_size);

Also applies to: 2530-2542

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/filter_kubernetes/kube_meta.c` around lines 2426 - 2438, In the
cache-miss handling in kube_meta.c, `flb_hash_table_add()` can fail and leave
`hash_meta_buf` uninitialized, but the code still continues into
`msgpack_unpack_next()`. Update the cache insertion flow in the affected helpers
(including the matching path around the second occurrence) so that on `id < 0`
you either return/bail out early or explicitly fall back to a safely owned
buffer before unpacking. Use the `flb_hash_table_add()`,
`flb_hash_table_get_by_id()`, and `msgpack_unpack_next()` call sites to ensure
only valid buffer/size pairs are passed forward.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@plugins/filter_kubernetes/kube_meta.c`:
- Around line 2426-2438: In the cache-miss handling in kube_meta.c,
`flb_hash_table_add()` can fail and leave `hash_meta_buf` uninitialized, but the
code still continues into `msgpack_unpack_next()`. Update the cache insertion
flow in the affected helpers (including the matching path around the second
occurrence) so that on `id < 0` you either return/bail out early or explicitly
fall back to a safely owned buffer before unpacking. Use the
`flb_hash_table_add()`, `flb_hash_table_get_by_id()`, and
`msgpack_unpack_next()` call sites to ensure only valid buffer/size pairs are
passed forward.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e165a48c-b07b-42bc-a747-533197d23771

📥 Commits

Reviewing files that changed from the base of the PR and between c2b1cfd and 8e6f2ca.

📒 Files selected for processing (8)

• plugins/filter_kubernetes/kube_conf.h
• plugins/filter_kubernetes/kube_meta.c
• plugins/filter_kubernetes/kube_meta.h
• plugins/filter_kubernetes/kubernetes.c
• plugins/in_fluentbit_logs/fluentbit_logs.c
• tests/runtime/data/kubernetes/local/namespace
• tests/runtime/data/kubernetes/local/token
• tests/runtime/filter_kubernetes.c