KAFKA-20541 : Move 42/generated from static/ to content/

[muralibasani]

• Adds a catch-all module.mounts entry in hugo.yaml so future Kafka releases that ship docs/generated/ under content/en/<v>/generated/ (the layout used by apache/kafka) are served at /<v>/generated/ URLs automatically
• no per-release edits to hugo.yaml, shortcode, or script changes needed.
• /42/generated/connect_rest.yaml still downloads the original file
• a new config in hugo.yaml to exclude the empty files in 4.2 (these empty files exist only in 4.2). Without this, empty stubs would shadow real files because of catch all config.
• Tested with a new version content/en/99/generated/test.html which is served at /99/generated/test.html

[mimaison]

Thanks for picking this up.

I guess I should put some thoughts in the Jira when opening it. While your PR resolve the duplication, I'm not sure it's the approach we should take to resolve this. When we generate docs for a release, the generated folder is there alongside apis, configurations, etc. This is why it was there under 42.

I wonder if the website should pick generated from that location instead of static.

[muralibasani]

Thanks for picking this up.

I guess I should put some thoughts in the Jira when opening it. While your PR resolve the duplication, I'm not sure it's the approach we should take to resolve this. When we generate docs for a release, the generated folder is there alongside apis, configurations, etc. This is why it was there under 42.

I wonder if the website should pick generated from that location instead of static.

@mimaison pls let me know if the below approach is better to handle this situation.

Currently there are a few empty files in content/en/42/generated too.
We can move files from static/42/generated to content/en/42/generated. (With this, generated dir will be deleted from static)
A mount config change in hugo.yml /42/generated/ pointing to content/en/42/generated/. This will keep the connect_rest.yaml downloadable file url as is.
include-html (shortcode) also to be pointed to new content location and fallback to static//generated/ so versions prior to 4.2 keep working as well.

Finally update the release playbook, so for 4.3 onwards, we can keep the generated/ with the docs. We don't have to copy it to the static folder.
With this, there are no URL changes and everything works the same way. Pages, downloads, urls, etc

I tested this locally and it works.

[muralibasani]

@mimaison pushed the changes too. It's a large PR, but mostly moving of files to content dir, and handling the include-html shortcode.

[mimaison]

Yes I think something along these lines would work. Thanks for figuring it out.

I'm wondering whether we should keep all existing versions as is, at least for now, and only start with the new scheme for newer versions. So we'd do the changes in the kafka repository instead.

Also it seems the new process requires editing hugo.yaml for each release, could we invert the process to include mapping all existing versions, and not require changes for new releases?

[muralibasani]

Yes I think something along these lines would work. Thanks for figuring it out.

I'm wondering whether we should keep all existing versions as is, at least for now, and only start with the new scheme for newer versions. So we'd do the changes in the kafka repository instead.

Also it seems the new process requires editing hugo.yaml for each release, could we invert the process to include mapping all existing versions, and not require changes for new releases?

@mimaison new changes now handle only newer versions and no edits are required for hugo config going fwd.
Added one excludeFiles config to skip empty stubs in 4.2. (if we don't need this config, need to delete those empty files and they only exist in content/4.2/generated)

And for new releases: copy docs/ into content/en// and images/javadoc into static//, catch-all will take care.
Tested with a new version locally.