License violation due to submodule

[zbirenbaum]

Hi! I'm working on a feature this package is perfect for but our PR bot notified me of a license violation in this repo (Apache 2.0) due to the submodule kaitai_struct_formats

The following ksy files in it are copyleft licensed:
firmware/broadcom_trx.ksy - GPL-2.0-or-later
scientific/nt_mdt/nt_mdt.ksy - GPL-3.0-or-later
filesystem/vdi.ksy - GPL-3.0-or-later
image/pif.ksy - LGPL-2.1

Additionally they seem to be violating their own license since that repo is unlicensed, but it has a submodule for workflows owned by the same org which is GPL'd (I made a post here)

I found polyfile because I was looking for a non-GPL alternative to libmagic. I guess I can always fork this and the kaitai_struct_formats to solve the issue but was wondering if there is a way to fix it officially

[ESultanik]

Hello! Thanks for bringing this to our attention. I don't think we realized that each of the .ksy definitions in the kaitai_struct_formats submodule was licensed differently. Kaitai struct itself is MIT licensed.

Those .ksy files are not distributed with PolyFile; they are used at build time to automatically generate pure Python parsers. I am not a lawyer, but I think it is a bit of a legal gray area whether those resulting parsers would be considered "derivative works" under the GPL. If not, then the resulting parsers would not be "tainted" by the GPL copyleft of the .ksy files. In general, having GPL-licensed code in a submodule (or even the actual repo) does not taint the rest of the codebase unless that code is linked against by the other code. So the presence of those files alone shouldn't be an issue.

PolyFile does not use all of the .ksy definitions in the submodule. In the immediate term, we will have to do some digging to check whether any of those GPL'd definitions are used by PolyFile. If so, we will have to disable those features. If not, then there shouldn't be an actual licensing violation. If the PR bot continues to have an issue with the license voilation, then worst-case we could consider vendoring the non-GPL definitions into PolyFile and removing the submodule, but this would be less than ideal as it would prevent easily updating the definitions from upstream.

[generalmimon]

@ESultanik:

I don't think we realized that each of the .ksy definitions in the kaitai_struct_formats submodule was licensed differently.

FWIW, you can see all the individual licenses at https://formats.kaitai.io/xref.html. Since the format specifications are authored by external contributors, it is up to each author to decide which license to use. I personally try to discourage them from doing this because it restricts direct legal use of the spec, but ultimately it's their decision, and we don't take copyleft licenses as a reason for not accepting them into the format gallery.

Kaitai struct itself is MIT licensed.

Kaitai Struct is a project that consists of multiple components. These are licensed differently. A quick overview can be found at https://kaitai.io/#download under "Licensing" - the compiler is released under GPL-3.0-or-later (though if you're just using it in an unchanged form to compile .ksy specs, you usually don't need to care about that), runtime libraries except the JavaScript one are released under MIT.

I am not a lawyer, but I think it is a bit of a legal gray area whether those resulting parsers would be considered "derivative works" under the GPL. If not, then the resulting parsers would not be "tainted" by the GPL copyleft of the .ksy files.

I'm not a lawyer either, but in my opinion the generated modules are definitely derivatives of the original .ksy specification, just as an executable file is a derivative of the source code. I don't see how this is in a legal grey area. To me, this is the simplest case of a "derivative work" - the generated code doesn't reflect any originality of the person using the Kaitai Struct compiler to "compile" someone else's .ksy specification.

Since the GPL .ksy specification yields a GPL-generated parsing library, the software that links to this library must be GPL-licensed, see https://www.gnu.org/licenses/gpl-faq.html#IfLibraryIsGPL:

If a library is released under the GPL (not the LGPL), does that mean that any software which uses it has to be under the GPL or a GPL-compatible license? (#IfLibraryIsGPL)

Yes, because the program actually links to the library. As such, the terms of the GPL apply to the entire combination. The software modules that link with the library may be under various GPL compatible licenses, but the work as a whole must be licensed under the GPL.

@ESultanik:

In general, having GPL-licensed code in a submodule (or even the actual repo) does not taint the rest of the codebase unless that code is linked against by the other code. So the presence of those files alone shouldn't be an issue.

Exactly my thoughts. Otherwise you couldn't even have a monorepo with some GPL components and other (let's say completely independent) non-GPL components just because they exist in the same repository, which doesn't make any sense.

As I understand it, for the concept of copyleft to apply to you, then pieces of software A and B must form a "single combined program". It's worth noting that for example the standard fork-exec method to invoke external programs does not establish "intimate communication" between them (see https://opensource.stackexchange.com/a/6942), so the programs remain separate and the copyleft condition doesn't apply.

@zbirenbaum:

Additionally they seem to be violating their own license since that repo is unlicensed, but it has a submodule for workflows owned by the same org which is GPL'd (I made a post here)

I don't see any non-GPL component linking to a GPL component to form a single combined program, so I don't see a problem with that. The strictest interpretation is that the .ksy specs and the https://github.com/kaitai-io/push_artifacts submodule are separate "programs" that are called from our GitHub Actions workflow main.yml, but I would even argue that this workflow only handles the .ksy specs as data and doesn't actually "use" them (just converts them to other formats - by formats I mean generated source code and the Graphviz diagram). Either way, there shouldn't be any issue.

[zbirenbaum]

Thank you both for the responses!

Since the GPL .ksy specification yields a GPL-generated parsing library, the software that links to this library must be GPL-licensed, see https://www.gnu.org/licenses/gpl-faq.html#IfLibraryIsGPL:

@generalmimon
It sounds like if we include Polyfile as a dependency for our package as things stand we do have to be under GPL as is since the compiled results which are linked against for detection are in-fact GPL?

Update:
@ESultanik
I talked with our security team and they said we definitely couldn't use Polyfile as-is. I PR'd a filter + recompiled version which excludes all the copyleft licensed code in kaitai.

We can rely on my fork for now but would like to upstream this. Let me know if you'd like this to be a flag or something at install.

It sounded like you were ok with removing the GPL content entirely but if you'd like to make this an install flag or something let me know and I'll work on it!