Improve license handling#662
Open
masami256 wants to merge 2 commits into
Open
Conversation
The find_layer() returns layer list that sort order is definition order in bblayer.conf. However, it would be useful that list is sorted by layer priority to know which is most prioritized. Signed-off-by: Masami Ichikawa <masami.ichikawa@miraclelinux.com>
masami256
force-pushed
the
improve-license-handling
branch
5 times, most recently
from
5bfc7d1 to
f6af9dd
Compare
This commit improbe license handling with following changes.
1. Allow dictionaly data in licenses.yml file
2. Create extra license information by using external_references data
3. Supoort to have licenses.yml file in each layers
The 1st change allow dictionary data in licenses.yml file to write license text into sbom file.
Current format is following list.
```
debian binary package name:
licenses: [
"LICENSE A",
"LICENSE B"
]
```
New fommat allows following 3 formats.
A: Current format style
```
debian binary package name:
licenses: [
"LICENSE_A",
"LICENSE_B"
]
```
B: Anather list format
```
debian binary package name:
licenses:
- LICENSE_A
- LICENSE_B
```
C: Dictionary format
```
debian binary package name:
licenses:
license:
- LICENSE_A
- LICENSE_B
- LICENSE_C
text:
LICENSE_C: <debian binary package name>/LICENSE_C.txt
```
A and B is same meaning that only diffrence is style.
So, C is new format.
If LICENSE_A and LICENSE_B is SPDX ID, you don't have to add license text in the sbom.
However, if LICENSE is not in SPDX ID list (https://spdx.org/licenses/), you need to add license text in the sbom.
Format C support this feature.
You can put license text file in <your layer>/conf/licenses/license_texts directory. So, you need to create directory in the license_texts directory then put license text.
If path is not absolute path, script makes file path to <your layer>/conf/licenses/license_texts/<debian package name>/<license text file name>.
If path si absolute path, scripte read from it by given path.
For example, libbpf1 package contains License line "GPL-2 with Linux-syscall-note exception" which is not in the SPDX license id list.
So, we create extra license using LicenseRef- prefix.
```
"licenseConcluded": "(LGPL-2.1-only OR BSD-2-Clause) AND GPL-2.0-only AND LicenseRef-libbpf1-GPL-2-with-Linux-syscall-note-exception-d5d4d27955c443937234e4e20fd0c460abf55feca458146b2da47b4ab4bcd3e4 AND GPL-2.0-or-later",
```
Also, we insert license text into the sbom in hasExtractedLicensingInfos section in SPDX fomat sbom.
```
{
"extractedText": "NOTE! This copyright does *not* cover user programs that use kernel services\nby normal system calls - this is merely considered normal use of the kernel,\nand does *not* fall under
the heading of \"derived work\". Also note that the\nGPL below is copyrighted by the Free Software Foundation, but the instance of\ncode that it refers to (the Linux kernel) is copyrighted by me and others who\
nactually wrote it.\n\nAlso note that the only valid version of the GPL as far as the kernel is\nconcerned is _this_ particular version of the license (ie v2, not v2.2 or v3.x\nor whatever), unless explicitly ot
herwise stated.\n\nLinus Torvalds\n\nThis program is free software; you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation; vers
ion 2 of the License.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program; if not, write to the Free Software Foundation, Inc.,\n51 Franklin
St, Fifth Floor, Boston, MA 02110-1301, USA.\n\nOn Debian and systems the full text of the GNU General Public\nLicense version 2 can be found in the file\n\"/usr/share/common-licenses/GPL-2\".",
"licenseId": "LicenseRef-libbpf1-GPL-2-with-Linux-syscall-note-exception-d5d4d27955c443937234e4e20fd0c460abf55feca458146b2da47b4ab4bcd3e4",
"name": "Custom License for libbpf1 LicenseRef-libbpf1-GPL-2-with-Linux-syscall-note-exception-d5d4d27955c443937234e4e20fd0c460abf55feca458146b2da47b4ab4bcd3e4"
},
```
CycloneDX format sbom also contains license text.
```
{
"license": {
"name": "LicenseRef-libbpf1-GPL-2-with-Linux-syscall-note-exception-d5d4d27955c443937234e4e20fd0c460abf55feca458146b2da47b4ab4bcd3e4",
"text": {
"content": "NOTE! This copyright does *not* cover user programs that use kernel services\nby normal system calls - this is merely considered normal use of the kernel,\nand does *not* fall under the heading of \"derived work\". Also note that the\nGPL below is copyrighted by the Free Software Foundation, but the instance of\ncode that it refers to (the Linux kernel) is copyrighted by me and others who\nactually wrote it.\n\nAlso note that the only valid version of the GPL as far as the kernel is\nconcerned is _this_ particular version of the license (ie v2, not v2.2 or v3.x\nor whatever), unless explicitly otherwise stated.\n\nLinus Torvalds\n\nThis program is free software; you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation; version 2 of the License.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program; if not, write to the Free Software Foundation, Inc.,\n51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.\n\nOn Debian and systems the full text of the GNU General Public\nLicense version 2 can be found in the file\n\"/usr/share/common-licenses/GPL-2\".",
"contentType": "text/plain"
}
}
}
```
We create above data as far as we can parse Debian's copyright file. If we cannot parse it, maybe copyright file format is old style, we need to license information in licenses.yml file.
Even if we cannot get license informaion from copyright file and licenses.yml file, we set license to "NOASSERTION".
By this improvement, even if we don't have licenses information from licenses directory, we can create sbom without errors.
For test, rename the licenses directory to any name then create sbom.
```
$ mv licenses/ aaa
```
Create SPDX format sbom.
```
2026-07-09 05:17:56,235:WARNING: x11-xkb-utils: Cannot get licenses from Copyright file and user defined license mapping file. So, set NOASSERTION as license.
2026-07-09 05:17:56,235:WARNING: xkb-data: Cannot get licenses from Copyright file and user defined license mapping file. So, set NOASSERTION as license.
2026-07-09 05:17:56,235:WARNING: xserver-common: Cannot get licenses from Copyright file and user defined license mapping file. So, set NOASSERTION as license.
2026-07-09 05:17:56,235:WARNING: xwayland: Cannot get licenses from Copyright file and user defined license mapping file. So, set NOASSERTION as license.
2026-07-09 05:17:56,235:WARNING: xwayland-init: Cannot get licenses from Copyright file and user defined license mapping file. So, set NOASSERTION as license.
2026-07-09 05:17:56,405:INFO: Create spdx format sbom for emlinux-image-weston
2026-07-09 05:17:56,695:INFO: sbom was created to /home/build/work/build/tmp/deploy/sbom/emlinux-image-weston-emlinux-bookworm-imx8mpevk/emlinux-image-weston-spdx.json
build@235302c2d5ac:~/work$ view /home/build/work/build/tmp/deploy/sbom/emlinux-image-weston-emlinux-bookworm-imx8mpevk/emlinux-image-weston-spdx.json
```
Create CycloneDX format sbom.
```
2026-07-09 05:15:46,184:WARNING: x11-xkb-utils: Cannot get licenses from Copyright file and user defined license mapping file. So, set NOASSERTION as license.
2026-07-09 05:15:46,184:WARNING: xkb-data: Cannot get licenses from Copyright file and user defined license mapping file. So, set NOASSERTION as license.
2026-07-09 05:15:46,184:WARNING: xserver-common: Cannot get licenses from Copyright file and user defined license mapping file. So, set NOASSERTION as license.
2026-07-09 05:15:46,184:WARNING: xwayland: Cannot get licenses from Copyright file and user defined license mapping file. So, set NOASSERTION as license.
2026-07-09 05:15:46,184:WARNING: xwayland-init: Cannot get licenses from Copyright file and user defined license mapping file. So, set NOASSERTION as license.
2026-07-09 05:15:46,355:INFO: Create spdx format sbom for emlinux-image-weston
2026-07-09 05:15:46,644:INFO: sbom was created to /home/build/work/build/tmp/deploy/sbom/emlinux-image-weston-emlinux-bookworm-imx8mpevk/emlinux-image-weston-spdx.json
```
Both show warnings because we couldn't read lincense information from copyright file, but sbom files were created.
Change 3 is support to have layer's own licenses.yml file to manage licneses for layer's own packages.
Direcotry structure is same as meta-emlinux's one.
In custom layer, you can have following data.
```
├── licenses.yml
└── license_texts
└── bash
└── FOO.txt
```
The licenses.yml is following text.
```
bash:
licenses:
license:
- FOO
text:
FOO: bash/FOO.txt
```
Foo.txt file contains "Test LICENSE" in it.
So, sbom has following data.
```
{
"extractedText": "Test LICENSE\n",
"licenseId": "LicenseRef-bash-FOO-d5d4d27955c443937234e4e20fd0c460abf55feca458146b2da47b4ab4bcd3e4",
"name": "Custom License for bash LicenseRef-bash-FOO-d5d4d27955c443937234e4e20fd0c460abf55feca458146b2da47b4ab4bcd3e4"
},
>>> snip <<<
"licenseConcluded": "LicenseRef-bash-FOO-d5d4d27955c443937234e4e20fd0c460abf55feca458146b2da47b4ab4bcd3e4",
"licenseDeclared": "LicenseRef-bash-FOO-d5d4d27955c443937234e4e20fd0c460abf55feca458146b2da47b4ab4bcd3e4",
```
Signed-off-by: Masami Ichikawa <masami.ichikawa@miraclelinux.com>
masami256
force-pushed
the
improve-license-handling
branch
from
f6af9dd to
8d351af
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains following 2 commits.
Changes in "scripts: Sort layer list by its priority"
The find_layer() returns layer list that sort order is definition order in
bblayer.conf.
However, it would be useful that list is sorted by layer priority to know which
is most prioritized.
Changes in "sbom: Improbe license handling"
This commit improbe license handling with following changes.
The 1st change allow dictionary data in licenses.yml file to write license text into sbom file.
Current format is following list.
New fommat allows following 3 formats.
A: Current format style
B: Anather list format
C: Dictionary format
A and B is same meaning that only diffrence is style.
So, C is new format.
If LICENSE_A and LICENSE_B is SPDX ID, you don't have to add license text in the sbom.
However, if LICENSE is not in SPDX ID list (https://spdx.org/licenses/), you need to add license text in the sbom.
Format C support this feature.
You can put license text file in /conf/licenses/license_texts directory. So, you need to create directory in the license_texts directory then put license text.
If path is not absolute path, script makes file path to /conf/licenses/license_texts//.
If path si absolute path, scripte read from it by given path.
For example, libbpf1 package contains License line "GPL-2 with Linux-syscall-note exception" which is not in the SPDX license id list.
So, we create extra license using LicenseRef- prefix.
Also, we insert license text into the sbom in hasExtractedLicensingInfos section in SPDX fomat sbom.
CycloneDX format sbom also contains license text.
We create above data as far as we can parse Debian's copyright file. If we cannot parse it, maybe copyright file format is old style, we need to license information in licenses.yml file.
Even if we cannot get license informaion from copyright file and licenses.yml file, we set license to "NOASSERTION".
By this improvement, even if we don't have licenses information from licenses directory, we can create sbom without errors.
For test, rename the licenses directory to any name then create sbom.
Create SPDX format sbom.
Create CycloneDX format sbom.
Both show warnings because we couldn't read lincense information from copyright file, but sbom files were created.
Change 3 is support to have layer's own licenses.yml file to manage licneses for layer's own packages.
Direcotry structure is same as meta-emlinux's one.
In custom layer, you can have following data.
The licenses.yml is following text.
Foo.txt file contains "Test LICENSE" in it.
So, sbom has following data.
To do this, we collect license information and create license information. In the process we create license ids that are SPDX ID style license ID because CycloneDX allows SPDX style ID. Then, create SBOM for SPDX or CycloneDX.
Test
Create SPDX and CycloneDX sbom
Build emlinux-image-weston with following setting.
Then run following commands.
create_sbom.py
--distro bookworm
--image emlinux-image-weston
--supplier "Cybertrust Japan Co., Ltd."
--product EMLinux3
--sbom-format spdx
Build emlinux-image-weston with following setting.
Then, run following commans.
Check license text
This check doesn't matter what MACHINE and DISTRO pair, and sbom format are used.
After creating sbom files, check license text.
Create sbom without any license mapping file
This check doesn't matter what MACHINE and DISTRO pair is used.
Rename conf/licenses directory to conf/aaa in the meta-emlinux directory.
Then create sbom file.
Use user defined licenses file
This check doesn't matter what MACHINE and DISTRO pair, and sbom format are used.
Case 1, put licenses.yml in a custome layer
Prepare:
2 Setup following directory structure in meta-custom/conf.
The licenses.yml file is here.
license_texts/bash/FOO.txt is here.
Then create sbom.
Case 2, pass path to the licenses.yml by command line argument
Prepare:
licenses.yml is here
Foo.txt is here
Then create sbom such as following command.
Use user defined license mapping file
This check doesn't matter what MACHINE and DISTRO pair, and sbom format are used.
Before create sbom, check dash's license from already created sbom. It has publi-comain license.
Case 1, put license-mapping.yml in a custom layer
Prepare:
2 Put license-mapping as meta-custom/conf/licenses/license-mapping.yml.
File is here. It replace public-domain to CC-BY-1.0.
Then create sbom.
Case 2, use license-mapping.yml passing by command line
File is here.
Then create sbom such as following command.
Test result
Create SPDX and CycloneDX sbom
Sbom files were created without any warnings.
SPDX sbom for emlinux-bookworm.
CycloneDX sbom for emlinux-bookworm.
SPDX sbom for emlinux-trixie.
CycloneDX sbom for emlinux-trixie.
Check license text
In the kmod package, there is no LicenseRef- ID.
SPDX sbom
CycloneDX sbom
In the systemd-sysv, there are license text are stored that are extracted from Copyright file.
SPDX sbom
CycloneDX sbom
Create sbom without any license mapping file
In the libflite1, there are license text that copied from text file specified in the licenses.yml.
SPDX sbom
CycloneDX sbom
Create sbom without any license mapping file
We got many warnings but sbom was created.
Some license is not parsed so these license is set to NOASSERTION in the sbom.
Use user defined licenses file
Case 1, put licenses.yml in a custome layer
bash's license was our test license.
Case 2, pass path to the licenses.yml by command line argument
bash's license was our test license.
Use user defined license mapping file
Case 1, put license-mapping.yml in a custom layer
public-domain was replaced to CC-BY-1.0 in dash's license information
Case 2, use license-mapping.yml passing by command line
dash's license contains CC-BY-2.0 instead of public-domain.