There is a common saying among people that manage storage backups: If you don't test your backup, you probably don't have one. The same logic applies to automated systems designed to help you with open source.
Trust, but verify
We always advise our clients to automate as much of the open source management process as possible. There is no way without automation to reasonably maintain compliance. This includes the systems designed to provide your customers or end users with the "compliance bundle" - the required notices, licenses, and source code (for copyleft licenses) for all the open source projects you use.
Automation is necessary, but your organization needs to periodically check your generated compliance bundles and verify that the systems are working correctly. Automated systems can fail for various reasons, such as hardware malfunctions, software incompatibilities, or human errors. By testing your systems you can identify and address any issues before you are called on for compliance. Even if you find just one incomplete or incorrect compliance bundle, that probably means that there are deeper and more pervasive issues with your automated systems.
Some specific issues to look for:
- Does your disclosed open source version match the version that is actually used in your software?
- Are the different types of licenses being identified correctly, including in hard-to-automate situations like multi-licensed packages?
- Does every source code disclosure include a copy of the license?
- Do disclosures that include the Linux kernal also include all the necessary configuration and installation scripts?
- Are notices and license files being put into the compliance bundle?
It's not just copyleft
It's important to highlight that last point about noticies and license files. Many organizations know about and have plans for required source code disclosures under copyleft licenses like the GPL. But your systems should be prepared to handle every open source package you use. Essentially every open source license - even permissive licenses - still have compliance requirements. A compliance bundle needs to include all the necessary copies of licenses and author attributions for all the open source that you use. For many open source developers, credit is a primary motivating factor. Don't take that away.