Packaging of Perl Modules ========================= This file explains the packaging guidelines that I am attempting to follow for Free Libre Open Source Software (FLOSS) Perl modules from [CPAN](https://metacpan.org/) These guidelines are for RPM packages, not for installing modules via the native CPAN installer (which also works). YJL RPM Macros for Perl ----------------------- The RPM Macros used when building Perl and Perl modules are defined in [`/usr/lib/rpm/macros.d/macros.perl`](SOURCES/rpm-macros-perl-5.36) which is owned by the `perl-devel` package. For packaging modules from CPAN, the following macros are used: * `%{perl5_vendorlib}` --- The base directory for RPM packaged perl modules that are architecture independent. * `%{perl5_vendorarch}` --- The base directory for RPM packaged perl modules with compiled components. * `%{perl5_API}` --- Used in YJL to limit a `noarch` package to the series of Perl it was packaged for (e.g. 5.38.x) * `%{perl5_ABI}` --- Used in YJL to limit a binary package to the series of Perl it was packaged for. * `%{perl5_cpanlic}` --- Used in YJL for the handful of cases where a Perl packages specifies a license but does not actually include the license in the package tarball. ### Non-Perl Specific Macros Virtually every Perl module has tests that should be run, but to avoid circular dependency issues one should *always* be able to build a Perl module without running the tests. The `%{runtests}` macro is a Boolean macro that YJL uses to determine whether or not tests should be run. To conditionally run tests, I use the following in my `~/.rpmmacros` file: %runtests fubar When I do not wish to run tests, I just comment that out my `~/.rpmmacros` file. More information on that macro can be found at (00-NON-STANDARD-MACROS.md#the-runtests-macro) The `%{_fixperms}` macro is a standard RPM macro that expands to `/usr/bin/chmod -Rf a+rX,u+w,g-w,o-w` Perl likes to install stuff without the write permission bit set, and that causes problems when RPM wants to strip a binary library. That macro is only needed for binary modules and is a standard part of RPM itself. ### Spec File Portability Note Of the five YJL Perl specific RPM macros, `%{perl5_API}`, `%{perl5_ABI}`, and `%{perl5_cpanlic}` should be used in such a way that the spec file still builds on other systems where they will not be defined. For `%{perl5_vendorlib}` and `%{perl5_vendorlib}`, to build a YJL Perl module RPM spec file on another system the user will have to define those macros on their system. Usually just adding the following to a `~/.rpmmacros` file will suffice: %perl5_vendorlib %{perl_vendorlib} %perl5_vendorarch %{perl_vendorarch} While I could have just used the more standard convention for the purpose, it is my opinion a good idea to restrict the macros to the Perl5 just so that it makes it easier to have a future Perl 7 installed on the same system as Perl 5. Scripting language specific macros should be versioned, and I chose to not follow the incorrect mainstream naming scheme just because that is what all (or most) distributions do. Other distributions LOVE to have distribution-specific Perl and Python macros that are difficult to identify what it is they are trying to do, making building their RPM spec files a nightmare on another system, so I do not feel bad about this deviation one bit *especially* since this deviation from common practice is cake to work around. First Line Of The RPM Spec File ------------------------------- The very first line should define the CPAN name of the module in a macro name `cpanname`. Usually it is the Perl module but replacing any occurrences `::` with a `-`, e.g. the Perl module `Test::More::UTF8` would have a CPAN name of `Test-More-UTF8`. Precisely, it has the name of the tarball on CPAN without the version and tarball extension. Example first line: %global cpanname Text-Template RPM Spec File Metadata Tags --------------------------- The RPM spec file `Name:` metadata field then contains the following: Name: perl-%{cpanname} For the RPM spec file `Summary:` metadata field, it is best to use the summary provided after the __NAME__ heading on the CPAN web page for the Perl module. For example: Summary: Expand template text with embedded Perl When a Perl module only contains text files as opposed to binary files (this is usually the case), the Perl module package __MUST__ be defined as a `noarch` package: BuildArch: noarch While not strictly required, I do like to have an empty line between that first group of metadata tags and the next group. For the RPM spec file `Group:` metadata tag, how to categorize groups in YJL has not yet been determined. Generally for the present I am just using `Development/Libraries` as such: Group: Development/Libraries To me that does not quite feel right as most Perl modules are both development *and* run-time libraries. I will figure that out later. It seems on my CentOS 7.9.2009 (running the ancient Perl 5.16.3) that many Perl modules just do not specify a group. Anyway... For the RPM spec file `License:` metadata tag, the [SPDX License Identifier](https://spdx.org/licenses/) should be used. Many (but not) Perl modules on CPAN specify they use the same license as Perl5 itself, and on CPAN those packages are labeled as License: perl_5 Perl 5 is dual-license, giving the user the option of using either the GPL 1.0 (or newer) or the Perl Artistic license. The correct way using SPDX to specify the `License:` metadata tag in those cases: License: GPL-1.0-or-later or Artistic-1.0-Perl For the RPM spec file `URL:` metadata tag, it should point to the URL on CPAN for the module. For example: URL: https://metacpan.org/pod/Text::Template The RPM spec file `SOURCE:` metadata tag needs to point to the CPAN link for the module source tarball: Source0: https://cpan.metacpan.org/authors/id/M/MS/MSCHOUT/%{cpanname}-%{version}.tar.gz The path on `cpan.metacpan.org` will differ according to the author of the package. BuildRequires ------------- Every Perl module needs BuildRequires: perl-devel That is the package that owns the file defining the Perl specific RPM macros that are used when building a Perl module. Any package that builds using `Makefile.PL` (most of them) will need BuildRequires: perl(ExtUtils::MakeMaker) >= 6.76 The reason for that, some of the options that are passed to `Makefile.PL` are not defined in earlier versions of `ExtUtils::MakeMaker`. Of course if the package itself specifies an even newer version of `ExtUtils::MakeMaker` is needed, then specify the newer version. Note the version of `ExtUtils::MakeMaker` in Perl 5.36.1 is 7.64, for YJL itself that version requirement will always be met regardless of whether or not it is specified, but it still needs to be specified so that people trying to build the package for another distribution (such as CentOS 7 where it is only at version 6.68 if not updated). ### Determining Build Requirements For other build requirements, a packager needs to distinguish between those that are actually needed to build the module and those that are only needed if running tests. For `noarch` Perl modules that make use of `Makefile.PL`, usually `perl-devel` and `perl(ExtUtils::MakeMaker) >= 6.76` are the only `BuildRequires` needed outside of the test suite. Inside the module source, there will be a file usually called `META.json` or `MYMETA.json` that can be used to find out the proper build requirements. There will be a JSON entry called `"prereqs"` that has the information needed. Within that entry, ignore what it is `"develop"` but generally anything that is `"configure"` should be added as a RPM spec `BuildRequires:` meta tag, wrapped of course in `perl()`. For example: "prereqs" : { "configure" : { "requires" : { "ExtUtils::MakeMaker" : "6.78" }, "suggests" : { "JSON::PP" : "2.27300" } }, [...] }, That would translate to: BuildRequires: perl(ExtUtils::MakeMaker) >= 6.78 BuildRequires: perl(JSON::PP) >= 2.27300 Note that in that example, `perl(JSON::PP) >= 2.27300` is not *strictly* required to build it as it is only listed as `"suggests"` but with a few exception, I do `BuildRequires:` the `"suggests"`. Those are the `BuildRequires:` that *must* be present on the RPM build system regardless of whether or not tests are run. Some (but not all) binary Perl modules will need a `BuildRequires:` added for a library dependency, e.g. for XML::Parser which links against `libexpat.so`: BuildRequires: expat-devel ### Conditional Test Requirements To run tests, the RPM build system will usually need quite a few additional Perl modules available---modules used by the test scripts themselves. Those `BuildRequires:` meta tags should be wrapped inside a conditional: %if 0%{?runtests:1} == 1 BuildRequires: perl(Foo::Bar) >= 3.14159 [...] %endif That way, the RPM build system only needs to install them if the `%{runtests}` macro is defined so that circular dependencies where A requires B for tests, B requires C for tests, and C requires A for tests can be worked around by building them without running tests first. Again in the `"prereqs"` entry of the JSON file, there will be a `"runtime"` and a `"test"` entry. Everything in there (except for the specified minimum version of `perl`) should be added within the `%{runtests}` conditional. For example: "prereqs" : { [...] "runtime" : { "requires" : { "Carp" : "0", "Data::Section" : "0", "File::Spec" : "0", "IO::Dir" : "0", "Module::Load" : "0", "Text::Template" : "0", "parent" : "0", "perl" : "5.006", "strict" : "0", "utf8" : "0", "warnings" : "0" } }, "test" : { "recommends" : { "CPAN::Meta" : "2.120900" }, "requires" : { "ExtUtils::MakeMaker" : "0", "File::Spec" : "0", "Test::More" : "0.96", "Try::Tiny" : "0" } } }, would translate to: %if 0%{?runtests:1} == 1 BuildRequires: perl(Carp) BuildRequires: perl(Data::Section) BuildRequires: perl(File::Spec) BuildRequires: perl(IO::Dir) BuildRequires: perl(Module::Load) BuildRequires: perl(Text::Template) BuildRequires: perl(parent) BuildRequires: perl(strict) BuildRequires: perl(utf8) BuildRequires: perl(warnings) BuildRequires: perl(CPAN::Meta) >= 2.120900 BuildRequires: perl(Test::More) >= 0.96 BuildRequires: perl(Try::Tiny) %endif Note that I left out `"perl" : "5.006"`, `"ExtUtils::MakeMaker" : "0"`, and the second `"File::Spec" : "0"`. Requiring specific versions of Perl itself is problematic because of `Epoch:` metadata tags that were necessary due to RPM evaluating version strings as integers delimited by a `.` while Perl evaluated everything after the dot as fractional. There is already a `BuildRequires: perl(ExtUtils::MakeMaker)` that precedes the `%{runtests}` conditional, so a second is not needed. And obviously do not also need two `BuildRequires: perl(File::Spec)` tags. Run-Time Requirements --------------------- For the RPM spec file `Requires:` tags, RPM is actually pretty good at figuring that out automatically *however* I always set it up manually anyway. Use the `"runtime"` section of the `"prereqs"` from the `META.json` or `MYMETA.json` file. For example: "prereqs" : { [...] "runtime" : { "requires" : { "Carp" : "0", "Data::Section" : "0", "File::Spec" : "0", "IO::Dir" : "0", "Module::Load" : "0", "Text::Template" : "0", "parent" : "0", "perl" : "5.006", "strict" : "0", "utf8" : "0", "warnings" : "0" } }, }, That translates to: Requires: perl(Carp) Requires: perl(Data::Section) Requires: perl(File::Spec) Requires: perl(IO::Dir) Requires: perl(Module::Load) Requires: perl(Text::Template) Requires: perl(parent) Requires: perl(strict) Requires: perl(utf8) Requires: perl(warnings) Notice again I left out the explicit Perl version requirement. An RPM package archive restricts the building of the module to a specific series of Perl versions (e.g. 5.36.x) or on some distributions, an even more specific of Perl. For YJL, this is handled with the `%{perl5_API}` and `%{perl5_ABI}` RPM macros, but they should be used in such a way that the package still works when built for another distribution. For `noarch` packages, use the following: %if 0%{?perl5_API:1} == 1 Requires: %{perl5_API} %endif For binary packages, instead use the following: %if 0%{?perl5_ABI:1} == 1 Requires: %{perl5_ABI} %endif In both cases, those `Requires:` are provided by the YJL build of Perl (see [perl.spec](SPECS/perl.spec)). By wrapping the `Requires:` in a conditional, the spec files will still build installable packages on other GNU/Linux distributions as well. The RPM Package Description --------------------------- For the `%description` part of a CPAN Perl Module, when available I just take the description offered on CPAN. Sometimes it does require some redaction for the purpose of a an RPM package. The RPM Spec File Build Preparation Section ------------------------------------------- Generally this will just be the two lines: %prep %setup -n %{cpanname}-%{version} In once case, the `Changes` file which I like to package with the `%doc` macro had an execution bit set on it in the source package. In that case, I fixed it within `%prep`: %prep %setup -n %{cpanname}-%{version} %__chmod -x Changes The RPM Spec File Build Section ------------------------------- Most (but not all) Perl modules on CPAN build using a `Makefile.PL` file to generate the Makefile. There are other build systems. I will cross that bridge when I come to them and describe them here, so they can be water under the bridge. ### Makefile.PL Build System When a Perl module has a `Makefile.PL` the `%build` section should *almost always* look like this: %build %__perl Makefile.PL INSTALLDIRS=vendor NO_PACKLIST=1 NO_PERLLOCAL=1 OPTIMIZE="$RPM_OPT_FLAGS" make %{?_smp_mflags} The `INSTALLDIRS=vendor` tells `Makefile.PL` that this is being packaged for installation and should use the directories defined when `%__perl` was built for distribution vendor-provided Perl modules. The `NO_PACKLIST=1` tells the `Makefile.PL` not to create a packlist. A packlist interferes with RPM package installation and the functionality is provided by RPM itself. The `NO_PERLLOCAL=1` tells the `Makefile.PL` that `perllocal.pod` should not be updated. That is for packages installed through the CPAN utility itself (or manually built on the system). The `OPTIMIZE="$RPM_OPT_FLAGS"` is really only needed for binary Perl modules but it does not hurt anything when used with `noarch` Perl modules. It allows the optimization flags set by RPM to be used. This is important for binary modules as it includes things like `-D_FORTIFY_SOURCE=2` that help protect against things like buffer overflows from being exploited. With the `make` command, the `%{?_smp_mflags}` again is really only needed for binary Perl modules but does not hurt anything with `noarch` Perl modules. It just speeds up the build process on systems that have more than one CPU core available for building. The RPM Spec File Check Section ------------------------------- In almost all cases, the `%check` section of the RPM spec file should look like this: %check %if 0%{?runtests:1} == 1 make test > %{name}-make.test.log 2>&1 %else echo "make test not run during package build." > %{name}-make.test.log %endif The `%if` block only runs the test if the RPM build system has defined the `%{runtests}` macro. The `%else` block indicates that tests were not run when that macro has not been defined on the build system. YJL policy is to log test results and package them with `%doc` just in case the end user ever wants or needs to review them, so that is why it is important to indicate tests were not run when they are not run. On the production build server they will always be run but to package the results of tests with `%doc` the results file needs to exist even if the tests were not run. In some cases, such as when an X11 server is needed to run the tests, the above `%check` system may need modification. The RPM Spec File Install Section --------------------------------- Both `noarch` and binary Perl modules generally need the following `%install` section: %install make install DESTDIR=%{buildroot} Perl tends to install modules without the write bit set. for noarch packages that is not a problem, but for binary Perl modules it interferes with the ability for RPM to strip the binaries. Binary Perl modules thus need the following addition: %{_fixperms} %{buildroot}%{perl5_vendorarch} That will set the write bit so that RPM stripping works. The RPM Spec File Files Section ------------------------------- The `%files` should always set the `%defattr`: %files %defattr(-,root,root,-) The package should own all directories it installs inside of the `%{perl5_vendorlib}` (for noarch) or `%{perl5_vendorlib}` (for binary) directories, even if those directories are already owned by a module that is dependency. With a dependency, it is all possible the dependency is installed in either `%perl5_corelib` or `%perl5_sitelib` for `noarch` dependencies, or in `%perl5_corearch` or `%perl5_sitearch` for binary dependencies. In those cases, the Perl module being packages would leave empty directories behind when uninstalled if it does own all directories inside of its module install directory. Since Perl likes to install without the write bit, I like to manually specify the attributes of installed files without the write bit. Due to a bug in current versions of RPM (see (00-RPM-POST-INSTALL-BUG.md)) the attributes of binary modules should set manually anyway. Honestly the RPM bug may not matter for binary Perl modules, they may not need the execution bit set to work on GNU/Linux, I do not know. Anyway what I do is manually set the the attributes for text files inside the Perl module directory to `0444` and I manually set the attributes for binary files inside the Perl module directory to `0555`. Thus, the attributes match what they looked like after the install but before `%{_fixperms}` and before the RPM scriptlets that automatically run after `%install`. Example: %files %defattr(-,root,root,-) %dir %{perl5_vendorarch}/XML %attr(0444,root,root) %{perl5_vendorarch}/XML/Parser.pm %dir %{perl5_vendorarch}/XML/Parser %dir %{perl5_vendorarch}/XML/Parser/Encodings %attr(0444,root,root) %{perl5_vendorarch}/XML/Parser/Encodings/Japanese_Encodings.msg %attr(0444,root,root) %{perl5_vendorarch}/XML/Parser/Encodings/README %attr(0444,root,root) %{perl5_vendorarch}/XML/Parser/Encodings/*.enc %attr(0444,root,root) %{perl5_vendorarch}/XML/Parser/Expat.pm %attr(0444,root,root) %{perl5_vendorarch}/XML/Parser/LWPExternEnt.pl %dir %{perl5_vendorarch}/XML/Parser/Style %attr(0444,root,root) %{perl5_vendorarch}/XML/Parser/Style/*.pm %dir %{perl5_vendorarch}/auto/XML %dir %{perl5_vendorarch}/auto/XML/Parser %dir %{perl5_vendorarch}/auto/XML/Parser/Expat %attr(0555,root,root) %{perl5_vendorarch}/auto/XML/Parser/Expat/Expat.so ### Man Pages For man pages, the standard `0644` permissions are fine: %attr(0644,root,root) %{_mandir}/man3/*.3* ### Make test log The output of `make test` should be packaged as documentation: %doc %{name}-make.test.log ### Package Documentation Any documentation file within the Perl module source that is potentially useful to the end user should be packaged with the `%doc` macro. This usually includes the `Changes` file, a `README` file, any license files, and with some packages the `examples` directory. RPM Packaging of the License File(s) ------------------------------------ Note that specification of `%license` is part of the `%files` section. Every Perl module __MUST__ package the upstream-provided License file(s) using both the `%license` macro *and* the `%doc` macro. Furthermore, the packager __MUST__ ensure that the `License:` meta-tag matches the upstream-provided License file(s). Unfortunately, a small handful of Perl modules on CPAN do not include the applicable license files. It is probably illegal and certainly bad form for anyone other that the upstream maintainer to add a license file to a package. For YJL there is a workaround. YJL has a packaged called `common-CPAN-licenses` that contains *most* of the various Open Source licenses used in Perl modules on CPAN. When a Perl module does not include the license text *and* the text of the licenses specified by the package are included in that package, add the following to the Run-Time `Requires:` RPM spec file meta tags: %if 0%{?perl5_cpanlic:1} == 1 Requires: common-CPAN-licenses %endif Then in the spec file `%install` section, add something like the following: %if 0%{?perl5_cpanlic:1} == 1 cat > Perl5-Licenses.txt << "EOF" This package specifies it uses the Perl 5 licenses but did not include them in the package source. They can be found in the following directory: %{perl5_cpanlic}/Perl5/ EOF %endif Obviously if the package does not specify the Perl 5 license, then that needs to be tailored to the license it does specify. The created `Perl5-Licenses.txt` (or whatever if it is a different specified license) can then be conditionally included in `%files`: %if 0%{?perl5_cpanlic:1} == 1 %license Perl5-Licenses.txt %doc Changes README samples Perl5-Licenses.txt %else %doc Changes README samples %endif %doc %{name}-make.test.log Disabling Makefile.PL Prompts ----------------------------- Sometimes a Makefile.PL script will want user interaction. To just accept the defaults, add the line: PERL_MM_USE_DEFAULT=1 \ directly above the `perl Makefile.PL` line. See the `ExtUtils::MakeMaker` documentation for more information. End Notes --------- These are general guidelines. There are conditions that are not covered by these guidelines, and conditions that may require deviation from them.