Poudriere Guide
This blog post describes how to set up and use the Poudriere Build System.
The port's system is one of FreeBSD's greatest advantages for users who want flexibility and control over their software. It enables administrators to easily create and manage source-based installations using a system that is robust and predictable.
While the benefits of this feature are great, some of the most common complaints against port-based management are the time and resources it takes to compile each software program. This becomes even more of a problem when you manage many servers, each compiling its ports. While FreeBSD packages offer an alternative that speeds installation, they sacrifice control that ports allow.
To resolve this issue, administrators can use an application called Poudriere to create and manage custom packages. While technically designed to package a wide variety of architectures, Poudriere is often used as a packaging environment to package and host an entire infrastructure of FreeBSD servers.
Install the required packages
In the beginning, we will install all the required ports.
First, we have to install Poudriere.
$: pkg install poudriere
Finally, we also want to install a web server. This will serve two purposes. First, this will be the method by which our machines will be able to download the packages that we will be compiling. Second, Poudriere provides a web interface, so we can track the build process and monitor logs. In our case, we use the NGINX web server:
$: pkg install nginx
Create an SSL certificate and key
When we build packages with Poudriere, we want to be able to sign them with a private key. This ensures that all of our machines legitimize the packets created and that nobody intercepts the connection to the built computer to send malicious packets.
We're going to make sure we have an SSL directory that has two subdirectories called keys and certs. We can do this in one command by typing:
$: mkdir -p /usr/local/etc/ssl/{keys,certs}
Our private key, which must be kept secret, is stored in the key directory. This will be used to sign the packages we are going to create. We can lock the directory so that users without root or sudo privileges cannot interact with the directory or its contents:
$: chmod 0600 /usr/local/etc/ssl/keys
Next, we create a 4096-bit key, called poudriere.key, and place it in our key's directory by typing:
$: openssl genrsa -out /usr/local/etc/ssl/keys/poudriere.key 4096
After the key has been generated, we can create a public certificate from it by typing:
$: openssl rsa -in /usr/local/etc/ssl/keys/poudriere.key -pubout -out /usr/local/etc/ssl/certs/poudriere.cert
We now have the SSL components we need to sign packets and verify the signatures. Later, we will configure our clients to use the generated certificate for package verification.
Configuring Poudriere
Now that we have our SSL certificate and key, we can start configuring Poudriere.
The main configuration file located at /usr/local/etc/poudriere.conf. We open this file:
$: nano /usr/local/etc/poudriere.conf
The Poudriere configuration file is very well commented and most of the settings must be predefined. We'll be making some specific changes, but leaving the majority of them intact.
If we use UFS as the file system, the following option must be commented out:
NO_ZFS=yes
On the other hand, if our server uses ZFS, we can configure Poudriere to use a specific pool by setting the ZPOOL option. In this pool, we can specify the trunk that Poudriere should use for packets, protocols, etc. with the ZROOTFS option. Note that these two options should not be set when the NO_ZFS option is set to “yes”:
# NO_ZFS=yes
ZPOOL=tank
ZROOTFS=/poudriere
When building software, Poudriere uses some type of jail to separate the build system from the main operating system. Next, we need to fill in a valid host where the build machine can download the software it needs to do the jails. This is configured via the FREEBSD_HOST option.
This option should already exist, even though it is not currently set to a valid host. We can change this to the default path. ftp://ftp.freebsd.org or use a narrower mirror if we know one:
FREEBSD_HOST=https://download.freebsd.org
Next, we want to be sure that our data directory is set correctly within the Poudriere root. This is controlled with the POUDRIERE_DATA option and should be set by default. However, we'll comment on the option to be certain:
POUDRIERE_DATA=${BASEFS}/data
The next options that we should comment on are the CHECKCHANGEDOPTIONS and CHECKCHANGEDDEPS options. The first option tells Poudriere to rebuild packages if the options for them have changed. The second option tells Poudriere to rebuild packages if dependencies have changed since the last compilation.
Both options exist in the form we want in the configuration file. We just have to comment them out:
CHECK_CHANGED_OPTIONS=verbose
CHECK_CHANGED_DEPS=yes
Next, we're going to point Poudriere at the SSL key we created so that packages can be signed during the build. The option used to specify is called PKGREPOSIGNING_KEY. We will uncheck this option and change the path to reflect the location of the SSL key we created earlier:
PKG_REPO_SIGNING_KEY=/usr/local/etc/ssl/keys/poudriere.key
Finally, we can set the URL_BASE string to the domain name or IP address where our server can be reached. This is used by Poudriere to create links in the output that can be clicked. We should include the log and end the value with a slash:
URL_BASE=http://server_domain_or_IP/
When we're done making changes, we'll save and close the file.
Create the built environment
Next, we need to design our built environment. As mentioned earlier, Poudriere will build ports in an isolated environment with jails.
For our purposes, our jails build command looks like this:
$: poudriere jail -c -j 13-0x64 -v 13.0-RELEASE
This will take a while, so be patient. When done, we can see the one installed by typing:
$: poudriere jail -l
Output:
JAILNAME VERSION ARCH METHOD TIMESTAMP PATH
13-0x64 13.0-RELEASE-p2 amd64 ftp 2021-01-06 20:43:48 /usr/local/poudriere/jails/13-0x64
Once we've created a jail, we need to install a port structure.
We can use the -p flag to name our ports tree. We will name our tree HEAD because it summarizes exactly using this tree (the “head” or the most current point of the tree). Likewise, we will update it regularly so that it corresponds to the most current version of the available ports structure:
$: poudriere ports -c -m git+https -B main -p HEAD
This procedure will also take a while because the entire port structure has to be fetched and extracted. When this is done, we can view our port's tree by typing:
$: poudriere ports -l
Now that this step is complete, we have the structures to compile our ports and build packages. Next, we can start by putting together our list of ports to create and configure the options we need for each software.
Create a port building list and set port options
We are going to make a list of ports that we can pass directly to Poudriere.
The file should list the port category followed by a slash and the port name to reflect its position in the ports tree:
port_category/first_port
port_category/second_port
port_category/third_port
All necessary dependencies are also created automatically. So, we don't need to track down the entire dependency structure of the ports that we want to install. We can create this file manually, but if our base system already has most of the software installed, we can create the file automatically.
Before we do this, it is usually a good idea to remove any unnecessary dependencies from our system to keep the port list as clean as possible. We can do this by typing:
$: pkg autoremove
Thereafter, we can get a list of the software that we have explicitly installed on our build system:
$: pkg query -e "%a==0" "%o" | sort -d | tee /usr/local/etc/poudriere.d/port-list
If there are ports that we don't want to add, we'll remove the associated line. This is also an opportunity to add additional ports that we may need.
If we use certain make.conf options to create our ports, we can create a make.conf file for each jail within our /usr/local/etc/poudriere.d directory. For example, we can create a make.conf file for our jail with this name:
$: nano /usr/local/etc/poudriere.d/13-0x64-make.conf
I prefer to create a global make.conf that is valid for all jails.
$: nano /usr/local/etc/poudriere.d/make.conf
Inside, we can specify all the options we want to use when creating our ports. For example, if we prefer not to build documentation, samples, native language support, or X11 support, we can specify:
OPTIONS_UNSET+= DOCS NLS X11 EXAMPLES
We can also specify Default_Versions. In my case, it looks like this:
DEFAULT_VERSIONS+= mysql=8.0 php=8.0
In this case, all packages and/or their dependencies are built with the specified version.
Subsequently, we can configure any of our ports that will create files with the options selected.
We can configure anything that has not yet been configured with the options command. We should pass both the port tree we created (with the -p option) and the jail for which we set these options (with the -j option). Not only that, but we also need to provide the list of ports we want to configure with the -f option.
$: poudriere options -j 13-0x64 -p HEAD -f /usr/local/etc/poudriere.d/port-list
We see a dialog for each of the ports in the list and all dependencies for which no corresponding options are set in the -options directory. The information in our make.conf file is preselected in the selection screens. We'll select all the options we want to use.
If we want to reconfigure the options for our ports in the future, we can run the above command again with the -c option. This will show us all the available configuration options, regardless of whether we have made a selection in the past:
$: poudriere options -c -j 13-0x64 -p HEAD -f /usr/local/etc/poudriere.d/port-list
Build the ports
Now we're finally ready to start building ports.
We enter the following to update the jail:
$: poudriere jail -u -j 13-0x64
Then we enter the following to update the ports structure:
$: poudriere ports -u -p HEAD
Once that is done, we can start the build process.
Note: this can be a very long process. If we are connected to the server via SSH, we recommend installing the screen package and starting a session:
$: pkg install screen
$: rehash
$: screen
To start the build, we just need to use the bulk command and point to all of the individual parts that we have configured. If we've used the values in this guide, the command looks like this:
$: poudriere bulk -j 13-0x64 -p HEAD -f /usr/local/etc/poudriere.d/port-list
This starts with many workers (depending on our poudriere.conf file or the number of CPUs available) and starts building the ports.
At any time during the creation process, we can get information about the progress by holding down the CTRL key and pressing t.
Certain parts of the process produce more output than others.
Setting up NGINX for providing the front end and the repository
For this step, the NGINX must be set up with virtual hosts.
The following hostname entered /etc/hosts:
127.0.0.1 bsd. «Domain»
For Poudriere to run under NGINX, the following file created under /usr/local/etc/nginx/vhosts/ under the name poudriere.conf with this content:
server {
listen 80 default;
server_name bsd.><domain>>;
root /usr/local/share/poudriere/html;
location /data {
alias /usr/local/poudriere/data/logs/bulk;
autoindex on;
}
location /packages {
root /usr/local/poudriere/data;
autoindex on;
}
}
Then restart the NGINX:
$: service nginx restart
Next, we're going to make a small change to our mime.types file. If we click on a log in the web browser with the current settings, the file is downloaded and not displayed as normal text. We can change this behavior by marking files with the extension .log. as plain text files.
We open the file mime.types in our text editor:
$: nano /usr/local/etc/nginx/mime.types
We find the entry indicating the text / plain content type, and append the log to the end of the current list of file types separated by a space:
text/mathml mml;
text/plain txt log;
text/vnd.sun.j2me.app-descriptor jad;
Now, we can display the Poudriere web interface by going to the domain name or the IP address of our server in our web browser, bsd.«domain».
Configure package clients
Now that we have created packages and configured a repository to host our packages, we can configure our clients to use our server as a package source.
Configure the build server to use the own package repo
We can start by configuring the build server to use the packages it built.
First, we need to create a directory for our repository configuration files:
$: mkdir -p /usr/local/etc/pkg/repos
In this directory, we can create our repository configuration file. It must end in .conf, so we'll call it poudriere.conf to make its purpose cleaner:
$: nano /usr/local/etc/pkg/repos/poudriere.conf =>
poudriere: {
url: "file:///usr/local/poudriere/data/packages/13-0x64-HEAD",
mirror_type: "srv",
signature_type: "pubkey",
pubkey: "/usr/local/etc/ssl/certs/poudriere.cert",
enabled: yes,
priority: 100
}
If we only select packages that we have created ourselves (the more secure option), we can omit the priority setting, but we should disable the default repositories. We can do this by creating another repo file that will overwrite the default repository file and disable it:
$: nano /usr/local/etc/pkg/repos/freebsd.conf =>
FreeBSD: {
enabled: no
}
Regardless of our configuration choices, we should now be ready to use our repository. We'll update our package list by entering:
$: pkg update
Now, our server can use the pkg command to install packages from our local repository.
Configure remote clients to use your build machine's repository
One of the most compelling reasons to set up Poudriere on a build machine is to use that host as a repository for many other machines. All we have to do to get this working is to download the public SSL certificate from our build machine and set up a similar repository definition.
To connect to our build host from our client computers, we should start an SSH agent on our local computer to store our SSH key credentials.
We need to add our SSH key by typing:
$: ssh-add
Then, we first have to create the directory structure (if it doesn't exist) so that we can save the certificate. We'll go ahead and create a directory for keys so we can use it for future tasks:
$: mkdir -p /usr/local/etc/ssl/{keys,certs}
Now we can connect to our build machine with SSH and send the certificate file back to our client machine. Since we've forwarded our SSH credentials, we should be able to do this without asking for a password:
$: ssh user@server_domain_or_IP 'cat /usr/local/etc/ssl/certs/poudriere.cert' | tee /usr/local/etc/ssl/certs/poudriere.cert
This command connects to the build machine from our client computer using our local SSH credentials. As soon as the connection is established, it shows the content of your certificate file and directs us through the SSH tunnel back to our remote client computer. From there, we use the tee combination to write the certificate into our directory.
Once this is done, we can create our repository directory structure, like on the build machine itself:
$: mkdir -p /usr/local/etc/pkg/repos
Now, we can create a repository file similar to the one used on the build machine:
$: nano /usr/local/etc/pkg/repos/poudriere.conf =>
poudriere: {
url: "https://server_domain_or_IP/packages/13-0x64-HEAD/",
mirror_type: "https",
signature_type: "pubkey",
pubkey: "/usr/local/etc/ssl/certs/poudriere.cert",
enabled: yes,
priority: 100
}
If we just want to use our compiled packages, the file should look something like this:
poudriere: {
url: "https://server_domain_or_IP/packages/13-0x64-HEAD/",
mirror_type: "https",
signature_type: "pubkey",
pubkey: "/usr/local/etc/ssl/certs/poudriere.cert",
enabled: yes
}
If we only want to use our packages, we must remember to create a different repository configuration file to override the default FreeBSD repository configuration:
$: nano /usr/local/etc/pkg/repos/freebsd.conf =>
FreeBSD: {
enabled: no
}
After we're done, let's update our pkg database to start using our custom compiled packages:
$: pkg update
Cron job
So that Poudriere automatically updates the jails and builds the packages, we will set up a cron job.
We create the following file under /usr/local/etc/poudriere.d/scripts:
$: mkdir /usr/local/etc/poudriere.d/scripts
$: nano /usr/local/etc/poudriere.d/scripts/poudriere-cron.sh =>
#!/bin/sh
SCRIPTNAME=`basename "$0"`
# check for running script
STATUS=`ps ax | grep "$SCRIPTNAME" | grep -v grep | wc -l`
# compare to 2 because the ` create a subprocess
if [ "$STATUS" -gt 2 ]; then
echo "already running ... exit"
exit 0
fi
# The build
POUDRIERE="/usr/local/bin/poudriere"
PORTLIST="/usr/local/etc/poudriere.d/port-list"
JAILS="13-0x64"
REPOS="HEAD"
URL="https://bsd.<<domain>>"
poudriere_build() {
for JAIL in $JAILS; do
for REPO in $REPOS; do
echo "Started $JAIL / $REPO / $SET ("`/bin/date | /usr/bin/tr -d '\n'`")"
"$POUDRIERE" bulk -j "$JAIL" -z "$SET" -p "$REPO" -f "$PORTLIST" > /dev/null
echo " Cleaning $REPO ("`/bin/date | /usr/bin/tr -d '\n'`")"
"$POUDRIERE" pkgclean -j "$JAIL" -z "$SET" -p "$REPO" -f "$PORTLIST" -y > /dev/null
echo " Finished $REPO ("`/bin/date | /usr/bin/tr -d '\n'`")"
done
done
}
repos_update() {
echo "[$SCRIPTNAME] Updating ports tree..."
for REPO in $REPOS; do
echo "[$SCRIPTNAME] Updating ports tree... $REPO"
"$POUDRIERE" ports -p "$REPO" -u > /dev/null
if [ $? -ne 0 ]; then
echo " Error updating ports tree."
exit 1
fi
echo " Ports tree has been updated."
done
}
echo "This is a log of poudriere. Details: $URL"
echo ""
repos_update
poudriere_build
echo "[$SCRIPTNAME] Cleaning distfiles..."
"$POUDRIERE" distclean -p "$REPOS" -f "$PORTLIST" -y > /dev/null
echo "[$SCRIPTNAME] Finished. ("`/bin/date | /usr/bin/tr -d '\n'`")"
exit 0
Then make it executable:
$: chmod +x /usr/local/etc/poudriere.d/scripts/poudriere-cron.sh
Then we should run the script once to test whether everything works correctly.
If everything fits, we can include it in the /etc/crontab as follows:
§: nano /etc/crontab =>
10 0 * * * root /usr/local/etc/poudriere.d/scripts/poudriere-cron.sh
The build process will then run every night. If we have packages such as Chromium where the build takes longer than 24 hours, it can be adjusted here.
Command list
Poudriere make.conf
$: nano /usr/local/etc/poudriere.d/make.conf
Poudriere conf
$: nano /usr/local/etc/poudriere.conf
Portlist
$: nano /usr/local/etc/poudriere.d/port-list
Create a new jail
$: poudriere jail -c -j 13-0x64 -v 12.0-RELEASE
Update a jail
$: poudriere jail -u -j 13-0x64
Update ports
$: poudriere ports -u -p HEAD
Remove unused packages
$: poudriere pkgclean -j 13-0x64 -p HEAD -f /usr/local/etc/poudriere.d/port-list
Set options for the port list
$: poudriere options -c -j 13-0x64 -p HEAD -f /usr/local/etc/poudriere.d/port-list
Set options for a package
$: poudriere options -c -j 13-0x64 -p HEAD databases/mariadb103-client
Build packages without a cronjob
$: poudriere bulk -j 13-0x64 -p HEAD -f /usr/local/etc/poudriere.d/port-list
When upgrading the system, copy the options to the new version
$: cp -R /usr/local/etc/poudriere.d/12-2x64-HEAD-options/**.* /usr/local/etc/poudriere.d/13-0x64-HEAD-options/