Fiddling with OpenBSD ports

This post is about me trying to get a hang of the OpenBSD ports system by making a newer version of one of the existing ports. This is more a bunch of notes to myself coz I attempted something, and probably not very interesting to anyone else…

I run Gitea on my OpenBSD.amsterdam VM. I have subscribed to the Gitea releases and noted that as of date they are on version 1.13.0 while the OpenBSD 6.8 package is still on 1.12.4. The -current ports version is only a version behind at 1.12.6 while the -stable ports is still at 1.12.4 (expected as OpenBSD only updates -stable in case of security issues). I figure if there were any critical security vulnerabilities between 1.12.4 and 1.13.0 there would have been an update to the package or port so it was probably fine; but I was also concerned maybe it wasn’t something critical and so ignored.

Anyways, I decided to download -stable ports and get it to use the latest version of Gitea instead. This would probably work well for something simple like Gitea but might not for more complex packages that might have version dependencies for packages not present in -stable ports. I figured this would be a good way to dip my toes into ports too; I could see what the Gitea port maintainer has done and work my way backwards.

As usual a good place to start with the Ports system is the OpenBSD docs. Also this page on AnonCVS as OpenBSD uses CVS for version management as opposed to Git which is more common now. The bsd.port.mk manpage is a good reference on the variables one can use within ports. And the Porting Guide is a good intro on how to get started submitting ports. Everything I am doing below is based on what I picked up from these.

Here’s what I did. As a first step I downloaded a zip of the ports tree and then updated it to -stable. Ports have various flavours. The zip file that I download is the -release flavour, which is the ports tree as the release date of my current OpenBSD install. Any updates to that are in what you’d call the -stable flavour. This is important to remember because unlike say FreeBSD ports, the branch/ flavour of OpenBSD ports that you use is tied to your OS version. You can’t use the latest version of ports with whichever OS version you are on – if you want the latest (-current) version then you must be running OpenBSD -current too.

Next I installed a package called portslist to let me find where the Gitea port is located.

I changed to this directory above and edited Makefile to change the Gitea version to 1.13.0. Then I ran the following which downloaded the new version and created checksums in distinfo:

(It is also possible to do make checksum after this to verify the checksums will work fine if you are actually making this change to the real ports tree – which I am not).

Then I did the following:

This extracted the downloaded source to cd /usr/ports/pobj/gitea-1.13.0/.

Something I noticed with the existing gitea version in ports is that it had a patches directory with two files and one of these was to do with the config file. It was targetting a file called custom/conf/app.ini.sample in the Gitea source. When I looked up this file in corresponding location at the extracted directory above (/usr/ports/pobj/gitea-1.13.0/gitea-src-1.13.0/custom/conf/) I saw that the file name had changed to app.example.ini. Looking at the patch I also saw that one of the things the port creator had done – apart from making a lot of sensible changes to lock things down (nice!) – was to change the paths. For example he would change a config value such as one below:

Interesting. From the bsd.port.mk manpage I saw that ${LOCALSTATEDIR} was a reference to OpenBSD’s location for the state information of any port/ package. It is in turn derived from a bunch of other variables, but essentially it translates to /var by default. So for example if we want Gitea to not store its state in tmp/local-repo as the above snippet in the default config suggested, but instead put it in a proper place under /var we would modify it as above.

This is very cool actually and for some reason I like it. This highlights what “porting” a package does. It is not just about making sure the package just compiles or works on OpenBSD but also about ensuring it fits in well with the system. You don’t want packages to have their state info for instance strewn around wherever they feel like; no, you want it to be in the correct place for your system so things are consistent and the end-user of the software finds things in the expected locations. It’s good that the Gitea port maintainer for OpenBSD went through the config file and made such changes and I admire that.

I was curious when these variables get substituted and noticed the following in the Makefile:

The ${SUBST_CMD} is a reference to pkg_subst (see manpage) and this substitutes all the variables with what they are actually supposed to be. So during this phase of the build process {$LOCALSTATEDIR} above would be converted to /var and subsequently the Gitea build process is none the wiser. As part of this the original file is backed up with a .beforesubst suffix and that threw light on this part of the Makefile:

We are cleaning up these files after everything’s done.

From the porting guide I saw the following section on how to make your own patches (which is what I wanted to do for the new example file):

In my case make show=WRKSRC pointed me to /usr/ports/pobj/gitea-1.13.0/go/src/code.gitea.io/gitea but that didn’t exist. I had /usr/ports/pobj/gitea-1.13.0/gitea-src-1.13.0 and that’s where all the source files seemed to be.

According to bsd.port.mk the variable ${WRKSRC} is supposed to be a sub-directory of ${WRKDIR}. I found that ${WRKDIR} points to /usr/ports/pobj/gitea-1.13.0 (where the source is extracted to) so looks like something else moves things to ${WRKSRC} later. Digging into this I came across port-modules(5) and learnt that as part of the Go support in ports during a pre-configure stage the source code is moved from ${MODGO_SUBDIR} to ${WKRSRC}. pre-configure is a hook to the configure target in the ports Makefile, and I know that make build (which is what compiles the source code) invokes make configure as part of its dependencies. In fact, the flow is that make build calls make configure which calls make patch which calls make extract which calls make checksum which calls make fetch… you’ll remember I had manually done the last three earlier.

At this point I realize I am getting a bit too in the weeds. :) What I want to do is create a patch for the new example file. The official steps for that require me to go into ${WRKSRC} but it sounds like to get there I have to do make configure first. That will apply any patches from the patches folder so first things first let’s get rid of the existing patch file so it doesn’t give any errors (as the file doesn’t exist any more). Thus I did the following (I am currently in /usr/ports/www/gitea).

The first time I ran this I got an error:

Ah, that’s because of the pre-configure target in the Makefile of Gitea itself which we saw above (remember it does the pkg_subst) so I removed the reference to that file for now:

Then I tried make configure again and continued with the rest of the steps:

Next I edited custom/conf/app.example.ini and incorporated all the changes manually from the patch file I had previously copied to /tmp. Some elements of the patch were no longer needed as it looked like the newer config file incorporated these as defaults now.

After this I am supposed to do the following:

This is supposed to work but I got the following error:

Doh! that folder does not exist coz it got moved as part of the pre-configure step. Hmm. So it sounds like all the steps I did above are not really needed? Or maybe it’s a typo in the docs and I am supposed to go to ${WRKDIST} rather than ${WRKSRC}? Not that I am complaining coz I got to learn some stuff; but time to restart.

I made a copy of the app.example.ini file and then started afresh.

I copied over the app.example.ini previously created into custom/conf and continued as before:

And whoo hoo! it worked. :) It showed me the patch and created a file called patches/patch-custom_conf_app_example_ini. Beautiful!

I edited Makefile again and changed the pre-configure step to include the new example file:

Now it’s time to cleanup and start afresh, but this time try and build to see if the patching succeeds:

It did. Yay!

The next step according to the docs is to do a fake install and also update the packaging list. This is a file pkg/PLIST in the ports directory. It lists all the files and also has entries to create new accounts etc. What I did was make a copy of it and then do the fake install and update:

Then I copied this stuff from PLIST.copy to PLIST (actually, I replaced whatever was there in PLIST because the original file had some extra stuff which I’ll talk about below):

This bit is what tells the package to create the users and groups, add the rc script, and also copy over the initial config file. Note: One thing I did here while copying from the original PLIST was rename app.ini.sample to app.example.ini.

The original file had these two sections which is what copies over the initial config (I’ve split them into four via a line break):

What this does is:

  1. Target the directory share/gitea/conf and then set the directory mode and owner etc. as detailed. Then the @sample ${SYSCONFDIR}/gitea/ basically copies that directory with the mode and owner to ${SYSCONFDIR}/gitea (i.e. /etc/gitea).
  2. The mode and owner etc. are then set back to defaults for any subsequent operations.
  3. Next, the share/gitea/conf/app.ini.sample folder is targetted. Again the mode and owner etc. are set as detailed and this time the @sample ${SYSCONFDIR}/gitea/app.ini says copy the above sample file with the mode and owners set to ${SYSCONFDIR}/gitea/app.ini.
  4. The mode and owner etc. are then set back to defaults for any subsequent operations.

This explains how the initial config file was created, but where the heck did share/gitea come from? The config file I patched was under custom/conf. The answer lies in the Makefile:

As part of the install I am copying the contents of this folder to share/gitea. And later as part of the actual install file, the PLIST file tells the ports/ packages system to copy these to /etc. Nice!

I did one more change to the PLIST file. The line share/gitea/conf/app.example.ini appears twice in the PLIST file – once from make update-plist and again from when I added it above. So delete the line that already exists.

OK, hopefully I am nearing the end now as I am starting to get tired. :) I cleaned up everything so there’s no leftovers and did a fake install and made a package:

Success!

Originally I didn’t have success got I got an error like this:

I think this is because I had some leftover build stuff from before even thought I did a make clean. So I went to /usr/ports/plist/amd64 and found two instances of the gitea-1.13.0 folder there and deleted them. After this I was able to create the package.

I can install the package now via make install.

Yay!

This has been a good learning experience. I don’t think I’ll ever be packaging anything myself but I have a better idea now of what goes on behind the scenes. I did want to package TailScale for OpenBSD at some point, maybe if I have free time in the future I’ll give it a shot as a learning exercise. At this point I’ve worked my way backwards to understand what’s happening, if I ever package something I’ll have to work forwards – which is an even bigger task.