This post is about me trying to get a hang of the OpenBSD ports system by making a newer version of one of the existing ports. This is more a bunch of notes to myself coz I attempted something, and probably not very interesting to anyone else…
I run Gitea on my OpenBSD.amsterdam VM. I have subscribed to the Gitea releases and noted that as of date they are on version 1.13.0 while the OpenBSD 6.8 package is still on 1.12.4. The -current
ports version is only a version behind at 1.12.6 while the -stable
ports is still at 1.12.4 (expected as OpenBSD only updates -stable
in case of security issues). I figure if there were any critical security vulnerabilities between 1.12.4 and 1.13.0 there would have been an update to the package or port so it was probably fine; but I was also concerned maybe it wasn’t something critical and so ignored.
Anyways, I decided to download -stable
ports and get it to use the latest version of Gitea instead. This would probably work well for something simple like Gitea but might not for more complex packages that might have version dependencies for packages not present in -stable
ports. I figured this would be a good way to dip my toes into ports too; I could see what the Gitea port maintainer has done and work my way backwards.
As usual a good place to start with the Ports system is the OpenBSD docs. Also this page on AnonCVS as OpenBSD uses CVS for version management as opposed to Git which is more common now. The bsd.port.mk manpage is a good reference on the variables one can use within ports. And the Porting Guide is a good intro on how to get started submitting ports. Everything I am doing below is based on what I picked up from these.
Here’s what I did. As a first step I downloaded a zip of the ports tree and then updated it to -stable
. Ports have various flavours. The zip file that I download is the -release
flavour, which is the ports tree as the release date of my current OpenBSD install. Any updates to that are in what you’d call the -stable
flavour. This is important to remember because unlike say FreeBSD ports, the branch/ flavour of OpenBSD ports that you use is tied to your OS version. You can’t use the latest version of ports with whichever OS version you are on – if you want the latest (-current
) version then you must be running OpenBSD -current
too.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
# change to a /tmp directory and download the ports zip and verify its signature cd /tmp ftp https://cdn.openbsd.org/pub/OpenBSD/$(uname -r)/{ports.tar.gz,SHA256.sig} signify -Cp /etc/signify/openbsd-$(uname -r | cut -c 1,3)-base.pub -x SHA256.sig ports.tar.gz # change to the /usr directory and extract the package here. it will be extracted to /usr/ports cd /usr doas tar xzf /tmp/ports.tar.gz # change ownership to myself as I'll be doing most of the operations as myself doas chown -R rakhesh:users ports # use cvs to get the -stable flavour. the server below is in the UK, be sure to choose something closer to you. notice I am getting the OPENBSD_6_8 tag cvs -qd anoncvs@anoncvs.spacehopper.org:/cvs checkout -rOPENBSD_6_8 -P ports # add this to your login scripts so the CVSROOT variable is set for when you run cvs later echo 'export CVSROOT="anoncvs@anoncvs.spacehopper.org:/cvs"' >> .bash_profile # Note to self: if I want to update the ports tree later all I do is: # cd /usr/ports # cvs -q up -Pd -rOPENBSD_6_8 |
Next I installed a package called portslist
to let me find where the Gitea port is located.
1 2 3 4 |
doas pkg_add portslist cd /usr/ports make search key=gitea cd www/gitea/ |
I changed to this directory above and edited Makefile
to change the Gitea version to 1.13.0. Then I ran the following which downloaded the new version and created checksums in distinfo
:
1 |
make makesum |
(It is also possible to do make checksum
after this to verify the checksums will work fine if you are actually making this change to the real ports tree – which I am not).
Then I did the following:
1 |
make extract |
This extracted the downloaded source to cd /usr/ports/pobj/gitea-1.13.0/
.
Something I noticed with the existing gitea version in ports is that it had a patches
directory with two files and one of these was to do with the config file. It was targetting a file called custom/conf/app.ini.sample
in the Gitea source. When I looked up this file in corresponding location at the extracted directory above (/usr/ports/pobj/gitea-1.13.0/gitea-src-1.13.0/custom/conf/
) I saw that the file name had changed to app.example.ini
. Looking at the patch I also saw that one of the things the port creator had done – apart from making a lot of sensible changes to lock things down (nice!) – was to change the paths. For example he would change a config value such as one below:
1 2 3 4 5 |
# before LOCAL_COPY_PATH = tmp/local-repo # after LOCAL_COPY_PATH = ${LOCALSTATEDIR}/gitea/tmp/local-repo |
Interesting. From the bsd.port.mk
manpage I saw that ${LOCALSTATEDIR}
was a reference to OpenBSD’s location for the state information of any port/ package. It is in turn derived from a bunch of other variables, but essentially it translates to /var
by default. So for example if we want Gitea to not store its state in tmp/local-repo
as the above snippet in the default config suggested, but instead put it in a proper place under /var
we would modify it as above.
This is very cool actually and for some reason I like it. This highlights what “porting” a package does. It is not just about making sure the package just compiles or works on OpenBSD but also about ensuring it fits in well with the system. You don’t want packages to have their state info for instance strewn around wherever they feel like; no, you want it to be in the correct place for your system so things are consistent and the end-user of the software finds things in the expected locations. It’s good that the Gitea port maintainer for OpenBSD went through the config file and made such changes and I admire that.
I was curious when these variables get substituted and noticed the following in the Makefile
:
1 2 |
pre-configure: ${SUBST_CMD} ${WRKDIST}/{custom/conf/app.ini.sample,main.go} |
The ${SUBST_CMD}
is a reference to pkg_subst
(see manpage) and this substitutes all the variables with what they are actually supposed to be. So during this phase of the build process {$LOCALSTATEDIR}
above would be converted to /var
and subsequently the Gitea build process is none the wiser. As part of this the original file is backed up with a .beforesubst
suffix and that threw light on this part of the Makefile:
1 2 3 |
post-install: @find ${WRKINST} -type f \ \( -name '*.beforesubst' -o -name '*.orig' \) -delete |
We are cleaning up these files after everything’s done.
From the porting guide I saw the following section on how to make your own patches (which is what I wanted to do for the new example file):
1 2 3 4 5 6 |
cd `make show=WRKSRC` ; cp foo/bar.c{,.orig} # edit foo/bar.c to fix the error. cd - ; make update-patches # this will create patches/patch-foo_bar_c with your modifications. |
In my case make show=WRKSRC
pointed me to /usr/ports/pobj/gitea-1.13.0/go/src/code.gitea.io/gitea
but that didn’t exist. I had /usr/ports/pobj/gitea-1.13.0/gitea-src-1.13.0
and that’s where all the source files seemed to be.
According to bsd.port.mk
the variable ${WRKSRC}
is supposed to be a sub-directory of ${WRKDIR}
. I found that ${WRKDIR}
points to /usr/ports/pobj/gitea-1.13.0
(where the source is extracted to) so looks like something else moves things to ${WRKSRC}
later. Digging into this I came across port-modules(5)
and learnt that as part of the Go support in ports during a pre-configure
stage the source code is moved from ${MODGO_SUBDIR}
to ${WKRSRC}
. pre-configure
is a hook to the configure
target in the ports Makefile
, and I know that make build
(which is what compiles the source code) invokes make configure
as part of its dependencies. In fact, the flow is that make build
calls make configure
which calls make patch
which calls make extract
which calls make checksum
which calls make fetch
… you’ll remember I had manually done the last three earlier.
At this point I realize I am getting a bit too in the weeds. :) What I want to do is create a patch for the new example file. The official steps for that require me to go into ${WRKSRC}
but it sounds like to get there I have to do make configure
first. That will apply any patches from the patches
folder so first things first let’s get rid of the existing patch file so it doesn’t give any errors (as the file doesn’t exist any more). Thus I did the following (I am currently in /usr/ports/www/gitea
).
1 2 3 4 5 |
# move out the old patch file mv patches/patch-custom_conf_app_ini_example /tmp # run make configure so it moves the source code make configure |
The first time I ran this I got an error:
1 2 3 4 |
Can't rename /usr/ports/pobj/gitea-1.13.0/gitea-src-1.13.0/custom/conf/app.ini.sample to /usr/ports/pobj/gitea-1.13.0/gitea-src-1.13.0/custom/conf/app.ini.sample.beforesubst: No such file or directory *** Error 1 in . (Makefile:38 'pre-configure') *** Error 2 in . (/usr/ports/infrastructure/mk/bsd.port.mk:2903 '/usr/ports/pobj/gitea-1.13.0/build-amd64/.configure_done': @cd /usr/ports/w...) *** Error 2 in /usr/ports/www/gitea (/usr/ports/infrastructure/mk/bsd.port.mk:2584 'configure': @lock=gitea-1.13.0; export _LOCKS_HELD=" gi...) |
Ah, that’s because of the pre-configure
target in the Makefile of Gitea itself which we saw above (remember it does the pkg_subst
) so I removed the reference to that file for now:
1 2 3 4 5 6 7 |
# before pre-configure: ${SUBST_CMD} ${WRKDIST}/{custom/conf/app.ini.sample,main.go} # after pre-configure: ${SUBST_CMD} ${WRKDIST}/main.go |
Then I tried make configure
again and continued with the rest of the steps:
1 2 3 4 5 6 7 8 |
# try configure again make configure # change to the source code folder that now exists cd `make show=WRKSRC` # make a copy of the file I want to change cp custom/conf/app.example.ini custom/conf/app.example.ini.orig |
Next I edited custom/conf/app.example.ini
and incorporated all the changes manually from the patch file I had previously copied to /tmp
. Some elements of the patch were no longer needed as it looked like the newer config file incorporated these as defaults now.
After this I am supposed to do the following:
1 |
cd - ; make update-patches |
This is supposed to work but I got the following error:
1 2 |
WRKDIST=/usr/ports/pobj/gitea-1.13.0/gitea-src-1.13.0 does not exist *** Error 1 in /usr/ports/www/gitea (/usr/ports/infrastructure/mk/bsd.port.mk:2570 'update-patches': @toedit=`WRKDIST=/usr/ports/pobj/gitea-...) |
Doh! that folder does not exist coz it got moved as part of the pre-configure
step. Hmm. So it sounds like all the steps I did above are not really needed? Or maybe it’s a typo in the docs and I am supposed to go to ${WRKDIST}
rather than ${WRKSRC}
? Not that I am complaining coz I got to learn some stuff; but time to restart.
I made a copy of the app.example.ini
file and then started afresh.
1 2 3 4 5 6 7 8 9 10 11 |
# cleanup stuff (this will delete the extracted directory) make clean # extract it again make extract # go the directory where we extracted cd `make show=WRKDIST` # make a copy of the file I want to change cp custom/conf/app.example.ini custom/conf/app.example.ini.orig |
I copied over the app.example.ini
previously created into custom/conf
and continued as before:
1 |
cd - ; make update-patches |
And whoo hoo! it worked. :) It showed me the patch and created a file called patches/patch-custom_conf_app_example_ini
. Beautiful!
I edited Makefile
again and changed the pre-configure
step to include the new example file:
1 2 |
pre-configure: ${SUBST_CMD} ${WRKDIST}/{custom/conf/app.example.ini,main.go} |
Now it’s time to cleanup and start afresh, but this time try and build to see if the patching succeeds:
1 |
make clean && make extract && make build |
It did. Yay!
The next step according to the docs is to do a fake install and also update the packaging list. This is a file pkg/PLIST
in the ports directory. It lists all the files and also has entries to create new accounts etc. What I did was make a copy of it and then do the fake install and update:
1 2 3 4 5 6 |
cp pkg/PLIST pkg/PLIST.copy # do a fake install. this does to ${WRKINST} make fake make update-plist |
Then I copied this stuff from PLIST.copy
to PLIST
(actually, I replaced whatever was there in PLIST
because the original file had some extra stuff which I’ll talk about below):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
@newgroup _gitea:787 @newuser _gitea:787:_gitea:daemon:Gitea Account:/var/gitea:/bin/sh @rcscript ${RCDIR}/gitea @bin sbin/gitea share/doc/pkg-readmes/${PKGSTEM} share/gitea/ share/gitea/conf/ @mode 750 @owner _gitea @group _gitea @sample ${SYSCONFDIR}/gitea/ @mode @owner @group share/gitea/conf/app.example.ini @mode 640 @owner _gitea @group _gitea @sample ${SYSCONFDIR}/gitea/app.ini @mode @owner @group |
This bit is what tells the package to create the users and groups, add the rc
script, and also copy over the initial config file. Note: One thing I did here while copying from the original PLIST
was rename app.ini.sample
to app.example.ini
.
The original file had these two sections which is what copies over the initial config (I’ve split them into four via a line break):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
share/gitea/conf/ @mode 750 @owner _gitea @group _gitea @sample ${SYSCONFDIR}/gitea/ @mode @owner @group share/gitea/conf/app.example.ini @mode 640 @owner _gitea @group _gitea @sample ${SYSCONFDIR}/gitea/app.ini @mode @owner @group |
What this does is:
- Target the directory
share/gitea/conf
and then set the directory mode and owner etc. as detailed. Then the@sample ${SYSCONFDIR}/gitea/
basically copies that directory with the mode and owner to${SYSCONFDIR}/gitea
(i.e./etc/gitea
). - The mode and owner etc. are then set back to defaults for any subsequent operations.
- Next, the
share/gitea/conf/app.ini.sample
folder is targetted. Again the mode and owner etc. are set as detailed and this time the@sample ${SYSCONFDIR}/gitea/app.ini
says copy the above sample file with the mode and owners set to${SYSCONFDIR}/gitea/app.ini
. - The mode and owner etc. are then set back to defaults for any subsequent operations.
This explains how the initial config file was created, but where the heck did share/gitea
come from? The config file I patched was under custom/conf
. The answer lies in the Makefile:
1 2 3 4 5 6 |
do-install: ${INSTALL_PROGRAM} ${MODGO_WORKSPACE}/bin/gitea ${PREFIX}/sbin ${INSTALL_DATA_DIR} ${PREFIX}/share/gitea .for _d in custom/conf options public templates cp -Rp ${WRKSRC}/${_d} ${PREFIX}/share/gitea .endfor |
As part of the install I am copying the contents of this folder to share/gitea
. And later as part of the actual install file, the PLIST
file tells the ports/ packages system to copy these to /etc
. Nice!
I did one more change to the PLIST
file. The line share/gitea/conf/app.example.ini
appears twice in the PLIST
file – once from make update-plist
and again from when I added it above. So delete the line that already exists.
OK, hopefully I am nearing the end now as I am starting to get tired. :) I cleaned up everything so there’s no leftovers and did a fake install and made a package:
1 2 3 4 5 6 7 8 |
# cleanup everything make clean # do the fake install (this will rebuild things as I did a cleanup above) make fake # make the package make package |
Success!
1 2 3 4 |
===> Building package for gitea-1.13.0 Create /usr/ports/packages/amd64/all/gitea-1.13.0.tgz Creating package gitea-1.13.0 Link to /usr/ports/packages/amd64/ftp/gitea-1.13.0.tgz |
Originally I didn’t have success got I got an error like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
===> Building package for gitea-1.13.0 Create /usr/ports/packages/amd64/all/gitea-1.13.0.tgz Creating package gitea-1.13.0 Error: change in plist | Assuming the old and new builds were done correctly | (fully up-to-date ports tree including relevant MODULES), | then someone probably forgot to bump a REVISION. | (see bsd.port.mk(5), PACKAGE_REPOSITORY) --- /usr/ports/plist/amd64/gitea-1.13.0 +++ /usr/ports/plist/amd64/gitea-1.13.0-new @@ -1,4 +1,3 @@ -@comment $OpenBSD: PLIST,v$ @name gitea-1.13.0 @version 6 @comment pkgpath=www/gitea ftp=yes ... |
I think this is because I had some leftover build stuff from before even thought I did a make clean
. So I went to /usr/ports/plist/amd64
and found two instances of the gitea-1.13.0
folder there and deleted them. After this I was able to create the package.
I can install the package now via make install
.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
make install ===> gitea-1.13.0 depends on: git-* -> git-2.28.0 ===> Verifying specs: c pthread ===> found c.96.0 pthread.26.1 ===> Installing gitea-1.13.0 from /usr/ports/packages/amd64/all/ gitea-1.13.0: ok The following new rcscripts were installed: /etc/rc.d/gitea See rcctl(8) for details. New and changed readme(s): /usr/local/share/doc/pkg-readmes/gitea --- +gitea-1.13.0 ------------------- If you are upgrading from gitea-<1.7.1, please note the following changes: * Configuration file location has changed from /etc/gitea/conf/app.ini to /etc/gitea/app.ini * GITEA_CUSTOM directory location has changed from /etc/gitea to /var/gitea/custom * Default ROOT_PATH for logs has changed from /var/gitea/log to /var/log/gitea |
Yay!
This has been a good learning experience. I don’t think I’ll ever be packaging anything myself but I have a better idea now of what goes on behind the scenes. I did want to package TailScale for OpenBSD at some point, maybe if I have free time in the future I’ll give it a shot as a learning exercise. At this point I’ve worked my way backwards to understand what’s happening, if I ever package something I’ll have to work forwards – which is an even bigger task.