Another Year Another Crisis

Update, courtesy of Ars Technica:

Seems I underestimated Red Hat under IBM. According to Ars Technica, Red Hat has a plan to begin offering free RHEL subscriptions to production deployments of 16 servers or fewer. I stand by my plan to diversify my network, however this means I can spend more time getting familiar with FreeBSD instead of learning Debian. I may deploy some Debian servers in the future, however.

The year 2020, by all accounts, was a terrible year. Among its “gifts” were a new pandemic, civil unrest, and murder hornets, just to name a few. A quick perusal of Administrative Circumlocution betrays that this blogger was so dumbstruck as to neglect posting anything at all the entire year. Most relevant to my content, however, is the announced end of CentOS stable. As you may know from any of my prior ramblings, my home contains an approximation of a corporate network, with at least the following services deployed:

• OpenVPN
• FreeIPA (directory services)
• DNS
• DHCP
• PXE Boot
• Squid
• NFS and Samba file services

Each of these services is currently hosted on CentOS. Most of them, CentOS 8. I’ll need to remedy that fact sometime between now and December 31, 2021. There are a few ways that I could go about this.

CentOS Stream 🔗

CentOS Stream is Red Hat Enterprise Linux’s direct upstream project. I’ve seen it described as a rolling release, but in truth it’s no more a rolling release than RHEL and CentOS already are. Sure, if you’ve installed CentOS 8-Stream at launch, it will happily update through packages until it’s just ahead of RHEL 8.3. However, it is my understanding that when RHEL 9 launches, CentOS users will need to rebase on CentOS 9-Stream.

By far, the easiest option is to simply migrate to CentOS 8-Stream. The CentOS team has published steps for converting existing CentOS 8 installs to 8-Stream, and given that the earliest form of my network ran on Fedora - a system far more bleeding-edge than 8-Stream ever will be - I could most likely get away with it. However, the idea of running ostensibly production systems on a less-than-stable distribution sets off alarm bells now as it did when I made the decision to change to CentOS in the first place. No, I’d much prefer a more stable alternative.

Rocky Linux 🔗

Shortly after the announcement that CentOS stable would be discontinued, IBM received the lesson Oracle was given when it decided to overturn Sun’s decision to release Solaris 10’s source code. Just as IllumOS rose from the ashes of OpenSolaris, so too does Rocky Linux rise from the ashes of CentOS. I have a lot of hope for that distribution as a drop-in replacement for anyone who wants to deploy a clone of Red Hat Enterprise Linux as a form of studying for EX200. Unfortunately, as of January 2021, there is no announced release date for Rocky Linux.

With any luck, Rocky may be available by the end of 2021. The CentOS project more or less did the ground work for a rebuild of Enterprise Linux 8 in releasing CentOS 8 to begin with, however that effort does not render a rebuild of CentOS with all branding stripped a trivial task. Rocky Linux will need a lot of funding and manpower to succeed. Perhaps the Rocky team should court Fermilab. In any case, I’ll be keeping an eye on Rocky as I genuinely hope it succeeds, if only to teach IBM a lesson. With no release date, however, I cannot hope to rebase my system on Rocky Linux.

Debian 🔗

I’ve played around with Debian a fair bit lately. I find it’s a good platform for tinkering with old X11 window managers, such as Motif Window Manager. Hobbies aside, Debian has an excellent reputation as an extremely stable distribution, well suited to servers and workstations alike.

Debian does not ship with a mandatory access control framework installed. A quick look at the table of contents for the Debian Administrator’s Handbook¹ reveals that SELinux can be installed, however the standard policy package was removed in Debian Jessie as its maintainer hadn’t updated it in a satisfactory timespan. Debian’s package database, however, shows the package is available in the latest release, Debian Buster. More research into the prospects of an SELinux-protected Debian deployment is in order.

Overall, I struggle to find any fault with Debian that amounts to more than just “It’s not Red Hat”. And if this blog is anything to go by, even that can be mitigated.

FreeBSD 🔗

If departing from my comfort zone is worth doing, it’s worth over-doing. I know practically nothing of unix-like operating systems outside Linux, and I consider this a weakness of mine. FreeBSD has several attractive features that I am interested in. Because the FreeBSD operating system is not bound by an infective license² like the GPL, it is able to ship with software provided under other free and open-source licenses, including the CDDL under which OpenZFS is distributed.

Indeed, contrary to Linux, which is only a kernel and not itself an operating system, FreeBSD is developed and released as a cohesive, fully functional operating system. In fact, while it may be of little practical consequence, FreeBSD is directly descended from Version 7 Unix, and its predecessor, the 4.x-BSDs contributed greatly to Unix System V, itself an ancestor of Solaris. The FreeBSD foundation doesn’t tend to advertise this fact because Unix is a trademark of The Open Group, which charges royalties for the use of the Unix name, and FreeBSD has built enough of its own reputation to shake the necessity of such an association.

Also of interest in FreeBSD is the feature of jails, which are a sort of proto-container technology introduced years before Docker and Kubernetes. The idea is that a full system is installed into a folder, minus the kernel, and a kernel call “boots” that system in an isolated namespace, similar to the concept of a chroot jail, but far harder if not impossible to escape from. Jails are a fairly mature feature in FreeBSD, and thus utilies exist to manage creation and maintenance of jails, along with bringing jails up and down on command. Some of these utilities are even capable of maintaining a single base system instance for multiple jails to build upon. Certainly this would prove far more efficient than my current virtual machines setup.

Diversification 🔗

Hitherto in this post, I have maintained - with purpose - a fundamental mistake characteristic of my prior system. Shortly after I began in my current role, I realized that no corporate datacenter is homogenous. There is, likely, no such thing as an all-Red Hat server farm, and the latest move from CentOS illustrates very well the reason for that. Placing all dependence for the continued operation of mission critical systems in one operating system vendor creates a single point of failure. This failure can manifest itself in discontinuance of the software, as is the case with CentOS 8, or in the kinds of common vulnerability that can result in quickly spread malware infestation.

In the above sections, I expressed ambivalence about whether I would rather switch my operation to Debian or FreeBSD. Deploying both systems serves to strengthen the security of my system overall, provide an exercise in interoperability, expose my shell scripting skills to a need for portability, and demonstrate my ability to learn new Unix-like systems in a short timespan.

Next Steps 🔗

Selecting a few distributions is really only the first step in this process. My system has relied on CentOS for at least 5 years at this point, and parts of it weren’t terribly well documented. Before making major changes, I should examine each service I need to deploy and determine the best distribution for the task. I should look into using more modern deployment and configuration tools like Vagrant and Ansible. And I should meticulously document this entire process. I seem to have my work cut for me.