a blog
This week, the Fedora Project released updated images for its Fedora 24-based Atomic Host. Fedora Atomic Host is a leading edge operating system designed around Kubernetes and Docker containers.
Fedora Atomic Host images are updated roughly every two weeks, rather than on the main six-month Fedora cadence. Because development is moving quickly, only the latest major Fedora release is supported.
While (pretty much) everyone who’s using docker is running it on Linux, and while lots of people run docker on their laptops and desktops, most aren’t running it directly on Linux desktops and laptops. Instead, most individual docker users are relying on some sort of purpose-built Linux distribution running as a virtual machine on their Mac or Windows machine.
However, if you are (like me) running Linux on your desktop, you can run docker containers right on your bare metal, with no virtualization overhead in between. Yay, Desktop Linux!
But wait. If you are (like me) running Fedora Linux on your desktop, and if you (also like me) weren’t thinking about docker and its particular storage needs when you installed Fedora on your machine, you could be in for some perplexing issues or at least crap performance, because of the way that docker storage works on Fedora.
I’ve written about the general issue elsewhere:
…the AUFS backend that started out as Docker’s default storage option, but never made its way into the mainlain Linux kernel, posed a problem for Red Hat and our upstream first, no out-of-tree bits ways.
The settled-upon solution was device mapper thin provisioning, which takes a block storage device to create a pool of space that can be used to create other block devices for Docker containers and images. The device mapper backend can be configured to use direct LVM volumes or you can let Docker create a pair of loopback mounted sparse files to serve as the block devices.
from: Friends Don’t Let Friends Run Docker on Loopback in Production
When you install Fedora on your desktop or laptop, the installer divvies your entire disk up into a small boot partition and a big LVM partition, and then divides that LVM space up into a swap volume that varies in size based on how much RAM you have installed, a root volume of 50GB, and a home volume that takes over whatever’s left.
With no room left for the pool of space that the docker device mapper storage driver needs for containers and images, the storage driver will turn instead to crappily-performing loopback mounted files. Boo!
You can cut back the size of the home volume in Fedora without too much trouble. I like to use system-storage-manager to work with my disks:
NOTE: I guess I should add that whenever you’re mucking with your disks, you should make sure you have backups, and so on, but I have resized my own laptop partitions in just this way on more than one occasion, and I’ve tested the steps written here with a VM as I wrote this, so, yeah.
$ sudo dnf install -y system-storage-manager
Next, reboot your machine, and when you get back to the login screen, hit CTRL-ALT-F2 to get to a virtual terminal, and then log in as root. We need to do this in order to unmount the home directory before we shrink it. As root, you can use system-storage-manager to shrink down the home volume. Below I’m shrinking the home volume to 20G, because I’m testing these instructions on a VM with a 100GB drive. Substitute a value that makes sense for your rig.
# umount /home # ssm resize -s 20G /dev/fedora/home # reboot
If you’ve already installed and run docker, you’ll need to delete /var/lib/docker, where all of your containers and volumes live, so be prepared to rebuild those.
$ sudo systemctl stop docker $ sudo rm -rf /var/lib/docker $ sudo systemctl start docker
When docker starts up again, a script that comes bundled with Fedora’s distribution of docker will check to see that there’s space available in your volume group and will set up your storage correctly. If you want to grow your home volume later, it’s easy to do and doesn’t require unmounting anything. You’ll run the same ssm resize command from above, and swap in your desired volume size.
NOTE: If you’re using docker-engine from docker.com, check out these docs for setting up devicemapper driver correctly by hand.
If you haven’t yet installed Fedora, you can configure your system to accommodate this and other LVM storage scenarios moving forward by making your home volume smaller and modifying your “fedora” volume group from its default “Automatic” size policy to the “As large as possible” policy. This way, all your spare disk space will be ready for new volumes (such as the docker thin pool) or for growing your home or root volumes if you decide that you need the space later on. This is probably how Fedora partitioning should be configured by default, anyway, but it isn’t.
Finally, there’s another option on the horizon for docker storage on Fedora, an option that doesn’t require partition changes or planning: OverlayFS. I wrote about this in the post I linked above, too, but the TLDR is that OverlayFS and SELinux don’t work together yet, although that’s set to change. Stay tuned.
I’ve been wishing for this exact thing to happen.
Today we are excited to announce free HTTPS for all custom domains hosted on WordPress.com. This brings the security and performance of modern encryption to every blog and website we host.
Best of all, the changes are automatic — you won’t need to do a thing.
As the EFF points out as part of their Encrypt the Web initiative, strong encryption protects our users in various ways, including defending against surveillance of content and communications, cookie theft, account hijacking, and other web security flaws.
WordPress.com has supported encryption for sites using WordPress.com subdomains (like https://barry.wordpress.com/) since 2014. Our latest efforts now expand encryption to the million-plus custom domains (like automattic.com) hosted on WordPress.com.
The Let’s Encrypt project gave us an efficient and automated way to provide SSL certificates for a large number of domains. We launched the first batch of certificates in January 2016 and immediately starting working with Let’s Encrypt…
View original post 180 more words
I’ve been sort of trying to get blogging again on WordPress lately, and as part of that, I’ve been paying more attention to the blogs I’m following using the WordPress Reader function. Recently, on Om Malik’s blog, I saw this item, pointing to the blog of photographer Eric Kim:
I clicked because I’ve been thinking grumbling thoughts lately about Instagram and its Twitter preview hiding ways.
OK, yeah, algorithm hate — it seems that every social network eventually adds features for bubbling up, through the magic of computer science, potentially interesting items you may have missed. These features disrupt the straight chronological feed presentation that many users prize (for themselves or for their followers). I know about these sentiments. I don’t share them — I like the option of “smart” timeline presentation. But, to each their own.
Anyway, I was interested to read, lower down, his suggestion: Start your own blog, and use WordPress.
I agree with this. I too am using WordPress, in part because WordPress works well, and in part because WordPress is open source software, and I’m an open source software loving sort. I was intrigued, further down, to see that all of Kim’s site content was open source:
Awesome! I clicked this link, assuming I’d find a creative commons license, but found instead a pair of essays about Kim’s ideas around open source photography:
OK, that sounds good. But what about licensing? I couldn’t Ctrl-F my way to finding the word “license” anywhere, so I broke down and read a bit more closely. I found this:
The terms he described sounded like a creative commons non-commercial license, which I don’t really consider to be open source, but whatever. Interestingly, the license he’s actually using is more permissive than he describes. He’s chosen the Public Domain Mark 1.0 license, which states that, “You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission.”
As for the content on his site, there’s no license, but this chunk of text suggests that he may be talking about something else when he says “open source”:
So, interestingly, he’s at once ceding more control (over his pictures) and less control (over his blog posts) than he seems to intend. Even among open source software enthusiasts, it’s sort of common to downplay/misunderstand/dismiss licensing, and to insist that what really matters is something else: community, process, governance, etc. All of that definitely matters, but we can’t forget about licensing — copyright laws enforce a no-sharing-allowed default setting, so when we set out to collaborate freely, we need to explicitly change that setting, and open source licensing is how that’s done.
Speaking of which — I hadn’t yet put any license information of my own on this blog, which made it ALL RIGHTS RESERVED. I fixed that with a visit to the creative commons license chooser page and some wordpress widget box-fiddling.
I noticed today (maybe I’ve noticed before, but forgotten) that the version of flannel in Fedora 23 is older than what’s available in CentOS. It looks like this is because no one tested the more-recent version of flannel in Fedora’s Bodhi, a pretty awesome application for testing packages.
Why not? Maybe because it isn’t always obvious how to test a package like flannel, but I here’s how I tested it, and added karma to the package in Bodhi.
I use flannel when I cluster atomic hosts together with kubernetes. I typically use the release versions of centos or fedora atomic, but the fedora project also provides an ostree image built from fedora’s updates-testing repo, where packages await karma from testers. I prepare three atomic hosts with vagrant:
[my-laptop]$ git clone https://github.com/jasonbrooks/contrib.git [my-laptop]$ cd contrib/ansible/vagrant [my-laptop]$ export DISTRO_TYPE=fedora-atomic [my-laptop]$ vagrant up --no-provision --provider=libvirt
Next, I rebase the trio of hosts to the testing tree:
[my-laptop]$ for i in {kube-node-1,kube-master,kube-node-2}; do vagrant ssh $i -c "sudo rpm-ostree rebase fedora-atomic:fedora-atomic/f23/x86_64/testing/docker-host"; done
[my-laptop]$ vagrant reload --no-provision && vagrant provision kube-master
Reloading the hosts switches them to the testing image, and runs the ansible provisioning scripts that configure the kubernetes cluster. Now to ssh to one of the boxes, confirm that I’m running an image with the newer flannel, and then run a test app on the cluster to make sure that everything is in order:
[my-laptop]$ vagrant ssh kube-master [kube-master]$ rpm -q flannel flannel-0.5.4-1.fc23.x86_64 [kube-master]$ sudo atomic host status TIMESTAMP (UTC) VERSION ID OSNAME REFSPEC * 2016-02-03 22:47:33 23.63 65cc265ae1 fedora-atomic fedora-atomic:fedora-atomic/f23/x86_64/testing/docker-host 2016-01-26 18:16:33 23.53 22f0b303da fedora-atomic fedora-atomic:fedora-atomic/f23/x86_64/docker-host [kube-master]$ sudo atomic run projectatomic/guestbookgo-atomicapp
That last command pulls down an atomicapp container that deploys a guestbook example app from the kubernetes project. The app includes two redis slaves, a redis master, and a trio of frontend apps that talk to those backend pieces. The bits of the app are spread between my two kubelet nodes, with flannel handling the networking in-between. If this app is working, then I’m confident that
flannel is working.
[kube-master]$ kubectl get svc guestbook NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE guestbook 10.254.233.237 3000/TCP app=guestbook 55m [kube-master]$ exit [my-laptop]$ vagrant ssh kube-node-1 [kube-node-1]$ curl http://10.254.233.237:3000/info # Server redis_version:2.8.19 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:c0359e7aa3798aa2 ....
The app is working, flannel appears to be doing its job, so I marched off to bodhi to offer up my karma:
Atomic system updates are at least half of how “Atomic Hosts” earn their Fallout-flavored appellation. Where a standard Fedora, RHEL or CentOS host gets its updates from a sack of RPMs downloaded from various repositories and exploded out where appropriate, the Atomic editions of these distros consume this same software in pre-exploded-and-composed-into-an-image form.
One tricky element of consuming your RPMs in a single blob is choosing a package or two to add beyond what’s been composed into the image. I wanted to do this straightaway after learning about the atomic host concept, and I (semi)helpfully documented my progress with composing custom trees in a few different spots, most recently at: Compose Your Own Atomic Updates.
This works pretty well, but composing and rebasing to a tree of your own is sort of a heavy approach. Shouldn’t you be able to compose just part of a tree, and, like, overlay those packages on your atomic host?
OSTree mastermind Colin Walters has whipped up just such a utility, and today, I took it for a spin with CentOS Atomic Host.
I started with a CentOS Atomic Host vagrant box, which, as you’ll see, doesn’t include the fortune-mod package:
[laptop-host]$ vagrant init centos/atomic-host [laptop-host]$ vagrant up [laptop-host]$ vagrant ssh [atomic-vm]$ fortune bash: fortune: command not found
I need to grab Colin’s tool from git, which is also not included in the CentOS Atomic Host, but which is available in the friendly centos/tools container. For a bit of info about the Fedora flavor of this container, see here.
[atomic-vm]$ sudo atomic run centos/tools [tools-container]$ cd /root [tools-container]$ git clone https://github.com/cgwalters/atomic-pkglayer/ [tools-container]$ cd atomic-pkglayer [tools-container]$ git checkout v2016.1
atomic-pkglayer requires ostree to function, and this package is missing from the centos/tools container, so I need to grab it from the repo below. Also, fortune-mod lives in EPEL, so I’ll install that repo as well.
[tools-container]$ curl -O https://raw.githubusercontent.com/CentOS/sig-atomic-buildscripts/downstream/rhel-atomic-rebuild.repo [tools-container]$ mv rhel-atomic-rebuild.repo /etc/yum.repos.d/ [tools-container]$ yum install ostree epel-release -y
Now I need to grab all the rpms required for fortune-mod, and install them to a pkglayer, before exiting my tools container, rebooting my atomic VM, and logging back in to the rebooted atomic VM:
[tools-container]$ mkdir pkgs [tools-container]$ yumdownloader --resolve --destdir=pkgs fortune-mod [tools-container]$ /root/atomic-pkglayer/atomic-pkglayer pkgs/*rpm [tools-container]$ exit [atomic-vm]$ sudo reboot [laptop-host]$ vagrant ssh
Now, for some fortune:
[atomic-vm]$ fortune
Rune's Rule:
If you don't care where you are, you ain't lost.
You can see my local overlay:
[atomic-vm]$ sudo atomic host status TIMESTAMP (UTC) VERSION ID OSNAME REFSPEC * 2016-01-29 00:30:06 local 0aa16a3e42 centos-atomic-host <unknown origin type> 2015-10-01 09:32:09 7.20151001 1e9838ce88 centos-atomic-host centos-atomic-host:centos-atomic-host/7/x86_64/standard
The system is left in an un-upgradable state — I’ll need to rollback before I can grab updates again, so this overlay is temporary:
[atomic-vm]$ sudo atomic host upgrade error: No origin/refspec in current deployment origin; cannot upgrade via ostree [atomic-vm]$ sudo atomic host rollback Moving '1e9838ce8879112c47c72503bbade0830e6f06dc20f5cabbf6da40a373550f69.0' to be first deployment Transaction complete; bootconfig swap: no deployment count change: 0 Removed: fortune-mod-1.99.1-17.el7.x86_64 recode-3.6-38.el7.x86_64 Successfully reset deployment order; run "systemctl reboot" to start a reboot [atomic-vm]$ sudo systemctl reboot [laptop-host]$ vagrant ssh
Post-rollback, the fortune command is missing once again, and my system is ready for upgrades again:
[atomic-vm]$ fortune bash: fortune: command not found [atomic-vm]$ sudo atomic host upgrade Updating from: centos-atomic-host:centos-atomic-host/7/x86_64/standard
This is cool. Also, I’m trying out reblog.
TL;DR: We’ve improved the host version management in Fedora Atomic Host, and you can now use atomic host deploy $version to atomically switch to a well-known version.
Longer version:
The awesome Cockpit project has been working on a UI for managing Atomic Host/OSTree updates. See this page for some background on their design.
If you download the most recent Fedora Atomic Host release, then atomic host upgrade, you’ll get a new rpm-ostree release which in turn has a new “deploy” verb. This was created to help implement the above Cockpit design; it’s a command line talking to code equivalent to what the Cockpit UI pull request will use.
This is noteworthy for several reasons. First, it really unlocks the “server side history” aspect of OSTree for the host tree. This is similar to tagged builds in a Docker repository for a container.
In order to explain this, one…
View original post 419 more words
For the past few years, the only posts I’ve written in this blog have been about this blog, and this post is no different. I make up for it in lack of volume: I’m averaging about one post a year.
Two years ago, I wrote about how I’d finally (almost) gotten my static blog all set up the way I wanted it, complete with self-hosted, open-source commenting functionality. Yay!
One year ago, I dumped some quick notes about how I’d re-homed my blog to Github Pages and wired it up to Github’s Travis CI service such that pushing posts or updates to my blog’s git repo would trigger a build and refresh of my site. Yay!
Of course, I still had some things to figure out, and while I eventually figured out most of it, I wasn’t blogging — I’d think about my blog, and my thoughts would immediately turn to distraction over hosting and customization and maintenance. If I’m actually going to write here, I should try to make the experience as smooth as possible, with fewer non-writing avenues around to distract me.
A few weeks ago, I was reflecting on my too-static blog, and on my general blog desires, which haven’t really changed since I switched away from WordPress:
(listed in order)
With my Middleman-based setup, I had point one nailed. Static HTML FTW. Point two, also nailed. I wrote in either AsciiDoc or Markdown. Point three, nailed.
Point four… not really nailed. Maybe… stapled? Basic customization chores, like figuring out how to add a simple sidebar, took a long time for me to figure out, and each question I answered led to other questions. As I Googled around for answers and fiddled endlessly with CSS, my thoughts often turned to a “retreat” back to WordPress, with all its point-and-clickitude.
Point five, the easy publishing experience, also nailed. The git repo to CI to github pages process I set up worked really well, and the middleman blog-writing UI that Garrett wrote is really awesome.
Point six I had working, using Juvia comments, which, being open source software, kept me in line with point three, but took me out of compliance with point one — I didn’t want to maintain some dynamic, mysql-backed web application just for my blog, and yet, my Juvia comments instance was just that. And, as a bonus, since I first deployed it, the Juvia project has been orphaned.
I mentioned above that in dark moments, I felt tempted by a return/retreat to WordPress. I figured I could use the wordpress.com service, which would satisfy my nothing-to-admin, open source, commenting, and publishing ease requirements. A combination of the many nice-looking, built-in WordPress themes and widgets, along with the (paid) option of tweaking theme CSS and whatnot, promised to satisfy my customization needs, as well.
Still, I really didn’t want to write posts in HTML, or have to hand-edit the HTML of the WordPress WYSIWYG editor. I love how markdown can be pasted straight into a email, as plain text, and remain totally readable.
It turns out that about two weeks after I quit using WordPress, the project added a “write in markdown” option to the software. You check a box somewhere, and the write-in-HTML tab in the UI becomes a write-in-markdown tab. So, that’s point two, nailed.
I signed myself up for the $99 premium WordPress subscription, which seems like a lot of money, but comes out to less than my WWE Network and Marvel Unlimited subscriptions, and goes to a company that creates and supports open source software, so… I’ll give it a year.
Getting rolling again with WordPress was easy. I pointed my domain in the right direction, picked out a theme I liked, sucked in most of my old posts from WP backups, converted the very few new ones, and modified my static posts sitting on github pages to redirect permanently over here.
We’ll see how this return to WordPress, now with markdown support and full SaaSification, affects or does not affect my personal blogging. There’s a non-zero chance I’ll be back in November 2016 to tell you all about my new LiveJournal odyssey.
A single Atomic Host is a fine place to run your containers, but these hosts are much more fun when bunched into clusters, a task that we can manage with the help of Kubernetes.
There are a lot of great guides for setting up a kubernetes cluster, but my favorite involves ansible and vagrant, and lives in the kubernetes contrib repository on Github.
This install method can be used with the libvirt, virtualbox or openstack vagrant providers. You can also use the ansible scripts on their own, if vagrant isn’t your thing.
Fedora 22’s Atomic Host dropped most of packages for the web-based server UI, cockpit, from its system tree in favor of a containerized deployment approach. Matt Micene blogged about running cockpit-in-a-container with systemd, but people have expressed interest in learning how to start this container automatically, with cloud-init.
via Running a Containerized Cockpit UI from Cloud-init — Project Atomic.
jebpages 