One of my goals for the last year has been to get more competent with my shell. I've used oh-my-zsh for years and found it helpful, but I'd never really bothered to learn shell-scripting or much about how my shell actually works. I've remedied that a bit (which I'll write about separately), and it's meant that my shell dotfiles have become a lot more useful to me.
Once I started to have a real investment of time in my shell and other command-line configs, I started wanting my usual tools -- version control, history, etc. I also wanted to be able to keep various computers in sync; I have a personal macbook where a lot of my configs originate, then my work computer, then various servers I log into to work periodically.
I'd gotten to a similar place a few years ago when I worked at US News, and I'd followed Brandon Rhodes's method for using git to track your dotfiles; however, my needs have changed, so I decided to see what other people do.
Since I'm doing a good bit of development at home and at work, and I want to share some settings but keep others private to their respective environments. Brandon's method of using a single git repo for all my dotfiles has a lot of benefits, but it doesn't lend itself to this particular setup; I want the work stuff in a private account and my personal stuff in my public github repo.
I also wanted to be able to maximize sharing, which means not requiring that a person take my whole setup to use part. For example, I found Byron Peebles's dotvim really helpful in starting to use vim, and I wanted to be able to share my various settings with colleagues and friends. However, I expect that not everyone is going to want to use my many idiosyncratic aliases and utilities. This means making my dotfiles modular, which is a little tricky --- it's not totally weird to want two separate repos that both end up putting files in .vim; however, doing that with git out of the box isn't really obvious.
For myself, I wanted a reasonably braindead way to bootstrap new machines and keep them in sync. It's not too unusual that I'll be doing a lot of work on a server for a few weeks and then stop touching it for months, only to login again and be frustrated that I've evolved how I work on my laptop and can't use similar shortcuts on the new box. Similarly, I tend to "clean home" every few months and then just work with what I've got for a while, touching-up the edges for a year or two before doing some serious remodeling. That means I need to be able to automate a lot of how my setup works so that it "just works" during the time I'm not focused on sharpening my tools.
The tricky part is that adding modularity makes the bootstraping/sync issue harder; I don't want to have to remember which 10 git repos to pulldown or update on a given box. I played a little with using submodules, but as always happens to me, I ended up making a little mess of my submodules within a few days; this reminded me that that tool doesn't mesh well with how I work.
Finally, because where I can, I try to use the most-specific tool to manage dependencies. For example, in a python project, you could use git submodules and shell scripts to manage all your dependencies, but pip has always been a lot less annoying for me. Similarly, vundle in git handles more than just checking-out and updating git plugins.
Where I ended up
It turns out, there are a ton of tools for managing your dotfiles, and github has even curated them. Initially, I was really interested in rcs and the python dotfiles package. But with rcs, my modularity goals were tricky to think through, and with the python package, given that I want all my python packages in virtualenvs, and part of that setup depends on my dotfiles, I decided to try to keep things simpler if I could.
I ended up using a set of shell scripts called homeshick to manage the dotfiles themsleves and myrepos to manage the many git repos. Homeshick handles symlinking files from a gitrepo into your home. It depends on bash, which means I can just clone it and be on my way. Symlinking has its own downsides, but the approach homeshick takes allows you to have a bunch of repos that all link into the same directories in your home, which solves my modularity problems. Then, myrepos handles cloning and bootstrapping the several repos where I keep things. I can have separate configs for myrepos for work and home (and even have one include the other), which is really useful.
I use these utilities to bootstrap tool-specific package managers, like zgen for manage zsh plugins or vundle for vim plugins.
Now, I can run mr --trust-all bootstrap https://raw.githubusercontent.com/mattbowen/dotmyrepos/master/home/.mrconfig in my home and have all my basic tooling setup. Not only does this pull my personal tools, but also external dependencies like homeshick or spacemacs (which I've been playing with). Because myrepos is well-designed, at work I can have a .mrconfig that includes my home one and extends it. This allows for the sort of modularity and layering that I wanted.
I was able to pretty easily separate out my personal base setup from work-specific settings and keep all of this in sync. I can run homeshick pull and mr pull and have my tooling updated.
Also, with automated-setups, I'm starting to think about ways to make my own tooling more useful; for example, since I can create more "layers" of setup, I might want to break my .vimrc into some basic settings I want absolutely everywhere (like, even on my router), and then all the plugins I like using for programming and writing. I've got to think on this more though, since it will add complexity that I need to remember later.
First, I'm using many tools; maybe too many. The domain-specific plugin managers are useful since they usually do more than just manage git repos, but they also mean thinking about a bunch of tools in a specific order if I want to make sure I'm using the same vim plugins everywhere. This means it's not braindead enough yet; for example, if I've added some new zsh plugins, it's not always even obvious to me which tools I should run in which order to make sure they're updated in a given environment. I should be able to solve this with a shell-script or two, but I haven't done so yet (I didn't think of it until writing this, so +1 blogging I guess).
Having so many repos also makes sharing somehow harder; it's hard to point people to any one specific place if they want to use my tools. This blog post is in part to help with that, but I probably need to write another once my setup has settled down a little more to give a better overview.
Also, the homeshick approach of using symbolic links for everything means it can be hard to tell at a glance if I have untracked configs sitting around in my home. This is another thing that I can almost certainly fix with some shell scripting, but I haven't yet, and it's another place I can trip. Then there's all the problems that come with symbolic links generally; part of why I picked homeshick is that they have documented how to work around some of these problems, but just reading the docs should give you an idea of the woes you're headed for with the symlinking approach.
Finally, ironically, homeshick itself makes it slightly harder to share my configs, since homeshick requires that all the configs be in a REPO/home layout, which I don't think other people necessarily want. But, outside of work (where I don't think people will really mind), I don't really expect people to "fork" my dotfiles as much as copy them and use them for their own purposes, in which case, a slightly-weird directory-layout isn't so bad. And, on the positive, it makes it obvious where to put docs, if I ever write any.
Foremost, I need to just live with this for a few more weeks; I'm pretty happy with how things are working, but only time will tell whether the complexity is really worth it.
In the meantime, I want to smooth out some of the "what's uncool" — the biggest things are
- making it easy to get an environment totally updated without having to run a ton of commands
- being able to see which dotfiles aren't currently tracked somewhere
- coming up with some sort of documentation for the various "castles" (homeshick's name for a set of tracked files) I have tracked
Resources and my repos
As I worked through this, I found the following really helpful:
I've got the following repos available for now:
- .zsh -- zsh plugins and customization. This deserves its own blog post, because there's a fair bit of work in here.
- .vim -- a (now pretty distant) fork of Byron Peebles's vim config, with about a million plugins, because I sure do like plugins
- .myrepos -- the home of everything else
- .git -- basic git configs and Tim Pope's automation for generating ctags
I still need to get my screen and tmux configs and a few others in, but they'll come before long.