Well, for the last 2 months I have been using Mutt as my primary email client, and I have not missed gmail at all!!
I spend the majority of my day inside a shell based environment, and a large majority of my workflow/tooling has been heavily modded to allow me to do the majority of my computer related activities from a shell.
- I edit all my code in vim
- I manage keychain entries using the command line clients
- I have a easy to customize automation/expansion library that allows me to add new shell automation tasks easily
- For the rare times that I use twitter, I use a vim plugin for that!
- I compose blog entries in vim
You get the point!
I have occassionally messed around with using links as a browser client. In reality, between chrome+vimium or firefox+vimperator, I have my mouseless browsing covered. The only scenario this setup does not work well for me is when I am ssh’d into a machine and want to browse the web (hence the use of links in those scenarios).
Focused, specific tools that do one job and do it well
This post is mostly for my own journalling purposes, but I am sharing it in the event that it proves useful to others. If you are running on osx, I am going to strongly recommend that you install the homebrew package manager, as that is what I use, to install other programs.
This post will outline the way that I setup the following tools:
- mutt – Shell based email client
- offlineimap – Offline imap mail management
- urlview – View urls in files
- msmtp – Smtp client
- gnupg – Encyrption utilities
As noted above, you will more than likely want to install homebrew to follow along with these instructions the best. I love homebrew as it is a ruby based package management system for non gem based packages (think nodejs, sqlite …)
Installing and configuring offlineimap
After a brief 30 minute session messing around with mutt as a simple gmail client I was running into a lot of annoying “freezes”. I came to the conclusion that using mutt as a replacement for the Gmail web interface would not really bring me a lot of the benefits that I wanted from a shell based email interface. I decided that it would be handy to be able to manage email offline, which would allow me to not have to connect to gmail unless I actually needed to send emails or synchronize local changes. To that end I decided to go with offlineimap to download a complete copy of my gmail imap folders and be able to deal with mail completely offline if need be.
As is the case with the rest of the tools in this guide, the install process is a snap, the configuration process is a little involved. I installed the current version of offlineimap. To install offline imap just type the following in a shell:
1 2 3
I installed sqlite to use it as the storage for offlineimap mail status.
Ok, that was painless now lets configure it:
This just copies a stock sample configuration file into one of the standard places that offlineimap looks for configuration. To read more about offlineimap configuration go here. I spent quite a bit of time reading through the docs and messing around with a bunch of the settings, along with messing up my install a couple of times!! That is how I learn the best though!! The documentation that comes with the sample configuration file is pretty informative, so you should be able to tweak a lot of things. As far as configuration goes I leveraged the sample configuration outlined by Steve Losh, who also went through the joys of configuring this setup a while ago. I wanted to understand how all these pieces worked, so I just leveraged his guide to rip off his configuration. His guide is also a great walkthrough of how to get this setup going. I am of the mindset that I like to struggle on my own for a bit as I try to figure things out. Instead of using his and other awesome guides, I set aside a half day and did this myself!! In the docs it mentions that the offlineimaprc file can also include a regular python file that can have supplementary code you want to leverage!! Pretty awesome concept. Steve uses this in a clever way to get around the need to have plain text passwords in the config file. Read his post for the details, but it basically makes use of the security utility (usr/bin/security on most unix based systems), to access the password in a keychain item.
Ok, so I configured things appropriately and was ready to kick off offlineimap to download the mail locally onto my computer (only about 1.5GB, but seeing as how we live out in the boonies, this would take a while). I fired up offlineimap and was immediately hit with an error about an incorrect fingerprint for the gmail stmp server. I figured this had something to do with a gmail cert change. This was rectified by following the instructions here which resulted in my adding the following config item in my JPBoodhoo-Remote Repository section:
With that change in place (keep in mind, the fingerprint can change at a later time, which would require this line to be updated), things looked good to go, I reran offlineimap and went about with other work while our blazingly fast connection proceeded to choke on the download for the next couple of hours.
Here is the majority of the pertinent stuff from my current configuration:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
You can check out Steves post for a very thorough explanation of the configuration that matches. I took advantage of the python file to DRY up my configuration for the nametrans setting. When you are working with one of the remote repositories (jp_developwithpassion_gmail_remote) in one of the examples above; the nametrans setting determines how to map the remote imap folder names to names on your local machine. In the examples on the offlineimap site, they would set up reverse entries in the nametrans for the local and remote options. That would look something like this (lifted from some standard example online):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Of course, as you can see above, it seems a little redundant. In my config I dry it up as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14
This is done by leveraging the following code that exists in the python file that I include at the top of my configuration:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102
The name mappings array is the single place where the mappings are defined. That way I don’t have to worry about typos between sections. The config also shows how I am doing exclusions as well as shelling out to a custom ruby script to get passwords.
On my second imap sync I got an UID validity error on one of the folders I was trying to sync. This was resolved using the following doc.
As you can see from the above config, I have 2 different email accounts that are currently setup. The second account is starred out due to privacy reasons.
One of the cool things about the offlineimap configuration is that you can specify a python file that will get mixed into the current config file and you can leverage code in the python file to help with configuration. I am using this feature for both retrieving account passwords and for remote & local name translation.
Next up it was time to get going with mutt. Mutt at its core is a text-based mail client. For the purpose of my setup, I am going to be using it to primarily read and compose messages. It has out of the box support to be able to interface with gmail, and its own smtp utility, but I configured it to use msmtp for smpt. To that end, most of the configuration for mutt will be setting it up to basically be able to work with the local copy of messages that gets pulled down by offlineimap.
Let’s install it:
As I am currently working with a client that requires encrypted messages for all internal communications, it was necessary for me to setup and configure mutt with encryption. I am using GPGTools for OSX. After installing and setting up my necessary keys, here is the config that I needed to place in the file ~/.mutt/gpg.rc (your file location may vary), this is only a slight modification of the stock gpg configuration.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57
For more thorough explanations of the above settings, you can check out the guide that I followed here
As far as my mutt configuration goes, I added the following lines to my mutt configuration file:
1 2 3 4 5
The keybinding allows me to hit p after I have composed a message, and it will bring up all of the options I can apply to my mail (sign, encrypt, …)
Accessing GMail Contacts
Another piece of the puzzle was for me to be able to access my gmail contacts through mutt. To do that I leveraged a python utility called goobook.
Install python if you don’t already have it:
To leverage goobook as my contact lookup mechanism, I added the following lines to my mutt configuration file:
1 2 3
The first settings tells mutt which utility to use to query for contact completion. The second setting allows me to have vim be able to autocomplete google contacts when I am editing mail headers in Vim. The third setting is just a mutt keybinding that will trigger the goobook search when I hit tab in mail header fields.
Mail Searching With notmuch
One of the last pieces of the puzzle was being able to locally search email. I accomplished this by setting up notmuch. Getting going was as simple as running:
1 2 3
To hook into the search from inside of mutt, I added the following configuration line to my mutt configuration file:
Basically, when I am in mutt and want to search for mail, I can hit S and then enter in a search term. At that point it will shell out to a small ruby script that I wrote, that executes the notmutt search and builds up the appropriate directory structure for mutt to treat the search results as a mail folder that can be explored in mutt. Finally, I change the working folder to the folder containing the search results (for me /tempfiles/notmuch_search_results). Here is the script that I wrote to do the search and result dir creation:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69
Up to this point I have only talked about a scenario optimized for reading mail. One of the main things you will also want to be able to do is send mail. Mutt has support for smtp, but there are more robust programs out there that only focus on smtp. One of those programs is msmtp . This is the program that I configured mutt to use to do mail sending.
Installing is a simple as:
Here is what my current msmtp configuration file looks like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
I “dashed” out the information for my current client in the second account configuration. Notice that I am using the first account as the default. In my mutt configuration (for my local mail account), I have the following configuration line for using msmtp as the mail sender:
1 2 3 4
I have 2 aliases set up that allow me to start mutt with my local mail account, and the other for my client mail account. Each instance uses a client specific mutt configuration that is also configured with the correct msmtp account to use for sending. That is outside of the scope of this article, but it is just a matter of templates and aliases.
Handling mail attachments
One of the first snags I ran into was correctly handling mail that contained attachments (sending mail with attachments is a snap, so I won’t go into it in this article). By defaualt, mutt will use you mailcap file to try to determine how to handle email. The contents of my mailcap file look as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Notice that most of this script shells out to a custom ruby script that does the appropriate program launching. Here is that script:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56
As you can see, this solution makes the assumption that I write attachments to the following folder : /tempfiles/mutt_attachments
It took a couple of days for me to tweak my keybindings and customize my setup. After the initial rampup, I love that I can now compose, read, edit my email in Vim, here is a screenshot of what my gtd window in tmux looks like:
Needless to say, the screenshot says it all (from my perspective!!)