Wubba Duckies - Waves

Here is a link to the newest song from the Wubba Duckies.

Waves.

Currently, my son is deciding if he is going to stick with the Wubba Duckies moniker. It made sense when he thought his focus was going to be dubstep; now that he is moving toward house music, he is contemplating a rebranding :)

If you like the song, and also happen to have a soundcloud account, feel free to give him a follow!

Develop With Passion®

First Official Song From the Wubba-Duckies (Aka: My Oldest Son and His Friend)

For the last couple of months, our son has been on a journey to explore the world of creating different styles of electronic dance music; with a strong preference toward house music.

It has been awesome to see him exploring this avenue of creative expression!

Here is a link to his first official song. As an extremely proud dad, I am excited to see what he is going to able to accomplish as he dives deeper into learning all of this cool stuff!!

Long Live Us.

If you like the song, and also happen to have a soundcloud account, feel free to give him a follow!

Develop With Passion®

Happy Valentines Day Sweetheart

On the 17th of December, our family home was burned to the ground.

For the last 2 months, our family of 6 has been shuttling between hotels and generally just trying to figure out the steps to move forward.

We have run the gamut of emotions and are moving into a season of change and new possibilities.

For the last week as I was walking through malls and stores trying to think of things that I could do/buy to express my love for my wife on valentines day I realized that nothing seemed to jump out at me for her. So I thought I would try something different. This year I am writing this blog post as a big “permanent” out there proclamation of love for the most amazing women that I know.

The Backstory

We met each other when we were in grade 9, and one of the first things she ever said to me was, “Are you saved, do you know Jesus!” As you can imagine those words took me by a surprise. Over the years we hung out more and more, and ultimately she was the one who lead me into a personal relationship with Jesus. Bar none, the greatest thing anyone could have ever done for me.

After a couple of failed attempts to move out of the “friend zone” we made the transition on a seemingly ordinary night with a kiss that to this day I can only describe as magical.

As a friend I was amazing, as a boyfriend I was a bit of a dud. I, unfortunately, had not developed the strength to stand strong for an ‘us’ in the midst of pressure from good friends(Border Bullies as I would later read about in a book called The Dream Giver) This put a significant amount of strain on our relationship in what should have been one of our most fun years of high school.

I married my high school sweetheart only months after graduation. How mature was I? Well let’s put it this way, I used our wedding gift money to buy a nintendo 64 video game system. Yep, I was quite the catch.

Enter The Workaholic

My amazing wife worked so that I could go back to school and make something of myself, all the while being there to raise our first son , also pregnant with our second…who was born just after I graduated.

I had been a bit of a ghost during the school years, mostly tunnel visioned on building top tier skills. We both assumed that, of course, after graduating things would be different. This was just a necessary season to transition into another one where I would be more focused on our marriage and our growing family. After all, with a high tech job and good salary, there would be no reason for me to maintain my tunnel vision… right?

Well let’s just say that my tunnel vision with respect to growing mad ninja coding skills paid off huge in the work department; unfortunately I had developed an incredibly unhealthy attitude toward work-life balance. I had none and I was oblivious to it.

I’d love to say that the title of workaholic was not one that could ever have been attributed to me. I’d love to say that as God blessed us with our 3rd and 4th kids that I snapped out of my workaholic mode of operation. I’d love to say that as a guy who “supposedly” put Jesus at the center of his life that I was growing into a man that placed his family at the top of his list of priorities. I’d love to say that I was growing more and more into a husband who could be a blessing to the amazing wife and children I was blessed with.

In my mind; for 10 years I convinced myself that I was all the things that I had previously mentioned. I allowed myself to get into a place where my wifes repeated concerns about my absenteeism were seeming incorrect. I was providing a good life for my family, this is just the way things are done. I’m not really that busy.

The Workaholic Has His Eyes Opened

June 2008, the moment the scales fell off my eyes. In a conversation with my wife that started off seemingly simple, with statements that I had heard her express many times over the previous years, she started crying. In that moment I became vividly self aware of several things.

  1. There is no way that Jesus is actually the center of my life right now
  2. I have been lying to myself about who I really am for the last 11 years.
  3. I have been a terrible husband
  4. I have been a terrible father

Feeling the full weight of any one of these realizations would be enough, but for the first time in our 11 year marriage (at that point), to be feeling the weight of all of those realizations at once was huge. I remember expressing to her in that moment words that I had said many times before over the previous years:

  • I’m going to change
  • Things are going to be different from now on
  • I’m sorry that I have not been there for you and the kids as much as I should be

By now you are probably thinking to yourself “wait, isn’t this supposed to be a romantic love story”; followed up close by “his wife has amazing patience, or she is a sucker for punishment”. To that last statement let me just share an amazing verse from the bible:

Her children arise and call her blessed; her husband also, and he praises her: “Many women do noble things, but you surpass them all.” Charm is deceptive, and beauty is fleeting; but a woman who fears the Lord is to be praised. Honour her for all that her hands have done, and let her works bring her praise at the city gate.

Proverbs 31:10-31 Bible

Amazing things happen when you let jesus take center stage

  • I’m going to change
  • Things are going to be different from now on
  • I’m sorry that I have not been there for you and the kids as much as I should be

I was quickly realizing that the above words did not have any meaning to her anymore, as she had heard them hundreds of times before with no visible changes to my behaviour. In that moment I knew that God was giving me another chance to become the husband and father that I should have been our entire marriage. Even though I did not deserve it, God gave me a vision for what I needed to do. I expressed to my wife that I was not going to say these things anymore, that she would have to wait and see the changes. Over the course of the next 3 weeks we made some pretty radical decisions that were going to change the trajectory of our family in ways that, at the time, we could not comprehend. I’ll save the details of that story for another time.

The net result of these decisions resulted in our family of 6 (our youngest was just 3 years old at the time) travelling the world together for the next year. I took on work as I needed to, the whole while praying for healing and restoration to take place in my marriage and family. We had rented out our house and were jumping from location to location waiting patiently for God to reveal to us where we should “stop”. Most importantly, during this time, I was finally placing Jesus at the center stage of my heart; which ultimately impacted everything that I did. Even though I had accepted Jesus as a teenager I was painfully aware that it had really been me calling the shots for a long time.

Even with rough beginnings, you can have your fairytale

Thankfully, this story is far from over and is currently on an amazingly different course than its first 11 years!. They say that we marry the person that helps us close the gaps in ourself that we can’t see. Through amazing Godly love, and lots of patience, our marriage is now amazing, and vibrant. In many ways, the last 6 years have been what our “newlywed” period should have been. The workaholic in me died on that day back in June 2008, when I finally put Jesus at the center of my heart. I can say with 100% confidence that my words and actions as a husband and father reflect who I envisioned myself to be when I thought about how a godly man should be. And the only reason I can share this now “online” with the world is I know that my wife, kids, friends, and family will all be reading this. I could not even think to post this if it did not reflect what they and I see before me in my family and marriage of today. At 35 years old and with 17 years of marriage under my belt, it would be easy to say that I am a veteran. Much like in the world of work though, number of years does not mean anything. It’s the quality of those years that matters. Using that metric, I’m a lightweight at 6 quality years. I am also a vividly different person than I was during the first 11 years of our marriage. A fact to which both my wife and kids would shout out loudly!!

A proclamation of love to my wife

Honey, you are an amazing godly women. I could not have envisioned in a million years that God would see fit to bless me with such a beautiful and caring person. Your patience is amazing, your love and encouragement is a constant source of strength for me. Your love and care for our children and the huge number of their friends who entered our home on a daily basis is/was a constant humbling and amazing thing to witness. You are stunningly beautiful, athletic, and just plain amazing fun. Thank you so much for agreeing to take my hand and walk down this amazing road of life with me. Thank you for your enduring patience as you waited for me to become the man that God must have revealed to you that I would become. Thank you for daily teaching and modeling to our kids and I what it means to love others as Jesus does, and to accept and welcome anyone who crosses our path.

I know the next year looks like it will be a little bit of a roller coaster. Our house may have burned to the ground along with 17 years of pictures, baby books, and memorabilia that can’t be replaced. I know that together we are going to carve out another set of amazing memories as we walk hand in hand together down whatever paths God puts in front of us.

On this valentines day, know that you are loved more than these “permanent” words can ever express.

I love you yesterday, today, tomorrow, and forever.

Develop With Passion®

What We Really Remember

Just thought I would share an amazing devotion that I read this morning, written by Max Lucado. Helps to remind us of one of the things that really matters:

Name the ten wealthiest men in the world. Name eight people who have won the Nobel or Pulitzer Prize. How did you do? I didn’t do well either. With the exception of the trivia hounds, none of us remember the headliners of yesterday too well. Surprising how quickly we forget, isn’t it? And… these are the best in their fields. But the applause dies. Awards tarnish. Achievements are forgotten.

Here’s another quiz…Name ten people who have taught you something worthwhile. Name five friends who have helped you in a difficult time. Easier? It was for me, too. The lesson? The people who make a difference are not the ones with the credentials, but the ones with the concern.

Max Lucado

Develop With Passion®

Simple Strategy for Resolving New Automation Scripts

I have a large (and continually increasing) number of automation scripts in play on my machine. I usually keep a simple notebook page going where I will make a note of automation items that I seem to be missing, usually identified by things that I have to do multiple times that I think could be automated. Once a week I will look at that list and complete one of the automation items.

After doing this for a while, it became apparent that I needed to have a simple way to be able to have new automation scripts be recognized, so I could quickly start using them. As my automation scripts grew, I wanted to be able to organize the scripts by functional area, so I could quickly get a listing of my automation folder and see what types of automation items I had in play.

My automation scripts currently consist of predominantly bash, python, and ruby scripts, all of which are marked executable (the ones that are automation items anyway).

I won’t get into how I organize my dotfiles, but here is an important couple of lines in one of my dotfiles that ensures that all of my automation scripts (regardless of folder they are organized into) are picked up and available in my path:

Automation Scripts - scripts.sh
1
2
OLD_PATH=$PATH
export PATH="$OLD_PATH:$(find [my automation root folder] -name '.*' -prune -type d | tr "\n" ":")"

Obviously, you would replace [my automation root folder], with the path to the folder which is the parent of all of your automation script folders. For me that is ~/repositories/developwithpassion/devtools/automation.

Notice the use of the find command to find all folders underneath my automation root, which are then piped into the tr utility to convert every newline into the required path separator.

This allows me to either add a new executable automation script under an existing folder in my automation hierarchy; or I can add a new automation script under a brand new subfolder in my automation hierarchy. Either way it will be picked up.

Develop With Passion®

Simple Script to Create Directory Specific Gemsets

When I’m working on ruby based projects I use rvm to manage and switch between differing ruby versions and gemsets.

I don’t ever keep my .ruby-version or .ruby-gemset files (or the old .rvmrc) under version control. Which means that I have to recreate them each time I clone a copy of the project. Sometimes I will have the same project cloned out in differing folders. I have usually tried to keep folder specific gemsets; that way I don’t have to get creative with gemset names as the gemset is basically the full path of the folder that I am in.

Here is the script that I use to create a new .ruby-version and .ruby-gemset in the current folder that I am in:

RVM Gemset Create - rvmgsc
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
#!/usr/bin/env ruby
def rubies
  ruby_pattern = /\s*(ruby-.*)\s\[/

  rubies = `rvm list`
  rubies = rubies.split("\n")
  rubies = rubies.select{ |item| ruby_pattern =~ item }
  rubies = rubies.map{ |item| ruby_pattern.match(item)[1] }
  rubies.sort! { |left, right| right <=> left }

  rubies
end

def pick_ruby_version
  puts "Which Ruby Version?"
  versions = rubies
  versions.each_with_index do |version, index|
    puts "\t#{index + 1} - #{version}"
  end
  puts "Which?: "
  version = gets.chomp.to_i
  version = version - 1
  versions[version]
end

def ruby_version
  return pick_ruby_version if ARGV.length == 0

  version = ARGV[0]
  full_version = "ruby-#{version}"

  return full_version
end

def gemset_name(gemset)
  gemset = gemset.gsub(/[\/\-\s]/,'_')
  gemset = gemset.gsub(/_{2,}/,'_')
  gemset = gemset.slice(1, gemset.length)
  gemset.downcase
end

gemset = gemset_name(Dir.pwd)

`echo #{ruby_version} > .ruby-version`
`echo #{gemset} > .ruby-gemset`

This file is marked executable and in my path (I name it rvmgsc). Whenever I enter a new folder that I want a gemset for I can call it either with an argument of the ruby version that I want to use. If I don’t pass any arguments it will list the ruby versions that I have and let me choose. The script will then continue to make an .ruby-version and .ruby-gemset file, with the correct contents.

Develop With Passion®

Getting Up and Running With a Sane Mutt Setup

Well, for the last 2 months I have been using Mutt as my primary email client, and I have not missed gmail at all!!

I spend the majority of my day inside a shell based environment, and a large majority of my workflow/tooling has been heavily modded to allow me to do the majority of my computer related activities from a shell.

  • I edit all my code in vim
  • I manage keychain entries using the command line clients
  • I have a easy to customize automation/expansion library that allows me to add new shell automation tasks easily
  • For the rare times that I use twitter, I use a vim plugin for that!
  • I compose blog entries in vim

You get the point!

I have occassionally messed around with using links as a browser client. In reality, between chrome+vimium or firefox+vimperator, I have my mouseless browsing covered. The only scenario this setup does not work well for me is when I am ssh’d into a machine and want to browse the web (hence the use of links in those scenarios).

Focused, specific tools that do one job and do it well

This post is mostly for my own journalling purposes, but I am sharing it in the event that it proves useful to others. If you are running on osx, I am going to strongly recommend that you install the homebrew package manager, as that is what I use, to install other programs.

This post will outline the way that I setup the following tools:

Getting Started

As noted above, you will more than likely want to install homebrew to follow along with these instructions the best. I love homebrew as it is a ruby based package management system for non gem based packages (think nodejs, sqlite …)

Installing and configuring offlineimap

After a brief 30 minute session messing around with mutt as a simple gmail client I was running into a lot of annoying “freezes”. I came to the conclusion that using mutt as a replacement for the Gmail web interface would not really bring me a lot of the benefits that I wanted from a shell based email interface. I decided that it would be handy to be able to manage email offline, which would allow me to not have to connect to gmail unless I actually needed to send emails or synchronize local changes. To that end I decided to go with offlineimap to download a complete copy of my gmail imap folders and be able to deal with mail completely offline if need be.

As is the case with the rest of the tools in this guide, the install process is a snap, the configuration process is a little involved. I installed the current version of offlineimap. To install offline imap just type the following in a shell:

Install Homebrew - install.sh
1
2
3

brew install offlineimap
brew install sqlite

I installed sqlite to use it as the storage for offlineimap mail status.

Ok, that was painless now lets configure it:

Copy Initial Configuration File - copy_config.sh
1
2

cp $(brew --prefix)/Cellar/offlineimap/HEAD/offlineimap.conf ~/.offlineimaprc

This just copies a stock sample configuration file into one of the standard places that offlineimap looks for configuration. To read more about offlineimap configuration go here. I spent quite a bit of time reading through the docs and messing around with a bunch of the settings, along with messing up my install a couple of times!! That is how I learn the best though!! The documentation that comes with the sample configuration file is pretty informative, so you should be able to tweak a lot of things. As far as configuration goes I leveraged the sample configuration outlined by Steve Losh, who also went through the joys of configuring this setup a while ago. I wanted to understand how all these pieces worked, so I just leveraged his guide to rip off his configuration. His guide is also a great walkthrough of how to get this setup going. I am of the mindset that I like to struggle on my own for a bit as I try to figure things out. Instead of using his and other awesome guides, I set aside a half day and did this myself!! In the docs it mentions that the offlineimaprc file can also include a regular python file that can have supplementary code you want to leverage!! Pretty awesome concept. Steve uses this in a clever way to get around the need to have plain text passwords in the config file. Read his post for the details, but it basically makes use of the security utility (usr/bin/security on most unix based systems), to access the password in a keychain item.

Ok, so I configured things appropriately and was ready to kick off offlineimap to download the mail locally onto my computer (only about 1.5GB, but seeing as how we live out in the boonies, this would take a while). I fired up offlineimap and was immediately hit with an error about an incorrect fingerprint for the gmail stmp server. I figured this had something to do with a gmail cert change. This was rectified by following the instructions here which resulted in my adding the following config item in my JPBoodhoo-Remote Repository section:

Adding gmail cert fingerprint - cert_fingerprint.sh
1
2

cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042

With that change in place (keep in mind, the fingerprint can change at a later time, which would require this line to be updated), things looked good to go, I reran offlineimap and went about with other work while our blazingly fast connection proceeded to choke on the download for the next couple of hours.

Here is the majority of the pertinent stuff from my current configuration:

offlineimap configuration - configuration.ini
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

[general]
ui = TTYUI
accounts = jp_developwithpassion_gmail, jboodhoo------gmail
pythonfile = ~/repositories/developwithpassion/devtools/shared/dotfiles/offlineimap/offlineimap.py
fsync = False


[Account jp_developwithpassion_gmail]
localrepository = jp_developwithpassion_gmail_local
remoterepository = jp_developwithpassion_gmail_remote
status_backend = sqlite
postsynchook = notmuch new

[Account jboodhoo----gmail]
localrepository = jboodhoo----gmail_local
remoterepository = jboodhoo----gmail_remote
status_backend = sqlite
postsynchook = notmuch new


#developwithpassion
[Repository jp_developwithpassion_gmail_local]
type = Maildir
localfolders = ~/Dropbox/imap_mail/jp_developwithpassion
nametrans = get_remote_name


[Repository jp_developwithpassion_gmail_remote]
maxconnections = 1
type = Gmail
cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042 
remoteuser = jp@developwithpassion.com
remotepasseval = get_password("jp@developwithpassion.com")
realdelete = no
nametrans = get_local_name
folderfilter = is_included

#***
[Repository jboodhoo-----gmail_local]
type = Maildir
localfolders = ~/Dropbox/imap_mail/jboodhoo---
nametrans = get_remote_name


[Repository jboodhoo----gmail_remote]
maxconnections = 1
type = Gmail
cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042 
remoteuser = jboodhoo@----
remotepasseval = get_password("jboodhoo@----")
realdelete = no
nametrans = get_local_name
folderfilter = is_included

You can check out Steves post for a very thorough explanation of the configuration that matches. I took advantage of the python file to DRY up my configuration for the nametrans setting. When you are working with one of the remote repositories (jp_developwithpassion_gmail_remote) in one of the examples above; the nametrans setting determines how to map the remote imap folder names to names on your local machine. In the examples on the offlineimap site, they would set up reverse entries in the nametrans for the local and remote options. That would look something like this (lifted from some standard example online):

nametrans configuration - nametrans.ini
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
[Repository blah-Local]
type = Maildir
localfolders = ~/.mail/blah.com
nametrans = lambda folder: {'drafts':  '[Gmail]/Drafts',
                            'sent':    '[Gmail]/Sent Mail',
                            'flagged': '[Gmail]/Starred',
                            'trash':   '[Gmail]/Trash',
                            'archive': '[Gmail]/All Mail',
                            }.get(folder, folder)

[Repository blah-Remote]
maxconnections = 1
type = Gmail
remoteuser = blah@blah.com
remotepasseval = "password"
realdelete = no
nametrans = lambda folder: {'[Gmail]/Drafts':    'drafts',
                            '[Gmail]/Sent Mail': 'sent',
                            '[Gmail]/Starred':   'flagged',
                            '[Gmail]/Trash':     'trash',
                            '[Gmail]/All Mail':  'archive',
                            }.get(folder, folder)

Of course, as you can see above, it seems a little redundant. In my config I dry it up as follows:

Drying up the name translation - nametrans_dry.ini
1
2
3
4
5
6
7
8
9
10
11
12
13
14
[Repository jp_developwithpassion_gmail_local]
type = Maildir
localfolders = ~/Dropbox/imap_mail/jp_developwithpassion
nametrans = get_remote_name

[Repository jp_developwithpassion_gmail_remote]
maxconnections = 1
type = Gmail
cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042 
remoteuser = jp@developwithpassion.com
remotepasseval = get_password("jp@developwithpassion.com")
realdelete = no
nametrans = get_local_name
folderfilter = is_included

This is done by leveraging the following code that exists in the python file that I include at the top of my configuration:

OfflineImap Python Configuration - offline.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
#!/usr/bin/python
import subprocess
import re

class NameMapping:
  def __init__(self, local_name, remote_name):
    self.local_name = local_name
    self.remote_name = remote_name

class LocalName:
  def __init__(self, folder):
    self.folder = folder

  def matches(self, mapping):
    return mapping.remote_name == self.folder

  def mapped_folder_name(self, mapping):
    return mapping.local_name

class RemoteName:
  def __init__(self, folder):
    self.folder = folder

  def matches(self, mapping):
    return mapping.local_name == self.folder

  def mapped_folder_name(self, mapping):
    return mapping.remote_name



def get_password(account):
  program ='/Users/jp/repositories/developwithpassion/devtools/automation/keychain/get_keychain_password'

  command = "{0} --account:{1}".format(program, account)

  output = subprocess.check_output(command, shell=True)

  return output.rstrip()


def is_included(folder):
  result = True

  for pattern in exclusion_patterns:
    result = result and (re.search(pattern, folder) == None)

  return result

exclusion_patterns = [
  "efax",
  "earth_class_mail",
  "eventbrite",
  "gotomeeting",
  "moshi_monsters",
  "peepcode",
  "raini_fowl",
  "stuart_know",
  "training.*2008",
  "training.*2009",
  "training.*2010",
  "training.*2011",
  "training.*2012",
  "training.*nbdn",
  "training.*nothin_but_bdd",
  "unblock_us",
  "web_hosting",
  "webinars",
  "Gmail.*Important"
]

name_mappings = [
  NameMapping('sent', '[Gmail]/Sent Mail'),
  NameMapping('spam', '[Gmail]/Spam'),
  NameMapping('flagged', '[Gmail]/Starred'),
  NameMapping('trash',   '[Gmail]/Trash'),
  NameMapping('archive', '[Gmail]/All Mail'),
  NameMapping('drafts', '[Gmail]/Drafts')
]



def find_name_mapping(name):
  default_mapping = NameMapping(name.folder, name.folder)

  for mapping in name_mappings:
    if (name.matches(mapping)):
      return mapping

  return default_mapping

def get_name_mapping(name):
  mapping = find_name_mapping(name)
  return name.mapped_folder_name(mapping)

def get_remote_name(local_folder_name):
  name = RemoteName(local_folder_name)
  return get_name_mapping(name)

def get_local_name(remote_folder_name):
  name = LocalName(remote_folder_name)
  return get_name_mapping(name)

The name mappings array is the single place where the mappings are defined. That way I don’t have to worry about typos between sections. The config also shows how I am doing exclusions as well as shelling out to a custom ruby script to get passwords.

On my second imap sync I got an UID validity error on one of the folders I was trying to sync. This was resolved using the following doc.

As you can see from the above config, I have 2 different email accounts that are currently setup. The second account is starred out due to privacy reasons.

One of the cool things about the offlineimap configuration is that you can specify a python file that will get mixed into the current config file and you can leverage code in the python file to help with configuration. I am using this feature for both retrieving account passwords and for remote & local name translation.

Installing Mutt

Next up it was time to get going with mutt. Mutt at its core is a text-based mail client. For the purpose of my setup, I am going to be using it to primarily read and compose messages. It has out of the box support to be able to interface with gmail, and its own smtp utility, but I configured it to use msmtp for smpt. To that end, most of the configuration for mutt will be setting it up to basically be able to work with the local copy of messages that gets pulled down by offlineimap.

Let’s install it:

Install Mutt - install_mutt.sh
1
2

brew install mutt

Configuring gpg

As I am currently working with a client that requires encrypted messages for all internal communications, it was necessary for me to setup and configure mutt with encryption. I am using GPGTools for OSX. After installing and setting up my necessary keys, here is the config that I needed to place in the file ~/.mutt/gpg.rc (your file location may vary), this is only a slight modification of the stock gpg configuration.

GPG Configuration - gpg.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

# -*-muttrc-*-
#
# Command formats for gpg.
#
# This version uses gpg-2comp from
# http://70t.de/download/gpg-2comp.tar.gz
#
# $Id$
#
# %p The empty string when no passphrase is needed,
# the string "PGPPASSFD=0" if one is needed.
#
# This is mostly used in conditional % sequences.
#
# %f Most PGP commands operate on a single file or a file
# containing a message. %f expands to this file's name.
#
# %s When verifying signatures, there is another temporary file
# containing the detached signature. %s expands to this
# file's name.
#
# %a In "signing" contexts, this expands to the value of the
# configuration variable $pgp_sign_as. You probably need to
# use this within a conditional % sequence.
#
# %r In many contexts, mutt passes key IDs to pgp. %r expands to
# a list of key IDs.

set pgp_decode_command="gpg --status-fd=2 %?p?--passphrase-fd 0? --no-verbose --quiet --batch --output - %f"
set pgp_verify_command="gpg --status-fd=2 --no-verbose --quiet --batch --output - --verify %s %f"
set pgp_decrypt_command="gpg --status-fd=2 %?p?--passphrase-fd 0? --no-verbose --quiet --batch --output - %f"
set pgp_sign_command="gpg --no-verbose --batch --quiet --output - %?p?--passphrase-fd 0? --armor --detach-sign --textmode %?a?-u %a? %f"
set pgp_clearsign_command="gpg --no-verbose --batch --quiet --output - %?p?--passphrase-fd 0? --armor --textmode --clearsign %?a?-u %a? %f"
set pgp_encrypt_only_command="pgpewrap gpg --batch --quiet --no-verbose --output - --encrypt --textmode --armor --always-trust -- -r %r -- %f"
set pgp_encrypt_sign_command="pgpewrap gpg %?p?--passphrase-fd 0? --batch --quiet --no-verbose --textmode --output - --encrypt --sign %?a?-u %a? --armor --always-trust -- -r %r -- %f"

set pgp_import_command="gpg --no-verbose --import %f"
set pgp_export_command="gpg --no-verbose --export --armor %r"
set pgp_verify_key_command="gpg --verbose --batch --fingerprint --check-sigs %r"
set pgp_list_pubring_command="gpg --no-verbose --batch --quiet --with-colons --list-keys %r"
set pgp_list_secring_command="gpg --no-verbose --batch --quiet --with-colons --list-secret-keys %r"

# fetch keys
# set pgp_getkeys_command="pkspxycwrap %r"

# pattern for good signature - may need to be adapted to locale!

# set pgp_good_sign="^gpgv?: Good signature from "

# OK, here's a version which uses gnupg's message catalog:
# set pgp_good_sign="`gettext -d gnupg -s 'Good signature from "' | tr -d '"'`"

# This version uses --status-fd messages
set pgp_good_sign="^\\[GNUPG:\\] GOODSIG"

set pgp_timeout=9000

For more thorough explanations of the above settings, you can check out the guide that I followed here

As far as my mutt configuration goes, I added the following lines to my mutt configuration file:

Mutt GPG Config - gpg.sh
1
2
3
4
5

bind compose p pgp-menu

# Prepare GPG usage
source ~/.mutt/gpg.rc

The keybinding allows me to hit p after I have composed a message, and it will bring up all of the options I can apply to my mail (sign, encrypt, …)

Accessing GMail Contacts

Another piece of the puzzle was for me to be able to access my gmail contacts through mutt. To do that I leveraged a python utility called goobook.

Install python if you don’t already have it:

Install Python - install_python.sh
1
2

brew install python

Install goobook:

Install goobook - install_goobook.sh
1
2

pip install goobook

To leverage goobook as my contact lookup mechanism, I added the following lines to my mutt configuration file:

Goobook related settings - goobook.sh
1
2
3
set query_command="goobook query '%s'"
set editor = "vim --cmd 'let g:goobookrc=\"/Users/jp/.goobookrc\"'"
bind editor <Tab> complete-query

The first settings tells mutt which utility to use to query for contact completion. The second setting allows me to have vim be able to autocomplete google contacts when I am editing mail headers in Vim. The third setting is just a mutt keybinding that will trigger the goobook search when I hit tab in mail header fields.

Mail Searching With notmuch

One of the last pieces of the puzzle was being able to locally search email. I accomplished this by setting up notmuch. Getting going was as simple as running:

Installing notmuch- not_much_setup.sh
1
2
3
brew install notmuch
notmuch setup #setting up for first time use
notmuch new #run the first search

To hook into the search from inside of mutt, I added the following configuration line to my mutt configuration file:

Configuring mutt with notmuch - mutt_not_much.sh
1
macro index S "<enter-command>unset wait_key<enter><shell-escape>mutt_notmuch /tempfiles/notmuch_search_results<enter><change-folder-readonly>/tempfiles/notmuch_search_results<enter>" "search mail (using notmuch)"

Basically, when I am in mutt and want to search for mail, I can hit S and then enter in a search term. At that point it will shell out to a small ruby script that I wrote, that executes the notmutt search and builds up the appropriate directory structure for mutt to treat the search results as a mail folder that can be explored in mutt. Finally, I change the working folder to the folder containing the search results (for me /tempfiles/notmuch_search_results). Here is the script that I wrote to do the search and result dir creation:

Searching for mail - mutt_notmuch.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
#!/usr/bin/env ruby

require 'digest/sha1'
require 'optparse'
require 'fileutils'

class Hasher
  def initialize
    @hasher = Digest::SHA1.new
  end

  def get_email_hash(file_name)
    @hasher.hexdigest(IO.read(file_name))
  end
end

def build_results_folder(results_path)
  FileUtils.rm_rf(results_path)
  FileUtils.mkdir_p(results_path)

  %w/cur new/.each do |folder|
    combined_path = File.join(results_path, folder)
    FileUtils.mkdir_p(combined_path)
  end
end

def read_line(message)
  print "Query:"
  STDIN.gets.chomp
end

def get_unique_files(files)
  hasher = Hasher.new
  messages = {}
  files.each do |file|
    hash = hasher.get_email_hash(file)
    messages[hash] = file
  end
  messages.values
end

def get_matching_files(query)
  files = `notmuch search --output=files #{query}`
  files.split("\n")
end

def sym_link(files, target_path)
  files.each do |file|
    system("ln -s #{file} #{target_path}")
  end
end

def run(results_path)
  build_results_folder(results_path)
  query = read_line("Search Phrase?")
  files = get_matching_files(query)
  unique_files = get_unique_files(files)
  sym_link(unique_files, "#{results_path}/cur/")
end

def parse_arguments
  OptionParser.new do|options|
    options.banner = "Usage is: mutt_notmuch [RESULTS_PATH]"
  end.parse!

  path = ARGV.length == 0 ? "/tempfiles/notmuch_search_results" : ARGV[0]
end

run parse_arguments

Sending Mail

Up to this point I have only talked about a scenario optimized for reading mail. One of the main things you will also want to be able to do is send mail. Mutt has support for smtp, but there are more robust programs out there that only focus on smtp. One of those programs is msmtp . This is the program that I configured mutt to use to do mail sending.

Installing is a simple as:

Installing msmtp - msmtp_install.sh
1
brew install msmtp

Here is what my current msmtp configuration file looks like:

Configuring msmtp - msmtp_config.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
account jp
host smtp.gmail.com
port 587
protocol smtp
auth on
from jp@developwithpassion.com
user jp@developwithpassion.com
tls on
tls_trust_file ~/.mutt/equifax_gmail.cert

account jp_---
host smtp.gmail.com
port 587
protocol smtp
auth on
from jp@developwithpassion.com
user jboodhoo@------
tls on
tls_trust_file ~/.mutt/equifax_gmail.cert

account default : jp

I “dashed” out the information for my current client in the second account configuration. Notice that I am using the first account as the default. In my mutt configuration (for my local mail account), I have the following configuration line for using msmtp as the mail sender:

Mutt msmtp config - mutt_msmtp_config.sh
1
2
3
4
set from = "jp@developwithpassion.com"
set sendmail = "/usr/local/bin/msmtp -a jp" #using the account named jp from my msmtp configuration file
set sendmail_wait = 0
unset record # gmail records sent mail automatically, so we don't need to track

I have 2 aliases set up that allow me to start mutt with my local mail account, and the other for my client mail account. Each instance uses a client specific mutt configuration that is also configured with the correct msmtp account to use for sending. That is outside of the scope of this article, but it is just a matter of templates and aliases.

Handling mail attachments

One of the first snags I ran into was correctly handling mail that contained attachments (sending mail with attachments is a snap, so I won’t go into it in this article). By defaualt, mutt will use you mailcap file to try to determine how to handle email. The contents of my mailcap file look as follows:

Mailcap file - mailcap.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# MS Office files
application/msword; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-" --open_with:'/Applications/OpenOffice.org.app'
application/vnd.ms-excel; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-"
application/vnd.openxmlformats-officedocument.presentationml.presentation; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-" --open_with:'/Applications/OpenOffice.org.app'
application/vnd.oasis.opendocument.text; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-" --open_with:'/Applications/OpenOffice.org.app'

# Images
image/jpg; ~/.mutt/view_mail_attachment.rb --file:%s --type:jpg
image/jpeg; ~/.mutt/view_mail_attachment.rb --file:%s --type:jpg
image/pjpeg; ~/.mutt/view_mail_attachment.rb --file:%s --type:jpg
image/png; ~/.mutt/view_mail_attachment.rb --file:%s --type:png
image/gif; ~/.mutt/view_mail_attachment.rb --file:%s --type:gif

# PDFs
application/pdf; ~/.mutt/view_mail_attachment.rb --file:%s --type:pdf

# HTML
text/html; ~/.mutt/view_mail_attachment.rb --file:%s --type:html

# Unidentified files
application/octet-stream; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-"

Notice that most of this script shells out to a custom ruby script that does the appropriate program launching. Here is that script:

View Mail Attachment - view_mail_attachment.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
#!/usr/bin/env ruby

require 'fileutils'

class Arguments
  attr_accessor :file
  attr_accessor :type
  attr_accessor :open_with
  attr_accessor :temp_dir

  def temp_dir
    @temp_dir ||= "/tempfiles/mutt_attachments"
  end

  def base_file_name
    @base_file_name ||= File.basename(file)
  end

  def base_file_name_without_extension
    File.basename(base_file_name, File.extname(base_file_name))
  end
end

def build_arguments(arguments)
  result = Arguments.new

  arguments.each do|argument|
    pair = argument.split(":")
    name = pair[0].gsub(/-/,"")
    value = pair[1]
    result.send "#{name}=", value
  end
  result
end

arguments = build_arguments(ARGV)

system("mkdir -p #{arguments.temp_dir}")
system("rm -rf #{arguments.temp_dir}/*")


new_file_name = ""
if (arguments.type == "-")
  new_file_name = File.basename(arguments.file)
else
  new_file_name = "#{arguments.base_file_name_without_extension}.#{arguments.type}"
end

new_file_name = File.join(arguments.temp_dir, new_file_name)
FileUtils.cp arguments.file, new_file_name

if (arguments.open_with)
  system("open -a #{arguments.open_with} #{new_file_name}")
else
  system("open #{new_file_name}")
end

As you can see, this solution makes the assumption that I write attachments to the following folder : /tempfiles/mutt_attachments

Conclusion

It took a couple of days for me to tweak my keybindings and customize my setup. After the initial rampup, I love that I can now compose, read, edit my email in Vim, here is a screenshot of what my gtd window in tmux looks like:

Needless to say, the screenshot says it all (from my perspective!!)

Develop With Passion®

Game Crash Bootcamp Coming to Calgary in May!!!

The Develop With Passion® – Game Crash Bootcamp is coming to Calgary, AB, Canada on the week of the 27th May 2013.

The course will be led by the extremely skilled Jon Beltran de Heredia, and it will be held at the Residence Inn Marriott Airport.

Attendees of the first Game Crash Bootcamp in Austin, back in February, were blown away by the course content. Jon has worked hard to develop a curriculum that can ensure that developers are able to build the skills to feel confident building games from scratch by the end of the week.

As with all Develop With Passion® courses, the focus is on fundamentals. Nothing shiny here. Nuts and bolts, math, and programming techniques that are not going to fade away with platform changes and language changes.

Develop With Passion® has always strived to go “deep” with students. I personally feel that if more developers allowed themselves to really understand concepts, instead of just “Google/Stack Overflow –> Copy –> Paste”, that they would utlimately feel much more confident about their abilities as a coder. Jon shares my approach to this, which is why he is the perfect trainer for this course.

If you have ever had a desire to want to learn game programming. This course will teach you the math, programming, and war stories from someone who has built very popular games for different hardware platforms.

Along with the onsite course which will be held at the Residence Inn Marriott Airport, in Calgary. The course will also be offered for people to attend remotely via a gotomeeting setup. This will give people who can’t fly to Calgary a change to take the course in their PJ’s if they so wish.

The price breakdown for this new offering (prices are only applicable for this year, next year the price will most likely rise to match the regular Develop With Passion® bootcamps):

  • Onsite US & Canada: $2500
  • Onsite International: $3500
  • Remote: $1500

  • Early bird discount (Registering within 2 weeks of event being published): $500

For more information on the course go here . To register you can use one of the following links:

Jon and I both feel confident that this is going to become a hugely popular course, once more people participate and realize how empowered they will feel with the results they will come up with in one week!!

Develop With Passion®

Pursuing Mastery

It occurred to me a couple of years ago, that there really is no such thing as attaining mastery of software development. We are all just at different stages of beginning.

When I was running a Software Craftmanship bootcamp in the UK a couple of years one of the attendees was kind enough to give me a copy of the book Bushido – The Way of the Samurai. He felt that a lot of the concepts that I was both demonstrating and talking about, were reflected in a lot of the ideas in the book. It really is a fantastic book on the subject of “pursuing mastery”. Software developers in general spend a lot of time trying to “attain mastery”. The only problem with this endeavour is that they can often trick themselves into falsely feeling like they have “attained it”. This can often result in the following set of events happening:

  • Establishing themselves as a subject matter expert in one or more areas
  • Camping on their ideas for a long period of time.
  • Failure to want to be “the beginner” again, as they have worked hard to be perceived as an expert
  • Boredom, as their need to hang-on prevents them from moving ahead
  • Transitioning to management as the next logical step, more because they come to realize that their skills have atrophied

Of course, that little blurb above is a very generalized path, but one that I have been able to observe enough for it to not just be an accident.

The sooner we can start realizing that we “pursue” mastery, then we will realize that the learning never stops. We don’t need to pat ourselves on the back too long when we have made an accomplishment, we can focus on incremental, iterative growth over the entirety of our career. This is the kind of movement that can prevent boredom, stagnation and the unecessary “promotion” to management. Now let me be clear, I have no issue with programmers who make the move to management because they choose and have a desire to. I am speaking to the devs who made the move because they felt like they had no other progression paths.

Here is an excerpt from the Bushido book that sums up what I am trying to say:

An old, retired swordsman once said, “There are levels in the course of mastery throughout your life. At the lowest level of skill and ability, one thinks of himself and others as poor. He thinks this because he has mastered only a little. Needless to say, a person at this level is not at all useful. ”
“At the middle level, one is still useless, but he can at least understand that he and others have mastered only a little.”
“At a high level, since a person has made something his own, he is proud of his accomplishment. And he is also glad of the praise of others. He grieves over the shortcomings of others. This kind of person is at least useful.”
At a higher level, one pretends to know nothing, yet others understand that he holds an upper hand. The majority of people cannot get beyond this level.
Beyond this higher level, there is one further step: the level of the trackless road. If you travel deeper into the trackless road, infinite secrets will finally appear. Then you can never see the end of your mastery. Then you truly realize how lacking you are. You have only to go ahead with your intention of mastery in mind. You go forward without pride and without humility.

High Upon High Bushido – the Way of the Samurai

Favourite part of the excerpt:

Then you can never see the end of your mastery. Then you truly realize how lacking you are. You have only to go ahead with your intention of mastery in mind. You go forward without pride and without humility.

I pray that you can continue to enjoy the pursuit!!

Develop With Passion®