I spend the majority of my day inside a shell based environment, and a large majority of my workflow/tooling has been heavily modded to allow me to do the majority of my computer related activities from a shell.

• I edit all my code in vim
• I manage keychain entries using the command line clients
• I have a easy to customize automation/expansion library that allows me to add new shell automation tasks easily
• For the rare times that I use twitter, I use a vim plugin for that!
• I compose blog entries in vim

You get the point!

I have occassionally messed around with using links as a browser client. In reality, between chrome+vimium or firefox+vimperator, I have my mouseless browsing covered. The only scenario this setup does not work well for me is when I am ssh’d into a machine and want to browse the web (hence the use of links in those scenarios).

Focused, specific tools that do one job and do it well

This post is mostly for my own journalling purposes, but I am sharing it in the event that it proves useful to others. If you are running on osx, I am going to strongly recommend that you install the homebrew package manager, as that is what I use, to install other programs.

This post will outline the way that I setup the following tools:

• mutt - Shell based email client
• offlineimap - Offline imap mail management
• urlview - View urls in files
• msmtp - Smtp client
• gnupg - Encyrption utilities

Getting Started

As noted above, you will more than likely want to install homebrew to follow along with these instructions the best. I love homebrew as it is a ruby based package management system for non gem based packages (think nodejs, sqlite …)

Installing and configuring offlineimap

After a brief 30 minute session messing around with mutt as a simple gmail client I was running into a lot of annoying “freezes”. I came to the conclusion that using mutt as a replacement for the Gmail web interface would not really bring me a lot of the benefits that I wanted from a shell based email interface. I decided that it would be handy to be able to manage email offline, which would allow me to not have to connect to gmail unless I actually needed to send emails or synchronize local changes. To that end I decided to go with offlineimap to download a complete copy of my gmail imap folders and be able to deal with mail completely offline if need be.

As is the case with the rest of the tools in this guide, the install process is a snap, the configuration process is a little involved. I installed the current version of offlineimap. To install offline imap just type the following in a shell:

1
2
3

brew install offlineimap
brew install sqlite

I installed sqlite to use it as the storage for offlineimap mail status.

Ok, that was painless now lets configure it:

1
2

cp $(brew --prefix)/Cellar/offlineimap/HEAD/offlineimap.conf ~/.offlineimaprc

This just copies a stock sample configuration file into one of the standard places that offlineimap looks for configuration. To read more about offlineimap configuration go here. I spent quite a bit of time reading through the docs and messing around with a bunch of the settings, along with messing up my install a couple of times!! That is how I learn the best though!! The documentation that comes with the sample configuration file is pretty informative, so you should be able to tweak a lot of things. As far as configuration goes I leveraged the sample configuration outlined by Steve Losh, who also went through the joys of configuring this setup a while ago. I wanted to understand how all these pieces worked, so I just leveraged his guide to rip off his configuration. His guide is also a great walkthrough of how to get this setup going. I am of the mindset that I like to struggle on my own for a bit as I try to figure things out. Instead of using his and other awesome guides, I set aside a half day and did this myself!! In the docs it mentions that the offlineimaprc file can also include a regular python file that can have supplementary code you want to leverage!! Pretty awesome concept. Steve uses this in a clever way to get around the need to have plain text passwords in the config file. Read his post for the details, but it basically makes use of the security utility (usr/bin/security on most unix based systems), to access the password in a keychain item.

Ok, so I configured things appropriately and was ready to kick off offlineimap to download the mail locally onto my computer (only about 1.5GB, but seeing as how we live out in the boonies, this would take a while). I fired up offlineimap and was immediately hit with an error about an incorrect fingerprint for the gmail stmp server. I figured this had something to do with a gmail cert change. This was rectified by following the instructions here which resulted in my adding the following config item in my JPBoodhoo-Remote Repository section:

1
2

cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042

With that change in place (keep in mind, the fingerprint can change at a later time, which would require this line to be updated), things looked good to go, I reran offlineimap and went about with other work while our blazingly fast connection proceeded to choke on the download for the next couple of hours.

Here is the majority of the pertinent stuff from my current configuration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

[general]
ui = TTYUI
accounts = jp_developwithpassion_gmail, jboodhoo------gmail
pythonfile = ~/repositories/developwithpassion/devtools/shared/dotfiles/offlineimap/offlineimap.py
fsync = False


[Account jp_developwithpassion_gmail]
localrepository = jp_developwithpassion_gmail_local
remoterepository = jp_developwithpassion_gmail_remote
status_backend = sqlite
postsynchook = notmuch new

[Account jboodhoo----gmail]
localrepository = jboodhoo----gmail_local
remoterepository = jboodhoo----gmail_remote
status_backend = sqlite
postsynchook = notmuch new


#developwithpassion
[Repository jp_developwithpassion_gmail_local]
type = Maildir
localfolders = ~/Dropbox/imap_mail/jp_developwithpassion
nametrans = get_remote_name


[Repository jp_developwithpassion_gmail_remote]
maxconnections = 1
type = Gmail
cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042 
remoteuser = jp@developwithpassion.com
remotepasseval = get_password("jp@developwithpassion.com")
realdelete = no
nametrans = get_local_name
folderfilter = is_included

#***
[Repository jboodhoo-----gmail_local]
type = Maildir
localfolders = ~/Dropbox/imap_mail/jboodhoo---
nametrans = get_remote_name


[Repository jboodhoo----gmail_remote]
maxconnections = 1
type = Gmail
cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042 
remoteuser = jboodhoo@----
remotepasseval = get_password("jboodhoo@----")
realdelete = no
nametrans = get_local_name
folderfilter = is_included

You can check out Steves post for a very thorough explanation of the configuration that matches. I took advantage of the python file to DRY up my configuration for the nametrans setting. When you are working with one of the remote repositories (jp_developwithpassion_gmail_remote) in one of the examples above; the nametrans setting determines how to map the remote imap folder names to names on your local machine. In the examples on the offlineimap site, they would set up reverse entries in the nametrans for the local and remote options. That would look something like this (lifted from some standard example online):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

[Repository blah-Local]
type = Maildir
localfolders = ~/.mail/blah.com
nametrans = lambda folder: {'drafts':  '[Gmail]/Drafts',
                            'sent':    '[Gmail]/Sent Mail',
                            'flagged': '[Gmail]/Starred',
                            'trash':   '[Gmail]/Trash',
                            'archive': '[Gmail]/All Mail',
                            }.get(folder, folder)

[Repository blah-Remote]
maxconnections = 1
type = Gmail
remoteuser = blah@blah.com
remotepasseval = "password"
realdelete = no
nametrans = lambda folder: {'[Gmail]/Drafts':    'drafts',
                            '[Gmail]/Sent Mail': 'sent',
                            '[Gmail]/Starred':   'flagged',
                            '[Gmail]/Trash':     'trash',
                            '[Gmail]/All Mail':  'archive',
                            }.get(folder, folder)

Of course, as you can see above, it seems a little redundant. In my config I dry it up as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

[Repository jp_developwithpassion_gmail_local]
type = Maildir
localfolders = ~/Dropbox/imap_mail/jp_developwithpassion
nametrans = get_remote_name

[Repository jp_developwithpassion_gmail_remote]
maxconnections = 1
type = Gmail
cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042 
remoteuser = jp@developwithpassion.com
remotepasseval = get_password("jp@developwithpassion.com")
realdelete = no
nametrans = get_local_name
folderfilter = is_included

This is done by leveraging the following code that exists in the python file that I include at the top of my configuration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102

#!/usr/bin/python
import subprocess
import re

class NameMapping:
  def __init__(self, local_name, remote_name):
    self.local_name = local_name
    self.remote_name = remote_name

class LocalName:
  def __init__(self, folder):
    self.folder = folder

  def matches(self, mapping):
    return mapping.remote_name == self.folder

  def mapped_folder_name(self, mapping):
    return mapping.local_name

class RemoteName:
  def __init__(self, folder):
    self.folder = folder

  def matches(self, mapping):
    return mapping.local_name == self.folder

  def mapped_folder_name(self, mapping):
    return mapping.remote_name



def get_password(account):
  program ='/Users/jp/repositories/developwithpassion/devtools/automation/keychain/get_keychain_password'

  command = "{0} --account:{1}".format(program, account)

  output = subprocess.check_output(command, shell=True)

  return output.rstrip()


def is_included(folder):
  result = True

  for pattern in exclusion_patterns:
    result = result and (re.search(pattern, folder) == None)

  return result

exclusion_patterns = [
  "efax",
  "earth_class_mail",
  "eventbrite",
  "gotomeeting",
  "moshi_monsters",
  "peepcode",
  "raini_fowl",
  "stuart_know",
  "training.*2008",
  "training.*2009",
  "training.*2010",
  "training.*2011",
  "training.*2012",
  "training.*nbdn",
  "training.*nothin_but_bdd",
  "unblock_us",
  "web_hosting",
  "webinars",
  "Gmail.*Important"
]

name_mappings = [
  NameMapping('sent', '[Gmail]/Sent Mail'),
  NameMapping('spam', '[Gmail]/Spam'),
  NameMapping('flagged', '[Gmail]/Starred'),
  NameMapping('trash',   '[Gmail]/Trash'),
  NameMapping('archive', '[Gmail]/All Mail'),
  NameMapping('drafts', '[Gmail]/Drafts')
]



def find_name_mapping(name):
  default_mapping = NameMapping(name.folder, name.folder)

  for mapping in name_mappings:
    if (name.matches(mapping)):
      return mapping

  return default_mapping

def get_name_mapping(name):
  mapping = find_name_mapping(name)
  return name.mapped_folder_name(mapping)

def get_remote_name(local_folder_name):
  name = RemoteName(local_folder_name)
  return get_name_mapping(name)

def get_local_name(remote_folder_name):
  name = LocalName(remote_folder_name)
  return get_name_mapping(name)

The name mappings array is the single place where the mappings are defined. That way I don’t have to worry about typos between sections. The config also shows how I am doing exclusions as well as shelling out to a custom ruby script to get passwords.

On my second imap sync I got an UID validity error on one of the folders I was trying to sync. This was resolved using the following doc.

As you can see from the above config, I have 2 different email accounts that are currently setup. The second account is starred out due to privacy reasons.

One of the cool things about the offlineimap configuration is that you can specify a python file that will get mixed into the current config file and you can leverage code in the python file to help with configuration. I am using this feature for both retrieving account passwords and for remote & local name translation.

Installing Mutt

Next up it was time to get going with mutt. Mutt at its core is a text-based mail client. For the purpose of my setup, I am going to be using it to primarily read and compose messages. It has out of the box support to be able to interface with gmail, and its own smtp utility, but I configured it to use msmtp for smpt. To that end, most of the configuration for mutt will be setting it up to basically be able to work with the local copy of messages that gets pulled down by offlineimap.

Let’s install it:

1
2

brew install mutt

Configuring gpg

As I am currently working with a client that requires encrypted messages for all internal communications, it was necessary for me to setup and configure mutt with encryption. I am using GPGTools for OSX. After installing and setting up my necessary keys, here is the config that I needed to place in the file ~/.mutt/gpg.rc (your file location may vary), this is only a slight modification of the stock gpg configuration.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

# -*-muttrc-*-
#
# Command formats for gpg.
#
# This version uses gpg-2comp from
# http://70t.de/download/gpg-2comp.tar.gz
#
# $Id$
#
# %p The empty string when no passphrase is needed,
# the string "PGPPASSFD=0" if one is needed.
#
# This is mostly used in conditional % sequences.
#
# %f Most PGP commands operate on a single file or a file
# containing a message. %f expands to this file's name.
#
# %s When verifying signatures, there is another temporary file
# containing the detached signature. %s expands to this
# file's name.
#
# %a In "signing" contexts, this expands to the value of the
# configuration variable $pgp_sign_as. You probably need to
# use this within a conditional % sequence.
#
# %r In many contexts, mutt passes key IDs to pgp. %r expands to
# a list of key IDs.

set pgp_decode_command="gpg --status-fd=2 %?p?--passphrase-fd 0? --no-verbose --quiet --batch --output - %f"
set pgp_verify_command="gpg --status-fd=2 --no-verbose --quiet --batch --output - --verify %s %f"
set pgp_decrypt_command="gpg --status-fd=2 %?p?--passphrase-fd 0? --no-verbose --quiet --batch --output - %f"
set pgp_sign_command="gpg --no-verbose --batch --quiet --output - %?p?--passphrase-fd 0? --armor --detach-sign --textmode %?a?-u %a? %f"
set pgp_clearsign_command="gpg --no-verbose --batch --quiet --output - %?p?--passphrase-fd 0? --armor --textmode --clearsign %?a?-u %a? %f"
set pgp_encrypt_only_command="pgpewrap gpg --batch --quiet --no-verbose --output - --encrypt --textmode --armor --always-trust -- -r %r -- %f"
set pgp_encrypt_sign_command="pgpewrap gpg %?p?--passphrase-fd 0? --batch --quiet --no-verbose --textmode --output - --encrypt --sign %?a?-u %a? --armor --always-trust -- -r %r -- %f"

set pgp_import_command="gpg --no-verbose --import %f"
set pgp_export_command="gpg --no-verbose --export --armor %r"
set pgp_verify_key_command="gpg --verbose --batch --fingerprint --check-sigs %r"
set pgp_list_pubring_command="gpg --no-verbose --batch --quiet --with-colons --list-keys %r"
set pgp_list_secring_command="gpg --no-verbose --batch --quiet --with-colons --list-secret-keys %r"

# fetch keys
# set pgp_getkeys_command="pkspxycwrap %r"

# pattern for good signature - may need to be adapted to locale!

# set pgp_good_sign="^gpgv?: Good signature from "

# OK, here's a version which uses gnupg's message catalog:
# set pgp_good_sign="`gettext -d gnupg -s 'Good signature from "' | tr -d '"'`"

# This version uses --status-fd messages
set pgp_good_sign="^\\[GNUPG:\\] GOODSIG"

set pgp_timeout=9000

For more thorough explanations of the above settings, you can check out the guide that I followed here

As far as my mutt configuration goes, I added the following lines to my mutt configuration file:

1
2
3
4
5

bind compose p pgp-menu

# Prepare GPG usage
source ~/.mutt/gpg.rc

The keybinding allows me to hit p after I have composed a message, and it will bring up all of the options I can apply to my mail (sign, encrypt, …)

Accessing GMail Contacts

Another piece of the puzzle was for me to be able to access my gmail contacts through mutt. To do that I leveraged a python utility called goobook.

Install python if you don’t already have it:

1
2

brew install python

Install goobook:

1
2

pip install goobook

To leverage goobook as my contact lookup mechanism, I added the following lines to my mutt configuration file:

1
2
3

set query_command="goobook query '%s'"
set editor = "vim --cmd 'let g:goobookrc=\"/Users/jp/.goobookrc\"'"
bind editor <Tab> complete-query

The first settings tells mutt which utility to use to query for contact completion. The second setting allows me to have vim be able to autocomplete google contacts when I am editing mail headers in Vim. The third setting is just a mutt keybinding that will trigger the goobook search when I hit tab in mail header fields.

Mail Searching With notmuch

One of the last pieces of the puzzle was being able to locally search email. I accomplished this by setting up notmuch. Getting going was as simple as running:

1
2
3

brew install notmuch
notmuch setup #setting up for first time use
notmuch new #run the first search

To hook into the search from inside of mutt, I added the following configuration line to my mutt configuration file:

1

macro index S "<enter-command>unset wait_key<enter><shell-escape>mutt_notmuch /tempfiles/notmuch_search_results<enter><change-folder-readonly>/tempfiles/notmuch_search_results<enter>" "search mail (using notmuch)"

Basically, when I am in mutt and want to search for mail, I can hit S and then enter in a search term. At that point it will shell out to a small ruby script that I wrote, that executes the notmutt search and builds up the appropriate directory structure for mutt to treat the search results as a mail folder that can be explored in mutt. Finally, I change the working folder to the folder containing the search results (for me /tempfiles/notmuch_search_results). Here is the script that I wrote to do the search and result dir creation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69

#!/usr/bin/env ruby

require 'digest/sha1'
require 'optparse'
require 'fileutils'

class Hasher
  def initialize
    @hasher = Digest::SHA1.new
  end

  def get_email_hash(file_name)
    @hasher.hexdigest(IO.read(file_name))
  end
end

def build_results_folder(results_path)
  FileUtils.rm_rf(results_path)
  FileUtils.mkdir_p(results_path)

  %w/cur new/.each do |folder|
    combined_path = File.join(results_path, folder)
    FileUtils.mkdir_p(combined_path)
  end
end

def read_line(message)
  print "Query:"
  STDIN.gets.chomp
end

def get_unique_files(files)
  hasher = Hasher.new
  messages = {}
  files.each do |file|
    hash = hasher.get_email_hash(file)
    messages[hash] = file
  end
  messages.values
end

def get_matching_files(query)
  files = `notmuch search --output=files #{query}`
  files.split("\n")
end

def sym_link(files, target_path)
  files.each do |file|
    system("ln -s #{file} #{target_path}")
  end
end

def run(results_path)
  build_results_folder(results_path)
  query = read_line("Search Phrase?")
  files = get_matching_files(query)
  unique_files = get_unique_files(files)
  sym_link(unique_files, "#{results_path}/cur/")
end

def parse_arguments
  OptionParser.new do|options|
    options.banner = "Usage is: mutt_notmuch [RESULTS_PATH]"
  end.parse!

  path = ARGV.length == 0 ? "/tempfiles/notmuch_search_results" : ARGV[0]
end

run parse_arguments

Sending Mail

Up to this point I have only talked about a scenario optimized for reading mail. One of the main things you will also want to be able to do is send mail. Mutt has support for smtp, but there are more robust programs out there that only focus on smtp. One of those programs is msmtp . This is the program that I configured mutt to use to do mail sending.

Installing is a simple as:

1

brew install msmtp

Here is what my current msmtp configuration file looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

account jp
host smtp.gmail.com
port 587
protocol smtp
auth on
from jp@developwithpassion.com
user jp@developwithpassion.com
tls on
tls_trust_file ~/.mutt/equifax_gmail.cert

account jp_---
host smtp.gmail.com
port 587
protocol smtp
auth on
from jp@developwithpassion.com
user jboodhoo@------
tls on
tls_trust_file ~/.mutt/equifax_gmail.cert

account default : jp

I “dashed” out the information for my current client in the second account configuration. Notice that I am using the first account as the default. In my mutt configuration (for my local mail account), I have the following configuration line for using msmtp as the mail sender:

1
2
3
4

set from = "jp@developwithpassion.com"
set sendmail = "/usr/local/bin/msmtp -a jp" #using the account named jp from my msmtp configuration file
set sendmail_wait = 0
unset record # gmail records sent mail automatically, so we don't need to track

I have 2 aliases set up that allow me to start mutt with my local mail account, and the other for my client mail account. Each instance uses a client specific mutt configuration that is also configured with the correct msmtp account to use for sending. That is outside of the scope of this article, but it is just a matter of templates and aliases.

Handling mail attachments

One of the first snags I ran into was correctly handling mail that contained attachments (sending mail with attachments is a snap, so I won’t go into it in this article). By defaualt, mutt will use you mailcap file to try to determine how to handle email. The contents of my mailcap file look as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

# MS Office files
application/msword; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-" --open_with:'/Applications/OpenOffice.org.app'
application/vnd.ms-excel; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-"
application/vnd.openxmlformats-officedocument.presentationml.presentation; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-" --open_with:'/Applications/OpenOffice.org.app'
application/vnd.oasis.opendocument.text; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-" --open_with:'/Applications/OpenOffice.org.app'

# Images
image/jpg; ~/.mutt/view_mail_attachment.rb --file:%s --type:jpg
image/jpeg; ~/.mutt/view_mail_attachment.rb --file:%s --type:jpg
image/pjpeg; ~/.mutt/view_mail_attachment.rb --file:%s --type:jpg
image/png; ~/.mutt/view_mail_attachment.rb --file:%s --type:png
image/gif; ~/.mutt/view_mail_attachment.rb --file:%s --type:gif

# PDFs
application/pdf; ~/.mutt/view_mail_attachment.rb --file:%s --type:pdf

# HTML
text/html; ~/.mutt/view_mail_attachment.rb --file:%s --type:html

# Unidentified files
application/octet-stream; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-"

Notice that most of this script shells out to a custom ruby script that does the appropriate program launching. Here is that script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

#!/usr/bin/env ruby

require 'fileutils'

class Arguments
  attr_accessor :file
  attr_accessor :type
  attr_accessor :open_with
  attr_accessor :temp_dir

  def temp_dir
    @temp_dir ||= "/tempfiles/mutt_attachments"
  end

  def base_file_name
    @base_file_name ||= File.basename(file)
  end

  def base_file_name_without_extension
    File.basename(base_file_name, File.extname(base_file_name))
  end
end

def build_arguments(arguments)
  result = Arguments.new

  arguments.each do|argument|
    pair = argument.split(":")
    name = pair[0].gsub(/-/,"")
    value = pair[1]
    result.send "#{name}=", value
  end
  result
end

arguments = build_arguments(ARGV)

system("mkdir -p #{arguments.temp_dir}")
system("rm -rf #{arguments.temp_dir}/*")


new_file_name = ""
if (arguments.type == "-")
  new_file_name = File.basename(arguments.file)
else
  new_file_name = "#{arguments.base_file_name_without_extension}.#{arguments.type}"
end

new_file_name = File.join(arguments.temp_dir, new_file_name)
FileUtils.cp arguments.file, new_file_name

if (arguments.open_with)
  system("open -a #{arguments.open_with} #{new_file_name}")
else
  system("open #{new_file_name}")
end

As you can see, this solution makes the assumption that I write attachments to the following folder : /tempfiles/mutt_attachments

Conclusion

It took a couple of days for me to tweak my keybindings and customize my setup. After the initial rampup, I love that I can now compose, read, edit my email in Vim, here is a screenshot of what my gtd window in tmux looks like:

Needless to say, the screenshot says it all (from my perspective!!)

Develop With Passion®

The course will be led by the extremely skilled Jon Beltran de Heredia, and it will be held at the Residence Inn Marriott Airport.

Attendees of the first Game Crash Bootcamp in Austin, back in February, were blown away by the course content. Jon has worked hard to develop a curriculum that can ensure that developers are able to build the skills to feel confident building games from scratch by the end of the week.

As with all Develop With Passion® courses, the focus is on fundamentals. Nothing shiny here. Nuts and bolts, math, and programming techniques that are not going to fade away with platform changes and language changes.

Develop With Passion® has always strived to go “deep” with students. I personally feel that if more developers allowed themselves to really understand concepts, instead of just “Google/Stack Overflow -> Copy -> Paste”, that they would utlimately feel much more confident about their abilities as a coder. Jon shares my approach to this, which is why he is the perfect trainer for this course.

If you have ever had a desire to want to learn game programming. This course will teach you the math, programming, and war stories from someone who has built very popular games for different hardware platforms.

Along with the onsite course which will be held at the Residence Inn Marriott Airport, in Calgary. The course will also be offered for people to attend remotely via a gotomeeting setup. This will give people who can’t fly to Calgary a change to take the course in their PJ’s if they so wish.

The price breakdown for this new offering (prices are only applicable for this year, next year the price will most likely rise to match the regular Develop With Passion® bootcamps):

• Onsite US & Canada: $2500
• Onsite International: $3500

• Remote: $1500

• Early bird discount (Registering within 2 weeks of event being published): $500

For more information on the course go here . To register you can use one of the following links:

• Calgary Onsite
• Calgary Remote

Jon and I both feel confident that this is going to become a hugely popular course, once more people participate and realize how empowered they will feel with the results they will come up with in one week!!

Develop With Passion®

“Do everything you do to the glory of the one who made you!!”

Steven Curtis Chapman - Do Everything (Official Music Video) from steven-curtis-chapman on GodTube.

When I was running a Software Craftmanship bootcamp in the UK a couple of years one of the attendees was kind enough to give me a copy of the book Bushido - The Way of the Samurai. He felt that a lot of the concepts that I was both demonstrating and talking about, were reflected in a lot of the ideas in the book. It really is a fantastic book on the subject of “pursuing mastery”. Software developers in general spend a lot of time trying to “attain mastery”. The only problem with this endeavour is that they can often trick themselves into falsely feeling like they have “attained it”. This can often result in the following set of events happening:

• Establishing themselves as a subject matter expert in one or more areas
• Camping on their ideas for a long period of time.
• Failure to want to be “the beginner” again, as they have worked hard to be perceived as an expert
• Boredom, as their need to hang-on prevents them from moving ahead
• Transitioning to management as the next logical step, more because they come to realize that their skills have atrophied

Of course, that little blurb above is a very generalized path, but one that I have been able to observe enough for it to not just be an accident.

The sooner we can start realizing that we “pursue” mastery, then we will realize that the learning never stops. We don’t need to pat ourselves on the back too long when we have made an accomplishment, we can focus on incremental, iterative growth over the entirety of our career. This is the kind of movement that can prevent boredom, stagnation and the unecessary “promotion” to management. Now let me be clear, I have no issue with programmers who make the move to management because they choose and have a desire to. I am speaking to the devs who made the move because they felt like they had no other progression paths.

Here is an excerpt from the Bushido book that sums up what I am trying to say:

An old, retired swordsman once said, “There are levels in the course of mastery throughout your life. At the lowest level of skill and ability, one thinks of himself and others as poor. He thinks this because he has mastered only a little. Needless to say, a person at this level is not at all useful. ”
“At the middle level, one is still useless, but he can at least understand that he and others have mastered only a little.”
“At a high level, since a person has made something his own, he is proud of his accomplishment. And he is also glad of the praise of others. He grieves over the shortcomings of others. This kind of person is at least useful.”
At a higher level, one pretends to know nothing, yet others understand that he holds an upper hand. The majority of people cannot get beyond this level.
Beyond this higher level, there is one further step: the level of the trackless road. If you travel deeper into the trackless road, infinite secrets will finally appear. Then you can never see the end of your mastery. Then you truly realize how lacking you are. You have only to go ahead with your intention of mastery in mind. You go forward without pride and without humility.

Favourite part of the excerpt:

Then you can never see the end of your mastery. Then you truly realize how lacking you are. You have only to go ahead with your intention of mastery in mind. You go forward without pride and without humility.

I pray that you can continue to enjoy the pursuit!!

Develop With Passion®

Currently, the next big question to ask and answer is where the next course should be hosted?

Instead of us just arbitrarily picking the next location, we thought we would solicit some suggestions on where the course should go next. To that end, if you feel compelled to help drive the literal direction of this course, please respond to the following survey, to help us make our decision!

We are looking forward to bringing this amazing bootcamp to developers all over the world who are wanting to stop just “thinking” about writing a game, and give them the fundamental skills and knowledge with which they can actually “write” one. We are also hoping to demystify for lots of developers, what it means to do things like:

• Write your own physics engine
• Leveraging OpenGL for graphics primitives
• lots, lots more…!!!

Develop With Passion®

In that time God has blessed me with the opportunity to work in a whole host of development environments, business domains, and countries. One of the things that I am constantly trying to do is push and stay outside of comfort zones. This is a life principal that is not just localized to the realm of my career. God does not want you to camp in a comfort zone, you can’t grow in a comfort zone. You can only maintain.

For the majority of my consulting gigs I have been able to experience a very high degree of success and leave with glowing reviews from both the managers and developers that I spend my time with. As a developer my primary focus when engaging clients, has been to focus on the code side of the fence. That is where my work passion lives. I am not interested in iteration management or project management. Having this knowledge has allowed me to maintain a very high degree of focus in honing the “craft”.

As a perpetual student, I am constantly welcoming of the opportunity to work with teams and individuals of all skill levels. In this capacity, I have historically been brought in as a senior resource to help cut some crazy code. It has been a while, but for the first time in a long time, the current contract that I am on has me feeling like a complete beginner again!!

And I will tell you the truth, it is times like this when I am reminded at an even higher level, of how exciting the realm of software development is. I love having my ideas turned upside down, and even in the span of only one month (the contract just started 1 month ago), I am actively and happily questioning, and shifting a lot of the approaches that I have taken to developing software, and more importantly design in general!!

Every morning for the last 5 years, I have woke up and thanked God for the fact that my career was starting again, that very day. When you wake up in the morning you get to choose how you are going to react to the events that come along in your day. You get to choose to be the beginner again. To realize that what you know now can become a small fragment of what you will know in a month, a year, a decade.

If you let go any assumptions you have about your own skill level, and go forward with the “pursuit” and not attainment of mastery; every day will be an amazing opportunity to tap into the creativity of the One who created us to be creative. You can choose joy, choose to be thankful, and choose to make a difference in not just your attitude and approach, but of those you surround yourself with.

Thank you Lord for allowing me to start my career again today!! I choose the path of the beginner.

Develop With Passion®

Currently the course load is small, but you can expect to see additions as Jon and I coordinate further iterations of the Gamecrash bootcamp over the course of the year.

Personally, I will be hosting courses in the following locations:

• Calgary,AB
• Denver, CO
• San Diego, CA
• Manchester, UK
• Brussels, Belgium
• Nashville, TN
• Philadelphia, PA
• Edmonton, AB

This year I will also be mixing in more occurences of the Ruby Primer bootcamp, as the first iteration proved to be very successful.

With the announcement that Jon made about the Gamecrash bootcamp, we are looking forward to a flood of new students excited to get into the world of game programming.

Develop With Passion® looks forward, and is honoured, to be a part in kickstarting/reginiting the passion in the 2013 class!!

Develop With Passion®

As all Develop With Passion® courses, the primary focus will be on the fundamentals. We want to give you the core tools that you can use to apply in any environment that you may consider building games.

As time permits I will make myself available to deliver this course also, but this brings me to the other part of this exciting announcement!

The primary instructor for this course is going to be Jon Beltran de Heredia, who many of you know as the creator of the fantastic ViEmu. Jon has a long history with game programming, being the lead programmer for Commandos and Commandos 2, and has deep experience with all aspects of game development.

We are very excited to have Jon on board with Develop With Passion®, and can’t wait to have a whole new set of students get “put through the wringer”!!

Please join me in welcoming Jon to the Develop With Passion® team and Gamecrash to the awesome lineup of courses that Develop With Passion® is continuing to grow.

Develop With Passion®

Considering that the group was extremely small (2 people!, not a surprise for the first run) , and that they both had experience with ruby (1 - 3 years), the challenge level received was significant and there was lots of amazing feedback. This feedback has strengthened my resolve to take this officially on the road as there is lots for even seasoned Ruby devs to learn.

The plan will be to deliver more of the Ruby Primer Bootcamps next year along with continuing to deliver the Develop With Passion - .Net Bootcamp.

My hope is that as awareness of the course and its challenging content grows, word of mouth will circulate and the class sizes will start to get bigger, which ultimately makes it a much richer environment for learning.

Here are some journal notes made by the attendees of the course:

• software craftmanship
• level up

Having already done the .Net boot camp I was looking forward to the Ruby Primer course with lots of excitement and a little bit of fear. Excitement because I knew how much I’d be learning over the course of the week and fear because it was going to be lots of hard work. It was totally worth it, embrace the fear!

I love playing around with Ruby and naively thought I had a pretty good grasp of the language but that idea was wiped out on day one. The course takes you out of your comfort zone very quickly and this is where real growth happens.

JP’s method of teaching brings the theory and practice of the ruby fundamentals together in such a way that it really sinks in. I have been genuinely surprised at how much more I understand some of the gnarly meta programming code I’ve read since the course. Before the course the same code would have just made my eyes glaze over.

I would have no hesitation recommending this course to anyone, Ruby programmer or not as long as you have a genuine interest in becoming better at your craft. Seeing the patterns and practices attacked from another angle, Ruby rather than .Net has really helped my understanding and at the same time highlighted gaps in my knowledge. JP’s boot camps are not really a weeklong, they last for weeks, months, and years as you try and plug these gaps.

It wasn’t all about the technical, as well as keeping you incredibly well fed and watered the course also leads you to do some real soul searching about where your career and even life in general is heading.

Thank you JP for a truly life changing experience! Now how many ‘technical’ courses do you get to say that about?

I feel blessed to be able to reach another set of developers with a set of approaches and techniques that do truly transcend language and platforms!

Develop With Passion®

The guys at camug graciously asked if I would be interested in coming to give a small 4 hour intro to the Ruby programming language.

I am going to keep the concepts as simple as possible; and treat it as a basic introduction to the ruby programming language in the context of building a simple, but fun, application.

The presentation is completely free so if you are interested in attending you have to:

• Register for the event at the camug site (there are only 25 spots available)
• Show up next Saturday morning with you laptop in tow, so that you can code along and have the concepts solidify in your head.

Software Requirements

• Windows Based OS

  • Install the latest version of Ruby using the RubyInstaller

• Unix Based OS (OSX/Linux)

  • Install the most current version of RVM, using the instructions on its site.

Regardless of which environment you come with, make sure that you have it configured correctly and can run a basic ruby file from your console. I would also recommend that you have a good text editor at the ready to code along during the presentation.

If there are still spots available, come out for what I pray will be a fun even for all involved!!!

Develop With Passion®

Once again, this year we will be hosting the course at the Ramada Hyde Park. Registration details can be found here.

You can come expecting 5 fun filled days of coding madness. Come and learn fundamental software principles, patterns, and techniques that will help take you to the next level in your career as a computer programmer. The course will be using .Net as a delivery mechanism, but past students will iterate that the course really has nothing to do with .Net. It is a walkthrough of practical application design techiques and approaches that you can use regardless of what your primary development environment/platform is.

If you are up for a challenge, or you feel like your dev routine needs a bit of a kick in the pants, sign up for what many others have called a “game-changer” in both their life and career.

I look forward to seeing you there!

Register Now

Develop With Passion®

If you have ever used Pentadactyl or Vimperator, you are already familiar with the feature. In Pentadactyl, when a web page is being displayed, if you press the ‘f’ key, it will highlight all of the clickable links on the page. To click a link, you just have to type in the digits/letters that correspond to the link. As you can imagine, this feature can save quite a lot of time!!

Well, Keymando supports the same style of feture, except you can use it in any OSX based app that supports OSX Accessibility.

In the screencast I show you a sample usage scenario of this awesome feature!

Go here to get the latest version of Keymando, and get busy automating your way through OSX!

Along with a set of tried and tested: Develop With Passion® - .Net Bootcamps, this year I am also going to be launching a Develop With Passion® - Ruby Bootcamp! In the same vein as the .Net bootcamp, this course will be a deep dive into advanced fundamentals of software engineering using Ruby as the delivery language mechanism. Over the course of the week, you will learn the ins an out of design patterns, software engineering, automated testing and builds, all while trying to build a Rails clone from the ground up. If you are wanting yet another course that will teach you how to use Rails - well, this one won’t be for you. This is a course that (like the .Net bootcamp) is geared at making you a much better software developer overall. Of course we will dive into some pretty awesome usages of the language and it capabilities, but first and foremost these bootcamps are all about giving you a solid foundation with which you can use to propel your software goals to new heights.

One of the biggest changes I am personally making to the courses (based off of a couple of years worth of feedback now) is the addition of slide decks to complement the material that is covered. This has traditionally been a very code focused course set, with the majority of the five 12 hours days that make up a course week revolving around cutting code! That focus is not changing, but it is now going to be augmented with a set of material that will cater to the visual learners.

The automation for the course over the years has become really smooth. With GitHub being used as a central component for both versioning our changes, as well as rapidly switching between different views on a problem. Even if you have no git experience, the automation scripts that are setup for the course allow people to keep moving smoothly without git inexperience getting in the way.

Course weeks are recorded in ScreenFlow screencasting software, and students are able to copy the screencasts from me at the end of the course week.

Over the course of the past 4 years I have been blessed to be able to travel the world with my family and deliver this course to over 1500 people. Over that time, I have had the benefit of identifying patterns and practices of effective development that I have both observed from people I have worked with as well as effective devs who have taken the training course. My main personal goals for anyone attending one of the Develop With Passion® courses are as follows:

1. Provide you with approaches, resources, and examples that can help turn you into an amazing coder.
2. Give inspiration and insight that can help you set bigger goals both professionally and more importantly, personally.
3. Help you to remember why you became a coder in the first place.

Don’t take my word for it, here is a sampling of comments made by past students (good and bad!, sorry for the age of the links, collecting reviews has definitely been a weak point!):

• So What Do Past Students Really Think

The upcoming course schedule for the 2012 year can be seen here.

And just in case you don’t click on the link, here it is:

• .Net Bootcamp - New York, NY - 27/8/2012
• .Net Bootcamp - London, UK - 10/9/2012
• .Net Bootcamp - Brussel, Belgium - 17/9/2012
• .Net Bootcamp - Chicago, IL - 1/10/2012
• .Net Bootcamp - Remote - 15/10/2012
• .Net Bootcamp - Calgary, AB - 5/11/2012
• Ruby Bootcamp - Las Vegas, NV - 26/11/2012
• .Net Bootcamp - Edmonton, AB - 3/12/2012

I am pretty excited about the courses that are coming up, and look forward to seeing some new faces ready to learn and share together!

Come and learn how to Develop With Passion®

God Bless You!

The feature in question is the ability to swap out classes at test time. The most useful aspect for this is in both configuring and monitoring calls made to class methods. Up until now, I have just used the rspec (predominant testing library I use) out of box mechanism to do this. With this feature added now, I don’t need to use any of the out of box rspec mocking features.

Here is an example of using the capability to monitor a call made to delete a file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

class DeleteFileCommand
  def initialize(filename)
    @filename = filename
  end
  def run
    File.delete(@filename)
  end
end
context "when run" do
  let(:filename){'blah.rb'}
  let(:sut){DeleteFileCommand.new(filename)}
  before (:each) do
    fake_class File # I usually use multiple before blocks to separate setup from invoking the method I am testing, this could also be in the before block below
  end

  before (:each) do
    sut.run
  end

  it "should delete the file" do
    File.should have_received(:delete,filename)
  end
end

The use of the new fake_class method creates a fake instance (using the existing fake infrastructure) and uses it to be the value used for all class methods on the specified class. You can notice how in the assertion we are verifying that the File class received a call to the delete method with a specified argument. This mechanism is identical to how you can already verify calls made to fakes, with all of the goodness of the new arg matching capabilities also available to use.

When you make use of the fake_class method, from that point on any reference to the class that you specified is now talking to the fake. Which means you can also treat it like any other fake when you are setting up fake behaviour. Here is another example:

1
2
3
4
5
6
7
8
9

before(:each) do
  fake_class File

  File.stub(:exist?).with(arg_match.any).and_return(true)
end

it "should never have a file that does not exist" do
  File.exist?('totally_non_existent_file').should be_true
end

Hopefully you can see that is a contrived example, but it demonstrates how you can setup behaviours now fully using fakes and not needing to mix and match between fakes and (in my original case, RSpec mocks and stubs).

fakes-rspec has already been rebuilt against this library to take advantage of this feature. The only change that was added to it was literally the following lines of code:

1
2
3
4
5

RSpec.configure do |config|
  config.after(:each) do
    reset_fake_classes
  end
end

This just ensures that after each test run, the classes get reset back to their original behaviour. Which means if you are already using fakes-rspec, you can just start using by updating to the latest version!!

Develop With Passion®

The feature in question is the ability to do arbitrary matching of arguments during both the setup phase for a fake and also for the verification section.

Here is an example of using the capability in your before block to setup a fake to return values for a call:

1
2
3
4
5
6
7
8
9
10

context "when setting up a fake using a combination of real values and matchers" do
 before (:each) do
  fake_item.stub(:do_work).with(2,3,arg_match.regex(/y/).and_return(3)
 end

 it "should be able to return values when calls are made that match the matchers and the real values" do
   fake_item.do_work(2,3,'yes').should == 3
   fake_item.do_work(2,3,'y').should == 3
 end
end

This is a really basic example, but basically, when you are setting up a method call, you can now specify either the explicit arguments the method should be called with to return a certain value, or you can use argument matchers, or both!! The arg_match method is a gateway to a factory that can create matchers. In pattern terms its a gatway to a specification factory!! You can use one of the prexisting methods:

• nil
• not_nil
• any
• greater_than
• in_range
• regex
• condition

I could add more convienience methods, but they all end up calling into the condition method which just requires a block which matches the signaure:

1

  {|item| bool}

An example of using this to create an arbitrary matcher is as follows:

1
2
3
4
5
6
7
8
9
10
11
12

context "when setting up a fake using a combination of real values and matchers" do
 let(:fake_item){fake}

 before (:each) do
  fake_item.stub(:do_work).with(arg_match.condition{|item| [2,3].include?(item)},3,arg_match.regex(/y)).and_return(3)
 end

 it "should be able to return values when calls are made that match the matchers and the real values" do
   fake_item.do_work(2,3,'yes').should == 3
   fake_item.do_work(2,3,'y').should == 3
 end
end

Hopefully you can see that is a contrived example, as we already have the in_range method that can handle that. Of course in place of either of those existing methods, or the condition method, you can just pass in something that responds to: matches?(item). Here is another contrived example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

context "when setting up a fake using a combination of real values and matchers" do
 let(:fake_item){fake}

 class CustomArgMatching
   def self.matches?(item)
    item  != 42
   end
 end

 before (:each) do
  fake_item.stub(:do_work).with(CustomArgMatching,3,arg_match.regex(/y/)).and_return(3)
 end

 it "should be able to return values when calls are made that match the matchers and the real values" do
   fake_item.do_work(2,3,'yes').should == 3
   fake_item.do_work(2,3,'y').should == 3
   fake_item.do_work(42,3,'y').should == nil
 end
end

This same mechanism works identically when you are doing post invocation verification. Here is another example:

1
2
3
4
5
6
7
8
9
10
11
12
13

context "when verifying that calls are made, using argument matchers" do
 let(:fake_item){fake}

 before (:each) do
  fake_item.do_work(2,3,'yes')
 end

 it "should be able to make call verifications" do
   fake_item.should have_received(:do_work,2,3,'yes') # this is doing fully explicit matching on each argument
   fake_item.should have_received(:do_work,2,3,arg_match.regex(/y/)) # this is matching explicitly on the first 2 arguments and using a matcher on the last
   fake_item.should have_received(:do_work,arg_match.any,arg_match.any,arg_match.regex(/y/)) # this is matching using matchers for all 3 (even though the first 2 matchers are just a special case)
 end
end

Whether you are in the setup phase or the verification phase the model is consistent. You can mix and match the matchers in place of real arguments as you need to. This ultimately gives you much better control on focusing on what it is you care about in the particular interaction you are observing!!

Hopefully you will find this as useful as I do!!

Develop With Passion®

VMWare Fusion fully supports linked clones;unlike VMWare Workstation it does not yet have a UI to create and manage them.

This video covers the main points:

• How I organize and separate my vms from the vmdks.
• Creating linked clones

Shamless Plug - Aside from the first click, no other mouse usage occured during the course of this video, courtesy of keymando.

Develop With Passion®

Prior to doing this I also completely deleted the /Developer folder.

Upon successfully reinstalling RVM and homebrew using the devtools, I cd’d into a rvm managed folder and did the standard bundle install, upon which time I eventually received this error:

1

: fatal error: 'stdio.h' file not found

I had installed the Command Line Tools for Xcode, but I had not yet switched where it should resolve those tools. Running the following command fixed the issue:

1

sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer

Develop With Passion®

I’ll write another post later about how I use devtools to successfully run msys and cygwin side by side from the same set of dotfiles (customized per environment)!

I started with a brand new vm image with the software installed according to the post here.

If you don’t wish to read that previous post just know I installed cygwin to C:\utils\cygwin (for the purpose of this post it you are following along, I would suggest installing to that path also). I also included the following packages (some of these are not necessary for ruby compilation, but they have become my base for a cygwin install):

• Archive
  • unzip - Unzipping zip files
• Net
  • openssl - bin and sources
  • openssh - Only if you are not going to compile openssh yourself
  • curl - download internet resources
• Devl
  • colorgcc
  • gcc
  • gcc-core - compiler
  • git
  • git-completion
  • git-gui
  • git-svn
  • gitk
  • libtool - Shared library generation tool. You’ll need it when trying to compile rubies
  • libncurses-devel - Used when compiling several other tools I use
  • make
  • mercurial
  • openssl-devel - Required for compiling openssh (not necessarily required for rvm, but I always install it to compile openssh myself)
  • readline
• Libs
  • zlib
  • zlib-devel
• Utils
  • ncurses - Enabling better handling of terminal
  • patch - Apply a diff file to an original. Again, you’ll need it when rvm is trying to patch the ruby installs

Once the cygwin install completes we can continue.

Setting Up

• Open up a new cygwin session (C:\utils\cygwin\cygwin.bat).

• Issue the following commands in the cygwin session:

1
2
3
4
5
6

mkdir repositories
cd repositories
mkdir developwithpassion
cd developwithpassion
git clone git://github.com/developwithpassion/devtools
cd devtools

The results should look as follows:

• You should now be sitting in the devtools folder so you can now run the kick_off_script:

1

./osx_or_cygwin_kick_off

• Repeat the above step(the first time you run it, it creates a settings file for your user that can be edited further if you are going to make further use of devtools later on)

  For the curious, the script is here

  The script does the following:

  • kicks off an rvm installation script
  • updates to the latest rvm
  • installs a couple of rubies and sets 1.9.3 as the default ruby

The last step takes a little time to complete, but the end result is definitely worth it!!:

Develop With Passion®

Here is the information for the webinar:

Reserve your Webinar seat now at:

This webinar will be an opportunity for people to get an understanding of how I structure my devtools folder and how it is leveraged to facilitate working effectively between osx and windows.

Title: Develop With Passion® - devtools Date: Wednesday, March 28, 2012 Time: 12:00 PM - 1:00 PM MDT

After registering you will receive a confirmation email containing information about joining the Webinar.

System Requirements * PC-based attendees * Required: Windows® 7, Vista, XP or 2003 Server * Macintosh®-based attendees * Required: Mac OS® X 10.5 or newer

Develop With Passion®

Being our first time in Brussels, we are hoping that pushing back the date will provide more people with an opportunity to convince their bosses that the course is a good investment!

I apologize for any inconvienience this may cause.

God Bless You.

Develop With Passion®