Simple Strategy for Resolving New Automation Scripts

I have a large (and continually increasing) number of automation scripts in play on my machine. I usually keep a simple notebook page going where I will make a note of automation items that I seem to be missing, usually identified by things that I have to do multiple times that I think could be automated. Once a week I will look at that list and complete one of the automation items.

After doing this for a while, it became apparent that I needed to have a simple way to be able to have new automation scripts be recognized, so I could quickly start using them. As my automation scripts grew, I wanted to be able to organize the scripts by functional area, so I could quickly get a listing of my automation folder and see what types of automation items I had in play.

My automation scripts currently consist of predominantly bash, python, and ruby scripts, all of which are marked executable (the ones that are automation items anyway).

I won’t get into how I organize my dotfiles, but here is an important couple of lines in one of my dotfiles that ensures that all of my automation scripts (regardless of folder they are organized into) are picked up and available in my path:

Automation Scripts - scripts.sh
1
2
OLD_PATH=$PATH
export PATH="$OLD_PATH:$(find [my automation root folder] -name '.*' -prune -type d | tr "\n" ":")"

Obviously, you would replace [my automation root folder], with the path to the folder which is the parent of all of your automation script folders. For me that is ~/repositories/developwithpassion/devtools/automation.

Notice the use of the find command to find all folders underneath my automation root, which are then piped into the tr utility to convert every newline into the required path separator.

This allows me to either add a new executable automation script under an existing folder in my automation hierarchy; or I can add a new automation script under a brand new subfolder in my automation hierarchy. Either way it will be picked up.

Develop With Passion®

Simple Script to Create Directory Specific Gemsets

When I’m working on ruby based projects I use rvm to manage and switch between differing ruby versions and gemsets.

I don’t ever keep my .ruby-version or .ruby-gemset files (or the old .rvmrc) under version control. Which means that I have to recreate them each time I clone a copy of the project. Sometimes I will have the same project cloned out in differing folders. I have usually tried to keep folder specific gemsets; that way I don’t have to get creative with gemset names as the gemset is basically the full path of the folder that I am in.

Here is the script that I use to create a new .ruby-version and .ruby-gemset in the current folder that I am in:

RVM Gemset Create - rvmgsc
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
#!/usr/bin/env ruby
def rubies
  ruby_pattern = /\s*(ruby-.*)\s\[/

  rubies = `rvm list`
  rubies = rubies.split("\n")
  rubies = rubies.select{ |item| ruby_pattern =~ item }
  rubies = rubies.map{ |item| ruby_pattern.match(item)[1] }
  rubies.sort! { |left, right| right <=> left }

  rubies
end

def pick_ruby_version
  puts "Which Ruby Version?"
  versions = rubies
  versions.each_with_index do |version, index|
    puts "\t#{index + 1} - #{version}"
  end
  puts "Which?: "
  version = gets.chomp.to_i
  version = version - 1
  versions[version]
end

def ruby_version
  return pick_ruby_version if ARGV.length == 0

  version = ARGV[0]
  full_version = "ruby-#{version}"

  return full_version
end

def gemset_name(gemset)
  gemset = gemset.gsub(/[\/\-\s]/,'_')
  gemset = gemset.gsub(/_{2,}/,'_')
  gemset = gemset.slice(1, gemset.length)
  gemset.downcase
end

gemset = gemset_name(Dir.pwd)

`echo #{ruby_version} > .ruby-version`
`echo #{gemset} > .ruby-gemset`

This file is marked executable and in my path (I name it rvmgsc). Whenever I enter a new folder that I want a gemset for I can call it either with an argument of the ruby version that I want to use. If I don’t pass any arguments it will list the ruby versions that I have and let me choose. The script will then continue to make an .ruby-version and .ruby-gemset file, with the correct contents.

Develop With Passion®

Getting Up and Running With a Sane Mutt Setup

Well, for the last 2 months I have been using Mutt as my primary email client, and I have not missed gmail at all!!

I spend the majority of my day inside a shell based environment, and a large majority of my workflow/tooling has been heavily modded to allow me to do the majority of my computer related activities from a shell.

  • I edit all my code in vim
  • I manage keychain entries using the command line clients
  • I have a easy to customize automation/expansion library that allows me to add new shell automation tasks easily
  • For the rare times that I use twitter, I use a vim plugin for that!
  • I compose blog entries in vim

You get the point!

I have occassionally messed around with using links as a browser client. In reality, between chrome+vimium or firefox+vimperator, I have my mouseless browsing covered. The only scenario this setup does not work well for me is when I am ssh’d into a machine and want to browse the web (hence the use of links in those scenarios).

Focused, specific tools that do one job and do it well

This post is mostly for my own journalling purposes, but I am sharing it in the event that it proves useful to others. If you are running on osx, I am going to strongly recommend that you install the homebrew package manager, as that is what I use, to install other programs.

This post will outline the way that I setup the following tools:

Getting Started

As noted above, you will more than likely want to install homebrew to follow along with these instructions the best. I love homebrew as it is a ruby based package management system for non gem based packages (think nodejs, sqlite …)

Installing and configuring offlineimap

After a brief 30 minute session messing around with mutt as a simple gmail client I was running into a lot of annoying “freezes”. I came to the conclusion that using mutt as a replacement for the Gmail web interface would not really bring me a lot of the benefits that I wanted from a shell based email interface. I decided that it would be handy to be able to manage email offline, which would allow me to not have to connect to gmail unless I actually needed to send emails or synchronize local changes. To that end I decided to go with offlineimap to download a complete copy of my gmail imap folders and be able to deal with mail completely offline if need be.

As is the case with the rest of the tools in this guide, the install process is a snap, the configuration process is a little involved. I installed the current version of offlineimap. To install offline imap just type the following in a shell:

Install Homebrew - install.sh
1
2
3

brew install offlineimap
brew install sqlite

I installed sqlite to use it as the storage for offlineimap mail status.

Ok, that was painless now lets configure it:

Copy Initial Configuration File - copy_config.sh
1
2

cp $(brew --prefix)/Cellar/offlineimap/HEAD/offlineimap.conf ~/.offlineimaprc

This just copies a stock sample configuration file into one of the standard places that offlineimap looks for configuration. To read more about offlineimap configuration go here. I spent quite a bit of time reading through the docs and messing around with a bunch of the settings, along with messing up my install a couple of times!! That is how I learn the best though!! The documentation that comes with the sample configuration file is pretty informative, so you should be able to tweak a lot of things. As far as configuration goes I leveraged the sample configuration outlined by Steve Losh, who also went through the joys of configuring this setup a while ago. I wanted to understand how all these pieces worked, so I just leveraged his guide to rip off his configuration. His guide is also a great walkthrough of how to get this setup going. I am of the mindset that I like to struggle on my own for a bit as I try to figure things out. Instead of using his and other awesome guides, I set aside a half day and did this myself!! In the docs it mentions that the offlineimaprc file can also include a regular python file that can have supplementary code you want to leverage!! Pretty awesome concept. Steve uses this in a clever way to get around the need to have plain text passwords in the config file. Read his post for the details, but it basically makes use of the security utility (usr/bin/security on most unix based systems), to access the password in a keychain item.

Ok, so I configured things appropriately and was ready to kick off offlineimap to download the mail locally onto my computer (only about 1.5GB, but seeing as how we live out in the boonies, this would take a while). I fired up offlineimap and was immediately hit with an error about an incorrect fingerprint for the gmail stmp server. I figured this had something to do with a gmail cert change. This was rectified by following the instructions here which resulted in my adding the following config item in my JPBoodhoo-Remote Repository section:

Adding gmail cert fingerprint - cert_fingerprint.sh
1
2

cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042

With that change in place (keep in mind, the fingerprint can change at a later time, which would require this line to be updated), things looked good to go, I reran offlineimap and went about with other work while our blazingly fast connection proceeded to choke on the download for the next couple of hours.

Here is the majority of the pertinent stuff from my current configuration:

offlineimap configuration - configuration.ini
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

[general]
ui = TTYUI
accounts = jp_developwithpassion_gmail, jboodhoo------gmail
pythonfile = ~/repositories/developwithpassion/devtools/shared/dotfiles/offlineimap/offlineimap.py
fsync = False


[Account jp_developwithpassion_gmail]
localrepository = jp_developwithpassion_gmail_local
remoterepository = jp_developwithpassion_gmail_remote
status_backend = sqlite
postsynchook = notmuch new

[Account jboodhoo----gmail]
localrepository = jboodhoo----gmail_local
remoterepository = jboodhoo----gmail_remote
status_backend = sqlite
postsynchook = notmuch new


#developwithpassion
[Repository jp_developwithpassion_gmail_local]
type = Maildir
localfolders = ~/Dropbox/imap_mail/jp_developwithpassion
nametrans = get_remote_name


[Repository jp_developwithpassion_gmail_remote]
maxconnections = 1
type = Gmail
cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042 
remoteuser = jp@developwithpassion.com
remotepasseval = get_password("jp@developwithpassion.com")
realdelete = no
nametrans = get_local_name
folderfilter = is_included

#***
[Repository jboodhoo-----gmail_local]
type = Maildir
localfolders = ~/Dropbox/imap_mail/jboodhoo---
nametrans = get_remote_name


[Repository jboodhoo----gmail_remote]
maxconnections = 1
type = Gmail
cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042 
remoteuser = jboodhoo@----
remotepasseval = get_password("jboodhoo@----")
realdelete = no
nametrans = get_local_name
folderfilter = is_included

You can check out Steves post for a very thorough explanation of the configuration that matches. I took advantage of the python file to DRY up my configuration for the nametrans setting. When you are working with one of the remote repositories (jp_developwithpassion_gmail_remote) in one of the examples above; the nametrans setting determines how to map the remote imap folder names to names on your local machine. In the examples on the offlineimap site, they would set up reverse entries in the nametrans for the local and remote options. That would look something like this (lifted from some standard example online):

nametrans configuration - nametrans.ini
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
[Repository blah-Local]
type = Maildir
localfolders = ~/.mail/blah.com
nametrans = lambda folder: {'drafts':  '[Gmail]/Drafts',
                            'sent':    '[Gmail]/Sent Mail',
                            'flagged': '[Gmail]/Starred',
                            'trash':   '[Gmail]/Trash',
                            'archive': '[Gmail]/All Mail',
                            }.get(folder, folder)

[Repository blah-Remote]
maxconnections = 1
type = Gmail
remoteuser = blah@blah.com
remotepasseval = "password"
realdelete = no
nametrans = lambda folder: {'[Gmail]/Drafts':    'drafts',
                            '[Gmail]/Sent Mail': 'sent',
                            '[Gmail]/Starred':   'flagged',
                            '[Gmail]/Trash':     'trash',
                            '[Gmail]/All Mail':  'archive',
                            }.get(folder, folder)

Of course, as you can see above, it seems a little redundant. In my config I dry it up as follows:

Drying up the name translation - nametrans_dry.ini
1
2
3
4
5
6
7
8
9
10
11
12
13
14
[Repository jp_developwithpassion_gmail_local]
type = Maildir
localfolders = ~/Dropbox/imap_mail/jp_developwithpassion
nametrans = get_remote_name

[Repository jp_developwithpassion_gmail_remote]
maxconnections = 1
type = Gmail
cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042 
remoteuser = jp@developwithpassion.com
remotepasseval = get_password("jp@developwithpassion.com")
realdelete = no
nametrans = get_local_name
folderfilter = is_included

This is done by leveraging the following code that exists in the python file that I include at the top of my configuration:

OfflineImap Python Configuration - offline.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
#!/usr/bin/python
import subprocess
import re

class NameMapping:
  def __init__(self, local_name, remote_name):
    self.local_name = local_name
    self.remote_name = remote_name

class LocalName:
  def __init__(self, folder):
    self.folder = folder

  def matches(self, mapping):
    return mapping.remote_name == self.folder

  def mapped_folder_name(self, mapping):
    return mapping.local_name

class RemoteName:
  def __init__(self, folder):
    self.folder = folder

  def matches(self, mapping):
    return mapping.local_name == self.folder

  def mapped_folder_name(self, mapping):
    return mapping.remote_name



def get_password(account):
  program ='/Users/jp/repositories/developwithpassion/devtools/automation/keychain/get_keychain_password'

  command = "{0} --account:{1}".format(program, account)

  output = subprocess.check_output(command, shell=True)

  return output.rstrip()


def is_included(folder):
  result = True

  for pattern in exclusion_patterns:
    result = result and (re.search(pattern, folder) == None)

  return result

exclusion_patterns = [
  "efax",
  "earth_class_mail",
  "eventbrite",
  "gotomeeting",
  "moshi_monsters",
  "peepcode",
  "raini_fowl",
  "stuart_know",
  "training.*2008",
  "training.*2009",
  "training.*2010",
  "training.*2011",
  "training.*2012",
  "training.*nbdn",
  "training.*nothin_but_bdd",
  "unblock_us",
  "web_hosting",
  "webinars",
  "Gmail.*Important"
]

name_mappings = [
  NameMapping('sent', '[Gmail]/Sent Mail'),
  NameMapping('spam', '[Gmail]/Spam'),
  NameMapping('flagged', '[Gmail]/Starred'),
  NameMapping('trash',   '[Gmail]/Trash'),
  NameMapping('archive', '[Gmail]/All Mail'),
  NameMapping('drafts', '[Gmail]/Drafts')
]



def find_name_mapping(name):
  default_mapping = NameMapping(name.folder, name.folder)

  for mapping in name_mappings:
    if (name.matches(mapping)):
      return mapping

  return default_mapping

def get_name_mapping(name):
  mapping = find_name_mapping(name)
  return name.mapped_folder_name(mapping)

def get_remote_name(local_folder_name):
  name = RemoteName(local_folder_name)
  return get_name_mapping(name)

def get_local_name(remote_folder_name):
  name = LocalName(remote_folder_name)
  return get_name_mapping(name)

The name mappings array is the single place where the mappings are defined. That way I don’t have to worry about typos between sections. The config also shows how I am doing exclusions as well as shelling out to a custom ruby script to get passwords.

On my second imap sync I got an UID validity error on one of the folders I was trying to sync. This was resolved using the following doc.

As you can see from the above config, I have 2 different email accounts that are currently setup. The second account is starred out due to privacy reasons.

One of the cool things about the offlineimap configuration is that you can specify a python file that will get mixed into the current config file and you can leverage code in the python file to help with configuration. I am using this feature for both retrieving account passwords and for remote & local name translation.

Installing Mutt

Next up it was time to get going with mutt. Mutt at its core is a text-based mail client. For the purpose of my setup, I am going to be using it to primarily read and compose messages. It has out of the box support to be able to interface with gmail, and its own smtp utility, but I configured it to use msmtp for smpt. To that end, most of the configuration for mutt will be setting it up to basically be able to work with the local copy of messages that gets pulled down by offlineimap.

Let’s install it:

Install Mutt - install_mutt.sh
1
2

brew install mutt

Configuring gpg

As I am currently working with a client that requires encrypted messages for all internal communications, it was necessary for me to setup and configure mutt with encryption. I am using GPGTools for OSX. After installing and setting up my necessary keys, here is the config that I needed to place in the file ~/.mutt/gpg.rc (your file location may vary), this is only a slight modification of the stock gpg configuration.

GPG Configuration - gpg.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

# -*-muttrc-*-
#
# Command formats for gpg.
#
# This version uses gpg-2comp from
# http://70t.de/download/gpg-2comp.tar.gz
#
# $Id$
#
# %p The empty string when no passphrase is needed,
# the string "PGPPASSFD=0" if one is needed.
#
# This is mostly used in conditional % sequences.
#
# %f Most PGP commands operate on a single file or a file
# containing a message. %f expands to this file's name.
#
# %s When verifying signatures, there is another temporary file
# containing the detached signature. %s expands to this
# file's name.
#
# %a In "signing" contexts, this expands to the value of the
# configuration variable $pgp_sign_as. You probably need to
# use this within a conditional % sequence.
#
# %r In many contexts, mutt passes key IDs to pgp. %r expands to
# a list of key IDs.

set pgp_decode_command="gpg --status-fd=2 %?p?--passphrase-fd 0? --no-verbose --quiet --batch --output - %f"
set pgp_verify_command="gpg --status-fd=2 --no-verbose --quiet --batch --output - --verify %s %f"
set pgp_decrypt_command="gpg --status-fd=2 %?p?--passphrase-fd 0? --no-verbose --quiet --batch --output - %f"
set pgp_sign_command="gpg --no-verbose --batch --quiet --output - %?p?--passphrase-fd 0? --armor --detach-sign --textmode %?a?-u %a? %f"
set pgp_clearsign_command="gpg --no-verbose --batch --quiet --output - %?p?--passphrase-fd 0? --armor --textmode --clearsign %?a?-u %a? %f"
set pgp_encrypt_only_command="pgpewrap gpg --batch --quiet --no-verbose --output - --encrypt --textmode --armor --always-trust -- -r %r -- %f"
set pgp_encrypt_sign_command="pgpewrap gpg %?p?--passphrase-fd 0? --batch --quiet --no-verbose --textmode --output - --encrypt --sign %?a?-u %a? --armor --always-trust -- -r %r -- %f"

set pgp_import_command="gpg --no-verbose --import %f"
set pgp_export_command="gpg --no-verbose --export --armor %r"
set pgp_verify_key_command="gpg --verbose --batch --fingerprint --check-sigs %r"
set pgp_list_pubring_command="gpg --no-verbose --batch --quiet --with-colons --list-keys %r"
set pgp_list_secring_command="gpg --no-verbose --batch --quiet --with-colons --list-secret-keys %r"

# fetch keys
# set pgp_getkeys_command="pkspxycwrap %r"

# pattern for good signature - may need to be adapted to locale!

# set pgp_good_sign="^gpgv?: Good signature from "

# OK, here's a version which uses gnupg's message catalog:
# set pgp_good_sign="`gettext -d gnupg -s 'Good signature from "' | tr -d '"'`"

# This version uses --status-fd messages
set pgp_good_sign="^\\[GNUPG:\\] GOODSIG"

set pgp_timeout=9000

For more thorough explanations of the above settings, you can check out the guide that I followed here

As far as my mutt configuration goes, I added the following lines to my mutt configuration file:

Mutt GPG Config - gpg.sh
1
2
3
4
5

bind compose p pgp-menu

# Prepare GPG usage
source ~/.mutt/gpg.rc

The keybinding allows me to hit p after I have composed a message, and it will bring up all of the options I can apply to my mail (sign, encrypt, …)

Accessing GMail Contacts

Another piece of the puzzle was for me to be able to access my gmail contacts through mutt. To do that I leveraged a python utility called goobook.

Install python if you don’t already have it:

Install Python - install_python.sh
1
2

brew install python

Install goobook:

Install goobook - install_goobook.sh
1
2

pip install goobook

To leverage goobook as my contact lookup mechanism, I added the following lines to my mutt configuration file:

Goobook related settings - goobook.sh
1
2
3
set query_command="goobook query '%s'"
set editor = "vim --cmd 'let g:goobookrc=\"/Users/jp/.goobookrc\"'"
bind editor <Tab> complete-query

The first settings tells mutt which utility to use to query for contact completion. The second setting allows me to have vim be able to autocomplete google contacts when I am editing mail headers in Vim. The third setting is just a mutt keybinding that will trigger the goobook search when I hit tab in mail header fields.

Mail Searching With notmuch

One of the last pieces of the puzzle was being able to locally search email. I accomplished this by setting up notmuch. Getting going was as simple as running:

Installing notmuch- not_much_setup.sh
1
2
3
brew install notmuch
notmuch setup #setting up for first time use
notmuch new #run the first search

To hook into the search from inside of mutt, I added the following configuration line to my mutt configuration file:

Configuring mutt with notmuch - mutt_not_much.sh
1
macro index S "<enter-command>unset wait_key<enter><shell-escape>mutt_notmuch /tempfiles/notmuch_search_results<enter><change-folder-readonly>/tempfiles/notmuch_search_results<enter>" "search mail (using notmuch)"

Basically, when I am in mutt and want to search for mail, I can hit S and then enter in a search term. At that point it will shell out to a small ruby script that I wrote, that executes the notmutt search and builds up the appropriate directory structure for mutt to treat the search results as a mail folder that can be explored in mutt. Finally, I change the working folder to the folder containing the search results (for me /tempfiles/notmuch_search_results). Here is the script that I wrote to do the search and result dir creation:

Searching for mail - mutt_notmuch.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
#!/usr/bin/env ruby

require 'digest/sha1'
require 'optparse'
require 'fileutils'

class Hasher
  def initialize
    @hasher = Digest::SHA1.new
  end

  def get_email_hash(file_name)
    @hasher.hexdigest(IO.read(file_name))
  end
end

def build_results_folder(results_path)
  FileUtils.rm_rf(results_path)
  FileUtils.mkdir_p(results_path)

  %w/cur new/.each do |folder|
    combined_path = File.join(results_path, folder)
    FileUtils.mkdir_p(combined_path)
  end
end

def read_line(message)
  print "Query:"
  STDIN.gets.chomp
end

def get_unique_files(files)
  hasher = Hasher.new
  messages = {}
  files.each do |file|
    hash = hasher.get_email_hash(file)
    messages[hash] = file
  end
  messages.values
end

def get_matching_files(query)
  files = `notmuch search --output=files #{query}`
  files.split("\n")
end

def sym_link(files, target_path)
  files.each do |file|
    system("ln -s #{file} #{target_path}")
  end
end

def run(results_path)
  build_results_folder(results_path)
  query = read_line("Search Phrase?")
  files = get_matching_files(query)
  unique_files = get_unique_files(files)
  sym_link(unique_files, "#{results_path}/cur/")
end

def parse_arguments
  OptionParser.new do|options|
    options.banner = "Usage is: mutt_notmuch [RESULTS_PATH]"
  end.parse!

  path = ARGV.length == 0 ? "/tempfiles/notmuch_search_results" : ARGV[0]
end

run parse_arguments

Sending Mail

Up to this point I have only talked about a scenario optimized for reading mail. One of the main things you will also want to be able to do is send mail. Mutt has support for smtp, but there are more robust programs out there that only focus on smtp. One of those programs is msmtp . This is the program that I configured mutt to use to do mail sending.

Installing is a simple as:

Installing msmtp - msmtp_install.sh
1
brew install msmtp

Here is what my current msmtp configuration file looks like:

Configuring msmtp - msmtp_config.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
account jp
host smtp.gmail.com
port 587
protocol smtp
auth on
from jp@developwithpassion.com
user jp@developwithpassion.com
tls on
tls_trust_file ~/.mutt/equifax_gmail.cert

account jp_---
host smtp.gmail.com
port 587
protocol smtp
auth on
from jp@developwithpassion.com
user jboodhoo@------
tls on
tls_trust_file ~/.mutt/equifax_gmail.cert

account default : jp

I “dashed” out the information for my current client in the second account configuration. Notice that I am using the first account as the default. In my mutt configuration (for my local mail account), I have the following configuration line for using msmtp as the mail sender:

Mutt msmtp config - mutt_msmtp_config.sh
1
2
3
4
set from = "jp@developwithpassion.com"
set sendmail = "/usr/local/bin/msmtp -a jp" #using the account named jp from my msmtp configuration file
set sendmail_wait = 0
unset record # gmail records sent mail automatically, so we don't need to track

I have 2 aliases set up that allow me to start mutt with my local mail account, and the other for my client mail account. Each instance uses a client specific mutt configuration that is also configured with the correct msmtp account to use for sending. That is outside of the scope of this article, but it is just a matter of templates and aliases.

Handling mail attachments

One of the first snags I ran into was correctly handling mail that contained attachments (sending mail with attachments is a snap, so I won’t go into it in this article). By defaualt, mutt will use you mailcap file to try to determine how to handle email. The contents of my mailcap file look as follows:

Mailcap file - mailcap.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# MS Office files
application/msword; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-" --open_with:'/Applications/OpenOffice.org.app'
application/vnd.ms-excel; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-"
application/vnd.openxmlformats-officedocument.presentationml.presentation; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-" --open_with:'/Applications/OpenOffice.org.app'
application/vnd.oasis.opendocument.text; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-" --open_with:'/Applications/OpenOffice.org.app'

# Images
image/jpg; ~/.mutt/view_mail_attachment.rb --file:%s --type:jpg
image/jpeg; ~/.mutt/view_mail_attachment.rb --file:%s --type:jpg
image/pjpeg; ~/.mutt/view_mail_attachment.rb --file:%s --type:jpg
image/png; ~/.mutt/view_mail_attachment.rb --file:%s --type:png
image/gif; ~/.mutt/view_mail_attachment.rb --file:%s --type:gif

# PDFs
application/pdf; ~/.mutt/view_mail_attachment.rb --file:%s --type:pdf

# HTML
text/html; ~/.mutt/view_mail_attachment.rb --file:%s --type:html

# Unidentified files
application/octet-stream; ~/.mutt/view_mail_attachment.rb --file:%s --type:"-"

Notice that most of this script shells out to a custom ruby script that does the appropriate program launching. Here is that script:

View Mail Attachment - view_mail_attachment.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
#!/usr/bin/env ruby

require 'fileutils'

class Arguments
  attr_accessor :file
  attr_accessor :type
  attr_accessor :open_with
  attr_accessor :temp_dir

  def temp_dir
    @temp_dir ||= "/tempfiles/mutt_attachments"
  end

  def base_file_name
    @base_file_name ||= File.basename(file)
  end

  def base_file_name_without_extension
    File.basename(base_file_name, File.extname(base_file_name))
  end
end

def build_arguments(arguments)
  result = Arguments.new

  arguments.each do|argument|
    pair = argument.split(":")
    name = pair[0].gsub(/-/,"")
    value = pair[1]
    result.send "#{name}=", value
  end
  result
end

arguments = build_arguments(ARGV)

system("mkdir -p #{arguments.temp_dir}")
system("rm -rf #{arguments.temp_dir}/*")


new_file_name = ""
if (arguments.type == "-")
  new_file_name = File.basename(arguments.file)
else
  new_file_name = "#{arguments.base_file_name_without_extension}.#{arguments.type}"
end

new_file_name = File.join(arguments.temp_dir, new_file_name)
FileUtils.cp arguments.file, new_file_name

if (arguments.open_with)
  system("open -a #{arguments.open_with} #{new_file_name}")
else
  system("open #{new_file_name}")
end

As you can see, this solution makes the assumption that I write attachments to the following folder : /tempfiles/mutt_attachments

Conclusion

It took a couple of days for me to tweak my keybindings and customize my setup. After the initial rampup, I love that I can now compose, read, edit my email in Vim, here is a screenshot of what my gtd window in tmux looks like:

Needless to say, the screenshot says it all (from my perspective!!)

Develop With Passion®

Game Crash Bootcamp Coming to Calgary in May!!!

The Develop With Passion® – Game Crash Bootcamp is coming to Calgary, AB, Canada on the week of the 27th May 2013.

The course will be led by the extremely skilled Jon Beltran de Heredia, and it will be held at the Residence Inn Marriott Airport.

Attendees of the first Game Crash Bootcamp in Austin, back in February, were blown away by the course content. Jon has worked hard to develop a curriculum that can ensure that developers are able to build the skills to feel confident building games from scratch by the end of the week.

As with all Develop With Passion® courses, the focus is on fundamentals. Nothing shiny here. Nuts and bolts, math, and programming techniques that are not going to fade away with platform changes and language changes.

Develop With Passion® has always strived to go “deep” with students. I personally feel that if more developers allowed themselves to really understand concepts, instead of just “Google/Stack Overflow –> Copy –> Paste”, that they would utlimately feel much more confident about their abilities as a coder. Jon shares my approach to this, which is why he is the perfect trainer for this course.

If you have ever had a desire to want to learn game programming. This course will teach you the math, programming, and war stories from someone who has built very popular games for different hardware platforms.

Along with the onsite course which will be held at the Residence Inn Marriott Airport, in Calgary. The course will also be offered for people to attend remotely via a gotomeeting setup. This will give people who can’t fly to Calgary a change to take the course in their PJ’s if they so wish.

The price breakdown for this new offering (prices are only applicable for this year, next year the price will most likely rise to match the regular Develop With Passion® bootcamps):

  • Onsite US & Canada: $2500
  • Onsite International: $3500
  • Remote: $1500

  • Early bird discount (Registering within 2 weeks of event being published): $500

For more information on the course go here . To register you can use one of the following links:

Jon and I both feel confident that this is going to become a hugely popular course, once more people participate and realize how empowered they will feel with the results they will come up with in one week!!

Develop With Passion®

Pursuing Mastery

It occurred to me a couple of years ago, that there really is no such thing as attaining mastery of software development. We are all just at different stages of beginning.

When I was running a Software Craftmanship bootcamp in the UK a couple of years one of the attendees was kind enough to give me a copy of the book Bushido – The Way of the Samurai. He felt that a lot of the concepts that I was both demonstrating and talking about, were reflected in a lot of the ideas in the book. It really is a fantastic book on the subject of “pursuing mastery”. Software developers in general spend a lot of time trying to “attain mastery”. The only problem with this endeavour is that they can often trick themselves into falsely feeling like they have “attained it”. This can often result in the following set of events happening:

  • Establishing themselves as a subject matter expert in one or more areas
  • Camping on their ideas for a long period of time.
  • Failure to want to be “the beginner” again, as they have worked hard to be perceived as an expert
  • Boredom, as their need to hang-on prevents them from moving ahead
  • Transitioning to management as the next logical step, more because they come to realize that their skills have atrophied

Of course, that little blurb above is a very generalized path, but one that I have been able to observe enough for it to not just be an accident.

The sooner we can start realizing that we “pursue” mastery, then we will realize that the learning never stops. We don’t need to pat ourselves on the back too long when we have made an accomplishment, we can focus on incremental, iterative growth over the entirety of our career. This is the kind of movement that can prevent boredom, stagnation and the unecessary “promotion” to management. Now let me be clear, I have no issue with programmers who make the move to management because they choose and have a desire to. I am speaking to the devs who made the move because they felt like they had no other progression paths.

Here is an excerpt from the Bushido book that sums up what I am trying to say:

An old, retired swordsman once said, “There are levels in the course of mastery throughout your life. At the lowest level of skill and ability, one thinks of himself and others as poor. He thinks this because he has mastered only a little. Needless to say, a person at this level is not at all useful. ”
“At the middle level, one is still useless, but he can at least understand that he and others have mastered only a little.”
“At a high level, since a person has made something his own, he is proud of his accomplishment. And he is also glad of the praise of others. He grieves over the shortcomings of others. This kind of person is at least useful.”
At a higher level, one pretends to know nothing, yet others understand that he holds an upper hand. The majority of people cannot get beyond this level.
Beyond this higher level, there is one further step: the level of the trackless road. If you travel deeper into the trackless road, infinite secrets will finally appear. Then you can never see the end of your mastery. Then you truly realize how lacking you are. You have only to go ahead with your intention of mastery in mind. You go forward without pride and without humility.

High Upon High Bushido – the Way of the Samurai

Favourite part of the excerpt:

Then you can never see the end of your mastery. Then you truly realize how lacking you are. You have only to go ahead with your intention of mastery in mind. You go forward without pride and without humility.

I pray that you can continue to enjoy the pursuit!!

Develop With Passion®

Where Should We Host the Next Gamecrash Bootcamp?

It has been one week since the first Gamecrash Bootcamp completed. It was a great success and we are already in the midst of planning and refining based on feedback from this first iteration!

Currently, the next big question to ask and answer is where the next course should be hosted?

Instead of us just arbitrarily picking the next location, we thought we would solicit some suggestions on where the course should go next. To that end, if you feel compelled to help drive the literal direction of this course, please respond to the following survey, to help us make our decision!

We are looking forward to bringing this amazing bootcamp to developers all over the world who are wanting to stop just “thinking” about writing a game, and give them the fundamental skills and knowledge with which they can actually “write” one. We are also hoping to demystify for lots of developers, what it means to do things like:

  • Write your own physics engine
  • Leveraging OpenGL for graphics primitives
  • lots, lots more…!!!

Develop With Passion®

Be Willing to Become the Beginner Again

I have been a software developer for just over 12 years. Not a crazy long period of time, but long enough to have learned a couple of lessons!

In that time God has blessed me with the opportunity to work in a whole host of development environments, business domains, and countries. One of the things that I am constantly trying to do is push and stay outside of comfort zones. This is a life principal that is not just localized to the realm of my career. God does not want you to camp in a comfort zone, you can’t grow in a comfort zone. You can only maintain.

For the majority of my consulting gigs I have been able to experience a very high degree of success and leave with glowing reviews from both the managers and developers that I spend my time with. As a developer my primary focus when engaging clients, has been to focus on the code side of the fence. That is where my work passion lives. I am not interested in iteration management or project management. Having this knowledge has allowed me to maintain a very high degree of focus in honing the “craft”.

As a perpetual student, I am constantly welcoming of the opportunity to work with teams and individuals of all skill levels. In this capacity, I have historically been brought in as a senior resource to help cut some crazy code. It has been a while, but for the first time in a long time, the current contract that I am on has me feeling like a complete beginner again!!

And I will tell you the truth, it is times like this when I am reminded at an even higher level, of how exciting the realm of software development is. I love having my ideas turned upside down, and even in the span of only one month (the contract just started 1 month ago), I am actively and happily questioning, and shifting a lot of the approaches that I have taken to developing software, and more importantly design in general!!

Every morning for the last 5 years, I have woke up and thanked God for the fact that my career was starting again, that very day. When you wake up in the morning you get to choose how you are going to react to the events that come along in your day. You get to choose to be the beginner again. To realize that what you know now can become a small fragment of what you will know in a month, a year, a decade.

If you let go any assumptions you have about your own skill level, and go forward with the “pursuit” and not attainment of mastery; every day will be an amazing opportunity to tap into the creativity of the One who created us to be creative. You can choose joy, choose to be thankful, and choose to make a difference in not just your attitude and approach, but of those you surround yourself with.

Thank you Lord for allowing me to start my career again today!! I choose the path of the beginner.

Develop With Passion®

2013 Course Schedule

The Develop With Passion® 2013 course schedule is now online!

Currently the course load is small, but you can expect to see additions as Jon and I coordinate further iterations of the Gamecrash bootcamp over the course of the year.

Personally, I will be hosting courses in the following locations:

  • Calgary,AB
  • Denver, CO
  • San Diego, CA
  • Manchester, UK
  • Brussels, Belgium
  • Nashville, TN
  • Philadelphia, PA
  • Edmonton, AB

This year I will also be mixing in more occurences of the Ruby Primer bootcamp, as the first iteration proved to be very successful.

With the announcement that Jon made about the Gamecrash bootcamp, we are looking forward to a flood of new students excited to get into the world of game programming.

Develop With Passion® looks forward, and is honoured, to be a part in kickstarting/reginiting the passion in the 2013 class!!

Develop With Passion®

New Course Announcement - Gamecrash Bootcamp

I am happy to announce that Develop With Passion® is adding a new course to its roster of kick butt one of a kind developer training. Develop With Passion® – Gamecrash Bootcamp is a course focused on taking traditional developers with no prior experience in game programming and having them come out at the end of the week with the fundamental skills required to build games.

As all Develop With Passion® courses, the primary focus will be on the fundamentals. We want to give you the core tools that you can use to apply in any environment that you may consider building games.

As time permits I will make myself available to deliver this course also, but this brings me to the other part of this exciting announcement!

The primary instructor for this course is going to be Jon Beltran de Heredia, who many of you know as the creator of the fantastic ViEmu. Jon has a long history with game programming, being the lead programmer for Commandos and Commandos 2, and has deep experience with all aspects of game development.

We are very excited to have Jon on board with Develop With Passion®, and can’t wait to have a whole new set of students get “put through the wringer”!!

Please join me in welcoming Jon to the Develop With Passion® team and Gamecrash to the awesome lineup of courses that Develop With Passion® is continuing to grow.

Develop With Passion®