Being new to working in a software development environment, last week was a bit of a crash course in getting together all the pieces. Here are some notes, mostly as a future reference for me but hopefully useful to someone else out there getting started writing software. Everyone seems to use Mac OS X these days so this is a bit skewed towards Macs, but most things apply anywhere.

TextMate

Rails developers love them some TextMate. It's a clean text editor for Mac OS X that mostly stays out of the way and has helpful bundles that do things like auto-completion. If you want to use it for things like git commit messages, add these to your ~/.profile:

export EDITOR="mate -w"
export VISUAL="mate -w"

I'm a Vim person myself, but I've been using TextMate at work to get a feel for the workflow that most people seem to use.

Pig

I played with Pig a bit at SugarCRM, but never on a Mac. It's one of the best ways to ramp up number crunching and file processing. If you want to use it on a Mac, you'll need to set your JAVA_HOME environment variable. I did this by adding the following to my ~/.profile:

export JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Home"

Homebrew

It used to be Fink, then DarwinPorts, then MacPorts. Now it seems "Homebrew" is the popular way to get some things up and running that aren't that fun to get up and running on Macs. Example:

brew install mysql

There you go! MySQL is now installed and running. Watch the end of the output for instructions on starting it on boot using launchd, etc. Rails apps tend to use sqlite for development but sometimes you might want to use MySQL.

Ruby Gems

All the functionality you need has probably been written by someone else and stuck in a Ruby Gem. For example:

sudo gem install clarity

Will install a fancy web interface for looking at log files in real time. To prevent all the documentation from being installed on your machine, set up a ~/.gemrc file with the following contents:

gem: --no-ri --no-rdoc

Some people like putting this in /etc/gemrc, but it's always better to put user specific things in your home directory!

It is recommended that you always install gems, including rubygems itself, from source and not from your operating systems package manager. Additionally, once installed everything will probably need to be updated with something like "gem update --system".

RVM

RMV is the Ruby Version Manager. This lets you have multiple versions of Ruby installed side by side, as well as create sets of Gems that can be used in different applications. For example, once set up you can put a ".rvmrc" file anywhere in your directory hierarcy, with contents like "rvm 1.8.7@mygemset". Whenever you cd into this directory, rvm will set up your envrionment to use Ruby 1.8.7 and the "mygemset" gemset, and can update your prompt to remind you what is active. Getting this bit to work requires adding a few things to your ~/.profile:

[[ -s "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm"

export PS1="\$(~/.rvm/bin/rvm-prompt) $PS1"

GIT and Github

Git is the source code management tool of choice for a lot of people. It's fully distributed which enables some interesting workflows, and the Github website has done an awesome job of gluing together lots of pieces that enable effortless collaboration. The Git Reference covers (very well!) most of what you need to know about Git, but here are a few particularily useful things I've found.

Need to delete a remote branch? Delete the branch locally, then push 'nothing' to the remote branch by using the push command and leaving the front of the colon blank. Example:

git push origin :deletedbranch

Need to undo something? 'git revert' doesn't do what you think. Re-checkout the file(s) instead.

To get color in some of the command output, add the following to your ~/.gitconfig:

[color]
        diff = yes
        status = yes
        branch = yes

For auto completion and a more helpful prompt and tab completion of branch names and more, clone a copy of .git_completion.sh into your home folder, and put the following in your ~/.profile:

source ~/.git_completion.sh

function parse_git_dirty {
  [[ $(git status 2> /dev/null | tail -n1) != "nothing to commit (working directory clean)" ]] && echo "*"
}

function parse_git_branch {
  git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e "s/* \(.*\)/[\1$(parse_git_dirty)]/"
}

export PS1='\h:\W$(__git_ps1 "[\[\e[0;32m\]%s\[\e[0m\]\[\e[0;33m\]$(parse_git_dirty)\[\e[0m\]]")$ '

And read about using "git stash". It is your friend, especially when you forget to create branches to work on different features and ideas, which you _should_ be doing!

Rails

Ah Rails, I wouldn't be here without you. There is plenty of documentation out there on Rails, but a few points of interest:

• Models are named like "Thing" but in the database the table will be called "things". I'm used to naming my tables "thing" and both case and plurality matter here so thats taken some habit adjusting
• script/server starts up your rails app in webrick, while script/console sets up your app in an interactive ruby shell that you can use to poke around.
• If you need to debug something, add breakpoints to your code with "debugger" function calls, start script/server with "-u", and when your app hits the debugger call the script/server will drop you into a debugging session. It works a lot like GDB and is pretty handy for tracking things down.
• Making a new {anything}? There is probably a generator for that. Running "script/generate" will tell you about all your options
• "Rake" is "Make" for rails. "rake -T" will list all the available tasks. Looking at how requests are routed? Check out the output of "rake routes".

Rails does a lot of Magic and it can be a bit overwhelming, but overall it's been pretty friendly to me so far. Still need to know more? Watch all of the Railscasts and search the documentation at: http://api.rubyonrails.org/.

Capistrano

Capistrano is the most popular tool at the moment for deploying rails apps. A deploy to a remote server is as simple as "cap deoply" from your development machine, and if you're doing anything more involved than that for new software releases, you're doing it wrong. "cap -T" shows available capistrano tasks. That said, if you're doing more than just deploying a new version of an app...

Chef

Chef is the new Puppet. I used Puppet a lot at SugarCRM, and figured that I need to learn more about Chef so that I can pick the best tool to use, so I've spent some time with Chef over the last week. The "Opscode Platform" is a hosted Chef server that gives you a few things for free without having to set up the whole stack, and I recommend using that for at least your first round of testing. Also worth grabbing into your setup is a copy of Opscode's Chef Cookbooks that are very helpful for setting up pretty common things. Some pages worth reading:

• Chef Quick Start - start here
• Launch Cloud Instances with Knife - Use "Knife" which comes with Chef for firing up Amazon EC2 instances
• Deploy Resource - Capistrano, meet your New Fancyness

The end result of this tinkering, Add your Amazon EC2 ssh key to your ssh-agent by putting this in your ~/.profile:

ssh-add ~/.ssh/id_knife-test 2> /dev/null

Run a command like:

knife ec2 server create "role[${MYAPP}]" -i ${MYAMI} -G default -x ubuntu -S knife-test

And in a few seconds, MYAPP will be up and running on a new Amazon EC2 instance. Pretty sweet! Assuming of course that you got Chef working and set up a Recipe for your app similar to this one I wrote for deploying ithought.org, recoded as a Rails app, inside of Apache and mod_passenger: https://gist.github.com/aef17730ef8514799dc1.

Thats it for week one!

I've left out a lot, and I have a long way to go, but hopefully some of those snippets will be useful to someone. A few of them took me a little while to figure out and searching the internet for things about "knife" and "make those things stop doing that thing" can be tricky!

I carry a GPS with me on long bike rides and pull the resulting trace into Google Earth and Garmin's MapSource software. Google Earth is nice for looking at, but doesn't provide much useful information, and MapSource is pretty awful to look at (and will only run in Windows so I have to boot up VMware) but does provide elevation maps (as well as the ability to load maps). I recently started using a bike computer with cadence, and a heart rate monitor, and the last missing piece of information was total elevation gain over a ride. This information is nowhere in MapSource or Google Earth.

I can get GPX format (The standard interchangable format for GPS information) files out of MapSource and it's just XML, so after trying several tools online and several programs I downloaded that didn't work, I wrote a quick python script to get me the info I want. Hopefully this will help someone else:

from xml.dom import minidom

file = minidom.parse('./file.gpx')

min = 1000000
max = 0
gain = 0
loss = 0
last = 0

for node in file.getElementsByTagName("ele"):
        cur = float(node.childNodes[0].data)
        if (cur > max):
                max = cur
        if (cur < min):
                min = cur
        if (last != 0):
                if (cur > last):
                        gain = gain + (cur - last)
                elif (cur < last):
                        loss = loss + (last - cur)
        last = cur

print "max: %.2fft" % (float(max * 3.2808399))
print "min: %.2fft" %  (float(min * 3.2808399))
print "gain: %.2fft" % (float(gain * 3.2808399))
print "loss: %.2fft" % (float(loss * 3.2808399))

So for my 43 mile ride on sunday:
max: 1110.63ft
min: 773.16ft
gain: 3328.98ft
loss: 3232.78ft

Getting those numbers were a lot harder than it should have been! Good ride though..

I was going through some drawers and stumbled across my good old Griffin PowerMate that I got back before I started using Linux. It controlled iTunes in Mac OS 10.1 and was great because I could change volume and pause music without having to change programs or anything. These days I use Rhythmbox in Linux to listen to music and theres not a plugin for it. Yet!

Rhythmbox supports plugins written in python, a guy has some skeleton python code for talking to the powermate, and that means something could work out!

I got the powermate working by compiling and loading the powermate module for 2.6 linux kernels (In 2.6.23 it's in Device Drivers -> Input device support -> Miscellaneous devices -> Griffin PowerMate and Contour Jog support), adding a udev.d entry:

# /etc/udev/rules.d/45-powermate.rules
KERNEL=="event*", SYSFS{product}=="Griffin PowerMate", NAME="powermate", GROUP="users", MODE="0660"

I plugged it in, catted /dev/powermate, and with each twist or push it spit out garbage to the screen. Success!

A quick glance through everything shows that Rhythmbox doesn't support threads and the python code here uses polling so I'd need to delve into the Rhythmbox docs to figure out the best way to do that, but Rhythmbox also exposes itself through DBus and there are some examples of using this around the internet. In a few minutes, I hacked together something dirty to cover the basics and perhaps later on I'll make something that works as a Rhythmbox module. Right now pushing the button is play/pause, turning it adjusts the volume, and the LED shows volume when playing and pulses slowly when paused. Here ya go:

#!/usr/bin/python

import powermate
import dbus

EVENT_BUTTON_PRESS = 1
EVENT_RELATIVE_MOTION = 2

DBUS_START_REPLY_SUCCESS = 1
DBUS_START_REPLY_ALREADY_RUNNING = 2

bus = dbus.SessionBus()
(success, status) = bus.start_service_by_name('org.gnome.Rhythmbox')

proxy_obj = bus.get_object('org.gnome.Rhythmbox', '/org/gnome/Rhythmbox/Player')
         
player = dbus.Interface(proxy_obj, 'org.gnome.Rhythmbox.Player')

pm = powermate.PowerMate("/dev/powermate")
while 1:
	event = pm.WaitForEvent(-1)
	if (event[2] == EVENT_BUTTON_PRESS and event[4] == 0):
		player.playPause(1)
		if player.getPlaying():
			pm.SetLEDState((int)(player.getVolume() * 255), 0, 0, 0, 0)
		else:
			pm.SetLEDState(255, 252, 1, 1, 1);
	elif (event[2] == EVENT_RELATIVE_MOTION and player.getPlaying()):
		player.setVolumeRelative(event[4] * 0.02)
		pm.SetLEDState((int)(player.getVolume() * 255), 0, 0, 0, 0)

Download powermate.py and the code above, save the code above as whatever.py, run it, and you'll be able to control rhythmbox with your PowerMate in Linux!

I use F-Spot to manage my photographs. It's fast, clean, simple, and does everything in my current workflow which is JPG on camera -> YYYY/MM/DD folders -> Gallery on my website. Once I start shooting RAW it will get a little more complicated, but F-Spot keeps moving forward so hopefully they'll come up with a plan for that.

When uploading images to Gallery, I noticed that my photo timestamps were off. Conveniently, there was a discussion about this on the F-Spot mailing list at the same time and it turns out that every time you import an image in F-Spot, it adjusts the EXIF Timestamp information based on your timezone. Basically, if you're 5 hours away from GMT, on import F-Spot writes to the file that the image was taken 5 hours later than it actually was. Not only does it do this once, but if you re-import images into F-Spot for whatever reason it does this again, again, and again.

This was a bit of a surprise because EXIF information written by the camera shouldn't be changed by an import program! I thought I'd lost all the actual capture date/times of my ~30,000 photos, and was getting pretty upset that software would do this, but after digging through EXIF headers from all the cameras I've had, it turns out that the "DateTimeOriginal" was still good! I disabled F-Spots ability to write metadata to files (which means I'll have to stop tagging images until this is all resolved upstream) and wrote a little script to fix my files. If you've run into this and would like your original EXIF information back so that photos taken on New Years Eve as the year ticks over aren't at some hour after sunrise on Jan 1st, use this! Just replace $directory with the path to your photo library, store it to a file named "fixer.pl" and run "perl fixer.pl". Note that you'll need find and jhead installed.

EDIT: Note! I looked at this again with my 40D and new version of f-spot. It seems that now the correct EXIF header is "DateTimeDigitized" and _NOT_ "DateTimeOriginal". Please verify things on your setup before running this random script you found on the internet!

#!/usr/bin/perl -w 

use strict;

my $directory = "/media/photos/";

my %opts;
my @files;

@files = `find $directory -type f -iregex \'.*\\.\\(jpg\\|jpeg\\)\'`;

foreach my $file (@files) {
        chomp $file;
        my $dateline = `jhead -v "$file" | grep DateTimeOriginal`;
        if (defined($dateline)) {
                $dateline =~ /.*\"(.*)\".*/;
                my $date = $1;
                if (defined($date)) {
                        $date =~ s/ /-/g;
                        system("jhead -ts$date \"$file\"");
                        system("jhead -ft \"$file\"");
                }
        }
}

Last summer while in California I ordered a Dell Precision 390 workstation with a nVidia QuadroFX 3450because I wanted to be able to run 2 24" monitors on DVI (which is my current work setup at home). It's always been a little loud but I figured I would worry about it when getting back to Atlanta. School was busy and I never got around to it, but a few weeks ago when all was quiet in my new house, I could hear the frigging thing running. I tinkered with things and turns out the culprit was the fan on the video card. Lame!

I tried changing versions of the nvidia-drivers I'm using in Linux because there was some bug with a newer version that pegs the fanspeed on the card to 100%, but nothing helped. Then I stumbled across nvclock. Trying to set my fanspeed to auto resulted in "Error: This card doesn't support automatic fanspeed adjustments." Doh! It ran much quieter at 20% fan speed, but I didn't want to damage my video card if I was using it heavily for something. Enter a little bash scripting and problem is solved. Cron runs the following script every minute and adjusts the fan speed to keep the temperature below a certain threshold. It's not perfect, but I think it's good enough and it sure beats the fan running at 50% all the time!

With this, my fan speed bounces between 20 and 30 with a target temp of 57C which seems about right? I think I could run it a few degrees hotter with no problem but couldn't find any documentation on the card limits.

#!/bin/bash

target=57
minspeed=19

temp=`nvclock -i | grep "GPU temp" | cut -d':' -f2`
temp=${temp/C/ }

fanspeed=`nvclock -i | grep "Fanspeed" | cut -d':' -f2`
fanspeed=${fanspeed/\%/ }
fanspeed=`echo $fanspeed | sed -n 's/^\(..\).*/\1/p'`

if [ $temp -gt $target ]; then
        if [ $fanspeed -lt 91 ]; then
                nvclock -f -F +10 > /dev/null
        fi
elif [ $temp -lt $target ]; then
        if [ $fanspeed -gt $minspeed ]; then
                nvclock -f -F -10 > /dev/null
        fi
fi

Note: As of time of writing, you will need to get a snapshot of nvclock other than the latest release for it to work with these flags and this video card. If you're using Gentoo, this means using ~x86.

Apparently people are reading this because I got enough comments about me not having updated all week and me having "stopped blogging" that I'm here writing something about something.

So a while ago I wrote a little tool to parse my log files from Gaim and make a pretty graph: pretty talking graph. It shows how many lines I've talked to people on AIM in all of my log files, was a bit of a pain to write, and now that it's been around a few years, I look in the code and wonder what some parts are doing. Plus, I wanted newer, cooler, graphs and something that would play nicely with other programs (like Adium on the Mac), and I've been doing a lot of Python at work, so it was time for a rewrite.

So I wrote a Python tool that crawls all of my logs and puts them in a database, and a graphing tool that does things on the database and spits out html for all kinds of graphs. It runs automatically every night and makes a page with all the details: pretty talking graph page. Everything there is done with HTML/CSS (Thanks to the trends graphs on Google Reader for inspiration) and I think it's pretty cool. More graphs in the future (like rates of change, and predictions for when people will pass other people, etc)! I'd also like to figure out some way to graph some e-mail things... We'll see what I can come up with.