Command Line Mystery

13 Nov 2016 in Unix Toolbox

A while back, someone published the command line mystery, a whodunit that you investigate on the command line. I had a few hours to kill on a flight so I decided to work through it.

We started by reading the instructions with less instructions.

There's been a murder in Terminal City, and TCPD needs your help.

To figure out whodunit, go to the 'mystery' subdirectory and start working from there.

Let’s get down to it!

$ cd mystery
$ ls
crimescene interviews memberships people streets vehicles

The instructions mentioned that the sergeant marked everything down with CLUE, so let’s search for that using grep. We use grep -r to make the search recursive as we don’t know which file to look in, then search for the word “CLUE”.

$ grep -r "CLUE" *
crimescene:CLUE: Footage from an ATM security camera is blurry but shows that the perpetrator is a tall male, at least 6'.
crimescene:CLUE: Found a wallet believed to belong to the killer: no ID, just loose change, and membership cards for AAA, Delta SkyMiles, the local library, and the Museum of Bash History. The cards are totally untraceable and have no name, for some reason.
crimescene:CLUE: Questioned the barista at the local coffee shop. He said a woman left right before they heard the shots. The name on her latte was Annabel, she had blond spiky hair and a New Zealand accent.

Interesting. The perpetrator is a tall male, there was a wallet left behind and there was a witness. Let’s start with the witness.

They said her name was Annabel, so let’s search our list of known people for that name.

$ grep Annabel people
Annabel Sun F 26 Hart Place, line 40
Oluwasegun Annabel M 37 Mattapan Street, line 173
Annabel Church F 38 Buckingham Place, line 179
Annabel Fuglsang M 40 Haley Street, line 176

We can discount Oluwasegun Annabel and Annabel Fuglesang as the clue told us that our witness was female. Let’s go and visit Annabel Sun first by reading line 40 of the Hart Place location.

There are a few ways that we can do this. The first is by using a combination of head and tail like so:

$ head -40 streets/Hart_Place | tail -1

Here, we pass in -40 which is an alias for -n 40, saying give me the first 40 lines of the file. Then we take the last line from that result and display it (again, -1 is an alias for -n 1).

We can compress this down to one command using sed. sed -n makes it so that sed doesn’t print lines by default. By passing 40p as our sed script, we’re telling sed to print the 40th line.

$ sed -n '40p' streets/Hart_Place

Now we’re getting somewhere! Let’s read interview #47246024.

$ less interviews/interview-47246024
Ms. Sun has brown hair and is not from New Zealand. Not the witness from the cafe.

Ok, so this isn’t our witness. Let’s try the other one

$ sed -n '179p' streets/Buckingham_Place

$ less interviews/interview-699607
Interviewed Ms. Church at 2:04 pm. Witness stated that she did not see anyone she could identify as the shooter, that she ran away as soon as the shots were fired.

However, she reports seeing the car that fled the scene. Describes it as a blue Honda, with a license plate that starts with "L337" and ends with "9"

Let’s see which vehicles we know about that match that pattern. First, we need to know what format the vehicles file is in. We can use head -15 to get a sample of the data.

$ head -15 vehicles
Vehicle and owner information from the Terminal City Department of Motor Vehicles

License Plate T3YUHF6
Make: Toyota
Color: Yellow
Owner: Jianbo Megannem
Height: 5'6"
Weight: 246 lbs

License Plate EZ21ECE
Make: BMW
Color: Gold
Owner: Norbert Feldwehr

So, it looks like there’s lots of information available about each car. Let’s search for L337 in the license plate field.

$ grep L337 vehicles
License Plate L337ZR9
License Plate L337P89
License Plate L337GX9
License Plate L337QE9
License Plate L337GB9
License Plate L337OI9
License Plate L337X19
License Plate L337539
License Plate L3373U9
License Plate L337369
License Plate L337DV9
License Plate L3375A9
License Plate L337WR9

There’s a few cars that start with “L337” and end with a 9! We can use fgrep L337 vehicles | wc -l to see that there are in fact 13 different cars. We also know that it was a blue Honda. Let’s try and narrow that down by searching for a blue car.

We can show additional car information by adding context to our grep command with -A3. This tells grep to show three lines after the match.

$ grep L337 vehicles -A3

License Plate L337ZR9
Make: Honda
Color: Red
Owner: Katie Park

The first match is a Honda, but it’s red! Let’s narrow it down to blue cars. We still want to see the make of the car and the owner, so this time we pass -B1 (as the make is one line before the colour) and -A1 (as the owner is one line after the colour`.

$ grep L337 vehicles -A3 | grep Blue -B1 -A1
Make: Honda
Color: Blue
Owner: Erika Owens

This can also be accomplished with the option -C1, which stands for Context. This implies -B1 and -A1.

$ grep L337 vehicles -A3 | grep Blue -C1
Make: Honda
Color: Blue
Owner: Erika Owens
Make: Toyota
Color: Blue
Owner: Matt Waite

We have a Toyota in the list. We know it was a Honda, so let’s strip that out too. We use -A2 as the owner is 2 lines after the make.

$ grep L337 vehicles -A3 | grep Blue -C1 | grep Honda -A2

Make: Honda
Color: Blue
Owner: Erika Owens
Make: Honda
Color: Blue
Owner: Aron Pilhofer
Make: Honda
Color: Blue
Owner: Heather Billings
Make: Honda
Color: Blue
Owner: Joe Germuska
Make: Honda
Color: Blue
Owner: Jeremy Bowers
Make: Honda
Color: Blue
Owner: Jacqui Maher

Now we have our suspect list. We need to check what the interview log says for each of the suspects. Let’s start with Erika Owens.

We can use our people list to work out where she lives.

$ grep "Erika Owens" people
Erika Owens F 56 Trapelo Street, line 98

Then we use sed to work out which interview number is hers

$ sed -n '98p' streets/Trapelo_Street

Then finally, check her interview. It turns out that she has an alibi.

$ less interviews/interview-5455315
Owens has an alibi for the morning in question, she was in Toronto for the Mozilla All Hands Meeting. Multiple sources, including a person in a fox costume, corroborate this. Not a suspect.

If Erika Owens isn’t our culprit, we need to move down the list. We need to run grep, sed and less for every suspect. To make this easier, let’s chain together some standard command line tools to do the job for us.

We need to grep the people file to work out where our suspect lives. Then, we need to extract just their address. To do this, we can use the cut command. This splits a line based on a delimiter (which is a tab by default) and allows you to extract a specific field (or fields). Fortunately, our file is already tab delimited so we can just call cut -f4 to extract the address.

$ grep "Aron Pilhofer" people | cut -f 4
Claybourne Street, line 145

Claybourne Street. That’ll be our file name, and we want to read line 145. We can use cut again to extract just the street name. This time we’ll change our delimiter to a comma.

$ grep "Aron Pilhofer" people | cut -f 4 | cut -d',' -f1
Claybourne Street

Our street filenames have underscores, not spaces. Let’s make that change now using tr.

$ grep "Aron Pilhofer" people | cut -f 4 | cut -d',' -f1 | tr ' ' '_'

Almost there! Now all we need to do is read line 145 of that file. Everything we’ve done this far has been piping the output of one command into another. Now though, we need to run two different commands after we run cut -f4. To do this, we need to restructure a little and use a variable.

I’ve extracted the name that we’re searching for into a variable, and save the location that they live at as another variable. Let’s replace what we had with our new version - as you can see the output is identical.

$ NAME="Aron Pilhofer"; LOC=`grep $NAME people | cut -f 4`; echo $LOC | cut -d',' -f1 | tr ' ' '_';

We use the backticks ``` around our grep statement as we need it to run and return the value in a sub-shell - each shell can only run one command at a time. Instead of usingecho` to show the street, let’s store that in a variable too.

$ NAME="Aron Pilhofer"; LOC=`grep $NAME people | cut -f 4`; STREET_NAME=`echo $LOC | cut -d',' -f1 | tr ' ' '_'`;

Finally, we need to extract the line number too. There are a few ways to do this - we’ll use awk which is the swiss army knife of text processing. awk has a special variable $NF which is the final entry in a row.

$ NAME="Aron Pilhofer"; LOC=`grep $NAME people | cut -f 4`; STREET_NAME=`echo $LOC | cut -d',' -f1 | tr ' ' '_'`; LINE_NUM=`echo $LOC | awk '{ print $NF }'`;

At this point, we have both our street name and our line number, so the final thing to do is make our sed call.

$ NAME="Aron Pilhofer"; LOC=`grep $NAME people | cut -f 4`; STREET_NAME=`echo $LOC | cut -d',' -f1 | tr ' ' '_'`; LINE_NUM=`echo $LOC | awk '{ print $NF }'`; sed -n "${LINE_NUM}p" streets/$STREET_NAME


Perfect! We now have an interview number. We can use the same techniques to extract this and feed it into less so that we can read the interview.

$ NAME="Aron Pilhofer"; LOC=`grep $NAME people | cut -f 4`; STREET_NAME=`echo $LOC | cut -d',' -f1 | tr ' ' '_'`; LINE_NUM=`echo $LOC | awk '{ print $NF }'`; INTERVIEW_NUM=`sed -n "${LINE_NUM}p" streets/$STREET_NAME | cut -d '#' -f2`; cat interviews/interview-$INTERVIEW_NUM

As we can see, Aron isn’t a suspect either. Now to check the next suspect all we need to do is change our $NAME variable. That line is getting pretty long though. Let’s wrap it up in a function.

To do this, we use the function keyword. You can run this in your terminal to make it available for the existing session, or add it to your .bashrc file (or similar) to make it available to all sessions.

Notice that we changed $NAME to point to $1. This means that it will receive the first parameter when we call our function.

function interview { NAME=$1; LOC=`grep $NAME people | cut -f 4`; STREET_NAME=`echo $LOC | cut -d',' -f1 | tr ' ' '_'`; LINE_NUM=`echo $LOC | awk '{ print $NF }'`; INTERVIEW_NUM=`sed -n "${LINE_NUM}p" streets/$STREET_NAME | cut -d '#' -f2`; cat interviews/interview-$INTERVIEW_NUM }

We can re-interview Aron to make sure that it works.

$ interview "Aron Pilhofer"
Too short to match the camera footage. Pilhofer is not considered a suspect.

Great! Now that we know that it works we can interview all of the other suspects. We could do it by hand, but we’re developers! We don’t need to do that.

Going back to our car owner list, we can extract just a list of car owners with a combination of grep and sed.

$ grep L337 vehicles -A3 | grep Blue -C1 | grep Honda -A2 | grep Owner | sed 's/Owner: //'

Erika Owens
Aron Pilhofer
Heather Billings
Joe Germuska
Jeremy Bowers
Jacqui Maher

sed is used here to replace the Owner: at the beginning of the string with an empty string, leaving us with just our owners.

Finally, we can loop over this list calling our interview function to see if they have an alibi.

IFS=$'\n'; for w in `grep L337 vehicles -A3 | grep Blue -C1 | grep Honda -A2 | grep Owner | sed 's/Owner: //'`; do echo "----------"; echo $w; interview "$w"; done

The interview with Jeremy Bowers looks particularly interesting:

Home appears to be empty, no answer at the door. After questioning neighbors, appears that the occupant may have left for a trip recently. Considered a suspect until proven otherwise, but would have to eliminate other suspects to confirm.

As everyone else has an alibi, let’s check if we’re right.

$ cat solution
Checking Your Answer on Mac/Linux:

Copy and paste the following command, replace John Doe with the suspect's name you want to check, and execute it from inside the main clmystery directory:

echo "John Doe" | $(command -v md5 || command -v md5sum) | grep -qif /dev/stdin encoded && echo CORRECT\! GREAT WORK, GUMSHOE. || echo SORRY, TRY AGAIN.

This command takes a name and runs it through a hashing algorithm named md5. The call to command here calls either md5 or md5sum depending on which one is available. It then uses grep and tells it to be quiet (-q), case insensitive (-i) and to read from a file (-f) - which in this case is stdin. If grep returns successfully (there was a match) it congratulates us. Otherwise it tells us to try again.

Let’s run the command!

$ echo "Jeremy Bowers" | $(command -v md5 || command -v md5sum) | grep -qif /dev/stdin encoded && echo CORRECT\! GREAT WORK, GUMSHOE. || echo SORRY, TRY AGAIN.


There we go! We got the murderer!

Hopefully this bit of fun has introduced you to a few commands or options that you’ve not seen before that you’ll be able to use in the future.