After installing vim and upgrading coreutils, the final piece of the puzzle was getting an up to date version of zsh installed.

I gave the version in the repos a go first, and it came back with version 4.2.6.

$ sudo yum install zsh

$ zsh --version
zsh 4.2.6 (x86_64-redhat-linux-gnu)

Unfortunately, that's too old for me to use zsh-syntax-highlighting so it was time to compile from source again.

First, get and extract the files:

$ wget http://sourceforge.net/projects/zsh/files/zsh/5.0.2/zsh-5.0.2.tar.bz2/download && tar xvjf zsh-5.0.2.tar.bz2 && cd zsh-5.0.2

Configure, make and install:

$ ./configure && make && sudo make install

Check that it's installed fine. It installs to /usr/local/bin/zsh:

$ /usr/local/bin/zsh --version
zsh 5.0.2 (x86_64-unknown-linux-gnu)

If you want to use it as your login shell, you'll need to add it to /etc/shells:

$ echo "/usr/local/bin/zsh" | sudo tee -a /etc/shells
$ chsh -s /usr/local/bin/zsh

Log out and back in again, and you should be running zsh version 5.0.2.