I have a dream

My dream is to one day write my very own RTS-game Just kidding, only serious. Well, at least now I have completed one step in the process by writing an a-star path finding library

https://github.com/krumberg/astar

Podcastfs now works on OS X

I spent some hours this morning hunting down a few bugs that caused podcastfs not to work under OS X (MacPorts). I also verified that it still compiles and runs fine under Debian testing.

https://github.com/krumberg/podcastfs

RObject

RObject is a library I wrote a while ago that adds single inheritance object oriented programming support to ANSI C. It’s very small and can be customized and extended if so needed. Features that are not implemented but could easily be added include

• Reference counting
• “toString” Java-like method in base type RObject
• “hashCode” Java-like method in base type RObject
• Properties

https://github.com/krumberg/robject

Note that you need my Google Test-like unit testing library “rtest” to build the code.

git: the stupid NOSQL database

There are so many ways to use git. Manage your source code, handle large directories of files with git-annex, run a decentralized wiki or blog. View this talk and get inspired

The speaker is the author of Gollum, the wiki system used by Github. I tried it yesterday and it’s really easy to setup and seems to work very well.

To get gollum running, open a terminal and execute

gem install gollum
git init
gollum


This will install gollum, create a new git repository (preferably in a new and empty folder but I wanted to keep the example short) and starts a wiki webserver on port 4567 that is reading/writing to this repository.

Some git workflows

In this post I will describe some of the git workflows I use. Stay tuned for more of these. Please comment and tell me if I do something really stupid and also please add suggestions on how I can optimize these tasks.

Fork a branch, do some development and merge it back to master.

Assume you have written some code that should be developed further in another branch, or assume you are going are to begin writing such code.

Fork the current branch
git checkout -b mynewfeature

Do some hacking – commit, commit commit. You should know how this works.

Now you want to merge it back to master.
git checkout master
git pull --rebase . mynewfeature

‘git merge mynewfeature’ should be enough but you may have done some work in your local master and forgot to push it. In such case we get a merge which is a quite ugly compared to a rebase since we then get a nonlinear history.

Now your local master has your new code and it’s safe to delete mynewfeature
git branch -d mynewfeature

Rebase from the server and push the code to the server.
git pull --rebase # You may have to add ‘origin master’ here if your tracking config is missing
git push

Split a large commit into smaller commits before pushing

Assume that you have made a large commit that you
must split before you push it to the server.

Undo the commit
git reset --soft HEAD^1

Now all of your commit are in the index. Move them back to the unstaged area.
git reset

(1)

Now issue a git status and check what your commit contains
git status

Start adding parts using the -p flag. Answer ‘y’ or ‘n’ depending on whether the chunk should be added or not
git add -p # You may add a folder or file path here

Look at what’s in the index
git diff --cached # I have an alias 'git dc' for this

Once happy with your index, make a commit of it
git commit -m “A tiny feature”

Go back to (1) until there is nothing more to add. Then merge the branch back to master according to the previous workflow description.

On the importance of backups and distributed version control

I care a lot about my data (and so should you). Because of this I keep all of my code and all my documents under version control using git. Media files are not modified very often which makes version control less important.

Still losing all your music or your video collection is a horrible experience but here comes git-annex to the rescue. It allows you to manage your files from a single location, you can add new files, remove files, rename or move files and then propagate these changes to multiple disks or servers.

Many people don’t do backups at all. Some do but not regularly. Some do them regularly but they never try to restore their data. Just assuming that your backup system works in the reverse direction is naive and dangerous, but it’s all too common. If you use a distributed version control system the problem goes away. Every client then has a full backup of all your data. In the case of git-annex this is not necessarily the case but it allows you to configure a policy that forces you to store file X in at least Y different repositories.

Finally I use Time Machine too as it allows me to very quickly get back on track if I need to reinstall my complete system. For the moment Time Machine is also responsible for my photos and my music. If you don’t use a Mac then checkout rsnapshot or DeltaCopy. If you find these systems hard to use you can store your data in the cloud using Dropbox (2 GB free) or Box.net. Use the right tool for the right job!

If you lose all your family photos there is no one else who can help you get those memories back. Start doing backups!

Emulate Time Machine using rsync

Many people using a Mac love Time Machine. It’s a simple and (often enough) powerful tool for performing incremental backups with file history. It relies on hard links.

For people using GNU/Linux or for Mac users who desire a bit more control it’s possible to simulate almost all of Time Machines features (except for the nice GUI) using rsync. I have written such a script and I might just as well share it. Please give feedback if you have any suggestions for how to improve it. As usually you have to tweak it to match your folder structure. Please note that I do not take any legal responsibility regarding how it’s used or what it may do to your data. I just believe it works!

If you use Microsoft Windows then checkout DeltaCopy which is a popular front end to rsync.

Update:
The script below is a quick and simple hack but it seems to work just fine. A more complete and mature solution for *NIX that is also based on rsync and hard links is rsnapshot.

#!/bin/bash

dirs2back=(~/Music ~/Documents)
destdirs=(/Volumes/media)

function back {
BDIR=$1

if [ -d $BDIR ]
then
	foofile="$BDIR/foofile"
	if touch "$foofile"
	then
		echo $BDIR is writable

		if ln $foofile ${foofile}_lnk
		then
			rm ${foofile}_lnk
			rm "$foofile"
		else
			echo "Error: Sorry this filesystem does not support hard linking"
			rm "$foofile"
			exit
		fi

		RDIR=$BDIR/rsyncmachine
			
		TODAY=`date | awk '{print $4 "_" $3 "_" $5}' | sed s/://g` 
		TODAY=$RDIR/$TODAY
		echo $TODAY
	
		LASTTXT=$RDIR/last.txt

		if [ -d $RDIR ]; then
			echo "Running incremential backup"

			if [ -e $LASTTXT ]; then
				echo "Performing incremential backup"
				YESTERDAY=`cat $LASTTXT`
				echo $YESTERDAY

				if [ -d $YESTERDAY ]; then
					for item in ${dirs2back[*]}
					do
						rsync --progress -av --include=".*" --link-dest $YESTERDAY $item "$TODAY"
					done

					echo $TODAY > $LASTTXT
				else
					echo "Error: $YESTERDAY is not a directory"
					exit
				fi
			else
				echo "Error: No last backup, please update last.txt manually!"
			fi
		else
			echo "First time running rsyncmachine"
			echo $TODAY

			mkdir -p $TODAY

			for item in ${dirs2back[*]}
			do
				rsync --progress -av --include=".*" $item "$TODAY"
			done

			echo $TODAY > $LASTTXT
		fi

	else
		echo "$2 is not writable, not syncing"
	fi

else
	echo "Error: $BDIR is not connected"
fi
}

for item in ${destdirs[*]}
do
	back $item
done