Atomic operations over filesystem and database

By Adrian Smith21 Nov 2010700 words3 mins to read

I had the situation recently where I needed a "button"; when the user clicks the button, then:

The config files are a bunch of XML files; each file has a complex structure, and they themselves are in a complex directory structure. One option would be to put them into a database as well but I felt that a simpler solution was just to leave them on the disk where they could be parsed. Directories, files and XML files more naturally fit the hierarchical nature of the configuration in question. I didn't want to have multiple representations of this configuration (one being directories/XML; the other being a database).

This is all OK, but:

How to make all of that—config files update & database work—atomic?

That means:

I came up with a nice solution:

This has the consequence that:

I'm pleased with this solution :-)

One should really also write a clean-up script, deleting directories which (a) aren't in the database and (b) were created a certain amount of time ago e.g. a few days. That way old directories, or directories checked out which had errors or failed, are removed, but directories which are in the process of being checked out are not removed.

P.S. Allowing the user to submit new versions of the data by checking it into Subversion is a nice idea:

Update: User-interface for this algorithm described here:
/offer-ready-publisher/

This article was written by Adrian Smith on 21 Nov 2010

Follow me: Facebook | Twitter | LinkedIn | Email

More on: Software Architecture | Operations & Servers | Things I've Released | Work & Customers