Archive for April, 2007

Scripting languages are only for advanced programmers

Wednesday, April 25th, 2007

Why do people believe scripting languages are suitable for beginners to programming? This may be the way they were designed, but they end up having the opposite effect.

Essentially I think scripting languages are like languages for experts - like when you can play the piano and get a really hard piece to play, to show off how good you are. In programming, if you can already write bug free code using a normal language, but why not try the same thing using a scripting language? If you can do it, it shows your intellectual dexterity and cleverness (although the result would in that case be the same, i.e. a working program, and in the case you failed, it would be a less-working program).

Look at the following: http://at2.php.net/manual/en/function.stripos.php

I understand that “warning” completely. Essentially this function returns zero to indicate the string occurs at position zero and false to indicate the string does not occur in the text. But they are both the same if compared with the php == operator. And for that reason there is a === operator, called “exactly equals”, which also tests the type of the thing being compared. In the case of ===, not (0 === false).

I mean this is not simple stuff. Nor is it abstracting away from the details of programming.

How is this in any way better, or simpler, than:

  • C which returns -1 if the item is not found?
  • Java which
    • gives a compile error if you try and compare false to 0,
    • and also returns -1 if the item is not found
  • A hypothetical language which threw a SubstringNotFoundException, which gave a compile error if not checked or thrown?

Scripting languages should just be banned in my opinion.

Where are the dots?

Tuesday, April 24th, 2007

Where are the "…" on the "Advanced" button of "Display Properties"? This has annoyed me for ages. In fact I'm sure this was the case on Windows 2000, XP, and Windows 98 (but I might be wrong).

And of course on Windows you have to use this button each time you plug your laptop into a new monitor, as it will randomly (not even consistently) choose a resolution for you. And once you've changed that to a higher resolution, it chooses the lowest possible frequency for you, like 60Hz, even though it knows (demonstrable via the "hide modes that this monitor cannot display") that the monitor can support a higher frequency: so you have to change that via the "Advanced" button.

Ah and in researching this post, it seems to be that Windows Vista has got it right! Amazing. A good reason to upgrade!

In many ways, apart from the "…", that panel doesn't seem to have changed that much in the years since Windows XP (or Windows 2000) came out.

GUI Programming: Always perform network requests asynchronously

Monday, April 23rd, 2007

Why does one feel ones so much more in control, when using Firefox, than Internet Explorer?

When you select a slow link in Internet Explorer, the whole program hangs for about 1-2 seconds. Firefox doesn’t. Although 1-2 seconds is hardly a large % of ones life, it makes a big difference to the experience one has when using Firefox.

Recently I wrote something similar to an IM client (written in Java). It sits on the Windows tray. You can log in, and open a window where you can do various things. The data stored on a website (communication over XML-RPC).

MSN Messenger has one tray-icon for the user being logged out, a different one for the user being logged in, and amazingly (I thought), a third one for during the time the program spends communicating with the servers to log the user on.

In my system, log on is just one single XML-RPC call, with all the necessary data returned in the response. This was a design goal, to never have more than one client-server request to represent a particular user action.

The back-end to this XML-RPC call is a simple perl script which uses a few objects to represent things such as Users. These objects are simple enough, they just make a few SELECTs against our super-fast database. So I thought, as any request to our back-end takes say max 0.2 seconds, I needn’t make that asynchronous to the UI of the Java program. And I certainly don’t need a separate icon to display during that time!

If I’d ever stated that decision out loud, I would have heard myself saying it, and realized what a nonsense that is.

  • While it may only take 0.2 seconds on the server, there’s latency to consider, i.e. the time for the packets to flow from the client to the server and back again.
  • One can’t take into account how slow the user’s network connection might be.
  • There may well be more than one request from client to server, multiplying the latency. Just because there is one XML-RPC request doesn’t mean there are no other requests going on underneath, for example DNS lookup of the hostname to connect to.
  • If there is a queue of HTTP requests in Apache, waiting for the FCGI to answer the XML-RPC request, then the time the HTTP request to wait in the queue will also be added to the duration of the call perceived by the user.
  • What if there is a server-problem, and all requests take 2 seconds? A design not tolerant of things going wrong is a bad design.
  • Even 0.2 seconds is noticeable in a front-end.
  • Programming asynchronously in Java is not difficult. So it need not be avoided.

So now, every time I log on using that program, the dialog to log in opens, one clicks connect, and … wait … until the success or failure response is shown. And in that time, the program is just dead. It doesn’t even redraw its windows. It may only be 0.5 seconds, but you notice it.

Lesson - it may sound obvious - but it’s still worth stating: In a GUI Program (Windows, Mac OS X, etc.), any user interaction over a network, must be performed asynchronously (i.e. in a thread or in a separate process).

Exceptions: use them

Friday, April 20th, 2007

Exceptions have been around for a long time. There’s no reason not to use them.

I don’t want to ever see code such as this ever again.

if ( ! $user) return false;

We all know what happens with such code:

  • Nobody checks the return value
  • Especially if half the code is written using exceptions, and half using return values, then definitely nobody will check the return code
  • It breaks the linguistics of the language. A function called “getUser” should return a User, but a function called “deleteFile”: why does it return “true” or “false”?
  • Where are the log statements, stating why this function returned false? What if the function has multiple places to return false? Then (perhaps) the calling function can print the log “function deleteFile returned false” but that doesn’t tell you which of the multiple places failed.
  • What if you want to programmatically check the return result of the function? If all the errors return the same value, there’s no way you can programmatically respond to each error differently (as some might be permenant errors such as the file doesn’t exist, some might only be temporary errors such as network currently unavailable.)

Every language in use today, at least in the projects I work on, has ways of handling exceptions.

  • PHP5 has exceptions.
  • Java has exceptions.
  • C++ has exceptions.
  • Perl has “eval” and “die” which are like exceptions.
  • Javascript has exceptions.

One needs to simply be assertive, realize that exceptions are much better, and insist on using them.

Colour is good

Thursday, April 19th, 2007

Traditionally one does business documents, diagrams, and in fact all other things, in black and white. But colour allows one to express more things.

  • Headings stand out from the rest of the text much better when they're in colour.
  • You can communicate much more in diagrams if you use e.g. blue text for an explanation, red text for method names, and so on.
  • You can augment existing diagram notations, e.g. UML, with extra information using colour

There's no reason to only use black and white these days.

  • Colour printers are very affordable. Everyone and every office has one. I bought one for €50 recently.
  • Nobody uses black and white monitors or laptops any more.
  • All electronic formats support colour: bitmap formats like GIF, PDF, Word, emails.

Releasing working code

Tuesday, April 17th, 2007

I spend a lot of my time getting annoyed by errors in other people’s software (e.g. Windows). Errors which, when you see them, you wonder how on earth they could have been overlooked. But recently I released of a piece of software which contained a major bug (it was only a small mistake, but the consequences were big).

So I set about thinking, what sequences of actions lead, in my experience, to software which works? A lot of these are obvious, yet I’ve found myself often enough not following them, due to time or pressure reasons. And the result: is stuff which doesn’t work.

(1) Unit test scripts: Make them easy to run. For one product I work on a lot, there’s are a whole bunch of test scripts, testing all sorts of classes. In fact there are over 21k lines of unit tests! This is a good thing. But sometimes the person running them has to compare the value printed by the program with the expected value (i.e. has to know the expected value, not easy 2 years after the program was written). And not all classes are tested at all. But there are still a good few which do good tests and print “ok” if the result is correct. This is good, but it’s so much work to run them all. The solution is to chain them altogether, as is easy to do with JUnit, and create one simple command or click to test them all. If it’s simple and convenient and creates value, people will do it.

Also, having a framework into which to put tests - for example, having a convention that a class called “X” has a test class called “XTest”, and that methods like “operationY” on “X” have a method “testOperationY” in “XTest” - encourages people not to be scared to write tests. (But forcing people to write tests, e.g. one test per method, is a waste of time. Not every method needs a test.)

(2) Know what the important features are. Most websites really have many many features. It’s impossible to test them all, without restricting oneself to 6 month release cycles and 1 month test phases. But there are usually a bunch of features would would be show-stoppers if they didn’t work. Can a new user register? Can they upload a photo? (For a photo website). Can they send an SMS (For an SMS website). Write these show-stopping features down. Before the release, go through and test them on the pre-production server. After the release, test them again on the live system. Writing them down helps one not to forget the ones one can’t be bothered to test.

It doesn’t matter if this list is long. Maybe there really are a ton of features which simply cannot not work. Then you’d better have tested them all.

(3) For unimportant operations, ignore failure. Recently I wrote a program which writes a ZIP file. As a small extra feature, if a file in the ZIP hasn’t changed since the last time the program ran, the timestamp of the file in the new ZIP file should be the same as in the old one. This isn’t a very important feature, but it’s there. Once, when it ran, there was a file I/O problem reading the old file, and the program aborted. But this isn’t an important enough feature to abort execution: the program should have continued, and just given all files in the new ZIP a new timestamp.

Consider this when writing all code: if this goes wrong, does it matter? If not, when something goes wrong (any Throwable), log the exception and continue. You’ll kick yourself if failure of one part doesn’t matter, yet it brings down the whole program.

(4) Restart everything. You’ve only change one small piece of code, why incur the cost of restarting all Apaches and all robots? Well, software’s strange, and any change, however localized, can break any functionality. Any programmer knows this to be true. If you don’t restart everything, how will you everything still works? How will you test it?

(5) Look at the log files after releasing. Even if, after a restart of the live servers, everything seems fine, what are the users seeing? They’re testing different paths than you. If you log uncaught Exceptions, take a look at the log file before the release, and again after the restart after the release, and see if there are more errors. For example, SQL errors which weren’t there beforehand. This could alert you to a problem you’ve overlooked.

(6) Static checks are good. Programming languages such as Smalltalk and LISP popularized the notion that it’s cool to do everything, such as method lookup, at runtime. “It’s gives you more flexibility.” While this is certainly true, there are a lot of errors which you’ll then only find at runtime. (The same is true of SQL strings in program code: You will only know if you’ve misspelled a column name in the SQL when you run the particular piece of code.) This is not helpful to minimize your errors. I appreciate that taking code online which hasn’t even been run once is hardly a good idea, but I’ve seen it happen often enough.

Java and Hibernate are a good combination in this respect. If the Java program compiles then you know you’ve got all your variable names, function names, type-casts and Exception checking right. If the Hibernate program starts then you know the classes map to existing tables correctly. (But HQL, represented as strings within your program, are bad again, as you could make a spelling mistake, and it will only cause an exception when the particular code is executed.)

If one has to have SQL strings in the program, and thus an error in it will only be detected once the code path is executed, maybe a prepare of the statement can be placed in a static constructor of the class? That way at least when the class is loaded (at the start of the program’s execution, most likely) one will find out about the problem.

(7) Be aggressive about cleaning old code. The more code there is, the more complex a system is to understand. If one has a new chat system, why is code which communicates with the old chat system still there? What if that code relies on classes which you’re about to change? What if it communicates with an old chat server and the results aren’t displayed anywhere any more, and then that old chat server goes away? The motto “clean code that works” does not involve having 100k lines of old junk around, which no one understands, no one wants to take the time to learn (as it’s no longer relevant), and will break randomly.

(8) Compile all of the program. It’s obvious, before one releases a Java program, one does a “clean all” then a compile, just to check that one hasn’t changed a class and forgotten to recompile a client of it, which will result in a runtime error, e.g. a MethodNotFoundException. Why doesn’t one do the same in scripting languages? Admittedly scripting language compilers don’t check as much, but they still check some things (e.g. syntactic correctness). One “unit test” should be to go through every program file - every library, every CGI, every PHP page, and do a compile check on it.

(9) Release emails. It’s easier to delete an email which one’s not interested in, than to find out information from an email one didn’t receive and doesn’t know was ever sent. If a service like a website suddenly breaks, it’s important to fix it as soon as possible, and that probably means contacting the person who caused it to break. Before (not after) a release, write an email to all concerned - operations engineers, software developers, support agents, managers - and let them know that a change is going live.

(10) Be contactable. There’s nothing worse, for creating a perception of negativity, than when someone’s made an error, and you can’t contact them. Make sure mobiles are on loud. If you’re not reading email for some reason, make sure you’ve told everyone in advance, and set up an auto-responder. Want to be contacted less? Make fewer errors.

(11) Monitoring. For each robot and front-end program, one needs to define what acceptable conditions are and what not. E.g. what logs must be written by the correctly-running program, and which logs must not be written. Monitor them. This takes quite a lot of effort, a) the monitoring software b) defining what are acceptable and unacceptable conditions c) tuning the software to actually produce logs which are usefully monitorable. But it’s necessary. If it’s not done, there will be errors written to the logs and nobody will see them.

Mozilla Thunderbird sucks

Thursday, April 12th, 2007

Really, Thunderbird is a terrible mail client. I'd been using Outlook for about 5 years when I first tried it, so I thought maybe the reason I didn't like it was simply because it was different, in which case I should continue to use it to get used to it. One year on I still hate it and recently it just ate half my mail. So I'm going back to Outlook.

While downloading a large message using POP over a slow connection recently, the download bar (slowly progressing from 0% to about 50% at the time of the crash) simply went away (without error). Clicking "Get mail" button again did nothing (without error). Restarting the program showed the "Inbox" to be blank for a very long time, but it seemed to be doing something, and after about 1-2 minutes the list of messages appeared. But only the mails received between the time I started using Thunderbird and about mid 2006-10 were there. Mails from mid 2006-10 to now (mid 2007-04) are just gone. So that'll be the mailbox corrupted then. Imagine you relied on Thunderbird as the only storage place for all your mail. Well, thankfully I don't. And thankfully I won't even be using Thunderbird for one of the storage places for my mail in the future.

Here are the reasons I didn't like Thunderbird from the beginning.

  • When you click "reply", the cursor inviting you to type a response to the quoted mail is at the bottom of the mail, not the top. It turns out there is a preferences option where you can change that, but it took me about 6 months to find it.
  • The HTML mail composer sucks. You have the cursor blinking away somewhere, press a key expecting the character to be inserted where the cursor is, but no. The cursor suddenly moves somewhere different (e.g. a line down) and inserts the character there.
  • If you send a rich text message, it asks you "do you want to send this mail as plain text (recommended), html, or both?". Text is rarely so long that the bandwidth required for a multipart/alternative would be a problem. And multipart/alternative is there so you, as the sender, don't have to know what formats the recipient can read. So this dialog box is just broken. Also: why is plain text recommended, do we want to be stuck in the 70s forever? Let's all go to the disco and send (recommended) plain text emails using Firefox.
  • In Outlook, if you click "send" and you are offline, the message is stored locally temporarily. As soon as a connection is available, it is sent. With Thunderbird, however, the situation is more complex. At the time of sending, you have to select "send" (which yields an error if you are offline), or "send later" (which is available when you are online, even though you'd never want it). When you go online you have to select "send emails now", as opposed to that happening automatically. However, I thought I could make this all go away when I found the option "if you go online, Thunderbird can send offline emails immediately". I clicked that but it didn't work. It turned out "go online" referred to the Thunderbird menu options "go online". If, every time I connected to the internet, I had to go through each application and use its menu option "go online", well, that would be a bad situation. Probably why other applications don't work like that.
  • Search results are unsorted. Search happens in the background (good) and adds mails to the search results window as it continues and finds them. If you click on a column heading in the results, e.g. "date", to sort the (initially unsorted) search results, then during search (as more emails are found) they are simply added to the bottom of the search results. So you have to click the column heading again, to do a sort including the newly found emails.
  • The UI to do search is terrible. If you open the drop-down with the keyboard, allowing you to select "sender", "recipient" (i.e. which field must match in the search), use the cursor keys to select the field you want, then press tab to move to the text field (to type the value of the field which much match, which works in other applications), the drop-down list of fields closes, but the field you had selected is forgotten.
  • Full-text search takes ages. No indexing. Why?
  • If you are composing an email, and want to send it to someone whose address you've forgotten, you can go to another window, find a mail from them, right click their address and say "add to address book". Go back to your compose window and try and use the address book: it doesn't contain the new entry. You have to close the compose window, open a new open, copy/paste the entire body and all other recipients over, then the new window knows about the current address book.
  • Emails you send using the HTML editor are in Times (not Helvetica/Arial as in Outlook), which makes ones emails look terrible, and also marks one out as a person using "strange" non-Outlook technology, to all ones recipients.

SMTP is not new. POP is not new.Win32 is not new. Surely in the time between the creation of those technologies and now, one must have been able to do better than this.

Don't think that task tracking numbers are a replacement for documentation

Wednesday, April 11th, 2007

Do you think this is an appropriate and sufficient documentation for this function?

# 3978
#
sub is_contact_in_abook_for_user {

This number refers to the task number in a task/bug tracking system. The idea being, why write documentation, when that would simply duplicate what is already available.

There are a number of reasons why this sort of documentation is bad, but the main one is that a feature lives on a for a long time, as does reusable code which one creates in order to implement the particular feature. But a task is just a task, once it's done, no one cares about it any more. So if one sees e.g. a class modelling a user exposing methods such "fetch by user name", "fetch by id", "fetch by telephone number", it isn't really helpful to know that the first two were implemented as part of a "implement payment" feature, and the last for a "mobile phone shop" feature. There are so many aspects of those features which have nothing to do with a function which can be reused time and time again.

The other reasons are:

  • You cannot click "3978" and see the documentation. You have to open a browser window, search for the task, and so on. This is a lot of work, so many people will never do it, meaning they'll alter the code without understanding what it's doing. This may be their fault, as opposed to the fault of the process, but it's a reality nevertheless.
  • What about the other documentation for the function? What are the return types? What are the parameter types? It's not documented here and I bet it's not documented as a "comment" to task #3978.
  • Task trackers get changed over the life of the program. In this case, the original task tracker was maintained by another company with whom my customer no longer has a relationship. So I have no way whatsoever to find anything about task #3978 or any other. So all those comments littered throughout the program are literally useless.

I think the solution is the task tracker should reference the code, not the other way around. E.g. a comment by a programmer "adrian 20th dec 2006: committed file MyClass.java in svn revision 2938".

Heathrow

Tuesday, April 10th, 2007

I've just arrived at Heathrow airport, Terminal 1, from Hong Kong.

  • The way to the terminal from the aircraft was not clear: then I saw some temporary photocopied sign stuck up with tape that showed the way.
  • There is no wireless internet here, in contrast to all the other airports I use: Hong Kong, Dubai and Vienna.
  • As I am sitting here, inside in a terminal building, there is a pigeon walking around on the carpet.
  • One guy already complained to me surprised that "there is no complimentary telephone service here, in contrast to Hong Kong" (although I've never heard of that before!)

When I first went to Egypt I was amazed at the chaos in the airport. But we really shouldn't think that Heathrow is much different.

Holiday in China (Yangshuo)

Monday, April 9th, 2007

Christina and I went to China recently for a few days holiday. We had a great hotel room with a huge balcony. Here is a picture of us on some boating expedition!