When you want to enforce uniqueness, for example across a whole table (for example a document name needs to be unique), or across a particular part of a table (for example each user must have documents with unique names), you need to do that in the database.

There is only one acceptable way to do this with a SQL database:

1. Insert a new row
2. If the insert succeeds, there wasn’t a row there before, and now there is
3. If the insert throws a unique constraint violation, the row is already there
4. If you want to update the row (i.e. an “insert or update” operation to maintain a “lazy singleton” in the database), you can update the row with safety after the unique constraint violation, as you can be certain the row is already there.

The following methods are all not acceptable:

• Do a “select” to find out how many rows there are. If there aren’t any, do an “insert”. However someone may have inserted a row between your “select” and your “insert”.
• Do an “update” and if the database says that 0 rows have been updated, do an “insert”. Again, someone may have inserted a row between your “update” and the “insert”.
• Do a “select for update” statement (Oracle, Postgres, InnoDB) to check that there aren’t any rows while creating a lock, and then do an insert. However that statement only locks the rows it returns, so if it doesn’t return any rows, it doesn’t create any locks, so you still can’t be certain that no one has inserted a row between the “select” and the “insert”.
• Lock the whole table and do one of the above. This works, but it means that all write access is “serialized” i.e. happens after one another. Any other operation, writing something completely irrelevant, will now also have to wait until the end of your transaction, whereas it shouldn’t have. This reduces concurrency.

The way I program this is the following. On the “insert” statement, I catch the database error, and see if it’s a “unique constraint violation”-type error. If it is, I throw an (unchecked) Exception. The calling code can catch that and do something with it (or not, if the statement should not generate such an error, in which case it will propogate to the main loop like any other database exception). I have had the pleasure of introducing this to easyname.eu and now also the pleasure of introducing this to the WebTek framework.

It is extremely frustrating working with big complex frameworks, whose usage is much more complex than just writing SQL manually, and not being able to do the above properly.

• Hibernate clearly states in its documentation that if any database error occurs, the Session (main object managing all persistence) must be destroyed as its state will be out-of-sync with the database. But there is no way other than the above to do this sort of check (as far as I know).
• OptimalJ generates code that, if a database error occurs, sets the transaction to rollback. Including any other work you may have done.

I mean checking unique constraints is something every database application needs, so the fact it’s not supported by major frameworks is just unbelievable.

This is different to programming languages (and indeed other databases, at least MySQL), which means one has to be careful not to make a mistake when using a programming language to talk to the database. That it’s different from other systems is the main reason for the criticism.

However, how many times have you wanted the following to be allowed in your data schema, for example on a “first name” column:

• Writing nulls is not allowed
• Writing the empty string is allowed

I just wrote a program and the front-end framework helpfully noticed that the field was “not null” in the database and gave the user an error in the front-end if the field was empty. However when I altered the code slightly, it no longer gave an error. Because the field in the program was the empty string, and not null.

However, I assert, when dealing with data, checking only for not null is not useful; you also want to check that the string contains some data in that case.

I have now updated the framework, so that if the field is marked “not null”, then the error is presented to the front-end not only if the variable in the program is “null”, but also if it is the empty string.

(Note: I am not advocating e.g. Java lose the distinction between ==null and .isEmpty(): for some reason this is a useful distinction when doing programming and data manipulation—such as null indicating that a variable isn’t initialized yet—I just don’t think it’s a useful distinction in a system solely designed to model persistent data.)

But there’s a point hidden down at the end of Chapter 7 which is well worth repeating. And that’s that using a wide fixed-width font for text entry is much better than a thin proportionally-spaced font.

Recently I noticed that the invitation panel on Google Spreadsheets, allowing you to type in email addresses of people to invite to collaboratively edit the document, used a fixed-width font. And conveniently they didn’t alter their word processing program’s similar facility, making it easy to compare the two.

It really does just feels so much nicer to enter text using a fixed width font. So all projects I’m doing now, I make sure I use a fixed-width font for data entry. And I don’t even think it has to look ugly!

Whenever a class in my model contains a collection which requires that particular care be taken with its items, there’s an internal debate regarding how to expose it to other classes. And with this, there are two major schools: one, the paranoia-based approach which doesn’t allow external code to touch the collection’s internal items and two, the trusting approach which just returns the collection for everyone to deal with.

What are your thoughts on the matter? What do you use, when and why?

Definitely paranoia.

It may make certain things more difficult or require more code, but one of the key cornerstones of object-oriented programming is encapsulation and not exposing your internal data in a way that means that others can break it.

If one wants to go for the trust approach - sure it’s easier - but if easiness is the objective one can write a C program and just declare a struct. Then anyone can access the data anyway they want and there’s absolutely no code to write (not even getters and setters). But the world moved away from that model towards encapsulation as with N lines of code (or Nk LOC or NM LOC), without encapsulation, any of those can be responsible for the creation of inconsistent data.

Plan for the next few months:

• 8th May - 12th May: break in Thailand
• 13th May - 20th May: back to Macau: Christina and I are packing her belongings to move to Europe
• 20th May - approx 1st June: in London with my parents. Various visa things have still to be organized.
• Whole of June: in Vienna
• Approx 1st July - 19th July: more preparations for the wedding
• 20th July - 28th July: honeymoon Maldives
• Start of August onwards: Vienna

But then I was sent a new version which was cleverer,

• The content was stored in an iframe
• There was Javascript which dynamically altered the size of the iframe to be the size of the monitor resolution (i.e. independent of current browser size)

The result of this complexity was that the iframe nearly—but not quite—fitted into the browser’s content area (assuming you had your browser maximised).

Because the iframe didn’t sit quite at the top of the window, this meant:

• The bottom “scroll down” arrow button of the inner iframe (the one you need to actually see more content) was, by default, off the bottom of the screen
• As there was content beyond the bottom of the screen (the “scroll down” button of the iframe) the main browser window also displayed a scroll bar
• To scroll to see more data, you need the inner iframe. But instinctively one reaches for the right-most scrollbar, as that’s normally what you need to see more data, e.g. in a browser, Word document, etc.
• It only got worse if you don’t use the browser maximised.

I mean I don’t know what aspect of “the browser will display a scroll bar if there’s more data than will fit in the window, without needing Javascript and iframes” they didn’t understand.

I suppose the navigation will always be shown, but I think people are used to navigation scrolling with the content these days. And if the objective was to keep the navigation on the screen, one could have used a normal frame, which would have required no Javascript, and still resulted in only one scrollbar, in the place where the user expects it.

Really the whole “scrollbar within a scrollbar” concept (which unfortunately is pretty much mandatory if you have a text area within a webpage) is really so nasty.

I think both problems are actually the same problem. It can be traced back to the necessity to make exactly one of the following two decisions when creating data-processing systems:

1. Either you try and work out what the datatype of a piece of data is by looking at e.g. the data’s string representation. E.g. this data is “abcd” so it’s a string, this data is “123″ so it’s a number.
2. Or you explicitly store and state, external to the data, its data type. E.g. store not only that the data is “123″, but that it’s a number.

Option 1 seems attractive as it’s simpler as you only need to store one piece of data. It also feels more normalized, as one piece of data is generally better than two (e.g. what if they are inconsistent, e.g. data is “abcjzh” and type is “number”?)

But the trouble is you option 1 doesn’t work (see above.)

But it gets worse. Option 1 seems to work, yet does not actually work in all case (and you want your software to work in all cases). That’s more dangerous that if it simply and clearly didn’t work.

The authors of the SOAP library in my example presumably believed their software worked. And I believed my software, built on top of the SOAP library, worked. It worked in my unit tests and when I tested it by clicking-through the front-end. Only 0.6% of users had a code with a hex string that looked like an exponent, so it’s understandable that I just didn’t hit it when testing. But with e.g. 2M users in the database, some of my users will hit it. And that means that the software I released didn’t work (working meaning working 100% for everyone.)

But I like my software to work. The way I achieve that is to avoid errors which are difficult to detect. Making errors is human; if they are easy to catch, one can spot them and then correct them.

I had prepared lots of documents for the application process; however by the time we went over there this set of documents had grown to at least twice or three-times this amount.

Ironically there was some repeating-DVD on the TV screens in the waiting room, where some applicant actress says “wow, I didn’t know it would be that fast and easy!”. I thought this was extremely ironic.

But in fact, contrary to our expectations, it was indeed both fast and easy (not including preparation of the documents). They called us 45 minutes after we left the building to tell us Christina’s passport containing the visa could be picked up.

We had prepared all the documents with the originals and copies collated (as they said they needed to take both away, and would give us the copies back). I had thus prepared a large spreadsheet listing all the documents that I wanted back (e.g. original bank statements). I thought merely sorting all this stuff out to give back to us would take half a day! But when we got the documents back they were all collated just as we’d given them to them, i.e. they hadn’t even taken the copies out. I suppose they didn’t look at the documents that much, or even at all?

The only things they took out were the copy of the passports (incl. stamps of our entry/exit to/from Macau/Europe) and my covering letter explaining my financial and employment situation, our plan to live with my parents initially when in the UK etc.

So maybe we could have spared most of the effort of the preparation of the documents? (Or maybe not, maybe a precondition of granting the visa was a certain mandatory documentary weight?)

Anyway, it’s good news. We have to make an application in the UK to “give notice”. But I’ve had chats with the office that do that, they imply it’s easy, and that the Fiancée Visa is the difficult one. So let’s hope that’s true.

First up, Knowing what’s going on:

This is a terrible example. You are really arguing that PHP programmers don’t know how their language works while C programmers do. This is a horribly wrong-headed assertion. How about I counter your straw man with one of my own. I know plenty of new (as of the last 5 years) C programmers who have no idea that 0 is equivalent to NULL.

Yeah you’re right, this point is probably untrue.

At the time I wrote it I was getting frustrated with PHP programmers who didn’t know the difference between == and ===. I still have the feeling that Java and C books tend to concentrate firstly on the fundamentals of the available data types and operations, whereas introductions to PHP tend to focus on just writing code that looks OK and seems to do the right thing (an attitude which leads one to write programs with subtle bugs).

But, having thought about that a bit more, that probably has more to do with my exposure to books written for people who can already program, vs articles about PHP on the web. And probably really does have nothing to do with the language whatsoever.

Strict typing

You want the compiler to check that a method can only receive an object of type SomeObject while I want any method to be able to receive any object as long as it responds to (or has the same interface) as SomeObject.

I used to think this way for quite a time, when I was programming Objective-C: that it was cool to write code which took any object as long as it responded to a certain set of methods. And that asserting an object must be of a particular class or respond to a particular interface made my code less flexible and reusable.

However after a time, looking at both my own Objective-C code and that written by colleagues, you would see methods like saveToDb:anObject. That method assumed that the parameter anObject responded to certain methods (by virtue of the method’s body calling those methods on its parameter), yet this was not documented in the method’s prototype (although it could have been placed in a comment had the programmer decided to), and could not be checked at compile-time. It gets worse when anObject is simply passed to some other function, so you have to open that in the editor to determine what type of object you can pass there. And you’re out of luck if you don’t have the source code. And even if you do document the type in a comment, you can’t build an IDE where you can just click on the type and it opens the definition, immediately listing its methods and documentation.

C, Fortran, C++, Java and Pascal require static definitions and suffer greatly for it. C++ (again) and Java (again) have templates/generics to fake this kind of feature and suffer horribly for it.

I have to agree that what really has improved in modern languages and runtimes (post concerning improvements in the future) is that the runtime knows what type of object a reference points to. Using void* in C is nasty.

No, Perl isn’t strictly typed and can’t do what you’re saying. But once again, you can check things. You can validate that an Object is a particular class or descendant of a particular class. As with the variable bounds, you can validate your data.

This is true, you can do that. But it doesn’t happen at compile-time (which means if you didn’t unit test or click-through that code path, you don’t see the error), and other programmers may choose not to even put the acceptable ranges or types in comments, and then you’ve got code which takes $x and then you’re really stuck. (Although I suppose if you work with programmers who don’t like to make readable code, you’re stuck no matter what language they’re programming in; I mean you can make unreadable code in any language.)

Enumerated types

This is a great feature modern day languages have though maybe it isn’t called “enumerated type.” Ruby has symbols so you can say your types are :hot, :warm, :lukewarm, :cold. These symbols mean the same thing everywhere. To use your PHP example in Ruby, how about error_log(”user not found”, :user_not_found). In this example, you don’t know the languages you are criticizing.

Well that’s great that Ruby has such a feature, but Perl and PHP still do not have such a feature. If they did, PHP wouldn’t have defined its error_log function that way. So when I’m programming those two languages (which I do a lot, alas) I am forced to write less readable code. (Even after defining constants, i can still pass gender_male with a value of 3 to a function expecting a state where 3 means the user has been deleted, and it won’t even exit with an error, let alone give me a compile error: it will simply do the wrong thing.)

No Compiler

Please point me to a modern language that is slower with longer variable and method names. Ruby, Perl, Python, OCaml and Erlang all “compile” the code to an intermediate form (bytecodes) and then execute those.

What? Are you suggesting that a comment in a procedure is parsed every time the procedure executes? I don’t know a single interpreted language implementation that would do that. The only exception are calls to “eval” or similar functions. 

As Perl, PHP etc all take plain-text files as their input, it follows that they have to process these files, byte per byte. Agreed, the better ones parse the source to an intermediate form where e.g. execution of loops will not be slower for longer variable names or a more complex programming style, but they still have to take the hit once, during the conversion from the text form to the intermediate form.

I have experienced this first hand. Uboot has about 350k lines of code (which is not unreasonable, the system provides mail, sms, photo galleries, blogs, subscriptions, and many more features, some of which are not active any more.) That takes about 4 CPU-seconds to convert to intermediate code (maybe faster these days, that was about 2 years ago). On each server we have about 30 instances of that code running. That means when we restart a webserver, it’s down for about 2 minutes. It does 2 minutes of useless work!

I have been told often enough, since working at Uboot, that I use the language wrong, that my programming is too “Java style”. The solution, I’m told by experienced Perl web developers, is simply not to write 350k lines of reusable library code, but instead write a simple large script with all the code rolled together. It starts faster, runs faster, and consumes less memory. And I’ve tried it: on some performance-critical sections I have indeed manually copy-pasted sections of code together to form one simple script, and it really does compile and run orders of magnitude faster.

I’ve essentially manually done what I would like a compiler to do. But that’s not the way I want to program. I do not want to be rewarded at runtime for bad programming practice!

*Every* language bears this cost because they *all* to have to parse the code at some point to either turn it into bytes or machine code.

That is very true, but some languages do this on your build machine, not on your production machines when you start the service.

Also, doing this on your build machine means you can perform more expensive optimizations, as you don’t have to worry about how long those optimizations take, which you do if the compiling means your service starts slower.

No linker

Your argument here is about memory footprint. This is a total non-starter on any modern operating system that does demand paging. If huge sections of your ruby/perl/python/whatever library are not used, the OS will never page them into RAM.

This depends where you wish to deploy to. For sure, on a web-server, this doesn’t matter.

On Uboot I wrote the “Uboot Joe” which is a program you can download to your Windows computer. I made the mistake of writing it in Java. To distribute it, I distributed the whole JVM (as most users won’t have one) which includes all sorts of things I never used, I included XML-RPC libraries (which no doubt include methods I never used), as well as my own code. The entire bundle came to 15MB. Our users had to download that just to get a program sitting on the tray, connecting to the Uboot servers, and popping up a few notifications. The size of this download file was attributed to one of the reasons why the program was not successful.

Yet cutting out unused functions via a linker is not rocket science. All C linkers do this (as far as I know).

I don’t think including the JVM was an incorrect decision; the file would not have been so excessively big if the download had included the Java runtime, but only those classes and methods of the JVM which I, or the libraries I had used, could actually possibly call at runtime.

I don’t write massive GUI apps in Perl.

Unfortunately I do write massive apps in Perl (albeit not GUI ones). And I did use Java to write a downloadable GUI app (albeit a simple one).

Multiple compile errors

I prefer to write a test, watch it fail, write the code to make it pass.

Right, but I’m tired of having to write test cases for trivial methods.

If I write a setter, I have to write a test case in Perl, otherwise it might fail because I made a spelling mistake. (I know from experience, writing test cases for even such trivial things really does actually help in Perl.)

In Java I don’t bother testing trivial methods; they just work.

Formatted Strings

I went through a long period of time wondering this myself. I thought sprintf was good enough all these years, why should I bother with iostreams. Well, I experienced one too many crashes from the simple error of mismatching the printf format specifier with argument type (%s -> int). These instances usually occur in logging statements that you don’t always encounter in normal code paths. This problem goes away completely with iostreams, as the most important benefit is type safety.

Ah that’s true. And one of the good things about modern systems (article forthcoming) is that they know what the types of things are at runtime. If they don’t (C++ by default), then I agree with you completely.

I suppose my point more related to the needless leaving out of good things which existed in the past. Java had to wait till 1.5 to get printf (and 1.4 for regular expressions). One should be more aware of the history of programming languages, and what things have already been thought of.

Auto-creation of variables

I agree with you on this one. It should be noted this is considered horribly bad practice in Perl now. Adding one line, “use strict;”, stops this from happening and every program I write begins with that. I think the PHP folk have long since started declaring and initializing variables for the most part. So it didn’t work.

That is true, that “use strict” helps.

Alas many languages such as PHP do not have such a “use strict”.

However, even in Perl with “use strict”, you can still misspell a function/method name and that will only get picked up at runtime (assuming you unit-test or click-through that path, otherwise it will go unnoticed), and if you misspell an attribute name in a $self hash, that only gets picked up at runtime.

I mean the flexibility that Perl offers (i.e. you can fill the $self hash with anything, and write an AUTOLOAD method which gets called when a method does not exist) would mean that it would not be possible to check those things at compile-time. However for me the benefit of catching errors at compile-time outweighs the benefits of the flexibility. But that is a matter of opinion, for sure.

Several features are dropped from new languages because the designers consider it “very dangerous, no _real_ programmer would ever use that”. As that’s a matter of opinion, we lose several powerful features just because they are… hmm… powerful. For example: GOTOs and Multiple Inheritance.

That’s for sure true. However I would use that argument to say that the power which one gains from the totally dynamic runtimes and languages (such as Perl $self hash and AUTOLOAD mentioned above) are too powerful (and means certain static checks cannot be done). But that’s a matter of opinion for sure.

If it’s Turing-complete, your language is ultimately fine.

I’m not sure about that. For me, a programming language is firstly a communication tool from one programmer to another programmer (or to the first programmer, but later). Secondly it is a way to express as many invariants as possible. Only thirdly is it a way to command the machine (which, as you say, all languages, including assembler, are capable of).

In that respect, one should choose a language firstly giving you maximum expressiveness (e.g. using an object-oriented language to program an object-oriented design, using a language which does not penalise you for creating libraries even if not all functions in the library are used in every program, etc.).

And secondly one should choose a language which enables you to express as many invariants as possible (e.g. the object being passed here should always be a User, this number should always be between 2 and 20, this reference should never be null), serving both as mandatory documentation and as a way for a computation process (e.g. compiler) to check as many of these invariants as possible.

Everyone seems to think that this is fact. But surprisingly it’s not. There are many features in older programming languages which are not present in today’s languages. I predict these features will be re-invented by the next generation of programming languages authors, and everyone will think they are geniuses for having come up with these ideas. But at the same time those new languages will omit most of the good points of today’s languages. This cycle can go on forever.

It’s like the cycle that tends to take place of “the network” vs “the standalone computer”.

• Central - IBM used to make mainframe computers, which one would access from terminals, i.e. central computing power, distributed usage.
• Local - But those computers were slow because they were remote. Then e.g. Sun invented the “workstation”. The PC then followed. Local power to everyone.
• Central - Then the web happened. Suddenly everything was remote again. “All you need is a browser!”. No local software installation nightmare. (Perhaps) independence from the single operating system vendor.
• Local - And now “using the web offline” is back in fashion. So that’ll be local computing again then.

A few facts, for those who think there was no programming before Javascript, the web:

• 1957 - Fortran released: expressions, variables, loops, subroutines
• 1959 - LISP released: treating functions as data, enabling higher-order programming
• 1967 - Simula 67 released: Object-oriented programming

Consider the following:

• Variable Bounds. Ada, developed for the American military, with high emphasis on program correctness, allows one to define bounds to variables. For example “array with index between 1 and 100″ or “0 and 10″ or number “not more than 5″. Most variables, in reality, have allowed ranges. Why not express it in the program, it’s more self-documenting and it allows the run-time, and to an extent the compiler. to check the constraints. Isn’t minimization of bugs something that affects not just the military?
• Strict typing. If you know an object being passed to a function is a “User”, it’s no good being passed an “Email Address”. The set of operations those objects can perform are completely different, so even if the programming language is “advanced” enough to be able to accept the parameter, the first method call to the object will fail. Why not express that and let the compiler check that. C++ can do it (since 1983) so let’s use that not Perl which can’t do it. Recently I read an article making a joke about casting everything to a string, but in reality that’s the default behaviour (in fact the only behaviour) of all scripting languages.
• Knowing what’s going on. In C, it’s well defined what “0″ means or what the string “abc” in a program means, and so. Ask a C programmer if 0==NULL and as a PHP programmer if 0==null and see a) their reaction times b) if they’re correct. The C programmer will know fast and be correct, the PHP programmer will not. Who do you think writes programs with fewer subtle bugs?
• Enumerated types. Is a user “active”, “disabled”, “inactive”? Having such options are common to all domains. C can define an enumerated type since ANSI C (1989) and Lisp since 1959. Why did Java have to wait until Java 5.0 (in 2004), and why do we have to create unreadable programs with languages like Ruby which can’t do them at all? For example what does the function error_log(”user not found”, 2) do in PHP, what does the 2 mean?
• No compiler. Every byte in an interpreted language costs time to interpret. So it makes sense to have short variable names, fewer comments, for run-time efficiency. Is this the sort of programming style one should be encouraging?
• No linker. You can build big libraries in a linked language, and only those functions used by the program (or used by the functions used by the program) will be included in the final executable. In Java, PHP etc, all the code you use is available all the time, taking up memory. I am often criticized for writing “too many libraries”, or code being “too object-oriented” in scripting languages, which is a fair criticism, as that code will run slower. However is it really an improvement to remove this function-pruning feature, which means bad programming practices will produce more efficient code?
• Multiple compile errors. Why do modern programming languages such as PHP only tell you the first error in your program, then abort? This is laziness on the part of the compiler writer. Old compilers tell you all the errors in your program, so you can correct them all, without having to correct one, retry, correct next one, retry, and so on.
• Formatted strings. There is nothing wrong with the format concept behind C’s “sprintf” command, originating from 1972. You can print numbers, strings, specify precision, field length and so on. (Apart from the inability to reorder parameters.) Why did C++ introduce the “<<" notation? (At least you can still use printf in C++). Why is this re-invented, worse, in .net? Why did Java have to wait until Java 5.0 to get this feature? Why do we have to reinvent the wheel (worse) all the time?
• Auto-creation of variables. When programming languages like C were created, the authors made the decision that it was an error to use a variable without declaring it. This caught all sorts of errors such as misspellings of variables. Why have these decisions been forgotten, and every scripting language allows you to just use variables without declaring them? This means hours of searching for bugs when you simply misspell a variable name, something that’s going to happen to everyone at some point. We’re only human and we have to take that into account.

The above is a list of things that have got worse over the last 2 decades, I.e. they haven’t just not got better by staying the same, but these things have actually got worse.