http://thoughts.j-davis.com/2010/03/07/scalability-and-the-relational-model/

However, I wonder if it’s even reasonable to search for solutions to performance problems with the relational model; NoSQL, etc.

I think as computers get faster and faster (CPUs, SSDs, more memory, …) the set of problems which are “too slow” for a particular technology (or mindset, ..) get fewer and fewer.

I used to have the pleasure of having to optimize systems (based on SQL, some of our solutions involved leaving the relational model); e.g. community with 4M users (uboot), but nowadays I have no customer where even an open-source database installed on reasonably inexpensive hardware is insufficient.

Of course, that’s just my experience, and there certainly are many problems, companies, etc. today which require solving performance problems in the database, but I assert a lot of the people proposing NoSQL as a solution to performance problems with SQL databases don’t have the performance problems in the first place; I mean not everyone is implementing Facebook, Google, …

An error occurred during a connection to svn.example.com.
SSL received a record that exceeded the maximum permissible length.
(Error code: ssl_error_rx_record_too_long)

What does that mean!? There’s not a great deal of info on the web.

Fundamentally, in my case, the first thing to work out, is that that error message means (or meant, in my case at least) HTTP was being transmitted over the HTTPS port, i.e. it wasn’t valid HTTPS at all, thus the protocol error. This could be confirmed by surfing to http://…:443/ (i.e. not https://) and seeing that the content (the Subversion server in my case) was correct.

The question was why? I had a bunch of sites in the “sites-enabled” directory, and another one of them (not my Subversion site!) had a

<VirtualHost *>

whereas it should have been

<VirtualHost *:80>

i.e. the port was missing. I’m not quite sure why it had that effect, as the request to the Subversion HTTPS URL did deliver the Subversion content, just not over HTTPS any more. But perhaps without the :80, it decided all ports should be subject to NameVirtualHost, and as that’s not possible with HTTPS, switched HTTPS off for all ports and all sites?

Nightmare ….

See also: http://stackoverflow.com/questions/119336/ssl-error-rx-record-too-long-and-apache-ssl

It certainly didn’t have all the features it needed back then, it took us one extra week to add an address book to the messaging functionality, for example. And it didn’t have any photo sharing, video sharing, blogging, or the e-commerce functions that it has now.

It’s actually amazing how fast we did develop the first version: a team of three software developers, I started at the company on 3rd Jan 2000 and completed two other projects before starting Uboot; the first commit was on 19th Jan 2000.

With a few exceptions, I have been involved with Uboot software development, sometimes more, sometimes less, since then.

Computers need, from time to time, to address the user, for example “You have updated your setting successfully”.

Some programs use the word “you” to address the user, some use the word “me” on the grounds that the user is reading it, and to them, they are “me”.

However, using “me” to address someone is ridiculous! That’s as logical as a human using the word “I” or “me” to refer to the recipient of a piece of communication, “hey, do I fancy going to the pub?” on the grounds that, to the recipient, they are “I”.

“me” is the source of communication, “you” is the destination of communication; if a computer is communicating, the user is the recipient of the communication so should be called “you”.

Specifically Gmail is the main culprit for this in my life, it lists conversations between “me, Joe”; having “you, Joe” or “Adrian, Joe” would be much better! Gmail even, on the same screen that it lists conversations between “me” and other people, says “You are using 25% of your 7000MB”, so it’s not even consistent!

Be aware of this limit. I was unaware of it when I started this project. But it makes Java completely unsuitable for a whole class of problem.

My customer supplies me with a config file from time to time, this specifies a certain algorithm. When the user enters data, this algorithm must be applied. The algorithm is complex, so performance is an issue.

The solution I chose was to generate code to execute the algorithm, based on the information in the config file. This is a valid computer-science approach, and is used for similar problems. For example, language parsers are often expressed as a grammar, and code to parse documents in the grammar are generated. JSPs are turned into Java classes which are then compiled and executed. WebTek pre-compiles HTML templates containing macros into code which produces the resulting HTML when executed.

However, don’t try this in Java, unless you are only working with small problems. A single method in Java can only be 64KB in size, once compiled.
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4262078

This means, JSPs can only be of a certain length, parsers can only parse languages of a certain complexity, if WebTek were written in Java then templates could only be of a certain length and complexity, and so on. Do you want to place such restrictions on the software you produce?

My specific problem involves simulating one million variations to a particular solution. How can I fit that into 64K?

• That is 0.06 bytes per solution variation; yet the simulation of a single variation involves many lines of code (i.e. in total compiling to more than 0.06 bytes!).
• I could put each variation into its own method, and have a big method which calls them all—but a method call takes more than 0.06 bytes!
• I could have a hierarchy of methods: one main method which calls, say, 100 sub-methods, each of those call 100 sub-sub-methods, and finally those methods call the methods for the individual variations.

It’s not even possible to know how many bytes a method will generate to! So, as the complexity of the simulation of a variation is expressed in the config file, I would have to essentially have to do a “trial and error” approach: generate a method, compile it, if I get the error concerning the 64KB limit, split the problem up into slightly smaller methods, try the compilation again, repeat, etc. (And the Java compiler is not even very fast.)

This is all so wrong! This is complexity, which isn’t solving the customer’s problem. This complexity costs me time (and thus my customer money), complexity leads to bugs and difficulty of maintenance, etc.

So I have changed the language. Rather than generate Java, I generate C and compile it using the GNU gcc compiler. From the GNU coding standards:

Avoid arbitrary limits on the length or number of any data structure, including file names, lines, files, and symbols, by allocating all data structures dynamically.

This is a good standard! I like it. All programs should be written with this in mind. Your program may well be online in 10 or 20 years, and the hardware may well have changed: a 64KB limit may seem reasonable one year but is a real limitation 10 or 20 years later in software which would otherwise still be useful.

So, if you are solving this type of problem, don’t use Java.

P.S. On a separate project I used a similar approach using Perl, and that worked out fine too.

\o/ – arms raised in the air – success gesture
<3  – heart

$ php -r 'function foo() { foo(); } foo();'
Segmentation fault

I’m not saying that infinite recursion is a good idea, but during development it can happen by accident, and I don’t expect such a simple error to crash the PHP interpreter! (Also it took me about 20 minutes to debug this problem, as I had no idea where it happened, nor indeed what the problem was..)

PHP 5.2.6 on Linux 2.6.26 Debian

• Exceptions
• Assertions
• Inheritance
• Generic types

Generics are necessary to express concepts like List<List<MyType>>. If you are in to static typing, then you need to be able to express those concepts. (And if you are not, then a statically typed language like Go is not for you anyway.)

Obviously it has good features as well, such as its concurrency constructs and type inference, but without those features above, I can’t see how you can do any useful modeling in the language; or rather how can you can express a useful model in the language without losing a lot of information.

http://golang.org/doc/go_lang_faq.html

Perhaps those features will be added in future versions.

(I had a mobile internet subscription / device from them.)

Firstly – and this is not new – the program they supply with the computer to show you how many MB you’ve used (and thus how much you’ve got left before they charge you) is “inaccurate” (= underestimates the amount you’ve used), some support person from three told me that. But this program has a massive window that comes up, with the 3 logo plastered all over it – I mean it’s not exactly obvious that this program is inaccurate. Why don’t they just put a huge warning up saying “this data is inaccurate”?

The product is something like 1000 MB surfing for £10 a month. The main point, for me, being that you can use it in certain foreign countries, including Austria, and the price is the same as if you used it in the UK. This is why I bought it – so I could use it from both the UK and Austria. This was called “roam like home”.

Well – that’s no longer the case. I noticed a bill (thankfully only £15), called them, tried to cancel, they told me I’d have to pay a cancellation fee as I’m still in the contract period. I mean this is so totally outrageous.

So I spent 45 minutes on the phone, failed to get them to drop the early cancellation fee (which admittedly is only 20 pounds). They say then sent me an SMS informing me of the changes, giving me 30 days in which to cancel. As I hadn’t cancelled within that time, I had to pay the fee.

However a UMTS modem for a computer – which I only use very rarely, simply isn’t switched on when it’s not in use. Normal SMS expire if they aren’t delivered after a certain period of time (10 days?) but they assured me this was a “special” SMS which didn’t expire. And I have often had problems with SMS when roaming, but they assured me this couldn’t have been the case.) Even if that had been the case, if the device is off for those 30 days, I wouldn’t receive the SMS in time. And anyway, I didn’t receive the SMS at all (but they claim I “must have done”)

And they also said they posted it on their website – nice one, as if I surf to three.co.uk every day to hang out – it’s the new Facebook, so much fun to be had there….

They also say they are not obliged to provide this service, it’s not in the terms and conditions. I knew they’d say this. But there were massive advertisements for this feature when I went into the shop and bought it – I mean many huge person-size signs advertising the service. The guy in the shop told me this was the product, he didn’t say “this is the product until further notice”. I even have an email from 3 confirming this is the case, again with no mention of this being something which could expire or change:

Austria is a 3 like home network. If you use data in this country then it’ll be first deducted from your allowance provided you are latched unto 3 network.

Normally out of allowance charges in Austria is £3.00 per MB however, as Austria is a 3 like home country if you exceed your allowance and are roaming to 3 network, you’ll be charged 10 pence per MB.

I mean I understand services can change over their lifetime, and prices can change, etc. But I mean this is no minor service change:

• Before, when I bought it, I can use 1000 MB in Austria for £10
• Now, 1 MB in Austria costs £1.25, so 1000 MB would be £1,250

So I spoke to them, to all their “managers” etc, on the phone, and they told me there’s nothing they can do. I didn’t cancel it in time; they understand my frustration but their hands are tied.

They’ve fundamentally changed the service – making it useless to me – and now charge me for cancelling it

Thankfully it didn’t cost me much - £15 in extra MB used, and £20 cancellation fee. But that’s mainly due to the wisdom I acquired by being charged €2500 by another mobile operator. This guy got charged £500 due to this change by 3, for example.

I hate mobile operators. Three is now added to my list of companies I’ll buy anything from again, the other being T-Mobile.

I mean this is capitalism – the operators are happy when they can charge people a few thousand Euros because they didn’t understand they were roaming, or didn’t understand the T&C had changed. As a consumer one is powerless. Capitalism shouldn’t have come to this.

1. A server may serve multiple applications; each application may use different libraries or even different versions of the same library, “side by side”.
2. XML parsers, transformers (XSLT), etc., have a standard interface, and there may be different implementations of this interface from different vendors, open-source projects, etc.
3. Which XML parser, transformed etc. is actually used depends on a global system variable.

And it’s point 3 that’s the problem really. Points 1 and 2 are debatable: they certainly bring advantages, but they certainly bring complexity too.

I just had the problem that one of my web applications stopped working, but only intermittently. Restarting the server led to everything being OK, but later things would not be OK. I do hate environments where everything appears to work, yet in fact doesn’t. I mean how do you know when you’re “done” in such an environment? (Or how do you even know you are in such an environment?)

The bug was caused by:

1. Application one used the default XML parser, and didn’t have any extra JARs (libraries) for reading XML
2. Application two required a special XML parser, set the global variable so it would be used, and included the JARs necessary for the special XML parser

So when a request came to application 1, after a request had come to application 2, then the system would try to instantiate the special XML parser within application 1 (specified in the global variable set by application 2), but wouldn’t find it, as it wasn’t deployed in application 1 (and applications can’t use one another’s libraries, due to feature #1).

This seems obvious when one describes it, but looking at the logs, on a live server, with the system down and the clock ticking? – Far from obvious.

So now, I assert, every time you want to create an XML parser, do the following:

If you require a special XML library, use:

System.setProperty("javax.xml.parsers.DocumentBuilderFactory",
    "org.apache.xerces.jaxp.DocumentBuilderFactoryImpl");
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
...

If you require the standard XML library, use:

Properties systemProperties = System.getProperties();
systemProperties.remove("javax.xml.parsers.DocumentBuilderFactory");
System.setProperties(systemProperties);
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
...

There is also the possibility to pass a parameter to DocumentBuilderFactory to specify which XML parser technology to use. That’s a good option too, as it wouldn’t “corrupt” this global variable (“system property”). However I think one should be defensive, and always delete the global variable if one wishes to use the standard XML parser, and therefore it doesn’t matter if this global variable gets corrupted or not.

Never do the following:

DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();

This simply relies on whichever XML parser is currently set in the global variable. You have no way to guarantee that some other application running on the same server won’t set the global variable to use an XML parser you don’t have installed in your application. Even if you have control of the server and all applications, you don’t know what software you’ll be writing in the future. (In this case I installed a new application to a server which’d been running fine for 1 year, but due to setting the global variable, the old application broke..)

The same applies for all those other “factory” situations such as TransformerFactory.newInstance() etc.

This feels all quite inelegant to me, and has just cost me a lot of time, and it’s not as if I’m so new to programming Java. I am wondering if there is a better way to approach it? Or is Java just broken in this particular respect?

P.S. This is not the only thing that went wrong with the old application today. I upgraded from Java 5 to Java 6 and suddenly some XML was not compliant against a schema according to Java – I had hit this error.