The MAJESTIC SEO Blog Has Moved

September 22nd, 2010

to here
We have now archived this blog. In due course we will take it down or redirect the data. Please keep up to date with Majestic SEO on our newer blog at

Work finished for the day

January 28th, 2010

And it’s 23:29…

Visual Studio 2010 BETA 2 memory usage is rather high

October 31st, 2009

185 MB after startup on 32-bit Windows XP Pro, compared to 38 MB in Visual Studio 2008 – 15 minutes of simple usage (test code) and memory usage goes up to nearly 300 MB, that’s not a good sign…

5 years since start of the project

October 8th, 2009

5 years and 1 month ago I’ve received British permanent residency (officially called Indefinite Leave to Remain) – I resigned next day from my previous job at Phones 4U and one month later (had to work my notice period!) I started this project with first crawler code.

The road would have been a lot easier to walk if I’d known how far the project would go in just 5 years! We’ve got more tough years ahead of us, but this time I know how far we *can* go – somewhere past the horizon…

Microsoft Outlook 2003 “Compact” mail feature – perfect example of poor algorithm

September 19th, 2009

Presumably “compacting” is the way to reduce fragmentation of the database, drop unused blocks and thus avoid extra seeks, that’s a very good intention, but why such a poor implementation? It’s over 30 mins now and my Outlook is still “compacting” mailbox with just 1 GB size: low CPU usage but lots and lots of disk seeks, which is the reason for such appallingly slow performance. This means that the algorithm used for this feature is very poor and should have never been used on a poor random access IO subsystem such as magnetic hard disks.

Our index building process deals with tens of terabytes of compressed data so we designed it to be parallel, and also to avoid unnecessary disk seeks. It wasn’t easy, but defragmenting small mailbox is a trivial matter that does not need advanced algorithms, yet it appears that whoever implemented that function in Microsoft did not know what they were doing – this compacting should have run in under 1 minute max on my system, I can’t use email while it runs, so this function MUST be quick because otherwise it wastes valuable business time.

There is also indication of progress during this operation. It seems obvious that any non-instant processing job should indicate amount of work done and ETA, so that the user can decide if they want to cancel it or wait a bit longer.

By the time I finished this post it was still running, so I cancelled it and won’t use this feature again.

Adaptec 52445 (5 series) is officially S**T

August 18th, 2009

Installed Areca 1280ML with 24 ports into the same SuperMicro 4U chassis – there was 0.0001% chance that it was cables, PCI Express slot that were causing incompatibilities but now it is conclusively proven that it was the Adaptec card.

New card (Areca) perfectly with all the disks that were not recognised by Adaptec 52445 – this card is so s**t that my conscience prevents me from selling it to anyone (other than Adaptec), I just can’t bear a thought of people suffering it – so we’ve got something special in mind for this darned card: stay tuned – we will make an example of this card…

Areca ARC-1280 ML (24 ports) in JBOD mode bug

August 18th, 2009

There was a bug in ARC-1280 ML which prevented it from using 24 disks – it shows up as PCI Bus Error 0001 (or some other code) during booting ONLY if card was put into JBOD mode: even if firmware is said to be version 1.46 internally, it is dated mid August 2008, this is OLD firmware – get 1.46 from Areca website dated January 2009 and it fixes this bug.

Adaptec 52445 (5 series) does NOT work with Seagate 1.5 TB with SD15/SD17/SD18 firmware

August 17th, 2009

The disks are visible and some data can be read, but from time to time in many sectors IO error is shown despite disks being perfectly fine. Conclusion – STAY AWAY from Adaptec even if they give out those cards for free.

This card basically made the server nearly useless for us – new Areca 1280ML is on its way to replace pile of junk that is this Adaptec card.

ADSL sucks…

August 5th, 2009

If it works, then all is fine, however once it doesn’t and it is something on the line/exchange, then you are in trouble … 3rd day without my normal line, thank goodness for backup fiber optic cable line…

Adaptec 52445 (5 series) and Seagate 2Tb Barracuda LP does not work either

July 31st, 2009

I had high hopes that Adaptec 52445 will at least work with new Seagate 2 TB disk, but I should have known better – the controller (with the latest available firmware) does NOT see the disk at all. I’ve seen 10 dollar disk adapters have better compatibility than this overpriced crock of steaming (very hot processor used in it!) hardware.

New server uses Areca ARC-1231-ML and it works perfectly with all disks even Samsung F1 that do have known compatibility issues (they don’t work in Adaptec 52445 either, but that I expected).

Just how bad is Adaptec? Consider that back in April 2009 when I had 10 new Western Digital 2 TB disks on my hands, support claimed they only plan to test them in May because disks were not yet available! My suggestion to post them disk was not accepted (or even properly considered). Guess what Adaptec? You just lost on a fair few near future sales of rather expensive disk controllers.

And how about having manually going to turtle-slow BIOS options that make you “initialize”, which is a crazy term when it relates to new disk with data on it, so that it can be seen by Adapter even if it is a simple JBOD kind of disk that is seen instantly in Areca without any effort. Or any ten dollar controller that does not make you bend over backwards in order to see simple disk (without any RAID features) in OS.

Conclusion: NEVER buy Adaptec, go for Areca. I have not tried 3Ware, because better the devil you know (Areca is not perfect, that’s why we tried Adaptec adapter), but they do appear to have issue with JBOD mode, which made that controller essentially useless for us.

Any purchases of Adaptec hardware are now officially banned in our company.