Tuesday, June 26, 2012

IT meltdown at UK bank

We're all used to the occasional story about a bank crediting the wrong amount into someone's account and of them making similar small mistakes that are caused by human error - a bank teller keying in the wrong account details or the incorrect amount. However, for the last week in the UK  the NatWest bank has been virtually crippled by a systemic software failure.
    There's a very good article explaining what went wrong in the Guardian. Basically problems began last Tuesday night when NatWest "updated a key piece of software – CA-7, which controls the batch processing systems that deal with retail banking transactions – ahead of the regular nightly run". Somebody managed to corrupt the schedule in which the batch jobs are run so that they were run in an incorrect order. This doesn't sound like a big problem, but imagine you are expecting $500 to arrive in your account, which you will then use to pay your rent. Then imagine your rent is paid before the money arrives in your account causing you to become overdrawn because the processing schedule was incorrect. Now multiply this problem by millions and millions of transactions. The NatWest is apparently still trying to unravel the mess and play catch-up, whilst assuring customers that their money is safe.
   One theory stated in the Guardian piece is that the outsourcing the Batch Processing IT jobs to India has contributed to the problem because "unless you keep an army of people who know exactly how the system works, there may be problems maintaining it".