Not good by most measures but the real shock comes with the realisation that something as simple (cough) as data testing and development is a massive green no-no.

The standard practice of using copies of production data in testing and development can mean that many organisations are apparently making 24 copies of the primary databases annually with some even noting that demands have meant that up to 120 copies are made each year.

It can’t be a surprise then to learn that copies are a massive drain on power, cooling and storage.

If the size of the production database is 500 GB then ten non-production copies will weigh in at 5 TB. That’s big, scary and unwieldy, hey, just like the now-dropped-from-production Humvee…

I’m guessing that the demand for the ‘cooler hybrid options’ of the testing and development world will grow and that means more attention should be given to the brave new world of synthetic data generation.

The Welsh Assembly Government’s e-Crime Wales is a partnership of organisations, agencies and the police – that it has dedicated police business liaison officers is thought to be a world’s first.

Data loss and security breaches are a growing problem for businesses everywhere and Detective Constable John Cherry recently confirmed something most organisations are aware of when he said: “I have found that 70 per cent of threats come from within companies, either through malicious abuse of data or simple employee ignorance of existing threats.”

A Computer Security, Issues, & Trends report placed the risk of security breaches from employee and former employees even higher, at 81 per cent.

But when government gives with one hand it can certainly take with the other and data protection issues are ever pressing (no matter that government departments have breached their own DPA principles).

When the Nationwide Building Society was fined £980,000 by the FSA for failing to manage information security back in 2007, everyone sat up and took notice.

Now the UK Information Commissioner has made it clear that companies found wanting can be hit by ‘unlimited fines’ and that it is down to the ‘data controller to comply with the data protection principles’, day-to-day demands on data use are brought into sharp relief.

No matter the security polices that are drafted, unless operational integrity is in place, good intentions will come unstuck. And when it comes to testing and development, testing on copies of production data contains unavoidable risks.

Discussions have recently centered on using representative or “fake” data for testing and development. No, there is no other secure way – not even data obfuscation. Seeing as we’re now hearing that masking algorithms can easily be re-engineered (read: http://www.guardian.co.uk/technology/2010/jan/24/computer-security-crime-anonymous-datasets) the time is now to get on the train.

Even if you set aside the advantages of the time and space saved, synthetic data is the most secure and completely compliant – it seems the case for producing synthetic data is copper-bottomed.