The Welsh Assembly Government’s e-Crime Wales is a partnership of organisations, agencies and the police – that it has dedicated police business liaison officers is thought to be a world’s first.

Data loss and security breaches are a growing problem for businesses everywhere and Detective Constable John Cherry recently confirmed something most organisations are aware of when he said: “I have found that 70 per cent of threats come from within companies, either through malicious abuse of data or simple employee ignorance of existing threats.”

A Computer Security, Issues, & Trends report placed the risk of security breaches from employee and former employees even higher, at 81 per cent.

But when government gives with one hand it can certainly take with the other and data protection issues are ever pressing (no matter that government departments have breached their own DPA principles).

When the Nationwide Building Society was fined £980,000 by the FSA for failing to manage information security back in 2007, everyone sat up and took notice.

Now the UK Information Commissioner has made it clear that companies found wanting can be hit by ‘unlimited fines’ and that it is down to the ‘data controller to comply with the data protection principles’, day-to-day demands on data use are brought into sharp relief.

No matter the security polices that are drafted, unless operational integrity is in place, good intentions will come unstuck. And when it comes to testing and development, testing on copies of production data contains unavoidable risks.

Discussions have recently centered on using representative or “fake” data for testing and development. No, there is no other secure way – not even data obfuscation. Seeing as we’re now hearing that masking algorithms can easily be re-engineered (read: http://www.guardian.co.uk/technology/2010/jan/24/computer-security-crime-anonymous-datasets) the time is now to get on the train.

Even if you set aside the advantages of the time and space saved, synthetic data is the most secure and completely compliant – it seems the case for producing synthetic data is copper-bottomed.

Anyone who has had to test applications clearly understands that proper application design and functional testing is directly related to the availability of a representative dataset. Let’s face it, we all know that testing applications is vitally important but we often neglect performing any extensive testing because it is considered to be too difficult and time consuming to find, message, or generate any decent test data. Likewise, both application and database performance testing is often inadequate—inhibiting the ability intelligently modify database structures to approximate database performance for application usage.

It would seem to most that using real-world data would be the best for performing application and database performance testing. Unfortunately real-world data is not always available, may be hard to acquire, doesn’t exist in sufficient quantity, or doesn’t have or is missing important properties (PKs, FKs, etc.). For these reasons, synthetic data generation is a viable alternative that helps ensure representative data is always available for testing. Synthetic data generation can be extremely beneficial in that it can:

1. Data goes far beyond just raw data and encompasses relationships, properties, and nuances of data. It is this ability of a synthetic generation tool to build in these types of relationships that helps perform the fine-grain testing within applications.
2. With a complete and representative dataset, database parameters, queries, indexing strategies, table structures, etc. can be tested and modeled to help tune database performance.
3. Instead of using ‘caned’ data, it is much easier to model and introduce outlying data and ask important questions as to how an application might behave.
4. It is much easier to create standardized benchmarking datasets that can be reused as application or database design changes.

Objects and object-oriented design has become main-stream within many data centers. Compiling attributes into object entities and modeling interacting objects is almost second nature. Grid-Tools has brought the concept of objects to synthetic data generation—enabling users of their Datamaker tool to assemble data objects together for the generation of test data. For instance an application that requires data representative of an order system might link together separate objects that represent internet, phone, or walk-in orders; each type of order object containing fixed or variable attributes that uniquely define the object.

The specification of a data objects ensures the representation of data that includes, but not limited to, fixed and variable data, inheritance by other objects, have a defined structure, have attributes, can relate to external data, can contain expected results, and can contain test coverage data. Data objects is what drives the power of Grid-Tools Datamaker to generate a rich and meaningful spread of data that will be representative of the application intended to be tested—guaranteeing maximum coverage of a code base.