Retention policies / compliance

[maclellana]

Good afternoon / morning everyone!

I've also got a couple of discussions going on the LinkedIn forums but thought it would be a good idea to post here too.

I'm doing some research at the moment regarding data retention, specifically guidelines for UK companies and data backup, but a global perspective would be equally interesting. There's a wealth of regulations out there but everything is so vague and in many cases contradictory - e.g. Data Protection - keep certain personal data for minimum time possible vs perhaps certain FSA misconceptions that everything must be kept for 6 or 7 years.

The basics of data retention are fairly straight forward (e.g. keep x records for y years) but what seems to be confusing most companies we speak to is how many generations of data need to be kept? Are there any sensible guidelines that are based on more than guesswork?

I'm not a lawyer but I'm assuming the answer is "the more you keep the more you're protected in the event of litigation, providing you're not breaking any data protection / privacy laws in the process, so keep as much as you can afford to keep"? And by afford I don't just mean data storage costs but management of that data too - and that means everything from finding the right data when you need it through to ensuring the data can't have been tampered with - and proving it can't have been tampered with.

I'm really hoping I'm being incredibly naive and we're just missing something, but we can't be the first company to do some proper research in to this?? Every, and I mean every, company we speak to has a different interpretation of what they need to do when it comes to data backups or data retention in general, and we've seen some horrific examples of sheer stupidity from companies with no backups at all through to some who leave the same backup tape in their server forever and assume that's fine.

At the other end of the scale many organisations are getting immensely paranoid, and probably rightly so, about e-mail backup retention and their ability to roll back to a point in time 4 years ago to restore a critical individual e-mail, assuming it wasn't deleted before the nightly backup had run of course. Without careful observation of T's & C's this is even harder to control with cloud-based e-mail systems and people rarely ask the pertinent questions.

Andy

[aravindhaltruist]

In the backup environment,retention is meant for how long the backed up data is kept on the storage media tape/disk.We cannot predict when we require what data that was backed up.So reatining data for sometime is neccessary to restore/recover the same.But the retention is to some extent depended on the frequency of restore requests.

E.g: In ABC product company we have 10 servers that needs backup services.And one among the 10 servers is getting the frequent restore requests.In such cases we should plan for longer retention.

Aravindh
Backup Admin-Level 3

[WWallace]

My thoughts are you should keep data and or retentions of data to conform to what your record retention policy states. I subscribe to the motto that there are two (2) sins when it comes to data: you should have it and don't and you do have it and you shouldn't. When disk storage continuing down the pricing curve, people and companies, are even more inclinded to keep everything and that could be just as bad as keeping nothing. The courts in the US seem to be following the guideline that as long as you follow your own published Records Retention Policy, then you are covered. Woodie

[clayramsey]

perhaps this is an untenable ideal, but getting guidance from corp. counsel on what laws apply, what those laws state, and what they end up requiring seems like the way to go. If there is no corporate counsel, then having a quality, current, and graduate level Information Assurance text book will give you a pretty decent basis on which to form a RRP. My 2 cents.

[MattG]

Woodie was on-target. Don't keep everything just because maybe you can... If you are in a regulated field, ie- Banking, Mortgage Company, Stock Broker, etc... you MUST know your retention requirements for certain data. Where the regulations and guidelines are "grey" then, just as Woodie intimated, have a backup and records retention policy written and followed. Attorneys will point out, "don't save unneccesary or unregulated data" and for regulated data, "don't save 1 day longer than necessary or required". Regards to all- Matt

[rahrenstorff]

I agree with MattG. I worked for medical providers in the past and in order to mitigate risk to the company, we always followed those same guidelines "never keep more than compliance mandates".

[PavanVyas]

Matt is spot on with his observations. Some of the regulations provide very specific guidelines on data retention policies. If you are offering data protection services to customers in the healthcare, financial services, legal, pharma, education, or research sectors, you should take the time to review the regulations related to data protection, data security, data sovereignty, and data retention. Sometimes, these requirements vary by state - therefore checking both federal and provincial regulations is a good idea.

And yes, "the more you keep, the more you are protected" may not necessarily be true - there may be a requirement for you to destroy data beyond a particular period / data that meets certain conditions. In such cases, destroying that data and providing destruction certifications may be necessary.

So, if you are dealing with regulated data, there is no silver bullet. You are advised to read the fine print.

Regards, Pavan

[mnbenton]

I would like to follow-up on Matt's about tuning retention to protect whats needed but let other data fall out of retention. One thing I have run into in keeping under 9 generation, keep 1 gen every day for 6 days and keep 1 gen every week for 3 weeks, we risk not protecting an infrequently changed file that some one overwrites 6 months later for example. What are others doing to avoid this risk while minimizing amount of stored size?

Thanks, Mark

[TheBackupKnight]

I guess there is always a trade off between minimizing storage size and protecting all the information that you have. For data beyond 3 weeks, do you use an archiver tool to store to cheaper storage / tape?

[mnbenton]

Yes, using the BLM to protect specific backup sets beyond 3 weeks is an option. Could you tell me what is your standard retention policy for the DS-System?