Business dilemma: email retention policy; new SEC regulations to address storing and restoring

Messaging applications are crucial for modem business, but managing them is a never-ending headache. Messaging data generates huge volumes of storage: Microsoft Exchange alone processes more than 3 million email messages every day through corporate email servers, and analysts estimate that email and attachments are growing at a rate of 35% per year. Email archives are massive and unwieldy, resist classification schemes, ignore structured searches and frustrate policy-driven archiving. But managing email archives is vital to protect companies from serious management, storage, litigation and regulatory liabilities.

Management Support Cost

Without effective email archiving in place it is time-consuming and expensive to search and restore email databases from backup media. Lacking email management tools makes email applications such as Exchange especially challenging. When Exchange crashes (as it frequently does), administrators must undergo an ugly and labor-intensive restore operation. If they must search archived databases they are forced to dedicate a separate server to the task, and individually search and restore each separate backup tape that might hold a relevant message.

Storage is also a huge issue. Many companies try to stay on the side of the angels by retaining all their emails. But the huge and growing volume of email seriously impacts storage budgets and resources. The great expense is not so much buying it, since relatively cheap storage brings in the initial purchase price at just 20% of storage-related expenditures. But managing it is another story: Email administration tasks, like backups and restores, gobbles up 43% of IT support costs. Companies are adopting and growing their storage area networks to manage their email glut as much as their critical database storage.

Corporate email systems are even harder to manage because of the weight of "personal" emails. Users send overwhelming numbers of personal emails from work, but job hunters and moonlighters beware--court cases surrounding personal emails sent from work have favored the corporations, essentially ruling that when an employee uses the company email system to send an email it is no longer personal and First Amendment rights do not apply. (Tell that to Arapahoe County's Clerk & Recorder. The Colorado man is facing tax misappropriation charges for using the county's email system to send some 600 raunchy messages to his mistress--who is also his assistant. The electorate returned him to office anyway, but his own embarrassed party is trying to kick him out.)

Because email is so hard to manage, corporations largely ignore personal email floods. But not only do personal emails strain storage resources, ignoring emails with sexual or hate-filled content is an invitation to serious lawsuits.

Storage

Yet this is hardly unusual for companies and their email. This leads to the ironic conclusion that--like not shredding paper documents--companies must retain their emails, even if they're keeping a smoking gun on their hands. Legalities aside, IT departments struggle with storing huge volumes of messaging data. Microsoft Exchange alone processes more than 3 million email messages every day, most of it through corporate email servers, and analysts estimate that email and attachments are growing at a rate of 35% per year. Since corporations are reluctant to delete messages, this type of volume greatly impacts storage purchases and management. This cost can sneak up on a business: Storage hardware is relatively cheap, so initial purchases account for only 20% of storage-related expenditures. But managing all these volumes is another story: Email administration tasks gobble up 43% of IT support costs. These days, companies are adopting and growing storage area networks to manage their email glut as much as their critical database storage.

Litigation

Companies have lost lawsuits by keeping too much information. Robert F. Williams, president of consulting company Cohasset reported, "Not having any email retention policies inevitably will result in amassing vast volumes of communications that are costly to retain, even more expensive to search through in response to discovery requests, and may unwittingly supply information that is harmful to the organization if disclosed in response to discovery requests."

The opposite tack, deleting most messages, is also risky. This used to be a popular accounting and legal strategy (the idea was that they can't use evidence that doesn't exist). But as the government frowns on that concept now, large accounting and legal firms are aggressively pursuing email retention strategies and software tools to keep their clients (and themselves) off regulatory hit lists. Taufer commented, "These large law firms want to address the issue from both a storage and a policy and methodology standpoint: how do you do it, what do you do, and how do you cover yourself from a legal standpoint?"

In one lawsuit an IT professional divulged that he was storing hundreds of backup tapes in a closet. He had not told his lawyers. Regardless of whether the backups had anything to do with the lawsuit, the opposing lawyers had the right to order that the backups be read. Of course they did that, and the cost ran into the millions of dollars for the company. In another case, Prudential Life Insurance was involved in a class action suit and the court had ordered that it destroy no records during the proceedings. Unfortunately no one told the IT department, who happily went on deleting electronic records on its own retention schedule. A judge issued a $1 million penalty against Prudential for destroying data that supported its opponent's case, and required them to deploy a records management program with a multimillion dollar price tag. Although Prudential had not deliberately destroyed relevant data, it still lost huge sums of money over its inability to enforce a reasonable and consistent retention policy.

Regulatory Requirements

Finance, pharmaceutical, healthcare, telecommunications and government-related firms must observe strict electronic document retention requirements. Regulations aren't strict for nothing: Email retrieval was key to the government's case in the Enron scandal (Andersen's shredding party notwithstanding). For instance, the SEC insists that American securities firms retain their electronic documents for five years--and be sure they can search and restore specific messages and threads in a short turnaround. Elizabeth Schnitzer of Iron Mountain noted that nearly 10,000 brokerage firms must keep all correspondence regarding a stock trade for six years, while email related to general business issues must be kept three years. And in companies whose analyst and investment banking divisions may have grown too cozy, the government closely analyzed email communications to build its cases.

Last year the SEC, NY Stock Exchange and NSAD forced five major, Wall Street firms to cough up over $8 million in non-compliance fees. What did they do--or not do--that cost them $1.65 million a firm?

The firms backed up email as part of their regular backup routines. However, they discarded, recycled and overwrote the backup tapes and other media, often a year or less after backup occurred.

Each firm had spotty procedures and systems around retaining and restoring email data. Some firms simply assumed users would retain all their email on their own hard drives. Many users did, but the firms could not efficiently search these emails in time to satisfy the investigators.

There were no formal policies in place for users to retain their emails. When a user left the company IT erased his hard drive, deleting the email along with it.

The irony of the huge fines is that unlike bad guys Enron or WorldCom, the securities firms didn't do anything differently than most firms do with their email. That is why Boulder's President Lesley Taufer commented, "It's unclear if their processes were haphazard. That's why the fines were so significant." The firms may have been acting within traditional acceptable boundaries for email management. But what used to be acceptable will no longer do.

Solving the Problem

Companies must walk a narrow course between the expensive and risky extremes of email management: neither keeping all message stores (strains storage and management resources; preserves smoking guns) nor deleting email without a strict retention policy (violates regulatory laws; invites litigation). Ideally companies will assess their email records situation, generate intelligent business policies, communicate the policies to all end-users, and use email management applications to archive, manage, audit, restore and delete messaging data. Primary messaging compliance objectives would include:

* Instituting effective back-up and restore procedures.

* Initially capturing and storing all email, TM and attachments.

* Basing retrieval capabilities on primary index values such as unique message ID, date, from, to, subject line, and combinations.

* Providing full-text search capabilities against message text as well as attachments.

* Scheduling deletions according to compliance timelines.

* Preserving regulated data on non-rewritable, nonerasable formats (an important provision of the SEC regulations).

* Automatically verifying the quality and accuracy of the archiving process.

* Offering full audit capability of the email archives.

Sadly, current email technology is not optimized for email retention and few corporate systems are capable of doing it well. The world's largest corporate messaging applications--Microsoft Exchange, IBM Lotus Notes/Domino and Novell's GroupWise--provide few native resources for compliance and retention operations. (A primary example of this is Exchange's own developer Microsoft, who did bother deleting the smoking gun emails that made them look like pirates in anti-trust court.)

Several vendors offer email management software to the SOHO and mid-sized markets, but enterprise-level packages are rare. (Enterprise-level packages that work with all three major email applications are non-existent.) One increasingly popular package is Exchange Archive Solution (EAS) from EDUCOM TS. EAS archives multiple MS Exchange email stores to NAS and SAN and compresses the archived messages. Users can access, search and restore archived mail and perform compliance audits. Tivoli's Storage Manager for Mail manages both Exchange and Notes, while developers AXSOne, C2C and KVS offer similar applications. Storage hardware vendors are also getting into the act: EMC partners with EDUCOM to manage mail on their Celerra storage device while others develop their own email management suites. NetApp's Enterprise Vault, for example, manages Exchange data stores on NetApp Filers.

By managing message stores throughout their lifecycle, companies can establish email retention policies, protect corporate intellectual property, increase information retrieval speed, and reduce expensive email server overload. And maybe save themselves from a whopping big judgment or two.

www.bouldercorporation.com

www.cohasset.com

www.educomts.com.

www.ironmountain.com

Here is a breakdown of the primary regulatory body or regulation that applies to email retention at a variety of otherwise-regulated industries and general business.

* Banking: FDIC, OCC (Office of the Comptroller of the Currency)

* Telecommunications: Title 47, Part 42

* Pharmaceutical: FDA--Title 21, Part 11

* Healthcare: HIPAA (Health Insurance Portability and Accountability Act)

* Defense: DOD--5015.2 standard

* Brokerage firms: SEC--Rule 17a-3 and 17a-4

* General business oversight: Sarbanes-Oxley Act (The government's response to the Enron debacle, it contains provisions for record retention and audits. It strongly discourages anyone from altering, destroying, hiding or falsifying records in response to a federal investigation or bankruptcy proceeding. Penalties include whopping fines and/or imprisonment of up to 20 years.)