E-Mail and IM Archiving: Proven Strategies for the Enterprise
Topic Last Modified: 2007-12-26
Published: March 30, 2006
E-mail has become a ubiquitous form of electronic communication. For most businesses, e-mail is the standard mode of communication. According to a 2003 Meta Group survey, 74 percent of the executives surveyed believe that being without e-mail would present more of a hardship than being without phone service (META Group, Inc., Senior Executive Panel Survey Representing 387 Corporations, April 22, 2003). Yet most organizations’ e-mail system infrastructure is probably not as reliable as their phone system.
As e-mail has become more important to enterprises, so too has the need to archive electronic communications. There are several factors driving businesses to consider message archiving:
E-mail is a mission-critical application. Organizations and their employees are understandably intolerant of an inoperative e-mail system. In case of e-mail system failure, archiving provides important insurance and is a needed business continuity measure for an organization’s e-mail lifeline.
Organizations are facing the need to comply with regulations from regulatory bodies such as the Securities and Exchange Commission (SEC), the National Association of Securities Dealers (NASD), the Health Insurance Portability and Accountability Act (HIPAA), and Sarbanes-Oxley, which impose retention standards for archiving electronic communication, including e-mail and instant messages (IM).
Finally, organizations must now take a more proactive position in the management of their electronic communications because of their potential use in legal discovery and dispute resolution. An organization must consider the possibility of electronic communications used in internal disputes as
This paper explores the business drivers and needs for archiving electronic messages—including e-mail and instant messages (IM)—while highlighting common problems and misperceptions encountered by enterprises in this area. The paper concludes with an overview of requirements for effective archiving and a review of how Microsoft Exchange Hosted Archive meets these needs.
Most IT departments are discovering that e-mail servers have become de facto data storage vehicles for users within the organization. Employees commonly use the e-mail system as their default “filing cabinet” for retention of their business communication, contacts, calendar, and tasks. Use of the e-mail system as storage coupled with the high volume of e-mail has forced some organizations to impose mailbox size quotas to prevent users from keeping everything in their e-mail system. Size restrictions force employees to spend valuable company time cleaning out and filing old messages and attachments. This escalating problem also creates significant challenges for the IT departments that manage these systems.
Some organizations believe minimizing the retention time of historical e-mail reduces their business risk. They delete old electronic communications assuming that the messages will not be needed. As a result, organizations are surprised when they discover their employees have copied or moved their e-mail to a location where it can be retrieved. This is noteworthy as it leaves an organization at risk under legal discovery. If an organization is asked to produce e-mail in discovery for a lawsuit, it must assess all locations where messages might exist; without a central repository, a firm is left to sift through backup tapes, file cabinets, desktop and laptop computers, and other offline media where e-mail might be stored. Such a task can be enormously expensive and time-consuming.
Organizations also face the possibility of spoofed or forged e-mail messages from employees or other individuals who interact with the organization. For example, a disgruntled employee or an embittered partner could use his or her creative talents to “doctor” a message to read favorably in a lawsuit or dispute against the organization. Without a verified and complete archive, the organization is unable to dispute the doctored message.
Many organizations do not yet have a viable archiving process or enterprisewide solution to aid in the defense of these significant concerns surrounding e-mail. In many cases an archive may exist, but may still be combined with offline tapes or other media, while being neither centralized and complete, nor online and available. In these situations, management and retrieval of archived messages can be difficult and very costly.
Regulatory compliance challenges organizations to modify current business processes surrounding electronic communications. Regulations such as those coming from the NASD and SEC create additional workflow requirements for organizations. Furthermore, clarity in understanding the implications of HIPAA and Sarbanes-Oxley complicates issues of compliance even further. One thing is certain—the regulatory environment will continue to change and so must an organization’s electronic message archiving system.
With significant attention given to the bottom line, organizations are operating their IT departments with lean staffing. In many cases, IT staffs are overburdened with day-to-day support and maintenance activities with little time available to research and address message archiving concerns. When an employee needs to recover a past message, restoration puts additional stress on e-mail administrators, help desk staff, and employees. Frequently, e-mail administrators must search through backup tapes to find messages to be restored, sometimes taking days or weeks to complete the task. Foreseeing the delay in restoring messages, employees resort to backing up their e-mail on CDs or local hard drives. The proliferation of backup copies creates an error-prone environment of inconsistent and incomplete archives.
A capable message archiving solution could resolve many of the e-mail storage, archiving and restoration issues faced by organizations today. However, many message archiving solutions require a significant investment in both hardware and software. In many cases, the entry point for these resources is substantially beyond of the reach of small to midsize organizations.
An effective e-mail archiving solution requires permanently organized storage with interactive capabilities. Many organizations mistakenly believe that copies of e-mail stored on backup tapes represent an archive. Unfortunately, this is not the case. While it is true that another copy of e-mail might then exist within the organization, there are a number of business elements required for a message archiving solution to provide good value to an organization. They include:
Cost-Appropriateness The solution should be priced to scale with the size of the organization and offer the lowest total cost of ownership (TCO).
E-Mail System Compatibility The solution should be compatible with the corporate e-mail system and also provide interoperability with popular instant messaging (IM) networks.
Ease of Use The solution should be effortless to use while offering security and accessibility to users from dispersed geographic locations.
Business Continuity The solution should ensure that electronic communications continue uninterrupted if e-mail system outages occur.
User Empowerment The solution should empower end users within an organization to easily search, locate, and retrieve their information in the archive.
Organizations must consider the growing need to comply with an increasing number of regulatory requirements. While sectors such as financial services, pharmaceutical, and healthcare lead the way and carry the greatest regulatory burden, many other industries are starting to feel the effects of regulatory compliance and the changes required to sustain them.
Current regulatory drivers require a message archiving solution to include:
Retention Policies The solution should allow organizations to enforce retention rules for all electronic communications.
Accessible Archive in Redundant Systems The solution should allow organizations to keep archived messages online and in fully redundant systems.
Complete and Tamper-Proof Archive The solution should commit all electronic messages to verifiably tamper-proof media and preserved.
Supervisory System The solution should enable organizations to establish and maintain a supervisory system that includes regular review of incoming and outgoing electronic correspondence based on statistical sampling, as well as predictive and ad hoc surveillance capabilities.
Message archiving solutions exist on the market today. There are two basic approaches that dictate the way in which the message archiving system is delivered. The traditional method uses a client-server architecture where the application is installed on-site on both a server and each user’s desktop. Typically, this involves investing in hardware and software as well as engaging the IT staff to install, operate, and maintain the system.
Exchange Hosted Archive uses an alternative approach, delivering the message archiving solution as a service. The Microsoft solution is delivered with no additional hardware and software expenses for the customer. Additionally, the service-based solution transfers the burden of application support to Microsoft rather than the IT staff, resulting in lower costs to the organization and better use of IT resources.
Nearly all solutions tackle message archiving “after the fact.” Specifically, this means that message archiving options are addressed after the message is received by the organization. Unlike the solutions that use this method, Exchange Hosted Archive intercepts and makes copies of each message without any latency, so that they can be immediately archived and indexed. This is a critical point as it opens up significant archiving opportunities that resolve many of the requirements for an effective message archiving solution. An important benefit of this approach is the ability to offer business continuity and full disaster recovery should there be an outage within an organization’s e-mail system. Furthermore, Exchange Hosted Services can perform other real-time message services, such as spam filtering, virus scanning, and policy enforcement, ensuring that only legitimate and clean messages are stored in the archive and delivered to the organization’s e-mail system.
The following diagram shows how Exchange Hosted Archive works in conjunction with the Exchange Hosted Filtering service to copy and archive messages after they have passed through the upstream spam, virus, and policy filters.
Exchange Hosted Archive intercepts messages in stream
This allows message filtering—such as spam filtering, virus scanning, and policy enforcement—to be performed prior to archiving, improving the efficiency of archiving operations and eliminating unwanted content from the archive.
One of the important features of Exchange Hosted Archive is the carefully crafted, secure, and easy-to-use archiving application. From the end user trying to recover an archived message to the e-mail administrator configuring an archiving policy, Exchange Hosted Archive provides a familiar interface that minimizes the need for formal end-user training. Users interact with Message Archive through a Web-based interface. The user interface is designed with the “look and feel” of Microsoft Outlook® messaging and collaboration client, making it familiar to many and intuitively easy to use for others. Secure communication with Exchange Hosted Archive is achieved through encryption techniques embedded within the desktop browser. The illustration below shows the Exchange Hosted Archive interface.
IT departments will appreciate the ability to define and facilitate a user’s role in the system, equivalent to their role within the organization. Predefined policies simplify the initial process of establishing access levels for each user while customized policies provide organizations with the flexibility to define appropriate levels of access and system usage.
All Exchange Hosted Message Archive functionality delivered through a unified Web-based interface
This includes end-user, administrative, and supervisory tools.
Organizations reap the benefits Exchange Hosted Archive delivers as a managed service. The Microsoft solution delivers the advantages of no upfront capital costs, predictable monthly service fees and a quick and painless implementation process. The significantly reduced initial investment makes the solution palatable for small and large organizations alike. Unlike client-server solutions, Exchange Hosted Archive requires no additional hardware or software for IT staff to install and maintain. Moreover, the service solution offers the organization the latest version of the application without burdening the user or IT department with updates to apply. Collectively, these benefits allow an organization to start addressing business continuity and regulatory compliance within hours of the initial deployment.
The Exchange Hosted Archive central repository ensures that the organization’s messaging data is accessible to all users including those in other geographic locations. This sets the stage in empowering end users to find and retrieve historical e-mail (IMs and other electronic communication included) on their own. The familiar, easy-to-use interface and powerful search capabilities make this all possible. Organizations benefit from improved employee productivity and the reduced reliance on IT staff.
Regulatory bodies such as the SEC, FDA and other federal, state and local government agencies have imposed requirements for documentation, retention and transparency of an organization’s electronic communication. Although many organizations find the broad range of recent regulations onerous and burdensome, in many cases the regulatory requirements push companies to adopt better business practices. Exchange Hosted Archive helps organizations comply with many of these regulations.
Exchange Hosted Archive allows an organization’s electronic communication to be retained online as far back as the company has historical electronic communications. This directly supports many regulatory rules where electronic communication records must be retained for a specified duration.
Another set of features help organizations monitor, filter, search and report on multiple types of electronic messages including e-mail, instant messaging and Bloomberg transactions . This is a key consideration for organizations facing regulatory compliance. For example, NASD requires broker-dealers to maintain a system to supervise transactions and correspondence between their brokers and clients. Using Exchange Hosted Archive, supervisors can quickly and efficiently perform supervision of electronic messages sent and received by their staff.
Exchange Hosted Archive makes it easy for managers to configure and filter messages for review. Messages are flagged in a review queue based on keyword matches and/or random percentage sampling. Designated management can configure the sampling rate as well as maintain the dictionary of keywords . Random sampling rates can be set on a person-by-person and/or employee-class basis (i.e., all “investment bankers” should have n% of their messages reviewed when speaking with “research”). In addition, sampling rates can be configured based on the type of electronic communication: external e-mail, internal e-mail, external instant messages or internal instant messages. Supervisors can then highlight a range of messages and traverse through each message, marking each as “reviewed” or “of concern.”
Search capabilities within Exchange Hosted Archive are superior to those found in many desktop applications. The Exchange Hosted Archive ad hoc search tools and rules-based supervisory tools allow for keyword and phrase matching in the message subject, body and attachments. Message Archive indexes the content of more than 200 attachments, including popular document formats such as Microsoft Word, Excel®, and PowerPoint® and Adobe Acrobat. The system’s keyword and key phrase matching allows the use of a “super wildcard” character that performs word stem matching. By comparison with other search applications that perform word completion, The Exchange Hosted Archive search technology can find all variations of a word. For example, in a typical application promise* would match promise or promised. With the Microsoft solution, $promise will match not only promise and promised, but also promissory, because all stems of the root of each word are found.
Conducting audits and reviews are necessary steps in the regulatory compliance process. Exchange Hosted Archive delivers a set of predefined reports along with the ability to create customized reports in support of supervisory responsibilities. For example, Microsoft includes a Supervisory Evidentiary Review report to allow supervisors and compliance managers to document and monitor compliance and attest to their adherence to the regulation. Following is a sample of a NASD 3010 evidentiary report:
All Exchange Hosted Message Archive functionality delivered through a unified Web-based interface
This includes end-user, administrative, and supervisory tools.
The dependence many users have on their e-mail systems has made it a mission-critical application to the organization. For this reason, business continuity and disaster recovery are critical topics when addressing messaging systems. Microsoft helps organizations plan for and resolve business continuity issue by providing a set of business continuity tools within the Exchange Hosted Archive service. When the organization’s e-mail system is down, the archive acts as a backup e-mail system so that business communications can continue without interruption. All inbound e-mail continues to be archived, even when the e-mail server is down. Further, e-mail can be composed, read, answered, and forwarded using the online archive interface until the primary e-mail system is restored. When the e-mail service is restored, all messages sent and received during the outage can be easily restored to the organization’s primary e-mail system.
There is no doubt electronic messaging plays a vital role in the success of an organization today. The pervasive use of e-mail as a critical business tool has upgraded the stature of e-mail systems to that of a true mission-critical application. As a result, organizations are driven to consider message archiving solutions for the appropriate backup and protection of mission-critical systems. Key requirements for an effective message archiving system include its ability to archive, index, retain, retrieve, and restore electronic communications as well as assist in legal discovery and regulatory compliance.
When an organization is considering message archiving solutions, Exchange Hosted Archive delivers exceptional value in several key areas:
Hosted Service Convenience Delivered as a managed service, Exchange Hosted Archive delivers complete message archiving for the entire enterprise with the lowest total cost of ownership (TCO), including no capital investment in hardware or software.
Regulatory Compliance All Exchange Hosted Archive functionality is designed to assist customers who are regulated by electronic message retention requirements. In addition, workflow tools for compliance supervision and reporting are fully integrated into the system.
Integrated Message Retrieval Exchange Hosted Archive includes tools to allow all users to search, retrieve, restore, and extract any message. All messages are fully text-indexed so they can be retrieved, restored, or extracted when needed, including for legal and compliance purposes.
Intuitive Web-based User Interface All aspects of the system from end-user interaction to administrative and supervisory tasks are handled via a familiar, easy-to-use Web interface.
E-Mail Continuity and Disaster Recovery A unique feature of Exchange Hosted Archive enables customers to continue to receive and have full access to e-mail, even when the corporate e-mail system is down.
Exchange Hosted Archive delivers a comprehensive electronic message archiving solution to meet the vital needs of today’s business enterprise.
Microsoft Exchange Hosted Services offer a cost-effective way for enterprises to actively ensure the security and availability of their messaging environment, while instilling confidence that their messaging processes satisfy internal policy and regulatory compliance requirements. A seamless extension of Microsoft Exchange that operates over the Internet as a service, the complete line of services includes hosted filtering for spam and virus protection; hosted archiving to satisfy compliance requirements and internal policies; hosted encryption to preserve e-mail confidentiality; and, hosted continuity for ongoing access to e-mail during and after disasters. Microsoft Exchange Hosted Services provide value to corporate customers by eliminating upfront capital investment, freeing up IT resources, and removing incoming e-mail threats before they reach the corporate firewall.
For more information, see Microsoft Exchange Hosted Services.
Current Enterprise Message System Challenges
Requirements for an Effective Message Archiving Solution
A Solution That Meets All the Requirements: Microsoft Exchange Hosted Archive
About Microsoft Exchange Hosted Services