Technical Case Study

Published: August 25, 2006

The Microsoft OEM Division's IT Center of Excellence (OEM IT), a global IT engineering organization within Microsoft, deployed Microsoft® Visual Studio® 2005 Team System. OEM IT now enjoys a fully integrated software development platform, realizes substantial and measurable savings, and is able to focus more on development and delivery of high-quality software and less on project administrative tasks.

To deliver high-quality software products on time, on schedule, and within budget, all large software development organizations must meet the same management and administrative challenges. They must plan, schedule, and manage the activities of the software development team, and they must do so efficiently, allowing the members of the development team to focus their efforts on developing software without becoming sidetracked on administrative tasks.

Until recently, project management, software and database design, software coding, testing, and packaging have usually been handled separately. Most development organizations accomplished these management and administrative tasks by using repetitious data entry into isolated documents and spreadsheets. These documents and spreadsheets were then distributed as e-mail attachments. Where automation was introduced into these tasks, the systems were narrowly focused on one activity. Integrating and coordinating these systems required ad hoc, manual programming, which then required customized maintenance and support.

Microsoft Visual Studio 2005 Team System was designed and constructed to address the beginning-to-end needs of large software development teams. Its components work together to form an integrated software development environment, from requirements gathering through design, coding, testing, packaging, and delivery of the software product.

This technical case study describes how Microsoft OEM IT implemented Visual Studio 2005 Team System. This paper can serve as a model for an organization that intends to implement Visual Studio 2005 Team System. It is intended for chief information officers, IT directors, software development executives and managers, solution architects, and technical decision makers.

Situation

The Microsoft OEM Division is part of the Microsoft worldwide sales and marketing group. It manages relationships between Microsoft and manufacturers of computers and other devices, including manufacturers of embedded systems. It handles all products that Microsoft can license directly to the OEM community, including all versions of the Microsoft Windows® and Windows Server® operating systems, Microsoft Office, Windows Mobile®, Windows CE, Microsoft Works, Microsoft Exchange, and Microsoft SQL Server™ 2005.

OEM IT is a global IT engineering organization that delivers enterprise business solutions to the Microsoft OEM Division and several other businesses within Microsoft; specifically, OEM operations, OEM sales, Microsoft Xbox® operations, System Builder channel operations, and Microsoft MapPoint® operations. OEM IT consists of approximately 200 people, including Microsoft employees and personnel from global vendor partners. It has offices in Redmond, Washington; Reno, Nevada; Dublin, Ireland; Beijing, China; and Hyderabad, India. It serves internal Microsoft business customers at their locations around the world.

The OEM IT development team maintains about 2.2 million lines of code, which consists of about half SQL code and half Web code. It delivers 40 line-of-business applications to its customers. OEM IT plans for four, quarterly software releases a year, scheduled to coincide with updates to the Microsoft implementation of SAP (the corporate enterprise resource planning system). Each quarterly release consists of multiple projects and contains from 65,000 to 220,000 lines of new or changed code. At any one time, OEM IT is typically working on three quarterly releases: one in the conception/envisioning stage, one in the design/build phase, and one in the stabilization phase (testing). When required, though, the group has simultaneously worked on four or five releases.

Before the implementation of Microsoft Visual Studio 2005 Team System, OEM IT was in the situation of many software development organizations. It used a set of separate, internally developed tools for work item tracking, version tracking, defect tracking, test case management, and build management. It also used a stand-alone product for source control. These tools operated on separate servers, were not highly integrated, and were difficult to manage. The disparate architectures and infrastructures made integrating these systems difficult.

The project teams had to deal with separate support teams for each tool. There was often some confusion about which support team was responsible for which problem. The project teams also had to manually manage their own build systems. They had to manage five different build configurations, including all of the dependencies of those configurations.

For code source control, the OEM IT development team used Microsoft Visual SourceSafe®. It worked well for its specific purpose, but it was not integrated with the manual build system or the build reporting system. Code branching strategy was complex and difficult to manage, and many problems existed with consolidating deliverable packages.

The internally developed defect tracking tool presented several problems. Project status reporting—determining the actual testing work done versus the testing work planned to that date—was time consuming, largely manual, and subject to errors. Data was manually entered into Microsoft Office Project and Microsoft Office Excel® spreadsheets. The project status information was difficult to determine on a daily basis and was impossible to generate in real time. The test team also found it very time consuming to plan test cases and import the test cases between releases by using manual systems.

Project managers accomplished reporting by running static report queries on a relational data store. They were unable to look at historical data or analyze trends. Adapting the system to support new metrics or processes required tedious SQL coding and report formatting. Also, it was difficult and expensive to comply with statutory reporting requirements, such as Sarbanes-Oxley, which requires proof of "an adequate internal control structure."

For a process methodology, the OEM IT development projects use Personal Software Process (PSP) and Team Software Process (TSP) from the Carnegie Mellon Software Engineering Institute. Engineers who use the PSP to develop software follow defined processes and collect detailed metrics on the time expended, the defects created and removed at various stages in development, and the size of the finished product. OEM IT engineers manually collected and analyzed the PSP/TSP metrics.

Solution

To bring automation, integration, and collaboration to the entire software development life cycle, Microsoft developed Visual Studio 2005 Team System. This client/server system consolidates project management, work item tracking, code source control, build and test environments, and bug tracking.

Components of Visual Studio Team System

Visual Studio Team System is a collection of client applications, Visual Studio Team Foundation Server, and several other related components.

Visual Studio Team Editions and Team Explorer

The Visual Studio Team System client applications are based on the roles that a person fills in a development project. The client applications are called Team Editions. There are three Team Editions for software development:

• Team Edition for Software Architects

• Team Edition for Software Developers

• Team Edition for Software Testers

There is also a Team Edition for database designers:

• Team Edition for Database Professionals

In addition, there is a client edition that delivers the functionality of all the client applications previously listed:

• Visual Studio Team Suite

Finally, there is a Team Foundation Server client, used to build projects and administer the server:

• Visual Studio Team Explorer

Figure 1 graphically shows the functionality that the software development client applications contain.

Figure 1. Visual Studio Team Editions functionality

Custom Clients

It is possible to use Microsoft Office Project 2003 and Microsoft Office Excel 2003 as Visual Studio Team System clients. Also, by using available Visual Studio Team System programming tools, it is possible to build custom clients, if required.

Visual Studio Team Foundation Server

Visual Studio Team Foundation Server, working with the Visual Studio Team Editions and other clients, integrates all of the core roles that compose the software development process. Team Foundation Server is a collaboration engine and a centralized data warehouse.

Team Foundation Server provides source control management, work item tracking, integrated analysis and reporting, and an integrated collaborative project Web site. It can also provide build and test rig management. In addition, Team Foundation Server can integrate with Office Project 2003 and Office Excel 2003 to extend reporting and tracking abilities. Team Foundation Server consists of two tiers (an application tier and a data tier), plus a team portal and team reports.

Application Tier The Team Foundation Server application tier primarily consists of Web services: Team Foundation Common Web services Registration, Security, Linking, Eventing, Classification), Version Control Web services, Work Item Tracking Web services, and Team Build Web services. The application tier communicates through an object model.

Data Tier The Team Foundation Server data tier consists of a SQL Server 2005 database server and databases specific to Team Foundation Server. The data tier also contains the data warehouse. The data warehouse uses SQL Server 2005 Integration Services and Analysis Services to extract needed information from the operational databases, transform it into a business-centered data model, and load it into an online analytical processing (OLAP), multidimensional database. In addition to serving as the source for report information, a multidimensional OLAP database system:

• Can analyze large amounts of data in a short time.

• Enables users to create summarized fields from detailed data or explore the details that contributed to creating a summarized field.

• Enables users to explore the data in new ways, to identify a trend or a problem area that would otherwise be overlooked.

Team Portal Team Foundation Server includes a team portal—a Windows SharePoint® Services Web site through which project personnel can access the project's data and the collaborative functions that Visual Studio Team Foundation Server provides.

Team Reports Team Foundation Server includes predefined reports. These can be modified, or new reports can be created using SQL Server 2005 Reporting Services.

Microsoft Team Foundation Service

To deliver the benefits of Visual Studio Team Foundation Server to all project teams at Microsoft and to enable effective management and reporting across multiple projects, Microsoft created the Microsoft Team Foundation Service. OEM IT used this managed service to handle setting up, hosting, and administering Team Foundation Server.

Architecture of Visual Studio Team System

Figure 2 is an architectural schematic of the Visual Studio Team System implementation that OEM IT deployed. This architecture is typical of Visual Studio Team System deployment.

Figure 2. Visual Studio Team System typical architecture

In Figure 2:

• Visual Studio Team Editions represents all of the Visual Studio Team System client products: Team Editions, Team Suite, and Team Explorer.

• Office System represents the integrations of Excel 2003 and Project 2003 to work as Visual Studio Team System clients. (Team Foundation Server includes templates for integrating Excel 2003 and Project 2003.)

• Microsoft Windows Internet Explorer® represents the users' Web access to the team portal.

• Team Foundation Server object model is the communication layer between the clients and the application tier.

• Team Foundation Server application tier includes the Web services. Also on the application tier are team reports (SQL Server Reporting Services functionality) and the team portal Web site (Windows SharePoint Services).

• Team Foundation Server data tier includes the SQL Server operational databases and the data warehouse.

Implementing Visual Studio Team System in OEM IT

While Visual Studio Team System was still in development, the managers at OEM IT learned of its capabilities and the plans to offer the Microsoft Team Foundation Service. OEM IT signed on to be among the first groups at Microsoft to employ the new product and service. OEM IT understood the value of the product's capabilities and wanted to deploy the product to its global team.

The timing of the Visual Studio Team System product release matched well with the beginning of a major release from OEM IT to its business customers. The OEM IT management team presented the new product, its functionality, and its capabilities to OEM IT personnel, and found great enthusiasm for deploying it. The personnel immediately saw how the product would eliminate a lot of administrative work and enable them to focus more on their substantive work.

The management team then presented its plan to the business partners. The management team found that it did not need to spend much time evangelizing Visual Studio Team System. The business partners, too, quickly saw the advantages of the integrated system.

" "When talking to our business partners, Visual Studio Team System sold itself. The longest it took for acceptance was 15 minutes." "

Al Buckingham

Setting Up the System

OEM IT worked with the Microsoft Team Foundation Service team to set up its projects in Team Foundation Server. The service had already set up two process model templates in Team Foundation Server: the Capability Maturity Model Integration (CMMI) process and the Microsoft Solutions Framework (MSF) for Agile Software Development process. OEM IT projects use the CMMI model, so the development team used that model to begin building its Team Foundation Server projects.

In the Team Foundation Server project management model, OEM IT set up work items, including work item type definitions (such as Defect, Task, and Issue), queries, and instance templates. Instance templates contain a predetermined set of tasks that must occur for every project, such as gathering requirements or writing a vision document. OEM IT was able to keep the same workflow and type definitions that they had been using in previous systems.

OEM IT used the reports that are prepackaged with Team Foundation Server, with slight modifications, and added a few custom reports. The development team also created a series of custom reports based on data read from the OLAP data warehouse into Excel PivotTable™ dynamic views for presentation and analysis. The custom reports enable the system to deliver to each person the information that he or she needs, in the form in which he or she needs it. These reports thus free each person from sorting through irrelevant information.

Then, starting with the SharePoint portal packaged with Team Foundation Server, OEM IT modified the portal according to its preferred themes. The development team also created a report dashboard for each OEM IT project. A dashboard is a graphic visual display of multiple information items that are essential for tracking a project.

OEM IT ported its defect tracking process into Team Foundation Server and enhanced it. OEM IT maintained the names and definitions that were familiar to the OEM IT team members.

The development team paid special attention to the build process. It customized the team build process so that the process could be automated and integrated with Team Foundation Server. The development team automated the packaging of files for delivery to user acceptance testing and to production.

Best Practices

While deploying and implementing the Visual Studio Team System, OEM IT identified the following best practices that may be of value to an organization that is planning a similar deployment.

Maintain the Processes and Definitions from Previous Systems

An organization should maintain as many processes and definitions as possible from its previous systems. For example, OEM IT ported the work item tracking process and work item type definitions from previous systems into Team Foundation Server. This action reduced the time required for team members to become productive on the new system. They only had to learn to use a new user interface.

The organization should begin refining processes and definitions after its personnel have mastered the new system.

Carefully Plan the Implementation of Visual Studio Team System to Testing Teams

Testing work starts only after the software design and coding is complete. By that time, the project managers, designers, and coders have been using the system for some time and have learned its details and how best to work with it. The testers, however, are just starting to use the system.

When planning a deployment of Visual Studio Team System, an organization should make sure that the testing teams and the end users involved in user acceptance testing are adequately trained and ready to use the system under real working pressures.

Carefully Design the Test Case Management Processes and Plan for Training

Team Foundation Server is very flexible and can be configured however the project team prefers. The project team, though, must carefully design and plan its configuration with its users in mind.

For OEM IT, implementing the test case management required special attention. The regular testing team consists of software test professionals who can learn to use a sophisticated test case management system in a relatively short period of time. However, the end users involved in user acceptance testing are professionals in their own fields and not necessarily highly technical.

When an organization is designing its test case management system in Team Foundation Server, it should keep in mind that relatively nontechnical people will use it. When planning training, the organization should pay special attention to the needs of nontechnical end users of the application.

Carefully Design the Software Release Processes

The software release process must ensure that the latest, approved versions of the source code files are compiled into a build to be released into testing or production. These processes touch most of the Team Foundation Server components. An organization should pay careful attention to the design, implementation, and testing of these processes.

Benefits

Visual Studio Team System delivers the following benefits to OEM IT:

• It automates numerous administrative tasks to save time and money in software development projects.

  It is an all-in-one, integrated, role-based system that combines and coordinates the entire software development life cycle. Personnel can therefore focus more on their substantive work instead of spending time on those administrative tasks.

• It provides powerful yet flexible project reporting capabilities.

  Information and metrics are automatically collected and organized into an OLAP data warehouse. Each person receives the latest information, and only the information that he or she needs, in the most appropriate and usable form. Team members can dynamically create summaries or explore detail data in real time to perform richer data analysis. Project managers discover possible problems before they can affect the project's progress. Senior management now has new visibility into projects' operations. The automated reporting makes meeting statutory requirements, such as Sarbanes-Oxley, easier and less expensive.

• It simplifies support.

  Because it is a consistent, all-in-one system, team members no longer have to deal with multiple support organizations for multiple tools. One support organization provides support for all components of Team Foundation Server—work item tracking, source control, version tracking, defect tracking, test case management, and build management.

• It automates the processes of build production and build reporting.

  The project teams no longer have to manually manage their own build systems.

• It provides a team portal for comprehensive, efficient team communications.

  The team portal provides a single point of access for all team members.

• It has improved the job satisfaction of team members.

  Because it eliminates a lot of administrative work and enables team members to focus more on their substantive work, it significantly improves their job satisfaction.

At the Microsoft corporate level, adoption of the Visual Studio Team System provides these benefits:

• It helps standardize project methodologies and taxonomies.

  It facilitates the use of standardized methodologies and taxonomies across all development projects.

• It enables coordinated, consistent, standardized, cross-project reporting.

  This reporting capability provides transparency to management and makes meeting statutory requirements easier and less expensive.

• It lowers support costs.

  The support organizations no longer have to support many instances of varied tool sets. Standardized hardware, software, configurations, and methodologies lower the load on the support organization.

Measurable Savings

Compared to using the previous set of tools, using Visual Studio Team System has delivered substantial, measurable savings to OEM IT. The tables that follow show the savings for key activities in the development (coding) and testing processes. The tables express the savings as the percentages saved, based on actual time and/or money savings.

Table 1. Visual Studio Team System—Development Savings

Table 2. Visual Studio Team System—Testing savings

Conclusion

OEM IT has found that using Visual Studio Team System has greatly changed how its members work. People use one comprehensive, integrated system daily as part of their regular workflow. The projects generate much less e-mail traffic, and much less of a need exists for meetings. The team portal provides one-stop access to all project information, from the project documentation to the latest test results. Reports, updated automatically, are available through one-click access. Information in the OLAP data warehouse can be analyzed and explored in new ways.

The Microsoft Team Foundation service provides support for all components of Team Foundation Server—work item tracking, source control, version tracking, defect tracking, test case management, and build management. If OEM IT has a support issue, it has one support organization to deal with, not multiple support organizations that support multiple tools.

By implementing Visual Studio Team System in its development organization, the Microsoft OEM Division's IT Center of Excellence has greatly improved communications in development projects, improved the job satisfaction of its personnel, enabled project managers and senior management to better monitor and troubleshoot the projects, and produced substantial and measurable savings.

For More Information

For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada information Centre at (800) 563-9048. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information through the World Wide Web, go to:

http://www.microsoft.com

http://www.microsoft.com/technet/itshowcase