Estimate performance and capacity requirements for Visio Services in SharePoint Server 2010

 

Applies to: SharePoint Server 2010

Summary: Test how long it takes to render a data-connected Visio drawing and how many drawings can be processed per second in a given scenario.

This article describes the effects of using Visio Services in Microsoft SharePoint Server 2010 on topologies that are running SharePoint Server 2010.

In this article:

  • Test farm characteristics

  • Test results

  • Recommendations

By using Visio Services, users can view Microsoft Visio Web drawings in SharePoint Server 2010. When Web drawings are connected to external data sources, Visio Services makes it possible for users to update the appearance of Web drawings, based on underlying data changes. For more information about Visio Services, see Visio Services overview (SharePoint Server 2010).

This article discusses the effect of topology on the overall service latency (how long it takes to render a drawing) and throughput (how many drawings can be processed in a second) when a typical mix of drawings is rendered. The test outlined in this article measures performance under recommended user load and maximum user load on an instance of Visio Services that is configured with default settings.

With this information, you can better scale deployments based on latency and throughput requirements. You can either scale up by increasing the capacity of the existing servers or scale out by adding additional servers to the topology. It is important to be aware that the specific capacity and performance figures that are presented in this article differ from the figures in real-world environments. The figures presented are intended to provide a starting point for the design of an appropriately scaled environment. After you complete the initial system design, test the configuration to determine whether the system supports the factors in your environment.

For general information about how to plan and run capacity planning for SharePoint Server 2010, see Performance and capacity management (SharePoint Server 2010).

Test farm characteristics

This section describes the dataset, workloads, hardware settings, topology, and test definitions that were used during the performance and capacity testing of Visio Services.

Dataset

Visio Services capacity and performance depends on, among other factors, the makeup of the Web drawings that are hosted on the service. File-related factors that affect performance are listed in the following table.

Factor Qualitative effect on performance

Drawing disk size

Larger files sizes increase latency on the SharePoint network.

Drawing complexity

Drawings that contain many complex shapes increase rendering latency and resource usage on the application servers.

Drawing data-connectivity

Visio Web drawings can be either static or data-connected.

For static drawings, Visio Services loads drawings and renders directly to the browser.

For data-connected drawings, Visio Services performs the additional step of polling the underlying data sources for fresh data and updating the drawing before rendering it to the browser. This additional step increases latency and resource usage on application servers.

To simulate the effect of these factors, files were categorized as shown in the following table.

  Small Medium Large

Number of shapes

20

50

100

Typical file size

300 KB

600 KB

900 KB

If drawings were data-connected, these additional categories were used.

Small Medium Large

Number of refreshable elements

10

120

180

Number of rows imported

10

40

80

Percent of data change

50%

50%

50%

The different file categories were then assembled into a test dataset by using the distribution shown in the following table.

Types of file Percentage

Small static

49

Small data-connected

21

Medium static

14

Medium data-connected

6

Large static

7

Large data-connected

3

Note

Data-connected files were connected to a Microsoft SQL Server data source by using Secure Store Service Authentication. This article does not cover the performance effect of drawings that are connected to other external data sources.

Workload

The following test procedure was used for each performance scenario. Note that this test assumes a farm that is dedicated to Visio Services (no other tests running on SharePoint Server) and that only one user is using the service.

Test ID Test name Test description

#1

Rendering a Typical Mix of Visio Web Drawings

  1. From the browser, a simulated user requests Visio Services to render a drawing.

  2. Visio Services renders the drawing by using the Portable Network Graphics (PNG) output format and returns it to the user via the browser. If the drawing is data-connected, the drawing is refreshed before it is rendered.

  3. Visio Services caches the drawing to reduce latency the next time that it is requested.

  4. Steps 1, 2, and 3 are repeated with files randomly selected from the Dataset without any think time between requests.

Green and red zone definitions

For each topology configuration a green zone and red zone user load was determined before throughput tests were run. These configurations are defined in the following table.

Configuration Definition

Recommended (green zone)

The user load at which the test being run consumes approximately half of the bottlenecking resource. In the case of Visio Services, this is the CPU usage on the front-end Web server (WFE).

You should expect the green zone throughput to be sustained for long periods of time in real-world deployments.

Maximum (red zone)

The user load at which maximum throughput was reached for a topology while latency was at minimum. After this point, throughput typically remained stable and latency increased.

You should expect the red zone throughput to be tolerated for a short time on a farm, but it should be avoided.

Hardware settings and topology

Lab hardware

To provide trend information, several farm configurations were used for testing with increasing computing capacity. Farm configurations ranged from two to five Web servers plus a single database server that was running SQL Server 2008. Testing was performed by using one client computer that generated all requests. All Web server computers and the database servers were 64-bit.

The following table lists the specific hardware that was used for testing.

DELL PE 2950 DELL PE 2950 DELL PE R900

Role

WFE

Application server

SQL Server-based server

Processor (CPU)

2pX4

(Xeon L5420@2.5GHz)

2pX4

(Xeon L5420@2.5GHz)

4pX4

(Xeon E7330@2.4GHz)

RAM in gigabytes (GB)

16

16

32

Operating system

Windows Server 2008 R2 Enterprise

Windows Server 2008 R2 Enterprise

Windows Server 2008 R2 Datacenter

Authentication

NTLM

NTLM

NTLM

Storage: Operating System

4x 146 GB, 10 K RPM, RAID 0

4x 146 GB, 10 K RPM, RAID 0

2x 146 GB, 15 K RPM, RAID 1

Storage: Backups

--

--

3x 300 GB, 15 K RPM, RAID 5

Storage: SQL Server data

--

--

9x 300 GB, 15 K RPM, RAID 5

Storage: SQL Server logs

--

--

6x 300 GB, 15 K RPM, RAID 5

Number of instances of SQL Server

0

0

1, SQL Server 2008 SP1 CU6

Number of network adapters

1

1

4

Network adapter speed

1 GB

1 GB

1 GB

Load balancer type

NLB

Not applicable

Not applicable

ULS logging level

Medium

Medium

Medium

Antivirus settings

Microsoft Forefront

Microsoft Forefront

Microsoft Forefront

Lab software

The following table lists the specific software that was installed on the lab computers during testing.

Software Description

Operating system

Windows Server 2008 R2 Enterprise version 6.1.7600

SQL Server version

SQL Server 2008 version 10.0.2531.0

IIS

Version 7.5.7600.16385

SharePoint Server

SharePoint Server 2010

Topology

The topologies shown in the following diagrams were used to detect scale-out performance trends.

1 WFE x 1 application server x 1 SQL Server data source (1x1x1)

Topology 1

2 WFE x 1 application servers x 1 SQL Server data source (2x1x1)

Topology 2

2 WFE x 2 application servers x 1 SQL Server data source (2x2x1)

Topology 3

3 WFE x 2 application servers x 1 SQL Server data source (3x2x1)

Topology 4

Test results

The following sections show the test results of Visio Services.

After a calibration run that is used to determine the green zone and red zone user loads, the Rendering a Typical Mix of Visio Web Drawings test was run repeatedly. Only the topology was changed, to show its progressive effect on farm performance. The requests per second (RPS) number that is reported in this article is the average RPS of a constant user load test.

Note

All the tests reported on in this article were conducted without think time, which is a natural delay between consecutive operations. In a real-world environment, each operation is followed by a delay as the user performs the next step in the task. By contrast, in this testing, each operation was immediately followed by the next operation, which resulted in a continual load on the farm. This load introduced database contention and other factors that can adversely affect performance.

For information about bottlenecks in Visio Services, see the Common bottlenecks and their causes section later in this article.

Overall scale

The following table summarizes the effect of adding front-end Web servers and application servers to the throughput of Visio Services.

  Recommended (green zone) requests per second Maximum (red zone) requests per second

1x1x1

97.5

126

2x1x1

172.5

192

2x2x1

195

218

3x2x1

242

266

The following graph shows that the addition of front-end Web servers and application servers increases both green zone and red zone throughput linearly. By slope comparison, the graph of these values also shows that the addition of a front-end Web server has a larger positive effect on the throughput than the addition of application servers. This indicates that front-end Web servers can be a bottleneck for Visio Services deployments.

Throughput vs. Topology

Throughput v. topology

Hardware cost per transaction

The following table highlights the hardware cost of running the Rendering a Typical Mix of Visio Web Drawings test across topologies in the green zone.

Scorecard dashboard 1x1x1 2x1x1 2x2x1 3x2x1

Average RPS

(requests/sec)

97.5

172.5

195

242

Average WFE server CPU resources

(%)

59.5

54.5

59.8

50.2

Average application server CPU resources

(%)

16.4

27.9

17.25

21.5

Failure rate

0.006

0.006

0.01

0.02

90% QoS availability

1

1

1

1

25th percentile latency

(seconds)

0.05

0.05

0.05

0.06

Percent time in garbage collector

WFE: 3.55

App: 6.6x10-7

WFE: 3.11

App: 0.00014

WFE: 3.27

App: 0.68

WFE: 3.13

App: 0.125

Number of WFE server crashes

0

0

0

0

Average memory used

(bytes)

WFE: 210,728,784

App: 210,172,592

WFE: 664,374,336

App: 810,444,288

WFE: 203,343,584

App: 229,627,536

WFE: 199,905,600

App: 259,536,336

Maximum memory used

(bytes)

WFE: 212,664,320

App: 211,578,880

WFE: 719,638,528

App: 1,309,839,360

WFE: 204,537,856

App: 229,969,920

WFE: 200,081,408

App: 262,713,344

The following table highlights the hardware cost of running the Rendering a Typical Mix of Visio Web Drawings test across topologies in the red zone.

Scorecard dashboard 1x1x1 2x1x1 2x2x1 3x2x1

Average RPS

(requests/sec)

124

190

216

264

Average WFE CPU resource

(%)

73.8

64

71.05

59.9

Average application server CPU resources

(%)

18.9

31

18.35

23.0

Failure rate

0.006

0.009

0.009

0.01

90% QoS availability

1

1

1

1

25th percentile latency

(seconds)

0.06

0.06

0.07

0.06

Percent time in garbage collector

WFE: 0.000036

App: 0.000074

WFE: 0.00036

App: 0.00014

WFE: 1.54

App: 0.805

WFE: 1.15

App: 0.4

Number of WFE crashes

0

0

0

0

Average memory used

(bytes)

WFE: 631,852,288

App: 820,075,648

WFE: 748,467,200

App: 884,640,512

WFE: 659,872,256

App: 511,670,960

WFE: 730,737,301

App: 827,111,104

Maximum memory used

(bytes)

WFE: 708,333,568

App: 1,446,760,448

WFE: 787,783,680

App: 1,350,569,984

WFE: 709,833,600

App: 1,319,833,600

WFE: 1,070,150,997

App: 1,450,207,232

Recommendations

This section provides general performance and capacity recommendations. Use these recommendations to determine which hardware, topology, and configuration settings best meet your particular capacity and performance needs.

Hardware recommendations

The basic hardware requirements for Visio Services are the same as those for SharePoint Server 2010, which can be found in the following article: Hardware and software requirements (SharePoint Server 2010).

Scaled-up and scaled-out topologies

To increase the capacity and performance of one of the starting-point topologies, you have two options. You can either scale up by increasing the capacity of the existing servers or scale out by adding additional servers to the topology.

As a rule, Visio Services is especially sensitive to scaling out. When planning to scale out, use the following general rules:

  1. When you, use the configurations described earlier, favor increasing WFE servers to increasing application servers. If your computers have performance specifications that are comparable to the specifications in this article, a ratio of three WFE servers to one application server is recommended.

  2. Remember that scaling out enables both an increase in throughput and an increase in user load that Visio Services can serve. Use the following graph to gauge the expected throughput on the topology at a particular user load. Notice that throughput tapers off at a certain user load, after which requests are served with increase latency.

Throughput vs. User Load

Throughput v. user load

Settings optimizations

One way to control the performance characteristics of Visio Services is to change the values of its performance-related service settings. The following table discusses the qualitative effect on farm performance of varying these settings.

Setting location Parameter Description Qualitative effect on performance

Central Administration

Maximum Web Drawing Size

The maximum size in MB of a Web drawing that can be rendered.

A larger size limit can lead to reduced throughput and increased latency.

A smaller limit can prevent more complex Web drawings from being rendered but has the opposite effect on performance.

Minimum Cache Age

The minimum number of minutes that a Web drawing is cached in memory.

This value is per user per drawing. The interval begins when a user views a Web drawing. That user cannot refresh that Web drawing until the interval expires. The interval begins for other users when they first view the Web drawing.

Smaller values allow for more frequent data refresh operations for users but increase CPU and memory usage on application servers, which reduces throughput and increases latency.

A larger value increases the refresh window but has the opposite effect on performance metrics.

Maximum Cache Age

The number of minutes after which cached Web drawings are purged.

Larger values decrease file I/O and CPU usage on application servers but increase memory usage on the server. Large values reduce latency for drawings that are rendered often.

A smaller value has the opposite effect on performance.

Maximum Recalc Duration

The number of seconds before the data refresh operations time out. This applies only to data-connected Web drawings.

Longer timeouts allow for more complex data-connected Web drawings to be recalculated but use more processing power, reduce throughput, and increase latency.

A shorter time-out decreases the complexity of the drawings that can be rendered but has the opposite effect on performance.

Web Part

Force Raster Rendering

Forces the Visio Web Access Web Part to render the drawing as a PNG even though Microsoft Silverlight might be installed.

Choosing to render all drawings by using a raster format reduces drawing visual fidelity but increases throughput slightly.

Choosing to render all drawings in XAML increases visual fidelity but reduces throughput slightly.

Common bottlenecks and their causes

The following sections discuss methods for improving farm performance by optimizing service settings and working around common system bottlenecks.

During performance testing, the following bottleneck was revealed. A bottleneck is a condition in which the capacity of a particular constituent of a farm is reached. This causes a plateau or decrease in farm throughput.

Bottleneck Cause Resolution

WFE CPU utilization

Because of caching on the application server layer, the throughput of the Visio Services application servers is larger than the front-end Web servers. This causes the WFE layer to be the bottleneck of the system.

Add more front-end Web servers to help reduce the effect of the WFE bottleneck on the overall throughput of Visio Services.

Performance monitoring

To help you determine when you have to scale up or scale out a Visio Services deployment, you can use performance counters to monitor its health. In addition to the performance counters that are included with Windows Server and that measure overall server health, you can use the following list of performance counters to gain a deeper understanding of the specific performance behaviors of Visio Services.

Visio Services performance counters

The following table lists performance counters that monitor key Visio Services application server metrics. All these counters are in the Visio Server: Visio Graphics Service performance counter category.

Logical category Counter Measurement

Aggregate counters

Requests Received per Second

Count of requests received per second.

 

Request Processing Time

Average processing time of a rendering request in milliseconds.

Drawing rendering pipeline detail

File Retrieval and Parsing Time

Average time, in milliseconds, to retrieve a drawing from the content database and to parse it.

 

Server Data Refresh Time

Average time, in milliseconds, to retrieve all external data for a data-connected drawing.

 

Server Data Binding Time

Average time, in milliseconds, to update a data-connected drawing.

 

Text Data Binding Time

Average time, in milliseconds, to update text in a data-connected drawing.

 

Server Rasterization Time

Average processing, in milliseconds, to create a PNG representation of a drawing.

Visio Web Access performance counters

The following table lists performance counters that monitor key Visio Web Access server (front-end Web server) metrics. All these counters are in the Visio Server: Visio Web Access performance counter category.

Logical category Counter Measurement

Aggregate counters

Request Processing Time

Average time, in milliseconds, to process a drawing render request from arrival to delivery.

 

Requests Received per Second

Count of requests received per second.

 

Requests with Errors per Second

Average number of requests that are returned with errors per second.

 

Average Content Initial Transmission Time

Average transmission time, in milliseconds, of initial HTML plus Java scripts to client computer.

See Also

Concepts

Configure Visio Services for a BI test environment