Power BI delivers custom data visualization to business teams

Article

May 2016

As data gets bigger and more complex, visualization becomes more important. Microsoft IT partnered with the Microsoft Power BI product development group and the larger Power BI community to bring the energy of custom visualizations to business users at Microsoft.

The Analog team at Microsoft uses custom Power BI data visualizations to improve productivity and effectiveness—letting the team quickly generate a wide variety of unique and nuanced visualizations for audiences ranging from engineers to executives.

Data drives decisions

Data is increasingly complex, powerful, and challenging to harness. You probably have multiple audiences in your enterprise, each of which requires very specific information. Data needs to be presented for both general understanding and to inform specific business and technical decisions. Nuance is required within summary data sets, to harvest meaningful outlier information.

Historically, business teams have relied upon customized and proprietary reporting systems and data science specialists. Not surprisingly, these custom reporting systems are expensive and time-consuming. Changes usually require formal software development cycles that last months. In this mode, it is hard for business to innovate, or even keep up.

We were no stranger to this scenario, and did not want to continue to depend on costly, slow, and resource-intensive custom reporting systems. We wanted to free our people’s valuable time to solve business problems, rather than building and maintaining custom reporting platforms.

We were chartered with bringing self-service BI tools to users internal to Microsoft. We used Power BI as a reporting platform so that internal teams could quickly analyze their own business data and produce their own reports and dashboards. At the same time, our teams benefited from the power of the larger community that was eager to use Power BI custom visualizations.

Bringing the power of data visualization to business users

We saw the potential of Power BI as a powerful and extensible business analytics platform. Analysts can easily build and publish rich Power BI reports that unlock new business data insights. Business users can easily discover deeper understanding of the data that matters to them in their job. In sum, users can quickly generate their own powerful analytics. Power BI tools are richer, so that the reports generated are also richer and more full-featured. Rapid iteration is supported—users can quickly make changes themselves. Three-month development cycles are a thing of the past.

Custom visualizations and community

Using the strength of community, we saw a way to partner to make Power BI even more powerful. Power BI was enriched with the excitement and impact of custom visuals. Because data visualization is now such a core user experience, Microsoft wanted to greatly expand the visual experience beyond proprietary choices.

Microsoft created an open, standards-based Power BI custom visuals gallery. GitHub hosted the source code, available for anyone to freely use and contribute to if desired. Microsoft develops on the same GitHub repository, so anyone can keep current on the Microsoft codebase.

Users can interact with the gallery in different ways. One option is to quickly access and download visuals from the gallery. At publication time, there were over 40 visuals in the gallery available for download. The Power BI team works with the community to add a new custom visual to the gallery each week. Alternatively, users can build their own custom visuals by either starting from scratch or by using a visual from the gallery as a template.

The gallery’s visuals are built on common Internet standards and use open-source libraries like D3.js. Using Power BI developer tools, the Power BI user community can code, test, and package new visuals to add to the gallery. To expand the gallery, Microsoft solicited ideas from the public through a competition. In four weeks, dozens of visuals were submitted, doubling the number of visuals available for anyone to download. The gallery continues to grow.

The available source code and visuals gallery means that Power BI can be extended with existing visuals, visuals from outside a company, or visuals built and customized to specifications. Business users can quickly build their own visuals. The public nature of the Power BI custom visuals creates new ways for business teams to operate, as they can now engage a vendor to build custom visuals.

Profile: The Analog team

The Microsoft Analog team makes computing more personal. It studies how humans interact with platforms and products that don’t involve a traditional keyboard, mouse, or monitor. Think HoloLens, the Windows Hello biometric security system, or Cortana, the Windows speech platform. The team is immersed in specialties such as display technologies and light science. Importantly, team members are not data scientists, and they don’t specialize in business analytics.

The team monitors the worldwide Windows Hello experience, where a user logs on to Windows 10 using facial recognition, a fingerprint, or iris recognition. Microsoft is keen to understand user behavior around these new ways to interact with a computer. Windows Hello data, surfaced through a Power BI data pipeline, helps the Analog team understand user behavior and product engagement. The data crosses a wide variety of geographies and markets.

The Analog team reports Windows Hello data to its executives twice a week, and the data is also used by engineers to address specific issues. Powerful custom visualizations let Analog engineers make more informed decisions based on how people are actually using products in the field. It also provides sophisticated data to executives on an ongoing basis to help guide business management decisions.

Power BI and custom visualizations let the Analog team make their business management and engineering team decisions more quickly. This, in turn, makes the entire team more productive, effective, and efficient. The following sections describe the Analog team’s business analytics evolution from a custom platform to Power BI.

Rich dashboards come at a cost

Before the transition to Power BI, business analytics reporting for the Analog team used proprietary dashboard websites built with ASP.net. The dashboards used technologies such as JavaScript, custom HTML, and D3.js. The Analog team invested significant resources—as many as five data science specialists—to maintain this rich data and reporting dashboard. The system was very powerful and provided deeply engaging visualizations and dashboards. However, it was also time-consuming and expensive to maintain.

Analysts and engineers wanted to dig deep into analytics data and act on it quickly, but their productivity was hindered. Data sets were not efficiently linked and cross-filtered. Any changes to reports or dashboards, even if minor, took weeks to implement, and required formal development cycles. The bottleneck meant that engineers ultimately stopped coming to the business analytics team to ask questions.

Powerful democratized data

Power BI was made immediately available to all Analog team members. This increased the number of people with access to powerful business analytics and reporting capabilities by a factor of six. Where there was once a subset of five data experts who could manage business analytics, now there are as many as 30. The bottleneck of queued dashboard requests and reporting changes simply no longer exists, and users can freely filter and manipulate their data sets to their own specifications. Team members can produce and quickly modify unique data visualizations that are discussed by the Analog leadership team twice a week. Custom dashboards can be developed in days, end to end.

Where the Analog team might run into limitations on reporting, creating custom visualizations allow the business to stretch those limitations. For example, if a particular data representation does not exist, the team can write the code and plugin that is required for that view, and then share it with the larger Power BI community. Before, a new data visualization took weeks to produce, and the result did not support robust filtering across data sets; this functionality is a click-and-drag function in Power BI.

One of the most powerful results of Power BI in the Analog team has been the level of engagement by engineers and engineering managers, and the quality of questions that are asked. “How many?” is now answered by Power BI users themselves. The closed loop of the self-service model means that problems can be addressed faster, because another data team does not have to be involved. Requests now dive deeply into the processing models behind the data. In other words, “How do I get these data points and structure them in a way that they can be queried?”

Below are two examples of how custom data visualizations have been created to serve both executive and engineering audiences in the Analog team.

Sankey bar chart. The Sankey bar chart creates a simplified representation of a visual decision flow process, showing both progression and drop-off points. When the Analog team is trying to associate a user action and the drop-off point, knowing exactly where to focus the effort to improve what’s being measured is critically important.

The Analog team tested a group of internal Microsoft users and studied the Windows Hello face detection and spoof detection algorithms on an experimental version of the software. They then compared the data with another group of internal Microsoft users, and their real-world use of Windows Hello.

Figure 1. Sankey bar chart click on image to view larger version

The chart helps the team make better product design decisions, and lets them tune facial detection and spoof detection algorithms, before being released to internal users for feedback. The Analog team can easily modify the chart. For example, sometimes filters need to be switched, or two filtered charts need to be viewed side by side. These help us see the impact of changes across the real-world population.

Box and whisker plot. The box and whisker plot shows outliers, clusters of data points, and varying volumes of data points between series—all potentially compelling information that can get lost within summary statistics. It allows you to break summary data down and see a number of different data points or outliers within a data series, like mean, median, and top and bottom 5th percentiles.

Here’s a practical application. Let’s say the Analog team wants to analyze summary statistics on Windows Hello sign-on times. For a group of 1,000 people, the average sign-on time might be two seconds. That number meets the goal, but perhaps ten of those users took longer than ten seconds to sign on using Hello. Engineers want to follow up on that data, which might reveal a problem for a significant number of users. If you were only looking at the average number, those ten users might not stand out.

Figure 2. Box and whisker plot click on image to view larger version

In another example, shown in Figure 2, the Analog team uses the box and whisker plot to help determine experience, code, and feature quality. They are measuring the latency of a set of operations in the Windows Hello feature, and comparing the algorithm in an A/B test.

In this case, the visualization really shines through. A standard bar chart expressing the average would show 859 milliseconds and 871 milliseconds. This would tell you that the new algorithm is, on average, slightly faster. The box and whisker plot instead shows that a group of devices is taking over 1,251 milliseconds, and are outliers. This indicates that there may be a bug or an issue with the algorithm under certain conditions that needs to be investigated before committing to the new algorithm change.

Looking forward with descriptive analytics

Today, the Analog team is using Power BI for descriptive analytics. Data drives business direction and investment decisions, such as the priority of new features and bug fixes, and ways to increase user satisfaction.

Going forward, the team wants to use Power BI to generate predictive analytics that will shape the direction of their business. Predictive analytics uses a variety of statistical, modeling, data mining, and machine learning techniques to study recent and historical data, allowing analysts to forecast what might happen in the future.

Evolving roles of data scientists

The role of data scientists on the Analog team continues to evolve. Because Power BI is largely self-service, engineering managers and group leaders have business analytics power in their own hands. This frees data scientists for other activities.

Data scientists can now address higher-level data quality issues and forward-looking investments, such as predictive analytics. Data scientists can also help more engineering and feature teams with their work.

Conclusion

Power BI and custom visualizations fuel our self-service BI culture. Power BI user community involvement continues to drive innovation in data visualizations. A large Power BI visuals gallery was quickly built on open-source technologies. Because custom data visualizations are built on an open-source platform, it creates new vendor engagement possibilities for how to build and manage your own business analytics system.

In the Analog team at Microsoft, custom visuals have increased the speed of the decision-making process, saving both time and money. Custom visualizations have brought nuanced views of actionable data that can be used across the organization. The self-service nature of Power BI has created faster iteration loops and deeper dives into data.

For more information

Power BI

Power BI visuals gallery

GitHub Power BI visuals

Power BI Developer Center

Microsoft IT

https://www.Microsoft.com/ITShowcase

© 2016 Microsoft Corporation. All rights reserved. Microsoft and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners. This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.