HPC and UFOs Explained! 

I had several discussions about High Performance Computing over the past week or so with people in different job roles.
Most of them wondered: “What is HPC for and where does Microsoft fit into it?”. I thought I’d spend some time trying to answer it. Here we go:    

HPC is for building UFOs

Now that Iʼve got your attention, let me try and explain: I would like to take a field of application - namely aeronautics - and take you through the design process that engineers in that field follow. Iʼll then try and explain where Microsoft technologies fit in that process. I am no expert in aeronautics - itʼs just a personal interest of mine; so, if you find any mistakes or just think I have smoked one too many, feel free to comment!

 
One of the hot topics in aeronautical engineering is how to overcome the design limitations of traditional airplanes. If you think about it, most commercial planes today look alike: a big pipe with arrow-shaped wings attached to it. This design is reaching its scalability limits: it is very difficult to make bigger planes that carry more people over longer distances, consuming less fuel than the current models. The recent problems and delays with both the Airbus A380, A350 and the Boeing 787 are just an example of this. One of the solutions being studied is a so called blended wing-body (BWB) aircraft.
Hereʼs an example of what Iʼm talking about:

Figure 1: An example of Blended Wing-Body aircraft.

Figure 2: A Manta ray, Natureʼs blended wing-body design

Aircraft designers typically use a variety of CAD / CAE programs to study the aircraft geometry on their workstations. A good example is  IBM's CATIA.

The geometry thus generated is then discretized for finite-element analysis. This step is often called pre-processing or mesh generation. Material properties and boundary conditions (loads, constraints) are associated to the mesh elements. A variety of mathematical models are produced to study structural stress, pressure distribution, heat distribution, lift & drag, etc...

Figure 3: Example of finite-element meshes.

Software like Ansys is often used for mesh generation. Several engineering outfits write their own mesh generators as well. These packages may take advantage of parallel computing.

The models thus created are then computed by separate packages (or modules within the same commercial package) called solvers or processors. These applications definitely benefit from parallel computing and are often written for high-performance computing clusters. To put it simply, the meshes are partitioned amongst computing nodes. Each of those solves equations on the elements that pertain to it, then communicates with the neighboring nodes over low-latency links to pass boundary results. Once every node has finished, we have an overall picture of the approximated solution. Several solvers are available in the public domain. Again, part of the “secret sauce” of engineering companies is in their own solvers, their algorithms, speed and precision.

The last step of the simulation is visualization or post-processing. The results of the computation are collated and displayed in a human-intelligible form. Here's an example of it:

Figure 4: Pressure distribution over a BWB glider

The visualization step is as important as the computation itself, because it is on the visualized data that design decisions are made. The visualization process itself may benefit from parallel processing, depending largely on the quantity of data at hand.
Figure 5 summarizes the process.

Figure 5: FEM Process

What has Microsoft got to do with it?
Well, some answers are evident: We provide the operating system where the geometry designand visualization will most likely occur.

Our platform, though, can do much more than that.

First of all, the data-handling requirements of such process are huge. Not only the file sizes involved are typically in order of gigabytes, but also those files do not mean much unless the design and simulation parameters that were used are associated to them.
Here's an immediate opportunity to build a data and metadata repository centered around SQL Server. Engineers must be able to retrieve past simulation data to work on statistical models. Regulators impose that design documentation be stored for the life of the aircraft, just in case something goes wrong and decisions must be re-visited and corrected.

Secondly, the workflow that I have presented is rather simplistic and is part of a much more complex design process. Here's an example:

Figure 6: Structural, Aerodynamic and Aeroelastic Design Concept Model

Microsoft's Workflow Foundation (WF) offers the building blocks for the application logic that automates such complex workflows. It also exposes several services, like persistence and tracking, that are extremely useful to manage the complexity of such processes, making interaction with them much easier. With Visual Studio one can construct such application logic workflows and then host them as services on Windows Server. The communication amongst those services can be handled by Windows
Communication Foundation (WCF).

The compute cluster per se can also be conceived as a “computing” service exposed to the application logic via a WCF interface. In fact, thatʼs what weʼre building into HPC Server 2008.

Thirdly, HPC Server 2008 offers a computing platform for your solvers that is easy to integrate in your engineering workflows, thanks to the variety of interfaces it exposes. Its management can be mostly demanded to a traditional IT organization, thus leaving the engineers free to focus on the design. It is a relatively small and conce