Chapter 2: Developing Phase: Process Milestones and Technology Considerations

Article
12/05/2007

This chapter introduces and discusses activities in the Microsoft® Solutions Framework (MSF) Developing Phase as they are performed in a migration project. The chapter addresses every code component of the solution and provides instructions on how to apply code changes for migrating to the Interix environment. This chapter looks at adapting and extending the components to meet the project requirements in the Interix environment with minimal changes.

Goals for the Developing Phase

For a migration project, the Developing Phase is the time when you build the solution components, code, and infrastructure, as well as prepare the documentation. Typically, to achieve this primary goal, you must modify existing code in a way that enables it to work within the new environment—in this case, Microsoft Windows® Services for UNIX 3.5. Generally, when new code is written, some aspect of the original component remains unchanged—for example, exposed application programming interfaces (APIs) or specific component behavior. In this context, both modifying the existing code and developing new code are considered to be migration activities. For this reason, when MSF is applied to a migration project, the MSF Developing Phase is equivalent to the actual act of migrating.

During this phase, all roles are active in building and testing the deliverables. The team continues to identify all risks throughout the phase and to address new risks as they emerge.

The phase formally ends with the Scope Complete Milestone. At this major milestone, the team gains formal approval from the sponsor and key stakeholders that all solution elements are built and that the solution features and functionality are complete according to the functional specifications agreed upon during the Planning Phase.

Figure 2.1 shows the Developing Phase, its associated interim milestones, and its major milestone—Scope Complete.

Figure 2.1. The Developing Phase in the MSF Process Model

Figure 2.1. The Developing Phase in the MSF Process Model

Table 2.1 describes the major tasks and deliverables associated with the Developing Phase and lists which roles are responsible for them.

Table 2.1. Major Tasks and Deliverables

Major Tasks and Deliverables	Owners
Starting the development cycle The team begins the development cycle by verifying that all tasks identified during the Envisioning and Planning Phases have been completed. Their main focus during this phase is to identify and analyze any risks that might occur during the Developing Phase.	Development team
Building a proof of concept Before development, the team finally verifies the concepts from the designs within an environment that mirrors production as closely as possible.	Development team
Developing the solution components The team develops the solution using the core components and extends them to the specific needs of the solution. The team also develops and conducts unit functional tests to ensure that individual features perform according to the specifications.	Development team, User Experience team, Test team
Developing the testing tools and tests The team develops the testing infrastructure and populates it with test cases. This ensures that the entire solution performs according to specification. This solution test suite typically incorporates, as a subset, the individual feature tests used by developers to build the solution components.	Test team
Building the solution A series of daily or frequent builds culminate with major internal builds and identification of points at which the development team will deliver key features of the solution. These builds are subjected to all or part of the entire project test suite as a way of tracking the overall progress of the solution and the solution test suite.	Development team, Test team
Closing the Developing Phase The team completes all features, delivers the code and documentation, and considers the solution complete, thus entering the approval process for the Scope Complete Milestone.	Project team

Note Refer to the UNIX Migration Project Guide (UMPG) for an overview of MSF, general information on the processes that belong to each phase, and additional information about the team roles responsible for the processes. The UMPG is meant to be used in conjunction with the technical and solution-specific information in this guide.

Starting the Development Cycle

This section focuses on identifying and addressing the risks in the Developing Phase. The Developing Phase can be the most volatile and trying, yet the most stimulating and challenging, part of any UNIX migration project. Major issues and risks become evident soon after this phase begins. For example, team members might later realize that Microsoft Windows Services for UNIX 3.5 does not support a particular function that their code is using. This can be categorized as a risk because replacement code might need to be written at that stage. Resolution of such issues is the distinguishing factor that determines if schedules will change, whether funding is sufficient, and if the project will be successful.

Design and technology choices involving various techniques and tools were discussed briefly as part of the Envisioning Phase and the Planning Phase in Volume 1: Plan of this guide. These same choices are discussed in detail in the chapters of this volume. If any item on the related task lists from the Envisioning Phase and the Planning Phase is not completely satisfied, it could present itself as a risk during the Developing Phase.

The following actions can help mitigate the risks:

Prepare a requirements specification document, which details the scope of the migration project and the design and architecture to be followed.
Perform an impact analysis of the changes and get customer sign-off on the project-execution approach for the requested changes.
Establish the existence of and procure licenses for source code to any necessary third-party libraries.

Implementing the preceding actions is easier if the risks are identified and mitigation plans are formulated and evaluated well ahead of time. Risk mitigation, as part of the risk management process, can be used to keep a project on track in adverse situations.

Note Information on Microsoft Office Solution Accelerator for Six Sigma is available at https://www.microsoft.com/office/solutions/accelerators/sixsigma/default.mspx.

Building a Proof of Concept

Typically, the proof of concept is a continuation of some initial pre-development work (the preliminary proof of concept) that occurred during the Planning Phase. Creating a proof of concept helps you to implement risk management. It also helps assess the worthiness and ease of the migration process through testing key elements of the solution in a nonproduction simulation of the proposed operational environment. Your development team tries to compile the source code and obtains a good idea of how hard the port will be after a few rounds of quick fixes and recompiles. The team guides operations staff and users through the solution to validate their requirements. Also, during such a review process, developers might discover design flaws and bugs in the original application being ported that need to be addressed.

There may be some solution code or documentation that carries through to the eventual solution-development deliverables. However, the proof of concept is not meant to be production-ready. The proof of concept is considered as throwaway development that gives the team a final chance to verify functional specification content and address any other issues before fully moving into development. The proof of concept also helps in obtaining metrics for projecting the developer effort required to port the overall application environment.

Interim Milestone: Proof of Concept Complete

Reaching this interim milestone marks the point where the team is fully moving from conceptual validation to building the solution architecture and components. The proof of concept should result in at least one prototype interoperation scenario being built before production development begins. There can be a number of strategies to accommodate interoperability:

Create a Windows-only environment. If the project is low risk, you may be able to run the Windows and UNIX environments in parallel just for the time it takes for users to make the transition to the migrated application.
Create a separate Windows environment alongside the UNIX environment. This approach is suitable for short-term parallel running or when the application has both UNIX and Windows modules.
Create an integrated cross-platform environment. This approach is ideal when the two environments have to coexist for a long time.

Developing the Solution Components

Developing the solution components identifies the suitable development environment for migrating applications. You can use the development environment for building and debugging the applications. Using the development environment, you can make changes to the UNIX code to run in the Interix environment. Because there are certain differences between UNIX and Interix, you must modify the UNIX code for it to work in the Interix environment. This volume addresses these potential coding differences by looking at the solution from various categories, which are described in detail in subsequent chapters of this volume. For each of these categories, this volume:

Describes the coding differences between UNIX and Interix.
Outlines options for converting the code.
Illustrates the options with source code examples.

These categories are:

Process management.
Thread management.
Memory management.
File management.
Infrastructure services.
Migrating user interface.
Functions to change for Interix.

The following subsection describes the purpose of the development environment and how to use the development environment for building and debugging applications.

Development Environment

The development environment is the environment in which the user develops and builds the solution. The development environment provides the necessary compiler, linker, libraries, and reference objects. In some cases, the integrated development environment (IDE) is also provided. Setting up the development environment is explained in “Setting Up the Development Environment” in Chapter 4, “Planning: Setting up the Development and Test Environments” of Volume 1. This section explains how to use the development environment after setting it up in the Planning Phase.

This section also discusses the important components of the development environment for Interix applications and using Interix for building and debugging an application’s environment.

Using Interix

You can use the Interix subsystem in conjunction with the Windows Platform SDK or Microsoft Visual Studio®. The Windows Platform SDK, which provides a front end for Microsoft Visual C++®, offers the benefits of the native compiler for Windows, while retaining a UNIX development environment. But the Interix environment also offers the benefits of the GNU gcc and g++ compilers.

Header Files Included in Interix

Table 2.2 lists the header files that are included with Interix in the /usr/include directory and also lists the header files found in the Linux and Solaris variants. Differences are indicated with an X. By identifying the differences between Interix and other environments, you can replace the unsupported features with third-party, or user-defined, libraries when migrating the applications to the Interix environment.

Table 2.2. Interix Header Files in /usr/include , with Linux and Solaris Variants

Header	Interix	Linux	Solaris
alloca.h
ar.h
assert.h
blf.h
cast.h
cpio.h
ctype.h
curses.h
db.h
dirent.h
dlfcn.h
err.h			sys/err.h
errno.h
eti.h
excpt.h
fcntl.h
features.h
float.h
fnmatch.h
form.h
fts.h
ftw.h
glob.h
grp.h
histedit.h
iconv.h
langinfo.h
libgen.h
limits.h
locale.h
malloc.h
math.h
md4.h
md5.h
memory.h
menu.h
monetary.h
mpool.h
ndbm.h
netdb.h
new.h
nl_types.h
nl_types_private.h
ohash.h
panel.h
paths.h
poll.h
pthread.h
pty.h
pwcache.h
pwd.h
regex.h
rmd160.h
sched.h
search.h
semaphore.h
setjmp.h
sha1.h
signal.h
skipjack.h
stdarg.h
stddef.h
stdio.h
stdlib.h
string.h
strings.h
stropts.h
syslog.h
tar.h
term.h
termios.h
time.h
tzfile.h
ucontext.h
ulimit.h
unctrl.h
unistd.h
utime.h
utmpx.h
va_list.h
varargs.h
vis.h
wait.h
wchar.h
wctype.h
xti.h

Table 2.3 lists the header files that are included with Interix in the /usr/include/sys directory and the header files found in the Linux and Solaris variants.

Table 2.3. Interix Header Files in /usr/include/sys , with Linux and Solaris Variants

Header	Interix	Linux	Solaris
cdefs.h
dir.h
endian.h
errno.h
fault.h
fcntl.h
file.h
fsid.h
ioctl.h
ipc.h
mkdev.h
mman.h
msg.h
param.h
procfs.h
queue.h
reg.h
regset.h
resource.h
select.h
sem.h
shm.h
siginfo.h
signal.h
socket.h
stat.h
statvfs.h
stropts.h
syscall.h
syslog.h
termios.h
time.h
time_impl.h
timeb.h
times.h
types.h
ucontext.h
uio.h
un.h
unistd.h
user.h
utsname.h
wait.h

Table 2.4 lists the header files that are included with Interix in the /usr/include/arpa directory and also lists the header files found in the Linux and Solaris variants.

Table 2.4. Interix Header Files in /usr/include/arpa , with Linux and Solaris Variants

Header	Interix	Linux	Solaris
ftp.h
inet.h
nameser.h
telnet.h
tftp.h

Table 2.5 lists the header files that are included with Interix in the /usr/include/netinet directory and also lists the header files found in the Linux and Solaris variants.

Table 2.5. Interix Header Files in /usr/include/netinet , with Linux and Solaris Variants

Header	Interix	Linux	Solaris
if_ether.h
if_ieee.h
igmp.h
igmp_var.h
ip.h
ip_info.h
mib_kern.h
udp.h
in.h
ip_icmp.h
ip_mroute.h
tcp.h
if_ieee.h
udp_var.h
igmp.h
in_systm.h
ip_igmp.h
ip_var.h
tcpip.h

Table 2.6 lists the header files that are included with Interix in the /usr/include/interix directory. These Interix-specific headers are useful for developers when they are looking for a particular functionality.

Note There are no man pages for the application programming interface (API) prototypes in the header files included in /usr/include/interix.

Table 2.6. Interix Header Files in /usr/include/interix

Header File	Description
registry.h	This contains the registry paths for Interix.
security.h	This has flags for the setuser API.
env.h	This has various environment variables values set.
interix.h	This contains routines to get information from the registry and to convert path names, specifically winpath2unix and unixpath2win.
ldsostartup.h	This contains details for ld.so library.
path_convert.h	This contains routines to convert path names.

Note The preceding tables do not list any header files that may exist on other platforms but not on Interix. However, header files present on systems other than Interix are often superseded by another header file or a set of header files. Moving to the header files present on Interix provides better portability across all UNIX platforms. Use the manual pages and the grep tool to find where the header file content is located.

Building the Application

Applications are built in the Interix environment in the same way they are built in the UNIX environment. Consider the following when building applications in the Interix environment:

The Interix SDK supports Visual C++ for compiling C programs. It does not support Visual C++ for compiling C++ programs. For C++ programs, the GNU g++ compiler is provided.
Compiler options and C language definitions may need a GNU gcc compiler or the cc/c89 tools. The cc/c89 compiler interface tools in turn invoke the Microsoft Visual C++ compiler cl.exe for compiling and Link.exe for linking. The gcc compiler of GNU should be used in these instances or the source code should be changed. (For example, "long long" should change to quad_t or some compiler options that are valid for gcc but not c89/cc, such as –x option for gcc.)
The version of make provided with the Interix SDK is based on the Berkeley Software Distribution (BSD) 4.4 make and supports all BSD features. It also conforms to Portable Operating System Interface (POSIX).2. The GNU make (gmake) tool can help when porting the makefiles of the application. The gmake tool is not included in the Interix SDK, but it can be downloaded at
https://www.interopsystems.com/tools/warehouse.htm.
To convert existing makefiles to run under the Interix subsystem, change macro definitions. Edit the value of $(CFLAGS) to use only flags supported by cc or c89.
The Interix SDK supports shared libraries or dynamically linked objects. Dynamic linking is supported by using standard calls, such as dlopen(). Dynamically linked applications and shared libraries can be created only by using gcc and other GNU compiler tools. The Interix GNU SDK should be installed to use the gcc compiler to create the shared libraries.
A typical problem when porting a UNIX application to Interix is that (on builds) gcc defaults to shared libraries in the Interix SDK. To force the compiler to link to the static libraries, use the compiler option –static. However, cc/c89/MSVC can only build static binaries.
Shared libraries should be linked to the libc.so library visible to them. It is usually incorrect to link shared libraries to libc.a.
The Interix SDK includes the liblock command, which locks a library to prevent the linker from using it. If any process attempts to use a locked library to link, the ld command reports a fatal error. There is no tool provided to unlock a library that has been locked by using liblock. However, locking is vulnerable and is not a high-security option. The syntax is as follows:
```
   % liblock lib.so
```

Although the gcc command is capable of handling these rules of linking, use extreme caution if you are invoking ld directly.

Note It is recommended that you make copies of the original libraries before using liblock.

Debugging the Application

The gdb debugger can be used to debug applications that are compiled using the gcc -g in the Interix environment. It cannot debug binaries that are created with the cc or c89. It provides a help command for the various operations. To see a list of help issues, type help at the gdb prompt. You can also refer to the man page for more details.

In addition to gdb, the Interix SDK includes pstat and truss tools to help in debugging programs. These tools are part of the base installation.

The pstat tool displays detailed information about a specified process.

Note If you have Administrator privileges (that is, if you have signed on as Administrator and are a member of the local Administrators group), you can view any process identifier (PID). If you do not have Administrator privileges, you can only view PIDs owned by your shell (that is, launched from your shell).

For example, this is the output for PID 10187 (a C shell process):

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

% pstat 10187
pid=10187 ppid=1 pgid=10187 sid=10187 state=3 Active
flags=40000022 execed pgrp asleep
signal trampoline = 0x77EA1F66,0x77EA1F8F nullapi =
0x77EA7A07,0x77EA7A4F
current syscall=sigsuspend()
IP=77f8224d SP=0088da1c BP=0088daf8 FL=00000246 AX=
00000000 BX=00000001
CX=00000000 DX=00000000 DI=0088eddc SI=0088da78
CS=0000001b DS=00000023 ES=00000023 FS=00000038 GS=
00000000 SS=00000023

Although it does not have the same functionality, the truss tool was inspired by the System V tool of the same name. The truss tool outputs information about system calls and signals.

For example, this output represents the system calls made on behalf of the cat tool:

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

% truss cat config.log
tracing pid 12491
null() null returned 0
open("config.log", 0x1) open returned 3
fstat(1, 0x1210650, 0x1200650) fstat ret: 0 dev: 0x0
ino: 0x00000000
read(3, 0x8228E0, 512) read returned 127 0x7F
write(1, 0x8228E0, 127) This file contains any messages
produced by compilers
while running configure, to aid debugging if configure
makes a mistake.
write returned 127 0x7F
read(3, 0x8228E0, 512) read returned 0
close(3) close returned 0
close(1) close returned 0
exit(0) process exited with status 0

Developing the Testing Tools and Tests

After developing the solution components, you need to test the code changes made as part of the development. The testing process helps identify and address potential issues prior to deployment. Testing spans the Developing and the Stabilizing Phases. It starts when you begin developing the solution and ends in the Stabilizing Phase, when the test team certifies that the solution components address the schedule and quality goals in the project plan. This also involves using the automated test tools and test scripts.

Figure 2.2 illustrates the positioning of testing activities within the phases of the MSF Process Model.

Figure 2.2. MSF Process Model: Testing throughout the Developing and Stabilizing Phases

This section discusses the unit testing activity that needs to be performed during the Developing Phase. The other necessary testing activities are discussed in Chapter 8, “Deployment Considerations and Testing Activities” and Chapter 9, “Stabilizing Phase” of this volume.

Testing in the Developing Phase is an integral part of the build cycle. It is not a stand-alone activity and is performed parallel with development. For instance, when building software, the development team designs, documents, and writes the code. Meanwhile, the test team designs and documents test specifications and test cases, writes automated scripts, and runs acceptance tests on components submitted for a formal round of testing. The test team assesses the solution, makes a report on its overall quality and feature completeness, and certifies that the solution features, functions, and components address the project goals.

Testing in migration projects involving infrastructure services is focused on finding discrepancies between the behavior of the original application, as seen by its clients, and that of the newly migrated application. All discrepancies must be investigated and fixed. It is better to add any new functionality to a migrated application—or new capabilities to a migrated service—in a separate project, initiated after migration is complete.

In the Developing Phase, testing entails a code review of the application, followed by unit testing.

Unit Testing

Unit testing is the process of verifying if a specific unit (which can be a class, a program, or a specific functionality) of the code is working according to its functional specifications. Unit testing also helps determine whether the specific unit will be capable of interacting with the other units as defined in the functional specifications.

Unit testing in a migration project is the process of finding discrepancies between the functionality and output of individual units in the Windows application and the original UNIX application. However, this might not always be the case; in some cases the design in Windows may differ from the UNIX design, thereby identifying units that are different from the UNIX units. Basic smoke testing, boundary conditions testing, and error testing are done based on the functional specification of the unit.

The test cases for unit testing include constraints on the inputs and outputs (pre-conditions and post-conditions), the state of the object (in case of a class), the interactions between methods, attributes of the object, and other units.

Building the Solution

By this stage, the individual components of the solution have been developed and sufficiently tested in the Interix environment to satisfy the project requirements. This stage helps you build the solution with the developed and tested components, and then make the migrated application ready for internal release.

As a good practice, MSF recommends that teams working on development projects perform daily builds of their solution. In migration projects, on the other hand, you typically have to examine large bodies of existing code to understand what they are intended for and to make changes to this code. However, code changes can happen only after addressing porting issues, hence daily builds may not be required. The process of creating interim builds allows a team to find issues early in the development process, which shortens the development cycle and lowers the cost of the project. Note that these interim builds are not deployed in the live production environment. Only when the builds are thoroughly tested and stable are they ready for a limited pilot release to a subset of the production environment. Rigorous configuration management is essential to keeping builds in synch.

Interim Milestone: Internal Release

Interim milestones help the team measure their progress in the actual building of the solution during the Developing Phase. Each internal release signifies a major step toward completion of the solution feature sets and achievement of the associated quality level. Depending on the complexity of the solution, any number of internal releases may be required. Each internal release represents a fully functional addition to the solution’s core feature set that is potentially ready to move on to the Stabilizing Phase. As each new release of the application is built, fewer bugs must be reported and triaged.

The subsequent chapters of this volume describe the necessary code changes required for the migration of the UNIX code to the Interix environment. You can use these instructions to develop the solution components in the Developing Phase.