Chapter 4: Developing Phase: Memory and File Management

This chapter discusses the similarities and differences in memory and file management between the Microsoft® Windows® application programming interface (API) and UNIX. It also provides various memory and file management-related functions and APIs that are available in both environments.

On This Page

Memory Management Memory Management
File Management File Management

Memory Management

For developers, memory management has always been one of the most important and interesting aspects of any operating system. Like UNIX, the Microsoft Windows operating system provides the standard heap management functions, as well as functions to manage memory on a thread basis. The basic functional mapping is covered in the next sections.

The information provided in this chapter will help you understand the memory and file management implementations in the UNIX and Windows applications. You will also be able to identify the areas that may need to be modified for compatibility in the Windows environment.

Heap

The heap is an area of memory that is separate from the program code and the stack. It is reserved for the memory allocation needs of the program that Windows provides similar to UNIX with respect to heap management. The standard C runtime in Windows includes comparable functions for such UNIX functions as calloc, malloc, realloc, and free. Windows has additional functions that may not be available in UNIX, which are covered briefly in the following sections.

Thread Local Storage

Thread Local Storage (TLS) is another area of memory management that defines memory on a per-thread basis. All threads of a process share its virtual address space, where the static and global variables are shared by all threads in the process, and the local variables of a function are unique to each thread that runs the function. With TLS*,* you can provide unique data for each thread that the process accesses by using a global index. One thread allocates the index, which can be used by the other threads to retrieve the unique data associated with the index.

The typical scenario in which TLS is used in Windows is within a dynamic-linked library (DLL), but this is not its only possible use. In the case of the DLL scenario, the use of TLS includes the following details:

  • When a DLL attaches to a process, the DLL uses TlsAlloc to allocate a TLS index. The DLL then allocates some dynamic storage to be used exclusively by the initial thread of the process. It uses the TLS index in a call to the TlsSetValue function to store the address in the TLS slot. This concludes the per-thread initialization for the initial thread of the process. The TLS index is stored in a global or static variable of the DLL.

  • Each time the DLL attaches to a new thread of the process, the DLL allocates some dynamic storage for the new thread and uses the TLS index in a call to TlsSetValue to store the address in the TLS slot. This concludes the per-thread initialization for the new thread.

  • Each time an initialized thread makes a DLL call requiring the data in its dynamic storage, the DLL uses the TLS index in a call to TlsGetValue to retrieve the address of the dynamic storage for that thread.

The following functions are used to manage TLS in UNIX and Windows:

  • Allocating memory. In the Windows environment, the TlsAlloc function allocates a TLS index. A TLS index is used by a thread to store and retrieve values that are local to the thread. The minimum number of indexes available to each process is defined by TLS_MINIMUM_AVAILABLE. TLS indexes are not valid across process boundaries. The prototype of the function is as follows:

    DWORD TlsAlloc(void);

    In the UNIX environment, pthread_key_create creates a thread-specific data key visible to all the threads in the process. Upon key creation, the value NULL is associated with the new key in all active threads. An optional destructor function may be associated with each key value. The prototype of the function is as follows:

    Note: The line has been split into multiple lines for readability.However, while trying it out on a system you must enter it as one line without breaks.

    int pthread_key_create(pthread_key_t *key,   
    

void   (destructor, void));

  • Deleting memory. In the Windows environment, TlsFree releases a TLS index. This, however, does not release the data allocated and set in the TLS index slot. The prototype of the function is as follows:

    BOOL TlsFree(DWORD dwTlsIndex);

    In the UNIX environment, the pthread_key_delete function deletes the thread-specific data key.

    int pthread_key_delete(pthread_key_t key);
  • Storing a value. In the Windows environment, the TlsSetValue function stores memory in a TLS index. The prototype of the function is as follows:

    BOOL TlsSetValue(DWORD dwTlsIndex, LPVOID lpTlsValue);

    In the UNIX environment, the pthread_setspecific function associates a thread-specific value with the key. The prototype of the function is as follows:

    int   pthread_setspecific(pthread_key_t key, const void *value);

  • Retrieving a value. In the Windows environment, the TlsGetValue function returns a memory element stored in a specified TLS index. The prototype of the function is as follows:

    LPVOID TlsGetValue(DWORD dwTlsIndex);

    In the UNIX environment, the pthread_getspecific function returns the values currently bound to the specific key. The prototype of the function is as follows:

    void *pthread_getspecific(pthread_key_t key);

Note Additional information is available at https://msdn.microsoft.com/library/en-us/winprog/winprog/windows_api_reference.asp.

Thread Local Storage (TLS) Example

The following section shows a portion of an example application. It illustrates allocation and access to a memory space on a per-thread basis. First, the main thread of the process allocates a memory slot. The memory slot is then accessed and modified by a child thread. If several instances of the thread are active, each thread procedure will have a unique TLS index value to ensure the separation and isolation of data and state.

UNIX example: Using TLS with pthread library

#include <pthread.h>
#include <stdio.h>
#include <string.h>
int main(void) 
{
      pthread_key_t k1;
      int ret;
      int val = 100;
      int *pval = NULL;
      ret = pthread_key_create(&k1, NULL);
      if (ret) 
    {
      printf("pthread_key_create: %s\n", strerror(ret));
      return -1;
      }
      ret = pthread_setspecific(k1, (void *) &val);
      if (ret) 
    {
         printf("pthread_setspecific: %s\n", strerror(ret));
         return -2;
      }
      pval =   (int*)pthread_getspecific(k1);
      if(pval == NULL)
      {
         printf("pthred_getspecific: NULL returned");
         return -3;
      }
      printf("pthread_getspecific value: %d\n",*pval);
      ret = pthread_key_delete(k1);
      if (ret) 
    {
     printf("pthread_key_delete: %s\n", strerror(ret));
     return -4;
      }
    return 0;
}

Windows example: Using TLS with Platform SDK

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

DWORD TLSIndex = 0;
DWORD WINAPI ThreadProc( LPVOID lpData)
{
HWND hWnd = (HWND) lpData;
LPVOID lpVoid = HeapAlloc( GetProcessHeap(), 0, 128 );
TlsSetValue( TLSIndex, lpVoid );
// Do your processing on the memory within the 
thread here...
HeapFree( GetProcessHeap(), 0, lpVoid );
Return(0);
}
LRESULT CALLBACK WndProc( HWND ...
{
switch( uMsg )
{
case WM_CREATE:
TLSIndex = TlsAlloc();
// Start your threads using CreateThread...
Break;
Case WM_DESTROY:
TlsFree( TLSIndex );
Break;
Case WM_COMMAND:
Switch( LWORD( wParam ))
{
case IDM_TEST:
// Do something with the TLS value by a call 
to TlsGetValue(DWORD)
break;
}
}
}

Memory-Mapped Files

Memory-mapped files offer a unique memory management feature that allows applications to access files on disk in the same way they access dynamic memory. It associates a file's contents with a portion of the virtual address space of a process. It can be used to share a file or memory between two or more processes. Windows supports memory-mapped files and memory-mapped page files. Memory-mapped page files are covered in the “Shared Memory” section as part of an exercise to port System V interprocess communication (IPC) shared memory to Windows using memory-mapped files. Creating and using shared memory in UNIX and Windows are conceptually the same activities, but syntactically different.

UNIX example: Creating and mapping a shared memory area

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

if ( (fd = open("/dev/zero", O_RDWR)) < 0)
err_sys("open error");
if ( (area = mmap(0, SIZE, PROT_READ | PROT_WRITE, 
MAP_SHARED, fd, 0)) == (caddr_t) -1)
err_sys("mmap error");
close(fd); // can close /dev/zero now that it's mapped

Windows example: Creating and mapping a shared memory area

hMapObject = CreateFileMapping(
INVALID_HANDLE_VALUE, // use paging file
NULL, // no security attributes
PAGE_READWRITE, // read/write access
0, // size: high 32-bits
SHMEMSIZE, // size: low 32-bits
"dllmemfilemap"); // name of map object
if (hMapObject != NULL) 
{
// Get a pointer to the file-mapped shared memory.
lpvMem = MapViewOfFile(
hMapObject, // object to map view of
FILE_MAP_WRITE, // read/write access
0, // high offset: map from
0, // low offset: beginning
0); // default: map entire file
if (lpvMem == NULL) 
{
CloseHandle(hMapObject);
}
}

Note For details on the CreateFileMapping and MapViewOfFile functions, refer to the Windows API documentation.

Shared Memory

Shared memory permits two or more threads or processes to share a region of memory. Interprocess communication (IPC) through shared memory provides the best performance among all the IPC mechanisms. This is because instead of copying the data, the same physical area of memory is accessed by both the client and the server. Windows does not support the System V IPC mechanisms for shared memory (the shm* APIs). However, it does support memory-mapped files and memory-mapped page files, which you can use as an alternative to the shm* APIs.

Note System V IPC support is available with MKS and Cygwin on Win32®/Win64.

The GlobalAlloc functions can also be used on Windows for sharing memory. The GlobalAlloc function allocates the specified number of bytes from the heap, which can be shared among processes.

Note Additional information on GlobalAlloc is available at https://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/globalalloc.asp.

Synchronizing Access to Shared Resources

The technical challenge of using shared memory is to ensure that the server and client do not attempt to access the shared resource simultaneously. This is particularly troublesome if one or both are writing to the same shared memory area. For example, if the server is writing to the shared memory, the client must not try to access the data until the server has completed the write operation.

To address this, several forms of synchronization are available in Windows. These are:

  • Semaphore

  • Mutex

  • Event

  • Critical section

UNIX has two of these mechanisms—the semaphore and the mutex—as well as an additional mechanism called file locking.

Note These mechanisms are also discussed in Chapter 3, “Developing Phase: Process and Thread Management” of this volume.

The first three mechanisms have two states: signaled and nonsignaled. The synchronization object is considered busy when it is in a nonsignaled state. When the object is busy, a waiting thread will block until the object switches to a signaled state. At this time, the pending thread continues to execute. The last form of synchronization is the critical section object. The critical section object is only for synchronizing threads within a single process. This synchronization mechanism only works for a single instance of the example application. Nonetheless, you can still consider its use as an IPC synchronization mechanism. This form of synchronization is appropriate for cases where you want to migrate your existing application from a multiprocessor architecture to a single multithreaded processor architecture.

The synchronization objects should be used carefully to prevent deadlocks. Deadlocks occur when some subset of the threads is waiting on one another so that none can execute.

Note Applications that need to control the size of the stack or heap space can use linker switches for this purpose. The default size for both the stack and heap on Windows is 1 MB. Use the /STACK linker option to set the size of the stack and the /HEAP linker option to set the heap size. Both options take the size in bytes. Additional information on how to use these linker options is available at https://msdn2.microsoft.com/en-us/library/f90ybzkh.aspx or
https://msdn2.microsoft.com/en-us/library/8cxs58a6(en-US,VS.80).aspx.

Further Reading on Memory Management

For further information about memory management, refer to the following:

  • Richter, Jeffrey. Programming Applications for Microsoft Windows, Fourth Edition. Redmond, WA: Microsoft Press®, 1999. (See Part III, Chapters 13–18.)

  • Stevens, W. Richard. Advanced Programming in the UNIX Environment. Reading, MA: Addison-Wesley Publishing Co., 1992.

File Management

This section discusses the differences in file handling, file access, file control, and directory operations. You can use this information to understand the file management implementations in UNIX and Windows applications. You can also identify the areas that may need to be modified for compatibility within the Windows environment.

Every program that runs from the UNIX shell opens three standard files. These files have integer file descriptors, provide the primary means of communication between the programs, and exist for as long as the process runs. You associate other file descriptors with files and devices using the open system call. Table 4.1 lists the UNIX standard file descriptors.

Table 4.1. UNIX Standard File Descriptors

File

File Descriptor

Description

Standard input

0

Standard input file provides a way to send data to a process. By default, the standard input is read from the keyboard.

Standard output

1

Standard output file provides a means for the program to output data. By default, the standard output goes to the display.

Standard error

2

Standard error is where the program reports any errors that occurred during the program execution. By default, the standard error goes to the display.

In Windows, when a program begins execution, the startup code automatically opens the following files (streams):

  • Standard input (pointed to by stdin)

  • Standard output (pointed to by stdout)

  • Standard error (pointed to by stderr)

These stream files are directed to the console (keyboard and screen) by default. Use freopen to redirect stdin, stdout, or stderr to a disk file or a device.

The stdout and stderr stream files are flushed whenever they are full or, if you are writing to a character device, after each library call. If a program terminates abnormally, output buffers may not be flushed, resulting in loss of data. Use fflush or _flushall to ensure that the buffer associated with a specified file or that all open buffers are flushed to the operating system, which can cache data before writing it to disk. The commit-to-disk feature ensures that the flushed buffer content is not lost in the event of a system failure.

Low-Level File Access

The low-level I/O functions directly invoke the operating system for lower-level operation, instead of the operation provided by standard (or stream) I/O. Function calls relating to low-level input and output do not buffer or format data. Low-level I/O functions can access the standard streams opened at program startup using the standard file descriptors. They deal with bytes of information, which implies that you are using binary files, not text files. Instead of file pointers, you use low-level file handles or file descriptors, which give a unique integer number to identify each file.

The read and write method used in UNIX have equivalents in the Windows environment as _read and _write. These methods can be used to read data from standard input and write data to standard output.

Information about the prototype and additional details about the low-level I/O routines available in Windows can be found at
https://msdn2.microsoft.com/en-us/library/40bbyw78.aspx.

Note Just as in UNIX, the Windows cmd.exe shell redirects standard error from the command line with the 2> operator.

Standard (Stream) File Access

The standard or stream I/O functions process data in different sizes and formats, and from single characters to large data structures. They also provide buffering, which can improve performance. These routines affect only buffers created by the run-time library routines and have no effect on buffers created by the operating system. The standard I/O library and its header file stdio.h provide low-level file I/O system calls. This library is part of ANSI standard C, so they can be ported directly to Windows.

Both UNIX and Windows support the standard I/O library functions for several file access purposes.

Note Additional information on the list of I/O library functions and their prototypes is available at https://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore98/HTML/_crt_stream_i.2f.o.asp.

ioctl Calls

The ioctl function performs a variety of control operations on devices and streams. For nonstream files, the operations performed by this call are device-specific control operations. The prototype of the ioctl function is as follows.

int ioctl(int fildes, int request, /* arg */ ...);

In Windows, a subset of the operations on a socket is provided by the ioctlsocket function. The ioctlsocket function supports only the SIOCATMARK command and does not have a command parameter equivalent to the FIOASYNC parameter of the ioctl function.

Windows ioctlsocket

The ioctlsocket function controls the I/O mode of a socket.

int ioctlsocket(SOCKET s, long cmd, u_long FAR *argp);

The ioctlsocket function can be used on any socket in any state. It is used to set or retrieve operating parameters associated with the socket, independent of the protocol and communications subsystem. Table 4.2 lists the supported commands to use in the command parameter and their semantics.

Table 4.2. ioctlsocket Parameters and Semantics

Command

Description

FIONBIO

Use with a nonzero argp parameter to enable the nonblocking mode of sockets. Set the argp parameter to zero if nonblocking is to be disabled. The argp parameter points to an unsigned long value. When created, a socket operates in blocking mode by default (nonblocking mode is disabled). This is consistent with BSD sockets. The WSAAsyncSelect and WSAEventSelect functions automatically set a socket to nonblocking mode. If WSAAsyncSelect or WSAEventSelect has been issued on a socket, then any attempt to use ioctlsocket to set the socket back to blocking mode will fail with WSAEINVAL. To set the socket back to blocking mode, an application must first disable WSAAsyncSelect by calling WSAAsyncSelect with the lEvent parameter set to zero or by calling WSAEventSelect with the lNetworkEvents parameter set to zero. The prototypes of the WSAAsyncSelect and WSAEventSelect functions are as follows:

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

int WSAAsyncSelect(SOCKET s, HWND hWnd
, unsigned int wMsg, long lEvent)
int WSAEventSelect(SOCKET s, WSAEVENT
hEventObject, long lNetworkEvents)

FIONREAD

Use to determine the amount of data pending in the input buffer of the network that can be read from sockets. The argp parameter points to an unsigned long value in which ioctlsocket stores the result. FIONREAD returns the amount of data that can be read in a single call to the recv function, which may not be the same as the total amount of data queued on the socket. If s is message oriented (for example, type SOCK_DGRAM), FIONREAD still returns the amount of pending data in the network buffer. However, the amount that can actually be read in a single call to the recv function is limited to the data size written in the send or sendto function call. The prototypes of the recv, send, and sendto functions are as follows:

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

int recv (int s, void *buf,
int len, int flags)
int send (SOCKET s, const char 
FAR* buf, int len, int flags)

int sendto (SOCKET s, const char FAR* buf, int len, int flags, const struct SOCK_ADDR* to, int tolen)

SIOCATMARK

Use to determine whether all out-of-band (OOB) data has been read. For a discussion of OOB data, refer to the "Windows Sockets 1.1 Blocking Routines and EINPROGRESS" section in the Microsoft Platform SDK: Windows Sockets 2 reference available on MSDN®. This applies only to a stream-oriented socket (for example, type SOCK_STREAM) that has been configured for inline reception of any OOB data (SO_OOBINLINE). If no OOB data is waiting to be read, the operation returns TRUE. Otherwise, it returns FALSE, and the next recv or recvfrom performed on the socket will retrieve some or all of the data preceding the mark. The application should use the SIOCATMARK operation to determine whether any data remains. If there is any typical data preceding the urgent (out-of-band) data, it will be received in order. A recv or recvfrom never mixes OOB and typical data in the same call. The argp parameter points to an unsigned long value in which ioctlsocket stores the boolean result.

File Control

File control in UNIX is implemented using the fcntl* *function. The fcntl function performs one of a number of miscellaneous operations on file descriptors.

#include <unistd.h>
#include <fcntl.h>
int fcntl(int fd, int cmd);
int fcntl(int fd, int cmd, long arg);
int fcntl(int fd, int cmd, struct flock *lock);

Table 4.3 lists the commands and semantics.

Table 4.3. File Control Commands and Semantics

Command

Description

F_DUPFD

Find the lowest numbered file descriptor available that is equal to or greater than arg and make it a copy of fd. On success, the new descriptor is returned.

F_GETFD

Read the close-on-exec flag. On success, the value of the flag is returned.

F_SETFD

Set the close-on-exec flag to the value specified by the FD_CLOEXEC bit of arg.

F_GETFL

Read the flags of the descriptor. All flags, as set by open, are returned.

F_SETFL

Set the flags of the descriptor to the value specified by arg. O_APPEND, O_NONBLOCK, and O_ASYNC may be set; the other flags are unaffected.

F_GETLK, F_SETLK, and F_SETLKW

These commands are used to manage discretionary file locks. The third argument lock is a pointer to a struct flock, which may be overwritten by this call.

F_GETLK

Return the first flock structure that prevents the requested lock from being created or set the l_type field of the lock to F_UNLCK if there is no obstruction.

F_SETLK

The lock is set (when l_type is F_RDLCK or F_WRLCK) or cleared (when it is F_UNLCK). If the lock is held by another user, this call returns -1 and sets errno to EACCES or EAGAIN.

F_SETLKW

Like F_SETLK, but instead of returning an error, wait for the lock to be released. If a signal that is to be caught is received while fcntl is waiting, it is interrupted and (after the signal handler has returned) returns immediately with a return value of -1 and errno set to EINTR.

F_GETOWN, F_SETOWN, F_GETSIG and F_SETSIG

These commands are used to manage I/O availability signals.

F_GETOWN

Get the process ID or process group currently receiving SIGIO and SIGURG signals for events on file descriptor fd. Process groups are returned as negative values.

F_SETOWN

Set the process ID or process group that will receive SIGIO and SIGURG signals for events on file descriptor fd. Process groups are specified using negative values. F_SETSIG can be used to specify a different signal instead of SIGIO.

F_GETSIG

Get the signal sent when input or output becomes possible. A value of zero means SIGIO is sent. Any other value (including SIGIO) is the signal sent instead. In this case, additional information is available to the signal handler if installed with SA_SIGINFO.

F_SETSIG

Sets the signal to be sent when input or output becomes possible. A value of zero sends the default SIGIO signal. Any other value (including SIGIO) is the signal to send instead. In this case, additional information is available to the signal handler if installed with SASIGINFO.

In Windows, the following equivalent functions are available for some, but not all, of the UNIX fcntl commands:

  • Use the _dup function in Windows for the F_DUPFD command in fcntl function in UNIX.

  • Use the LockFile, LockFileEx, and UnLockFile functions in Windows for the F_SETLK and F_SETLKW commands of fcntl function in UNIX. The following example demonstrates this.

UNIX example: Using fcntl

The following sample opens a file, sets the read lock, and unlocks the file. If any errors occur, an error message is output to the standard error file descriptor.

#include <unistd.h>
#include <fcntl.h>
int main()
{
struct flock l;
int fd = open("/tmp/locktest", O_RDWR|O_CREAT, 0644);
if (fd < 0)
{
perror("file open error");
exit(1);
}
l.l_type = F_RDLCK;
l.l_whence = SEEK_SET;
l.l_start = 0;
l.l_len = 0;
if (fcntl(fd, F_SETLK, &l) == -1)
{
perror("fcntl error – F_RDLCK");
exit(1);
}
l.l_type = F_UNLCK;
if (fcntl(fd, F_SETLK, &l) == -1)
{
perror("fcntl error – F_UNLCK");
exit(1);
}
exit(0);
}

(Source File: U_Fcntl-UAMV3C4.02.c)

Windows example: Using fcntl

The following sample opens a file, sets the read lock, and unlocks the file. If any errors occur, an error message is output to the console.

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

#include <stdio.h>
#include <fcntl.h>
#include <windows.h>
#include <io.h>
int main()
{
HANDLE hFile;
DWORD dwBytesRead, dwPos;
BOOL fResult;
// Open the existing file.
hFile = CreateFile("ONE.TXT", // open ONE.TXT
GENERIC_READ, // open for reading
0, // do not share
NULL, // no security
OPEN_EXISTING, // existing file only
FILE_ATTRIBUTE_NORMAL, // normal file
NULL); // no attr. template
if (hFile == INVALID_HANDLE_VALUE)
{
printf("Could not open ONE.TXT\n"); // process error
}
// Lock the file
fResult = LockFile(hFile, dwPos, 0, dwPos + 
dwBytesRead, 0);
if (fResult == 0)
{
printf("Could not lock ONE.TXT\n");
}
// Unlock the file
UnlockFile(hFile, dwPos, 0, dwPos + dwBytesRead, 0);
if (fResult == 0)
{
printf("Could not unlock ONE.TXT\n");
}
// Close file.
CloseHandle(hFile);
return(0);
}

(Source File: W_Fcntl-UAMV3C4.02.c)

Directory Operation

Directory operations involve calling the appropriate functions to perform directory scanning or to list the contents of a directory. Directory scanning involves traversing a directory hierarchy.

Table 4.4 provides details about the related functions for directory operations in UNIX and their replacements in Windows.

Table 4.4. Directory Operations Functions in UNIX and Their Replacements in Windows

UNIX Functions

Description

Suggested Replacement in Windows

getcwd

getwd

Gets the current working directory.

The _getcwd function does the same action. The prototype of the function is as follows:

char *_getcwd(char *buffer, int maxlen);

get_current_dir_name

Gets the current working directory.

The GetCurrentDirectory function gives the same result. The prototype of the function is as follows:

DWORD GetCurrentDirectory(DWORD nBufferLength, LPTSTR lpBuffer)

The following examples illustrate additional directory handling functions in UNIX and Windows.

UNIX example: Using directory handling functions

This sample prints out the current directory, and then recurses through subdirectories.

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

#include <unistd.h>
#include <stdio.h>
#include <dirent.h>
#include <string.h>
#include <sys/stat.h>
void ScanDir(char *dir, int indent)
{
DIR *dp;
struct dirent *dir_entry;
struct stat stat_info;
if((dp = opendir(dir)) == NULL)
{
fprintf(stderr,"cannot open directory: %s\n", dir);
return;
}
chdir(dir);
while((dir_entry = readdir(dp)) != NULL) 
{
lstat(dir_entry->d_name,&stat_info);
if(S_ISDIR(stat_info.st_mode)) 
{
/* Directory, but ignore . and .. */
if(strcmp(".",dir_entry->d_name) == 0 
|| strcmp("..",dir_entry->d_name) == 0)
continue;
printf("%*s%s/\n",indent,"",dir_entry->d_name);
/* Recurse at a new indent level */
ScanDir(dir_entry->d_name,indent+4);
}
else printf("%*s%s\n",indent,"",dir_entry->d_name);
}
chdir("..");
closedir(dp);
}
int main(int argc, char* argv[])
{
char *topdir, defaultdir[2]=".";
if (argc != 2) {
printf("Argument not supplied - using 
current directory.\n");
topdir=defaultdir;
}
else
topdir=argv[1];
printf("Directory scan of %s\n",topdir);
ScanDir(topdir,0);
printf("done.\n");
exit(0);
}

(Source File: U_Dir-UAMV3C4.01.c)

Windows example: Using directory handling functions

This sample prints out the current directory and then recurses through subdirectories. It uses the FindFirstFile, FindNextFile, and FindClose Windows API functions.

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

#include <windows.h>
#include <stdio.h>
void ScanDir(char *dirname, int indent)
{
BOOL fFinished;
HANDLE hList;
TCHAR szDir[MAX_PATH+1];
TCHAR szSubDir[MAX_PATH+1];
WIN32_FIND_DATA FileData;
// Get the proper directory path
sprintf(szDir, "%s\\*", dirname);
// Get the first file
hList = FindFirstFile(szDir, &FileData);
if (hList == INVALID_HANDLE_VALUE)
{
printf("No files found\n\n");
}
else
{
// Traverse through the directory structure
fFinished = FALSE;
while (!fFinished)
{
// Check the object is a directory or not
if (FileData.dwFileAttributes & 
FILE_ATTRIBUTE_DIRECTORY)
{
if ((strcmp(FileData.cFileName, ".") != 0) &&
(strcmp(FileData.cFileName, "..") != 0))
{
printf("%*s%s\\\n", indent, "", FileData.cFileName);
// Get the full path for sub directory
sprintf(szSubDir, "%s\\%s", dirname, 
FileData.cFileName);
ScanDir(szSubDir, indent + 4);
}
}
else
printf("%*s%s\n", indent, "", FileData.cFileName);
if (!FindNextFile(hList, &FileData))
{
if (GetLastError() == ERROR_NO_MORE_FILES)
{
fFinished = TRUE;
}
}
}
}
FindClose(hList);
}
void main(int argc, char *argv[])
{
char *pszInputPath;
char pwd[2] = ".";
if (argc < 2)
{
printf("Argument not supplied - using 
current directory.\n");
pszInputPath = pwd;
}
else
{
pszInputPath = argv[1];
printf("Input Path: %s\n\n", pszInputPath);
}
ScanDir(pszInputPath, 0);
printf("\ndone.\n");
}

(Source File: W_Dir-UAMV3C4.01.c)

Raw Device I/O

A raw device can be bound to an existing block device (for example, a disk) and can be used to perform raw I/O with that block device. Raw I/O does not involve the kernel caching that is normally associated with block devices. In UNIX, the system calls for character devices can be used to operate on raw devices. Device-specific files are created by the mknod system call. There is an additional system call, ioctl, for manipulating the underlying device parameters of special files. The prototype of the mknod and ioctl functions is as follows:

int mknod( char *pathname, mode_t mode, dev_t dev);
int ioctl(int fildes, int request, /* arg */ ...);

The following example describes the usage of mknod function for creating the device-specific files.

UNIX example: Raw device I/O

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/sysmacros.h>
#include <unistd.h>
int main(int argc,char **argv)
{
            int major =0,minor=0;
            char *path = "test.x";
            int mode = 0666 | S_IFBLK;
            char *end;
            /* get the major and minor 
            numbers for device files */
            major = strtol("01",&end,0);
            if(*end)
            {
                   printf("error 
                   in minor number:%d\n",minor);
                   return 1;
            }
            minor = strtol("01",&end,0);
            if(*end)
            {
                  printf("error in minor 
                  number:%d\n",minor);
                  return 1;
            }
            /* creating device file */
          if(mknod(path,mode,makedev(major,minor)))
          {
            printf("error in creating the device 
            file: %s\n",strerror(errno));
            return 2;
            }
            return 0;
}

Raw device I/O is common in Windows. Device I/O in Windows can be done with asynchronous I/O (or overlapped I/O). A process can open a file for asynchronous I/O with a call to CreateFile by specifying the FILE_FLAG_OVERLAPPED flag in the parameter. When the file is opened for asynchronous I/O, a pointer to an OVERLAPPED structure is passed into the call to ReadFile and WriteFile. You can also create an event and put the handle in the OVERLAPPED structure; the wait functions can then be used to wait for the I/O operation to complete by waiting on the event handle. The prototype of the Createfile, WriteFile, and ReadFile methods are as follows.

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

HANDLE CreateFile(LPCTSTR lpFileName, DWORD
dwDesiredAccess, DWORD dwShareMode, LPSECURITY_ATTRIBUTES
lpSecurityAttributes, DWORD dwCreationDisposition, DWORD
dwFlagsAndAttributes, HANDLE hTemplateFile);
BOOL WriteFile(HANDLE hFile, LPCVOID lpBuffer, DWORD
nNumberOfBytesToWrite, LPDWORD lpNumberOfBytesWritten,
LPOVERLAPPED lpOverlapped);
BOOL ReadFile(HANDLE hFile, LPVOID lpBuffer, DWORD
nNumberOfBytesToRead, LPDWORD lpNumberOfBytesRead, 
LPOVERLAPPED lpOverlapped);
BOOL DeviceIoControl(HANDLE hDevice, DWORD 
dwIoControlCode, LPVOID lpInBuffer, DWORD 
nInBufferSize, LPVOID lpOutBuffer, DWORD nOutBufferSize,
LPDWORD lpBytesReturned, LPOVERLAPPED lpOverlapped);

Windows example: Raw device I/O

The following example describes the usage of raw device I/O with the Windows API.

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

Note: Some of the lines in the following code have been displayed on multiple lines for better readability.

#include <windows.h>
#include <stdio.h>
int main()
{
 int nDiskNumber = 0;
char buf[64];
memset(buf,0,sizeof(buf));
sprintf ( buf, "\\\\.\\PHYSICALDRIVE%c", 
(char)nDiskNumber+'0');
//Open the physical disk handle
HANDLE hDiskHandle =   CreateFile ( buf , 
GENERIC_READ | GENERIC_WRITE
                        , FILE_SHARE_READ  
                         | FILE_SHARE_WRITE ,
NULL,OPEN_EXISTING, 0 , NULL );
if (hDiskHandle == INVALID_HANDLE_VALUE)
{
    printf ("PhysicalDisk:: can not open physical disk");
    return 1;
} 
int nSide,nTrack,nSector;
nSide = nTrack = nSector = 200 ;
LARGE_INTEGER li;
li.QuadPart = UInt32x32To64 (nSide*nTrack*nSector, 512);
//Move the physical disk pointer to the required position
DWORD dwPos = SetFilePointer ( hDiskHandle, li.LowPart,
&li.HighPart, FILE_BEGIN);
DWORD dwError = GetLastError();
if(dwError != NO_ERROR)
{
    printf("error in moving the file position\n");
    return 1;
}
unsigned long bytesRead;
char inBuffer[512];
memset(inBuffer,0,sizeof(bytesRead));
//Read the data from the physical disk
BOOL result = ReadFile( hDiskHandle,inBuffer,512 , 
&bytesRead , NULL );
 dwError = GetLastError();
 if( bytesRead <= 0 || dwError)
 {
     printf("error in reading the file\n");
     return 2;
 }
 CloseHandle(hDiskHandle);
 return 0;
}   

Note   More information on raw I/O and file systems in Windows is available at the MSDN Web site.

Download

Get the UNIX Custom Application Migration Guide

Update Notifications

Sign up to learn about updates and new releases

Feedback

Send us your comments or suggestions