Charting the Uncharted

A road map to porting shared memory, process management, and semaphore calls from Unix to Windows NT

Steve Niezgoda

If you develop Unix applications, market dynamics may eventually force you to come face to face with Windows NT. The good news is that the two operating systems have a lot in common: Both are based on abstractions like multiple processes, virtual memory, and networking. But while much Unix functionality exists in NT, the trick from a programming perspective is finding it.

Some Unix calls map effortlessly to Win32 counterparts. For example, Win32's WaitForSingleObject( ) and GetExitCodeProcess( ) replace Unix's waitpid( ) nicely. But many other substitutes are not obvious. I'll describe some of these subtleties here.

To help me identify important substitute calls, I wrote a custom back-end application. By back-end, I mean an application that doesn't contain a user interface but relies heavily on system calls to provide and control resources. Back-end applications are notorious users of low-level system calls, like process primitives, shared memory, and semaphores. My application is a process synchronization program where two processes--a producer and a consumer--share a common buffer. The producer places data into the buffer, and the consumer takes it out.

Shared Memory

In Win32, Microsoft combines shared memory and memory-mapped files into a single set of API calls. Thus, the Unix calls shmget( ), shmat( ), shmdt( ), and shmctl( ) have no direct counterparts (see the table). Win32's CreateFileMapping( ) maps a physical file into a block of memory. When CreateFileMapping( ) receives a NULL file handle, it behaves like shmget( ) and reserves a block of memory of specified size. However, unlike with shmget( ), the first call to CreateFileMapping( ) allocates memory. MapViewofFile( ) is analogous to shmat( ) in that it allows applications access to the shared memory.

In Unix, shmat( ) allows a process to map a piece of shared memory to its address space more than once; the Win32 MapViewofFile( ) provides similar functionality. The ubiquitous Win32 CloseHandle( ) detaches from (and, in the case of the last open handle, deallocates) shared memory. It replaces Unix shmdt( ) and shmctl(IPC_RMD). On Intel-based machines, Win32 requires memory-mapped files to start on 64-KB boundaries. This may be limiting: A Unix program depending on several contiguous, non-64-KB chunks of shared memory may need a face-lift.

Process Management

Unix developers use fork( ) for two purposes, and there is no single Win32 substitute for these tasks. Most often, developers use fork( ) in the course of loading other applications. In these cases, fork( ) immediately precedes exec( ) (or another member of the exec( ) famil y).

The Win32 CreateProcess( ) is a viable substitute for a fork-exec combination, but there are some important differences. First, Win32 imposes a 1024-byte limit on the command line. If the argument list requires more space, you should pass data through environment variables, shared memory, or files. Second, CreateProcess( ) is no match for exec( ) in building command-line arguments. Unix passes argv as an array of strings. Win32, in contrast, passes a single command-line string. This may cause parsing problems for strings that contain spaces or double quotes. Finally, CMD.EXE does not expand regular expressions as the Unix shell does.

Sometimes exec( ) does not follow fork( ). Implementing this flavor of fork( ) in Win32 is tricky. Microsoft recommends using threads because they offer multiple paths of execution inside a single address space. Threads use less overhead than processes do, but they require more synchronization. Because threads share variables, controlling access is important.

A fork( ) sans exec( ) can also be implemented with CreateProcess( ). This approach is attractive when the child process needs only a subset of the parent's resources. After a child process is created, the parent must copy all relevant handles and data to the child. Inheritance is a clean mechanism for transferring handles. Object handles become inheritable by setting bInheritHandle, located in the security descriptor, to true during creation. (By default, bInheritHandle is false.) When CreateProcess( ) is invoked and the InheritHandles argument is specified, all the parent's inheritable handles are duplicated for the child. Alternatively, DuplicateHandle( ) can be used to copy handles between processes--but then the child must be made aware of these handles. In both cases, you should pass global variables and data structures on the command line, in shared memory, or in the environment space.

The Win32 process structure is not hierarchical, so there is no Unix concept of the parent process. Conseq uently, applications cannot assume that killing the parent process automatically kills child processes. There is a kludge, however. Child processes become grouped if their parent is created with CREATE_NEW_PROCESS_ GROUP set. Then, GenerateConsoleCtrlEvt( ) can send Control-C or Control-Break signals to the group. However, only children who share the console with the parent process receive the signal.

Finally, the Win32 call TerminateProcess( ) is not a suitable replacement for Unix kill( ). Microsoft recommends terminating processes with the WM_CLOSE message. TerminateProcess( ) is only for extreme circumstances, because DLLs do not call all their exit routines.

Semaphores

Win32 supports two types of semaphores: mutual exclusion (mutex) and counting. In Unix, mutex semaphores are a special case of counting semaphores--the semaphore count is either 0 or 1. They port easily to Win32. CreateMutex( ) is the Win32 replacement to semget( ).

Unix semaphore operations are performed by set ting the sem_op parameter to an integer value and invoking semop( ). In Win32, WaitForSingleObject( ) and ReleaseMutex( ) perform down and up operations, respectively.

In Unix, you can treat multiple semaphore operations as a single atomic unit. The syntax is transparent: semop( ) accepts a pointer to an array containing one or more semaphores. When the array contains multiple semaphores, the operating system blocks until the program signals all semaphores. This functionality exists in Win32 but requires different syntax (see the table).

Counting semaphores are not as portable as mutexes. The Win32 calls CreateSemaphore( ), ReleaseSemaphore( ), WaitForObject( ), and CloseHandle( ) are comparable to the mutex calls described above. (There is also OpenSemaphore( ), which lets multiple processes share a single semaphore.) Win32's big weakness is that no API call consumes more than one semaphore.

Consuming multiple semaphores in Unix is trivial: You simply set sem_op to the desired value (e.g ., -2 or -3) and invoke semop( ). Win32 WaitForObject( ), however, can reduce the semaphore count by only one. (There is no limitation in the other direction: ReleaseSemaphore( ) can increase the semaphore count an arbitrary amount.) This limitation can wreak havoc. Consider an application where three reader processes and one writer process share a block of memory. The writer requires exclusive access; the readers require shared access. In Unix, the writer sets sem_op to -3 and blocks until the readers finish. Each reader sets sem_op to -1. In Win32, the readers are straightforward: Use WaitForSingleObject( ). However, there are only two alternatives for the writer, and both are unattractive. The first is to nest WaitForSingleObject( ) inside a for(i = 0; i < 3; i++) statement. The second is to redesign the code. Adding a second writer compounds the problem, because the for() statement must be protected by a mutex.

The persistence of semaphores and shared-memory resources also differs between Unix and Win32. Unix semaphore and shared-memory constructs remain in memory until explicitly deleted. In Win32, all of a process's open handles close automatically on exit.

APIs and Products

Porting applications from Unix to NT using the preceding is what I call the brute-force method. An alternative may be a commercial program that offers developers a common API. One such program, Consensys's Portage, provides a Unix System V release 4 interface for Win32. DataFocus's Nutcracker supports SVR4, Posix.1, and Berkeley 4.3 extensions, including sockets.

At press time, both Nutcracker and Portage address only the back-end aspects of applications, but the companies say they are working on libraries to help with user interfaces and other front-end elements. If minimizing time to market is not crucial for your Unix application, redesigning it from Win32 may be appropriate. However, if you need to port to NT quickly and can enhance front-end pieces later, consider one of these programs as a painless alte rnative to brute-force translations.

ACKNOWLEDGMENT

Alan Brown, a senior consultant at DataFocus, contributed information about substitute calls.

---

Key Substitutes For Translating Unix To Win32

UNIX            WIN32                        COMMENTS
Shared memory
shmget( )       CreateFileMapping( )         NT implements shared memory
                                             through memory-mapped files.
                OpenFileMapping( )
shmat( )        MapViewofFile( )
                OpenFileMapping( )
shmdt( )        UnMapViewofFile( )           Unattaches from shared memory.
                CloseHandle( )               CloseHandle( ) deallocates
                                             resources.
shmctl( )       No counterpart               Shared memory is deallocated with
                                             CloseHandle( ).
Process management
fork( )         CreateProcess( )             Good substitute for fork( )

                                             + exec( ).
                CreateThread( )              Can be used for fork( ) not
                                             followed by exec( ).
exec( )         CreateProcess( )             CreateProcess( ) is more like
                                             system( ) than exec( ).
waitpid( )      WaitForSingleObject( )
                WaitForMultipleObjects( )
                WaitForSingleObjectExt( )
                WaitForMultipleObjectsExt( )
                GetExitCodeProcess( )
getpid( )       GetCurrentProcessID( )
getppid( )      No counterpart               Process structure is not
                                             hierarchical.
kill( )         SendMessage (WM_CLOSE)       Use TerminateProcess( ) under
                                             extreme circumstances.
Binary semaphores
semget( )       CreateMutex( )
semop( )        ReleaseMutex( )              Increments semaphore count by one
                                             or
 more.
                WaitOnSingleObject( )        Decrements semaphore
                                             count by one.
semctl( )       No counterpart               Semaphore is deallocated with
                                             CloseHandle( ).
Counting semaphores
semget( )       CreateSemaphore( )
                OpenSemaphore( )
semop( )        WaitForObject...( )          Any flavor of WaitFor; however,
                                             consumes only one semaphore at a
                                             time.
                ReleaseSemaphores( )
semctl( )       No counterpart               Semaphore is deallocated with
                                             CloseHandle( ).

---

Steve Niezgoda is a member of the FBI Laboratory's Computer Analysis and Response Team and a graduate student at George Mason University. He can be reached on CompuServe at 76114,1542 or on the Internet or BIX c/o edito rs@bix.com .