Using IO Timeout and Interrupt on NT

This technical memo is a cautionary note on using NSPR's IO timeout and interrupt on Windows NT 3.51 and 4.0. Due to a limitation of the present implementation of NSPR IO on NT, programs must follow the following guideline:

If a thread calls an NSPR IO function on a file descriptor and the IO function fails with PR_IO_TIMEOUT_ERROR or PR_PENDING_INTERRUPT_ERROR, the file descriptor must be closed before the thread exits.

NSPR IO on NT

On Windows NT, NSPR IO is implemented using NT's overlapped (also called asynchronous) IO. When a thread calls an IO function, the thread issues an overlapped IO request using the overlapped buffer in its PRThread structure. Then the thread is put to sleep. In the meantime, there are dedicated internal threads (called the idle threads) monitoring the IO completion port for completed IO requests. If a completed IO request appears at the IO completion port, an idle thread fetches it and wakes up the thread that issued the IO request earlier. This is the normal way the thread is awakened.

IO Timeout and Interrupt

These two errors are generated by the NSPR layer, so the OS is oblivious of what is going on and the overlapped IO request is still in progress. The OS still has a pointer to the overlapped buffer in the thread's PRThread structure. If the thread subsequently exists and its PRThread structure gets deleted, the pointer to the overlapped buffer will be pointing to freed memory. This is problematic.

Canceling Overlapped IO by Closing the File Descriptor

The only reliable way to cancel outstanding overlapped IO request that works on both NT 3.51 and 4.0 is to close the file descriptor, hence the rule of thumb stated at the beginning of this memo.

Limitations

• It is difficult for threads to shared a file descriptor. For example, suppose thread A and thread B call PR_Accept() on the same socket, and they time out at the same time. Following the rule of thumb, both threads would close the socket. The first PR_Close() would succeed, but the second PR_Close() would be freeing freed memory. A solution that may work is to use a lock to ensure only one thread can be using that socket at all times.
• Once there is a timeout or interrupt error, the file descriptor is no longer usable. Suppose the file descriptor is intended to be used for the life time of the process, for example, the logging file, this is really not acceptable. A possible solution is to add a PR_DisableInterrupt() function to turn off interrupts when accessing such file descriptors.

Note: A related known bug is that timeout and interrupt don't work for PR_Connect() on NT. This bug is due to a different limitation in our NT implementation.

Conclusions

When we stop supporting NT 3.51, we can look into using NT 4's CancelIo() function to cancel outstanding overlapped IO requests when we get IO timeout or interrupt errors. If CancelIo() really works as advertised, that should fundamentally solve this problem.

If these limitations with IO timeout and interrupt are not acceptable to the needs of your programs, you can consider using the Win95 version of NSPR. The Win95 version runs without trouble on NT, but you would lose the better performance provided by NT fibers and asynchronous IO.