As ACE increases in popularity, we see more questions about how to run services that must manage thousands of concurrent network connections. Since many network services handle network connections by demultiplexing and dispatching using the ACE Reactor framework, the concerns and limitations center around handle limits and performance.

Although each computing platform limits the total number of open files (or handles), the demultiplexing facility used under each ACE Reactor implementation is often the most important factor in determining the maximum number of handles an application can use concurrently:

• WaitForMultipleObjects(). Available on Windows only, this is used in ACE_WFMO_Reactor. This system call has a limit of 64 handles per call. Because ACE uses two of them internally, users are limited to 62 handles.
• select(). Available on most platforms (including Windows), select() is the basis of both ACE_Select_Reactor and ACE_TP_Reactor. The number of handles useable by select() varies widely by platform.
• /dev/poll or epoll. While epoll is available on 2.6 and newer Linux kernels, /dev/poll is available on Solaris and HP-UX. These mechanisms are used by the ACE_Dev_Poll_Reactor. There's no hard limit to the number of handles used by either one.

With the variety of limits and facilities available, how do you choose a way to enable your application to serve thousands of connections simultaneously? Applications on Windows can choose to use the Proactor framework instead of the Reactor framework and escape the handle limits entirely. Although the Proactor framework is available on a number of UNIX/Linux platforms, the quality and completeness of the underlying AIO facilities are often not good enough to base a commercial application on. Therefore, we focus on Reactor usage in this article.

Windows: WFMO vs. select. Although the Proactor framework is often a better choice for managing large numbers of connections on Windows, the Reactor framework's familiarity and ease of porting often argue for its use on Windows as well. There are two basic choices:

1. ACE_WFMO_Reactor. As noted above, this reactor implementation is limited to 62 user handles. Clearly inadequate for serving large numbers of connections.
2. ACE_Select_Reactor (and its cousin, ACE_TP_Reactor) based on select(). The Windows implementation of select() has no limit on the number of handles that can be used. However, due to the internal layout of fd_set on Windows, it's very difficult to optimize scanning of the handle sets as ACE does on other platforms. Therefore, the handle limitations are avoided, but you need to test for performance to see if its impact on your particular application is too high. If so, you'd really need to change to the Proactor framework.

Non-Windows: select() vs. others. select() is nearly ubiquitous on today's most popular platforms, so it's often the first choice in demultiplexers. However, there are two issues affecting use of select():

1. Handle limits. The number of handles useable by select is limited, and it seems that no two platforms limit it the same way. Furthermore, the mechanism for raising the limit is platform-dependent (we show how to raise the limit on Linux in the linked article below).
2. Performance. select() is notorious for its poor performance characteristics as the maximum handle value is raised. In some situations, CPU resources can be consumed largely by select()'s scanning for ready handles and the application's scanning of the ready sets for handles to work on.

Cool! How can I use these?! You're probably thinking that, right? ACE does offer a new Reactor implementation, ACE_Dev_Poll_Reactor, that takes advantage of /dev/poll or epoll depending on the platform capabilities. To use this in place of the default reactor implementation, simply instantiate one and set it as the implementation in a new ACE_Reactor instance. Both C++NPv2 and APG contain examples of how to set up a new reactor. But wait... you may want to hold on before cranking up the editor just yet. ACE_Dev_Poll_Reactor was added to ACE 5.4 but classified as "experimental" since its suitability was limited. However, Riverace and other ACE users have cooperated to improve ACE_Dev_Poll_Reactor, particulary for use on Linux with epoll, for ACE 5.5. It has improved quite a bit, but still requires some work to be as universally useable as the other Reactor implementations. With another burst of development, we could finish this reactor and add a powerful tool to ACE's set of cross- platforms solutions. If you are interested in sponsoring completion of this work, please contact Steve Huston at Riverace to discuss it.

The ACE development community is hard at work on the next version of ACE. Here are some of the things you can look forward to (items in bold are new or changed since last month):

• Native library path-searching. Previous ACE versions's implementation of ACE_DLL (used in loading dynamic services) implemented a search of the configured path-search to try and locate the desired library file (DLL) and then used the full pathname to load the library. This skirted some platforms's rules for path search and security settings. This has been changed in the next ACE version to make use of all of the native platform's path-search and security facilities.
• Improved wide-character support. Wide- character support has been included in ACE for many years on Windows (also known as Unicode builds). However, wide-character support for POSIX platforms has been sorely lacking. As ACE's reach expands, this has become a more important issue. Riverace is leading the completion of wide-character support in the next version of ACE. This support is also available now in the ACE 5.4b fix kit.
• GNU autotools support. Riverace is the lead developer in charge of adding auto-configure support to ACE. For native-build, non-Windows systems, autoconf will very likely become the way to configure and build ACE. This will help insure that ACE uses the latest features available on each supported platform, and reduce the work required to build ACE. No more picking the right config.h or platform_macros.GNU file. Just do ./configure then make. Ahhh...
• New compiler support. Support for g++ 3.4 and Microsoft Visual C++ 2005 are being added.
• Newer C++ Features Being Used. As the range of ACE-supported compilers matures and older compilers are taken out of service, ACE can make more use of newer C++ features while still maintaining its stellar portability record. For example, many methods that returned 1 or 0 as an int now return bool. Small steps, yes, but significant ones. You'll see more modern C++ usage as time goes on.
• The ACE and ACE_OS classes, previously containing a number of static member functions, are now C++ namespaces. Similar ACE-internal classes, such as ACE_Sock_Connect, are no longer in use.

As you can see, the pace of changes is slowing and the ACE development team is beginning to focus more on testing in preparation for the release of ACE 5.5.

If you're planning a new project to start soon, or find yourself grappling with how to apply ACE to your current project, remember that Riverace provides world-class development and consulting services with special expertise in ACE. We can help you be sure that your new system is designed to take full advantage of ACE's power and flexibility, getting your system delivered in the shortest possible time and with the highest level of quality. Please contact Steve Huston at 888-384-8154 (toll-free in the US) or +1 508- 541-9180 to discuss how Riverace can help you.