Support for asynchronous disk I/O on Unix platforms suck. "Ready"-ness based APIs really just don't work for disk based access. Thus if your after a nice portable option, a thread pool is it.
G+: Support for asynchronous disk I/O on Unix platforms …
(+1's) 1
I've played this game before: https://plus.google.com/104708749502806774663/posts/6EVPAjbGbhc
It's a real shame - as you pointed out it takes a vast amount of engineering effort for something that seems pretty easy. There's several different asynchronous/event mechanisms in Linux, but they don't play well together - there's no real universal mechanism (for example, I'm still annoyed how hard it is to be notified of something like a child process being terminated). Something like completion ports would be nice for the various types of I/O, but I don't think that's the way Linux is going. For specific use-cases there are things like the proposed extension of the splice system call ( http://lwn.net/Articles/567086/ - useful if you're just copying from one file descriptor to another) but nothing universal.
My takeaway was "do I/O in a thread" which is exactly how [one of the flavors of] AIO is implemented in glibc.