Deprecate `asyncDoProc` and `asyncDoProc#` in base

[dcoutts]

The proposal is to:

• deprecate the primop asyncDoProc# and the corresponding wrapper asyncDoProc in base
• deprecate the corresponding BlockedOnDoProc special thread blocking status
• eventually remove them
• no replacement or compat shim
• deprecation message to point users to Haskell FFI and foreign import dynamic

The motivation is that as part of the ongoing (slow) modernisation of the GHC I/O managers, we will eventually want to remove the win32-legacy I/O manager, and doing so will necessitate doing something about asyncDoProc.

So what is this thing?

asyncDoProc :: FunPtr (Ptr a -> IO Int) -> Ptr a -> IO Int

(I can't point to haddocks since it is a Win32 only export)

It asynchronously executes the given C function with an argument. It is asynchronous with respect to other Haskell threads on the capability. It is synchronous with respect to the executing thread.

The history is that this thing has been there forever. It long predates the Haskell FFI and the threaded RTS with it's support for "safe" FFI calls. It made some sense at the time.

It is however unique to the win32-legacy I/O manager. This means it is Windows only, and only works on the single threaded RTS. It is not supported by the new WinIO I/O manager, nor has it ever been supported by the mio I/O manager used in the threaded RTS on Windows.

It is now redundant. The same effect can be achieved in a portable way using the Haskell FFI: using "safe" FFI calls using foreign import ccall "dynamic", at least on the threaded RTS. This allows invoking C function pointers (with arbitrary arguments) and of course "safe" FFI calls also do not block other Haskell threads (on the threaded RTS).

Unsurprisingly, it is used very little. A hackage search https://hackage-search.serokell.io/?q=asyncDoProc reveals only one non-trivial use. That is in cabal-install-bundle-1.18.0.2.1 where it is used to do accept asynchronously on Windows without using the threaded RTS.
There are also some trivial uses of the thread blocking state, mostly for rendering each state into English.

There is no great rush to remove. We could deprecate now/soon and wait a cycle or two before removing. There is not a pressing need to remove the win32-legacy I/O manager, since it remains the default for the non-threaded RTS. It will eventually be replaced as the default by either WinIO or some other new Windows I/O manager. Eventually removing win32-legacy will allow cleaning up a bunch of special things it does differently to all other I/O managers (like its custom thread blocking states), and unfortunate special hooks into other parts of the RTS.

[parsonsmatt]

Hell yeah, sounds great

[treeowl]

What will cabal-install-bundle have to do to adapt?

[parsonsmatt]

Most likely nothing, it appears to be totally unmaintained: https://github.com/Paczesiowa/cabal-install-bundle and with very few downloads on Hackage https://hackage.haskell.org/package/cabal-install-bundle

EDIT: To be clear, it appears to be a fork of cabal-install with vendored dependencies. If someone wanted this, then the easy thing would be to use a version of cabal-install as a base for a new vendored/bundled package which did not do the asyncDoProc thing

[dcoutts]

Actually, there is a problem here. I was about to say that cabal-install-bundle just has to use the latest network code which must not use asyncDoProc since network doesn't turn up in the hackage search. But this is false, the latest network still has more or less the same code using asyncDoProc and yet does not turn up in the hackage search. That's rather worrying for using that search as a basis for making decisions.

https://github.com/haskell/network/blob/master/Network/Socket/Syscall.hs#L217

Anyway, yes the current network code uses asyncDoProc for accept for the non-threaded case on Windows. It uses normal "safe" FFI calls on the threaded RTS and uses asyncDoProc on the non-threaded RTS. It does this because the network lib avoids using non-blocking mode for sockets on Windows (but uses them consistently on Unix).

[treeowl]

So to adjust my question, what does network need to do to adapt?

[dcoutts]

Indeed, that is the question. To support the new WinIO I/O manager the network lib will have to do something too. I presume for winio it will use an asynchronous accept directly (since winio works primarily with async I/O APIs). Currently winio is the presumed replacement for win32-legacy. But that approach doesn't allow removing asyncDoProc prior to removing win32-legacy itself, which I had been hoping to do.

Or the network lib could switch to using non-blocking sockets as it does on Unix, but that's probably rather an upheaval.

[dcoutts]

But this is false, the latest network still has more or less the same code using asyncDoProc and yet does not turn up in the hackage search. That's rather worrying for using that search as a basis for making decisions.

Update: if you leave it long enough (much much longer than I expected) then it does eventually finish and find the occurrence in network.

[dcoutts]

So currently, the network package only supports a small handful of non-blocking operations on Windows on the non-threaded RTS. That is, non-blocking in the sense that they block only the calling thread and do not block all other Haskell threads. The list is short: send, recv and accept. In particular it does not work for sendTo, recvFrom or other variants like sendMsg. The docs say to just use the threaded RTS if one wants to use those and blocking is a problem.

So one approach would be to just say that consistently for all the network operations on windows on the non-threaded RTS.

I guess that if network really doesn't want to use non-blocking sockets on Windows, and still wants to support non-blocking accept in the non-threaded RTS, then it can do more low level magic. It already needs some custom C code to use the async accept. It could change that C code to use its own simple Win32 thread-based mechanism: fork a new Win32 OS thread to make a blocking call to accept and on return put the result into an MVar (there's a C RTS API for that). The Haskell side can call the C function, passing the MVar and then block on the MVar. No need to use a thread pool, it doesn't need to be performant.

[hasufell]

@kazu-yamamoto

[kazu-yamamoto]

See haskell/network#602