In one of the projects I’m working on I hit a fairly nasty problem involving (as might be obvious from the title) both AppDomains and unmanaged code calling back into managed code. But first, a little background.
It all started with an unmanaged C++ class library. We’ve been using it for a while from other unmanaged C++ applications, but since we’ve been writing most new applications in C#.NET, the time had come to bring the library (kicking and screaming) into the .NET world. Naturally, the way forward was to write a compatibility layer in C++.NET, thereby taking advantage of its IJW framework (which makes C++ the best .NET language to use to interface with native code). (.NET 2.0, of course. Don’t even get me started about some of the problems interfacing native & managed code in 1.x.)
One of the key ingredients was an unmanaged callback. Since some of the work of the unmanaged base library was asynchronous, we gave it an object to call back through in order to tell us when the results were ready — meaning that the unmanaged code needs to call into the managed code. This is normally simple enough — while unmanaged classes can’t hold on to managed objects directly, they can through a GCHandle
(or the equivalent helper class, gcroot
). The code went something like this (paraphrased to protect the guilty):
class MyCallback : public INativeCallback { public: explicit MyCallback(ManagedClass^ managed) : m_Managed(managed) { } virtual void OnNativeCallback(const BYTE *data, DWORD length) { cli::array<Byte>^ bytes = gcnew cli::array<Byte>(length); Marshal::Copy(IntPtr(const_cast<Byte*>(data)), bytes, 0, length); ManagedData^ mdata = gcnew ManagedData(bytes); m_Managed->OnCallback(mdata); } private: gcroot<ManagedClass^> m_Managed; }; ref class ManagedClass { public: ManagedClass() { m_Callback = new MyCallback(this); NativeFramework::RegisterCallback(m_Callback); } void OnCallback(ManagedData^ data) { // ... } private: MyCallback *m_Callback; }; |
So, all well and good so far, right? No such luck. At first this was all going smoothly; the code was coming together and manual testing of the application showed that it was communicating properly with the library and with native applications that were also using the library. Once the proof of concept was in place it was time to ensure nothing went wrong later on through the use of unit testing. This is where the problems began, however.
The unit tests in question were written using the NUnit framework, which runs the tests in a separate AppDomain. This is useful because it permits the test runner to treat the application/library under test as a plugin — keeping it loaded only while actually running the tests, thereby allowing it to be recompiled and have the tests run again without having to exit the test runner.
The problem is that AppDomains are purely a managed construct. They’re intended to keep sections of managed code mostly isolated from each other (as in the above plugin-like case), and as such objects usually only exist in one AppDomain at a time. Unmanaged code of course knows nothing about any of this. Consequently, when calling managed code from unmanaged code, the compiler has to pick one AppDomain to use, and it appears to pick the first one. This is fine for most applications, since normally apps only use one AppDomain — which is why this code was working at first.
But when running the unit tests, it was executing in a second AppDomain, and the callback failed. Specifically, when trying to access the m_Managed object (from the gcroot
, which you’ll recall is a GCHandle
) an exception was thrown saying “Cannot pass a GCHandle across AppDomains”.
The solution is to use delegates. They’re not just function pointers — they also contain an object reference, a few other odds and ends, and (most importantly for us) a reference to the AppDomain that created it. However there’s a catch. Delegates themselves are managed objects, and so would have to be stored in a gcroot
if held in unmanaged code — and we already know that we can’t access anything in a gcroot
outside its original AppDomain.
Fortunately there’s a loophole: delegates can be marshalled into unmanaged function pointers (via a thunking layer). There is a downside to this though. Since the method signatures must match as closely as possible, and since a purely unmanaged function pointer can’t have anything to do with managed objects, the parameters must be restricted to native types. This required a bit of a redesign, but fortunately since you’re coming in from unmanaged code all the data you’re dealing with is going to be native anyway. So here’s the redesigned code. It’s possible there’s still something that could be improved in it; but this one does the trick, and maybe it’ll help someone else who has been struggling with this issue
delegate void CallbackDelegate(IntPtr data, unsigned int length); typedef void (__stdcall *CallbackNative)(const BYTE *data, DWORD length); class MyCallback : public INativeCallback { public: explicit MyCallback(CallbackDelegate^ callDelegate) : m_Delegate(callDelegate) { m_Native = (CallbackNative) Marshal::GetFunctionPointerForDelegate(callDelegate).ToPointer(); } virtual void OnNativeCallback(const BYTE *data, DWORD length) { m_Native(data, length); } private: gcroot<CallbackDelegate^> m_Delegate; CallbackNative m_Native; }; ref class ManagedClass { public: ManagedClass() { CallbackDelegate^ callback = gcnew CallbackDelegate(this, &ManagedClass::OnNativeCallback); m_Callback = new MyCallback(callback); NativeFramework::RegisterCallback(m_Callback); } ~ManagedClass() { this->!ManagedClass(); } !ManagedClass() { NativeFramework::UnregisterCallback(m_Callback); delete m_Callback; m_Callback = NULL; } void OnCallback(ManagedData^ data) { // ... } private: void OnNativeCallback(IntPtr data, unsigned int length) { cli::array<Byte>^ bytes = gcnew cli::array<Byte>(length); Marshal::Copy(data, bytes, 0, length); ManagedData^ mdata = gcnew ManagedData(bytes); OnCallback(mdata); } private: MyCallback *m_Callback; }; |
(Don’t forget to Dispose
the ManagedClass
when you’re done with it!)
And there we have it; my first Programming post. I hope at least someone finds it interesting! 😀
Ok, I’d like to get you started on Managed C++ in 1.x.
Seriously, I’m having the exact same problem as you did and luckily I found your post. Unfortunately, our code is still in the old MCPP syntax, although I’ve recompiled it on VS 2005 with the oldSyntax switch. Do you think I could do something similar to the above with the old syntax?
I’m not a C++ pro, so I’d rather not rewrite it in C++/CLI.
Thanks,
Jason
I haven’t tried it, but as far as I know anything you can do with the new syntax you can also do in the old syntax — it’s just a lot more verbose (and has lots of double-underscores). As long as you’re using VS2005, anyway (and targetting 2.0). (The above method won’t work in 1.x, since it uses functions added in 2.0 — but hybrid 1.x DLLs don’t work all that well anyway.)
If you take my code above and backport the syntax (eg. replace ^s with *s, refs with __gcs, etc) then you should be close.
Hi
I have exactly the same Problem and your code looks very fine. But I have a problem with INativeCallback and NativeFramework::RegisterCallback.
What is INativeCallback and NativeFramework?
Where can I get it?
One additional problem may be, my callback gets a very deep C++ class instead of an simple bytearray. But I think, that’s not so big problem
Many thanks for your help!
Franz
INativeCallback and NativeFramework are placeholders for the native classes you’re trying to interface to. In other words, that’s your external code, not part of .NET — so you’ve already got it
And no, there won’t be any issues with passing C++ classes through the callback, it’ll just work (bearing in mind what I said later). Though if you’re going to be interfacing with C# or VB code at some point then you should convert the data to managed classes along the way (after calling through the delegate; you can’t do it earlier).
Excellent Article!!
You saved my day (again even when we are no longer working together), Gavin!
I was trying to use AppDomain to load a native dll which receives phone calls, as I want to have control of unloading and reloading the dll. I was stuck at the callbacks part where your article just helped me fixed it.
Thank you so much. You are a real Guru!
[…] appdomain to select to execute? In general, no, it doesn’t — you must use delegates for that. See http://lambert.geek.nz/2007/05/29/un…main-callback/ — […]
Please post a link of full source code.
Why? All the relevant bits are there. The implementation details of the native framework aren’t interesting to the discussion.
I am planning to make another post soon along similar lines to this one, showing a different technique which puts more emphasis on the native code. Perhaps that’ll be more to your liking.
If ManagedClass was implemented in C#. how do I store the delegate in gcroot? is it automatic once i call somedelegate = new delegate(…)??
do i use GCHandle.Alloc(somedelegate) ?
The gcroot isn’t the problem then — the problem is the native framework call and storing a reference to the unmanaged handler class (MyCallback).
It’s not totally impossible for ManagedClass to be implemented in C#, but it’s far more hassle than it’s worth, at least for the scenario I’ve specified (implementing a C++ abstract callback class), since MyCallback has to be implemented in C++ anyway.
Thanks for that. It was just what I needed and the example code was a great help at moving me in the right direction.
I had the same issue. Your example helped me – thanks. My example may help others – see below. I show my code (before) and (after).
My code after:
Ok, but you can just pass the
std::string
(or yourRfa::UnManaged::RdmLogMessage
) through the function pointer and delegate (so it winds up inLogDelegateAdapter::Log
). You don’t need to get achar*
and useIntPtr
.(The downside of doing that is that you then won’t be able to call
LogDelegateAdapter::Log
from fully managed code, but then usually you don’t need to anyway. And if you do, then you can just define another overload.)Error: Cannot pass a GCHandle across AppDomains…
I’m currently working on adding easy to use Real Time Data server support to my Managed XLL Excel Addin system. This lets you use the =RTD() functionality of Excel to push real time data into your spreadsheets without needing to……
great ! it worked for me as well
.Net framework 3.5 does not have NativeFramework::
Thanks you
[…] This link is about creating a callback from unmanaged code into an AppDomain using a thunking trick. I’m not sure this can help you but maybe you’ll find this useful to create some kind of a workaround. […]
Pass C++/CLI delegate as function pointer into native C++ class…
To Pass C++/CLI delegate as function pointer into native C++ class, you can make use of this .net method: System::Runtime::InteropServices::Marshal::GetFunctionPointerForDelegate, the key point is: define the delagate (MyDelegate^ m_myDelegate in my ex…
I want to thank you for publishing this article. I had exactly this problem and would not have solved without your explanations and your sample code. I have seen other articles, but this is the most illustrative. Many, thank you very much!
Thanks you so much. You save my day. That was exactly what I needed !!!
[…] diane on: Unmanaged callbacks across AppDomains […]
Thanks sooo much.
It worked for us.
Thank you so much!!!!! Best article!
@dave dowd:
Quick note: In LogDelegateAdapter.h (after), you have the following line near the top:
using namespace System;
Doesn’t that pollute the global namespace when you include it in a header file?
Thanks for listing your code in detail like this. It is helping me figure out how to tackle my particular situation.
I’ve managed to work round the same problem by using the COM Callable Wrapper for the managed object works: instead of storing a
gcroot<ManagedObject>
, store the pointer as anIUnknown *
usingGetIUnknownForObject
, and do the reverse translation withGetObjectForIUnknown
before making the callback.I presume that the COM Callable Wrapper must wrap up the AppDomain information, because COM knows nothing about AppDomains itself. Certainly it seems to work in practice.
The downside is losing a little bit of type safety because
IUnknown *
loses the actual object type and you later have to downcast, but that seems like a small price to pay for the greater simplicity.(I also posted about this on StackOverflow)
Stephen: no, it doesn’t pollute the global namespace, it pollutes the
MarketData
namespace. But yes, in my own code I usually try to avoid doing that sort of thing even though it makes headers more wordy.Hi,
I have same problem. But my base c++ api expect object of specific type (object implement a interface which has one method “void OnReceive(vector<shared_ptr> obj)” which is called on callback ) not function pointer and any change in base libraries is difficult now. Any other way to solve this issue.
Nishant: there’s no problem in transferring any native type through the interface (at least as long as you’re doing it within the same module). See the followup post linked at the end of this post for more information and some examples.
Let me elaborate my problem a bit more.
I have “Receiver” class in C# and this is passed as input parameter to one of the api. This receiver object is used for callback. In C++/CLI I have created a Native /unmanaged class “ObjectBinder” which is same replica (has same methods) of managed Receiver class. It holds reference of managed receiver object in gcroot. When we call that api from C# it comes to CLI layer and app domain is “client exe”. we store the parameter “managed receiver object” in ObjectBinder in gcroot and pass reference of native ObjectBinder object to C++. Now the backend code (c++ and c) send an asyn callback (new thread) to c++ layer which use ObjectBinder object to send back call to CLI. Now we are in CLI layer in ObjectBinder object. BUT App domain has been changed (in case of WCF or NUNIT or any other service that creates it’s own App domain which is not known at compile time) . Now i want to access managed Receiver object which is stored in gcroot to send back callback to C# but it gave “Cannot pass a GCHandle across AppDomains”.
I have also tried IntPtr and IUnknown * instead of gcroot with Marshal::GetIUnknownForObject and Marshal::GetObjectForIUnknown but getting same error.
As discussed in the post, you have to make the call through an unmanaged function pointer derived from a Delegate. Again, see the followup post linked at the end for some better examples.
+1 for this valuable analysis.