Linking to a WinSxS DLL that might not be there

Sometimes it is necessary to link to a DLL that might not be installed on the user’s system. Usually this is to add support for new features, but to still gracefully handle the case where it is not available. For example, you might want to use DirectX 11 on Windows 7, but fall back to DirectX 9 on Windows XP.

This is generally quite straightforward to achieve by using LoadLibrary and GetProcAddress at runtime to load the required library if it’s available:

HMODULE d3d11 = LoadLibraryW(L"d3d11.dll");
if (d3d11)
    auto D3D11CreateDevice_ =
            GetProcAddress(d3d11, "D3D11CreateDevice"));
    // ...

This process becomes somewhat more complicated when using a side-by-side DLL, due to the way they are loaded. Linking to SxS DLLs requires you to include an entry in the application manifest which allows the SxS loader to find the right DLL. To use Windows Vista style common controls, for example, you have to insert the following as a dependent assembly into your manifest:


If the SxS DLL is not present on the user’s machine then the application will fail to start. However, you can’t simply remove it and try to load it at runtime instead, as then the call to LoadLibrary won’t be able to find it.

The solution to this catch-22 requires you to embed a separate manifest in the application resources which references the assembly, modify the activation context to load this manifest at runtime, then finally try to load the DLL. For the sake of clarity, I’ve omitted any error checking in the code below.

ACTCTXW actctx = { 0 };
actctx.cbSize = sizeof(actctx);
actctx.dwFlags =
actctx.lpSource = modulePath;
actctx.lpResourceName = manifest;
actctx.hModule = module;
HANDLE activationContext = CreateActCtxW(&actctx);

If the DLL isn’t available, CreateActCtx will return INVALID_HANDLE_VALUE and GetLastError() will return ERROR_SXS_CANT_GEN_ACTCTX. If the activation context is successfully created then it is simply a case of activating it and loading the library as normal:

ULONG_PTR cookie;
ActivateActCtx(activationContext, &cookie)
library = LoadLibraryW(L"YourLib.dll");

The activation context is no longer required after loading the DLL, so it’s a good idea to clean up after yourself:

DeactivateActCtx(0, cookie);

This technique is also useful if you want to support multiple different versions of the same library by including a separate manifest for each version of the library you support and trying each of them in order at runtime until one of them works.

A complete example which attempts to load three different versions of the Microsoft Flight Simulator SimConnect SDK can be downloaded here:

Posted in programming | Comments Off on Linking to a WinSxS DLL that might not be there

Ineffective small functor optimisation in Visual Studio 2012

C++11 introduced the std::function class, which acts as a generic wrapper around something callable (a functor). Among other things, these are very useful for callback functions, as they allow non-static member functions to be used, which isn’t possible with stock function pointers without some wrangling. A common idiom is to use an opaque context pointer along with a static member function that calls a non-static member function to do the actual work:

void longOperation(void (*callback)(void *context),
    void *context)
    // Do some work then invoke the callback function

class MyClass
    void doSomething()
        longOperation(onCallback, this);

    static void onCallback(void *context)
        static_cast<MyClass *>(context)->doOnCallback();

    void doOnCallback()
        // Handle the callback

std::function simplifies this as it can hold any functor, meaning it can hold a member function pointer along with a pointer to the associated object:

void longOperation(function<void ()> callback)
    // Do some work then invoke the callback function

class MyClass
    void doSomething()
        longOperation(bind(&MyClass::onCallback, this));

    void onCallback()
        // Handle the callback

When using std::bind, arguments can be bound, meaning their value is fixed when the functor is created, or unbound using placeholders, meaning their value is set when the functor is invoked:

void longOperation(function<void (int)> callback)
    // Do some work then invoke the callback function

class MyClass
    void doSomething()
        longOperation(bind(&MyClass::onCallback, this,
            // This argument is unbound as it is using
            // std::placeholders::_1
            // This argument is bound and the value is
            // stored inside the functor object

    void onCallback(int x, int y)
        // x == 123
        // y == 456

Implementations of std::function will typically optimise the common case of small functors (such as a member function pointer and an object, which would typically take 8 bytes on a 32-bit architecture) by allocating them on the stack, while larger functors will go on the heap, which is where the point of this post lies. The Visual Studio 2012 implementation allocates 16 bytes of stack space for small functors, which should be plenty of space for most use cases. Unfortunately, there is a lot of waste in the implementation. The functor is wrapped in a _Func_impl object, which, in addition to holding the functor, has an allocator and a base class with virtual methods. The virtual methods add 4 bytes to the class for the virtual function pointer table, and the allocator, although it is an empty std::allocator, must be addressable, and due to alignment requirements adds another 4 bytes. This means the 16 bytes of space available for the small functor optimisation is now cut down to just 8 bytes.

8 bytes is of course enough to hold a member function and object pointer, but we now run into an issue with the implementation of std::bind. The Visual Studio 2012 implementation of std::bind uses 4 bytes for every placeholder. This means the result of bind(&MyClass::onCallback, this, _1), for instance, takes 12 bytes, which, with the 8 bytes of overhead in std::function, means it is too big for the small functor optimisation and will end up on the heap. This effectively means the small functor optimisation will only work for member functions that have no arguments at all.

The Boost implementation fares much better in comparison. boost::function allocates 24 bytes on the stack for functors, and it does not wrap the functor in anything, meaning the full space is available. Additionally, placeholders in boost::bind take no space at all. This means there is a full 16 bytes available for bound arguments before the functor will be forced onto heap in boost::function.

The small functor optimisation can be important in applications that make heavy use of functors as it will avoid a large number of small heap allocations. As such, I would recommend sticking to the Boost versions if you are targeting Visual Studio 2012.

Note: The above is based on my analysis of the code in Boost and the Visual Studio 2012 headers and may contain errors. Please leave a comment if you think I’ve made a mistake! It also does not reflect behaviour in the 64-bit build, which I imagine would be similar, but haven’t tested. I’d be interested to hear if it’s implemented any differently in Visual Studio 2013.

Posted in programming | 5 Comments

Getting useful C++ exception information from Visual Studio

Trying to get useful information about a C++ exception from a crash dump in Visual Studio is often an exercise in frustration, with useless dialogs like this being typical:

In this post I’m going to present two ways of getting more information from the exception. The second method is more reliable and easier to use, but also requires a minor code change.

The first thing to do is find the _CxxThrowException frame in the stack trace. This contains the information you need to find the exception object.

In the local variables window navigate to ThisExceptionparamspExceptionObject. This carries the pointer to the object that was thrown.

Getting the type of the exception object may take a bit of guesswork, but can probably be determined by looking at pThrowInfo, or at where the exception was thrown in the code. Otherwise, the std::exception base class is usually fine anyway. In this case it is std::system_error.

Copy the address into the watch window and add an appropriate cast. You now have all the exception information you need! You may need to switch to the frame in which the exception was thrown in your application for the cast to work, otherwise the debugger won’t know about the type you specified.

We can see here that we failed to open some super-important file with an error code of 3, which we know means the file doesn’t exist.

Unfortunately, sometimes you might find your exception information has been optimised away:

If this happens you’re pretty much out of luck for any existing crash dumps, but if you switch to method two then future dumps should be more useful.

The second method involves installing your own unhandled exception filter and forcing the process to exit immediately, while still preserving the exception information. I generally prefer this method because it is more reliable and easier to work with.

First thing to do is define your unhandled exception filter:

LONG WINAPI unhandledExceptionFilter(EXCEPTION_POINTERS *exInfo)
        exInfo->ContextRecord, 0);
    return EXCEPTION_CONTINUE_SEARCH; // Not reached

int main()

    // ...

Here we use RaiseFailFastException to kill the process immediately and invoke Windows Error Reporting to create your crash dump.  This bypasses whatever the C runtime does which causes it to lose your exception information and you get a much more useful dialog when you enter the debugger:

This time we’ve got both the type of the exception and its address (which in this case is 0x0028FD38). Put this in your watch window in the same way as described before and you will have your exception object!

Bonus: You can get WER to generate full crash dumps when your program crashes and you don’t have a debugger attached by creating the DWORD registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\DumpType and setting it to 1 (for mini dumps) or 2 (for full dumps). This will create a dump file in %LOCALAPPDATA%\CrashDumps when your program crashes which you can import into Visual Studio. This is particularly useful when you are testing on someone else’s machine where you can’t debug directly.

Posted in programming | 1 Comment

Synchronising Audio Generation and Consumption

As my final year university project I am building an emulator and source-level debugger for the Game Boy and Game Boy Color; today’s post is about the troubles I have had whilst emulating the sound system for the Game Boy. The technical details about how the Game Boy sound hardware works is not important for this post; the problem is with getting the audio after it has been emulated to the sound card. Audio buffer underflows and overflows have been causing me much frustration ever since I wrote the initial implementation a few months back, as each time it happens it creates an audible skip in the playback. I initially just assumed it was because the SDL audio library requests audio from a separate thread (I’m still not convinced this is a good idea, even though it turned out not to be the cause of the problem). Because of this I moved the sound generation code into the audio thread so it was generated on demand.

This seemed to to work pretty well, mainly because the Game Boy sound system is almost completely isolated from everything else; the game just sets a couple of registers indicating what tone it wants to play then the sound hardware goes away and generates the appropriate tones. Of course, as is quite typical with old consoles, games like to push the hardware to the limit, which means timings must be extremely accurate in order for an emulator to work correctly. As it turns out, in Pokemon Yellow when the game plays the ‘Pikachu’ sound, the game forces the sound hardware to play the sample-based sound (something which it is not very good at as standard) by switching the sound on and off as fast as the CPU will allow. Naturally this doesn’t work at all when the sound emulation thread is only getting switched in every few hundred milliseconds, so I was forced to bring the sound emulation back into the main thread for the sound to be emulated correctly.

And so the skipping came back with it…

Still convinced that the threading was the problem I decided to rebuild the sound emulation on top of Microsoft’s WASAPI. Not only is WASAPI closer to the hardware than higher level abstractions like SDL, meaning reduced latency and better quality, but, crucially, it can all run in the same thread. So after spending several hours writing a WASAPI backend that ended up being five times longer than the SDL backend (it wouldn’t be a Microsoft API if it wasn’t significantly more complicated than everything else), it was still skipping… Not impressed.

After this I focused more on how fast the samples were actually being generated versus how fast they were being consumed. After adding a bit of instrumentation code I found that the samples were being generated about 200 Hz too slowly. Whoops. I should point at here that I had two synchronisation timers running in parallel in my emulator: a simple idle loop that ran every frame to keep from the emulator running too fast, and the audio timer which the operating system/audio hardware uses to request samples at the correct rate. I fixed up the idle loop timer and got it generating the sound at exactly the right rate +/- 1 Hz. This fixed the problem to a large extent, but it was skipping much more than I would like.

It’s about now that it hit me that the solution had been staring me in the face the entire time: use only one timer. The audio timer is required to make any sound, so the idle loop timer had to go. Instead I replaced it with a callback in the audio backend which is called at a fixed frequency derived from the audio clock based on when it requests new sound data. The sound samples could now be generated at exactly the rate they were required and overflows and underflows simply stopped happening. I still opted to keep the WASAPI backend because the SDL backend still seems to skip periodically, but even with SDL the problem is basically gone.

I’ve put this here in the hopes it might be useful to someone; I know I had trouble finding any useful information from my searches.

Posted in programming | Comments Off on Synchronising Audio Generation and Consumption

The Ambiguity of Flight Plans

Some time ago I wrote a utility to convert a flight plan route into a series of waypoints so it could be used by other applications. On the face of it, this seems like a pretty simple task; all you need to do is convert something like this:


Into this:


T420 here is an airway; all we need to do is find the waypoints on the airway between BUZAD and WELIN. Unfortunately, it turns out flight plan routes are actually quite ambiguous. For example, you can’t be certain whether T420 is the name of an airway, or if it is a waypoint that should be directly flown to. In this case it’s pretty easy as there are no waypoints named T420, so it must be an airway. Often times, however, there will be many waypoints and airways going by the same name throughout the world, so you just have to make a best guess based on the surrounding context. Much tweaking was necessary in order to get it to a state where it wouldn’t reject valid routes as invalid.

A particularly memorable issue I had was with this route, purely because it was the only route in the large test set I had which had the problem:

EGPH TLA N864 DCS [...]

Decoded, this means fly from Edinburgh Airport via a Talla SID to TLA, then follow the N864 airway to DCS. I don’t consider SIDs while parsing as there is usually more than one with the same endpoint; instead I just take the waypoint with that name closest to the airport then continue from there. However, for some reason with this particular route I was getting errors saying it couldn’t find N864 after TLA. As it turns out there is an NDB with the same name in the area, and as it happens the NDB is 0.1 NM closer to the airport than the VOR to which the airway is attached. This issue forced me to change the logic for handling direct routings such that it will look at all the potential waypoints for an attached airway with the name of the following token before giving up and just using the closest available.

Posted in aviation, programming | Comments Off on The Ambiguity of Flight Plans

Back Online

Finally, after a long period of downtime, I am back online. Back when the site was first set up it was hosted for free by a friend of mine in exchange for some programming help with C++. Unfortunately, I made the mistake of allowing said friend to register the domain in his name. Some time later he closed his hosting down, and I was stuck with a domain I couldn’t use and didn’t actually own. I tried to get him to transfer the domain ownership to me, but he was unable to do so for whatever reason, and he was always very difficult to get hold of, so I pretty much gave up.

However, the other day I checked the status of the domain on a which and discovered it had expired and was free to re-register, which was weird because he claimed to have re-registered the domain when attempting to transfer ownership. But regardless, I am finally in control of my own domain and have got a VPS set up with XILO, at the recommendation of another friend. Luckily I had a backup of the WordPress database from before the server went down, and I managed to find most of the uploaded media files still lying around on my computer, so not too much was lost.

I had a number of options for which Linux distributions could be pre-installed, which basically boiled down to the only two I am familiar with: CentOS and Gentoo. I initially set up Gentoo; however, after a couple of failed kernel upgrades I gave up and installed CentOS. The problem is that working on it remotely, you’re basically stuck if the system fails to boot. Perhaps for the best anyway; I’m more familiar with locking down CentOS from the excellent Guide to the Secure Configuration of Red Hat Enterprise Linux 5 produced by the NSA which I have used to great effect at work.

Setting up BIND was an interesting challenge, as I have never done anything with DNS before. An unusual aspect of XILO is that they don’t provide nameservers, so you have to set up your own, but that raises the question: how does one resolve the nameserver when trying to resolve Turns out that is achieved with the help of so called “glue records”. The IP address of the name server is stored in a glue record on the top-level domain. Now when someone needs to resolve, they use the glue record to resolve to and subsequently query my BIND server to resolve, which of course also comes out to as everything is running on the same server.

Posted in linux, self | Comments Off on Back Online

Homebrew Earpiece for the Yaesu FT-60

I’m going to the Bournemouth Air Festival this week, and of course will be bringing my HT for that extra ATC goodness. Unfortunately though, the Yaesu FT-60 doesn’t have a normal headphones socket. It does have a 3.5mm jack socket, but it has four contacts, and plugging in a normal three contact connector grounds the PTT. No good.

I checked the manual, and it seems I need the Yaesu MH-37A4B. I looked at various stores online and the cheapest I could find it for was about £25. No thanks; it’s a cheap earphone with an abnormal connector. So of course I built my own. I bought a four pole jack from Maplin (QT65V) for £2, and some el cheapo earphones for £6, chopped off the connector, ripped off the right earpiece and soldered on the new connector. It works perfectly and was a third of the price. I call that a successful mission.

Posted in amateur radio, elecronics | Comments Off on Homebrew Earpiece for the Yaesu FT-60

PC→Transceiver Connection for Data Modes

I’ve wanted to properly connect my radio to my PC for some time, enabling me to use data modes. I figured instead of getting a RIGblaster I’d build my own; it’s a fairly simple circuit. I spent a few days designing the circuit with the assistance of a couple of websites.[1. WM2U’s Sound Card Interfacing Schemes] [2. AGWPE Documentation] This is what I came up with:

This is actually four electrically separate circuits. The computer side is completely isolated from the radio side to prevent a ground loop from distorting the signal. I will explain each section of the circuit below.

Most (if not all) amateur radio data mode applications have an option of activating the RTS line on a serial port when transmitting; this is used to activate the PTT circuit on the rig. An opto-isolator is used to isolate the two sides of the circuit. This consists of an LED and a phototransistor. When current flows through the LED, current is allowed to flow through the transistor, grounding the PTT line and activating the transmitter. A 1.2kΩ resistor is used to reduce the current reaching the LED; the RS-232 specification allows for up to 25V, which is too much for the diode. The D1 diode acts as a shunt. When the RTS line is ‘off’, it can be up to -25V, which would damage the LED in the opto-isolator if allowed to flow through it. Instead the current flows through the diode and out the RTS line.

Next is the transmit audio input. This is connected to the line out connection on the PC sound card. This time an audio transformer is used to isolate the two circuits. The manual for my rig specifies the maximum input level is 40mV, so a resistive divider is necessary to reduce the ≈2V input. I generated a sine wave in Audacity on my computer, set the volume to maximum, set my voltmeter to AC mode and measured exactly 2.5V output. I used a resistive divider calculator to get the 62k and 1k resistor values.

Finally the receive audio is pretty much the same as the transmit audio, except the voltage is ≈1.5V, which is fine for the computer line in, so this doesn’t need anything more.

Having designed and had checked the circuit I bought all the components from Farnell and got to work. After spending pretty much all day on it, I finally finished in the dwindling light:

Everything went perfectly on the first attempt; I didn’t break anything, receive and transmit audio, and PTT all work perfectly. If you want to build your own I encourage you to check the links in the footnotes as they provide more detail. This circuit ought to work for any rig with a Mini-DIN 6 input (usually marked ‘data’). I’ll be happy to answer any questions if you leave a comment.

Posted in amateur radio, elecronics | Comments Off on PC→Transceiver Connection for Data Modes

Decoding the Yaesu FT-7800 Clone Data Format

I bought a data cable for my radio a few days ago to connect it to my computer so I could program the memories more easily. I figured instead of buying the programming software I’d try to decode the data format the radio uses and write my own. It’s just more fun that way.

The premise is simple enough, you need to make the application act like the radio and use clone mode to get the memory dump from the rig. Once you have made your changes you do the same thing in reverse to send the data back.

The transfer protocol is pretty simple, it’s essentially a file transfer protocol which always sends a file of the same size. Upon enabling clone transmit mode on the rig it sends the ASCII string “AH016$u$”. It’s not clear exactly what this means so I just check for it verbatim. Upon receiving this you send back an ACK character (0x06) and the radio starts spewing out its data.

The data is sent in 64 byte packets,[1. It’s not really a packet in the normal sense, there’s no header data or anything like that, it’s just a 64 byte block of raw data.] after each of which an ACK must be sent. The rig sends 493 of packets to make a total of 31 552 bytes. After the last packet a single byte checksum is sent, which must be acknowledged.

The Checksum

Figuring out the checksum algorithm was something I wasn’t really looking forward to, I figured it would be pretty hard to do; fortunately, this turned out not to be the case. I started by just accumulating all the bytes in the data (discarding overflow). This didn’t work, but after making several dumps with minor changes I noticed that the checksum was always at a certain offset from the accumulation of all the bytes. 0x23 less to be precise. What’s so special about 0x23 I have no idea, but it seems to work so it doesn’t really matter.


Something quite frustrating I have come across with this is that the radio echoes everything sent to it, even if it’s off. I figure there must be a direct connection between the receive and transmit lines or something. It means I have to read back everything after I have just written it. Annoying? Yes, but not a huge issue really.

Decoding the Data

Decoding the data format is simply a case of changing what you are looking for on the radio then comparing the dumps to see where they differ. I decided to start by looking for the memory names in a hex editor, they’re probably stored as ASCII right? Apparently not. I couldn’t find any ASCII strings in the data at all. I changed the name of one of the memories and made another dump to see what was what; it seems it uses its own 6-bit character set, which includes several weird glyphs that don’t seem to map to any actual characters. I have decoded and included the character set in the table below.

Dec Hex Oct Character Glyph
0 0 0 0
1 1 1 1
2 2 2 2
3 3 3 3
4 4 4 4
5 5 5 5
6 6 6 6
7 7 7 7
8 8 10 8
9 9 11 9
10 A 12 A
11 B 13 B
12 C 14 C
13 D 15 D
14 E 16 E
15 F 17 F
16 10 20 G
17 11 21 H
18 12 22 I
19 13 23 J
20 14 24 K
21 15 25 L
22 16 26 M
23 17 27 N
24 18 30 O
25 19 31 P
26 1A 32 Q
27 1B 33 R
28 1C 34 S
29 1D 35 T
30 1E 36 U
31 1F 37 V
32 20 40 W
33 21 41 X
34 22 42 Y
35 23 43 Z
36 24 44 SP
37 25 45
38 26 46
39 27 47
40 28 50 $
41 29 51
42 2A 52
43 2B 53
44 2C 54
45 2D 55
46 2E 56
47 2F 57 +
48 30 60
49 31 61
50 32 62
51 33 63 /
52 34 64 |
53 35 65
54 36 66
55 37 67
56 38 70
57 39 71
58 3A 72
59 3B 73
60 3C 74
61 3D 75
62 3E 76
63 3F 77 _

I have found the locations of a few other details, like the frequency and modulation mode, but I will leave those details for another day once I have got a better grasp of the format.

Posted in amateur radio, programming | Comments Off on Decoding the Yaesu FT-7800 Clone Data Format

SSTV Captures from the International Space Station

Today and tomorrow (July 15–16) the amateur radio station aboard the International Space Station is broadcasting SSTV images on 145.800 MHz.[1. MAI-75 activation planned for July 15-16] I have managed to capture a couple of images as it passed over northern France today. Neither is a complete image sadly, but it’s still very cool.

Something interesting that I haven’t experienced before with receiving data from satellites was that a slight movement of the antenna could mean the difference between a strong signal and complete noise; hence the thin lines of noise in the images. Not sure why it should be different this time, normally the signal fading is quite gradual.

Posted in amateur radio | Tagged | Comments Off on SSTV Captures from the International Space Station