Ineffective small functor optimisation in Visual Studio 2012

C++11 introduced the std::function class, which acts as a generic wrapper around something callable (a functor). Among other things, these are very useful for callback functions, as they allow non-static member functions to be used, which isn’t possible with stock function pointers without some wrangling. A common idiom is to use an opaque context pointer along with a static member function that calls a non-static member function to do the actual work:

void longOperation(void (*callback)(void *context),
    void *context)
{
    // Do some work then invoke the callback function
    callback(context);
}

class MyClass
{
public:
    void doSomething()
    {
        longOperation(onCallback, this);
    }

private:
    static void onCallback(void *context)
    {
        static_cast<MyClass *>(context)->doOnCallback();
    }

    void doOnCallback()
    {
        // Handle the callback
    }
};

std::function simplifies this as it can hold any functor, meaning it can hold a member function pointer along with a pointer to the associated object:

void longOperation(function<void ()> callback)
{
    // Do some work then invoke the callback function
    callback();
}

class MyClass
{
public:
    void doSomething()
    {
        longOperation(bind(&MyClass::onCallback, this));
    }

private:
    void onCallback()
    {
        // Handle the callback
    }
};

When using std::bind, arguments can be bound, meaning their value is fixed when the functor is created, or unbound using placeholders, meaning their value is set when the functor is invoked:

void longOperation(function<void (int)> callback)
{
    // Do some work then invoke the callback function
    callback(123);
}

class MyClass
{
public:
    void doSomething()
    {
        longOperation(bind(&MyClass::onCallback, this,
            // This argument is unbound as it is using
            // std::placeholders::_1
            _1,
            // This argument is bound and the value is
            // stored inside the functor object
            456));
    }

private:
    void onCallback(int x, int y)
    {
        // x == 123
        // y == 456
    }
};

Implementations of std::function will typically optimise the common case of small functors (such as a member function pointer and an object, which would typically take 8 bytes on a 32-bit architecture) by allocating them on the stack, while larger functors will go on the heap, which is where the point of this post lies. The Visual Studio 2012 implementation allocates 16 bytes of stack space for small functors, which should be plenty of space for most use cases. Unfortunately, there is a lot of waste in the implementation. The functor is wrapped in a _Func_impl object, which, in addition to holding the functor, has an allocator and a base class with virtual methods. The virtual methods add 4 bytes to the class for the virtual function pointer table, and the allocator, although it is an empty std::allocator, must be addressable, and due to alignment requirements adds another 4 bytes. This means the 16 bytes of space available for the small functor optimisation is now cut down to just 8 bytes.

8 bytes is of course enough to hold a member function and object pointer, but we now run into an issue with the implementation of std::bind. The Visual Studio 2012 implementation of std::bind uses 4 bytes for every placeholder. This means the result of bind(&MyClass::onCallback, this, _1), for instance, takes 12 bytes, which, with the 8 bytes of overhead in std::function, means it is too big for the small functor optimisation and will end up on the heap. This effectively means the small functor optimisation will only work for member functions that have no arguments at all.

The Boost implementation fares much better in comparison. boost::function allocates 24 bytes on the stack for functors, and it does not wrap the functor in anything, meaning the full space is available. Additionally, placeholders in boost::bind take no space at all. This means there is a full 16 bytes available for bound arguments before the functor will be forced onto heap in boost::function.

The small functor optimisation can be important in applications that make heavy use of functors as it will avoid a large number of small heap allocations. As such, I would recommend sticking to the Boost versions if you are targeting Visual Studio 2012.

Note: The above is based on my analysis of the code in Boost and the Visual Studio 2012 headers and may contain errors. Please leave a comment if you think I’ve made a mistake! It also does not reflect behaviour in the 64-bit build, which I imagine would be similar, but haven’t tested. I’d be interested to hear if it’s implemented any differently in Visual Studio 2013.

This entry was posted in programming. Bookmark the permalink.

5 Responses to Ineffective small functor optimisation in Visual Studio 2012

  1. kuhar says:

    Is the situation different on VC12?

  2. Navin Shajan says:

    If you post a test case, I can run it on VS 2013 (v120) and the Nov2013 CTP.

Comments are closed.