win3mu

Win3mu - Part 10 - Leveraging Reflection

Brad Robinson

— 6 min read
Win3mu - Part 10 - Leveraging Reflection

This is Part 10 in a series of articles about building Win3mu — a 16-bit Windows 3 emulator. If you’re not sure what this is or why I’m doing it you might like to read from the start.

In a previous post about the Module Loader I gave an example of what’s involved in calling a Windows API method from VM code. In that example I showed how parameters are read from the VM stack and type converted, the real Windows API called and then how the return value is passed back to the VM.

All this parameter and type conversion code is not only tedious to write but error prone if done by hand. This post explains how Win3mu leverages C# reflection to automatically do the heavy lifting.

What is Reflection?

Reflection is a mechanism by which a program can inspect itself. For example in C# all of a type’s methods can be found like this:

foreach (var mi in typeof(SomeClass).GetMethods())
{
    Console.WriteLine(mi.Name);
}

Similarly, a method’s parameters can be discovered:

foreach (var pi in mi.GetParameters())
{
    Console.WriteLine(pi.Name);
}

Also important to this discussion are Custom Attributes. Custom attributes are used to decorate code with additional information that can be retrieved at run time (via reflection).

For example, here is a method decorated with a custom attribute called “EntryPoint”:

[EntryPoint(0x0001)]
public void MessageBox()
{
}

In Win3mu all emulated modules (eg: Kernel, User, GDI etc…) derive from Module32 whose primary responsibility is to use reflection to provide the plumbing between the VM and the API methods that each module provides.

Creating API Thunks

The first thing Module32 does is create call thunks for each of the methods that the module exports. Exported API methods are decorated with an “EntryPoint” attribute that declares the 16-bit ordinal of the exported method.

// Create thunks for all exported methods
foreach (var mi in this.GetType().GetMethods())
{
    // Check it has an EntryPoint attribute
    var ep = mi.GetCustomAttributes<EntryPointAttribute>().FirstOrDefault();
    if (ep==null)
        continue;
        
    // Create a system thunk for it
    var vmptr = _machine.CreateSystemThunk(() => {
        // When thunk called, use reflection to call the method
        mi.Invoke(this);
    });
    
    // Connect the ordinal and the thunk (GetProcAddress will
    // return this ‘vmptr’ when asked for ordinal ‘ep.ordinal’)
    _mapProcAddress[ep.ordinal] = vmptr;
}

Any method with an EntryPoint attribute is now callable from 16-bit code.

Handling Parameters

The next task is to lift the parameter values from the VM stack and pass them to the C# function. Module32 looks at the function’s parameters (using reflection) and works out how to get these parameters from the VM stack.

The following code shows how two simple types “ushort” and “string” are read from the calling VM stack and passed to a C# function:

// The parameters that will be passed to the C# code via reflection
List<object> csharpParameters = new List<object();

// Position of next parameter on the VM stack (+4 because the
// caller’s return address is also on the stack)
ushort vmStackPos = _machine.sp + 4;

// Convert all parameters
foreach (var pi in mi.GetParameters())
{
    if (pi.ParameterType == typeof(ushort))
    {
        // Read 2 bytes from the stack
        csharpParameters.Add(_machine.ReadWord(_machine.ss, vmStackPos));
        vmStackPos += 2;
    }
    else if (pi.ParameterType == typeof(string))
    {
        // Read string pointer from the stack 
        var stringPtr = _machine.ReadDWord(_machine.ss, vmStackPos));
        vmStackPos += 4;
        
        // Read the string from VM memory
        string str = _machine.ReadString(stringPtr);
        csharpParameters.Add(str);
    }
}

// Call the C# function, passing parameters
var retv = mi.Invoke(this, csharpParameter.ToArray());

// Return value to VM
_machine.ax = (ushort)Convert.ChangeType(retv, typeof(ushort));

Although not shown in the above example, the return value is handled similarly by inspecting the method’s return type and updating the ax (or dx:ax) register after the function has been invoked.

The MessageBox function can now be implemented like this:

// The real API method imported from Windows
[DllImport(“user32.dll”)]
static extern uint MessageBox(IntPtr hWndParent, string msg, string title, uint options);

// Implementation method exported to 16-bit code
[EntryPoint(0x0001)]
public ushort MessageBox(ushort hWndParent, string msg, string title, ushort options)
{
    // No more messing around with the VM stack for parameters
    // nor with the ax register for the return value. Nice!
    return (ushort)MessageBox(IntPtr.Zero, msg, title, (uint)options);
}

This is starting to look a lot cleaner.

Type Widening and Narrowing

If you look closely at the above example you’ll notice there’s no actual code — it’s essentially one function directly calling another and its only purpose is to convert the options parameter from ushort to uint and the return value from uint to ushort.

This widening and narrowing of types is prevalent throughout the set of API methods that Win3mu needs to support so it’s worth including some automatic support for this through two new types “nint” and “nuint”.

I stole these names from Xamarin for iOS where the “n” stands for “native” (as in native to the platform on which it’s running). In Win3mu a nuint is a 16-bit on the 16-bit side and a 32-bit on the C# side.

[MappedType]
public struct nuint
{
    public nuint(uint value)
    {
        this.value = value;
    }

    public uint value;

    public static nuint To32(ushort value)
    {
        return new nuint(value);
    }

    public static ushort To16(nuint value)
    {
        return (ushort)value.value;
    }

    public static implicit operator nuint(uint value)
    {
        return new nuint(value);
    }

    public static implicit operator uint (nuint value)
    {
        return value.value;
    }

    public override string ToString()
    {
        return value.ToString();
    }
}

Mapped Types

Originally Module32 included direct support for nint and nuint but it didn’t take long to realize that there were quite a few types that needed similar special handling. Rather than hard code support for each of these into Module32, it now looks for types declared as mapped types.

Take a closer look the nuint type above. Notice the [MappedType] custom attribute? This tells Module32 that it’s a type that needs some special handling when transitioning between 16 and 32-bit worlds.

Mapped types must have two static methods named “To32” and “To16” which will be called to do the conversion. You can see these two methods in the above example.

Win3mu has MappedTypes for many of the Windows API types. eg: POINT, SIZE, RECT, HWND, HDC etc…

Revisiting MessageBox

Applying all of the above reduces the MessageBox function to just its declaration. No actual code!

[EntryPoint(0x0001)]
[DllImport(“user32.dll”)]
public static extern nuint MessageBox(HWND hWndParent, string msg, string title, nuint options);

Other Parameter Types

There are a few other cases that Module32 can cope with like ref and out parameters, special handling for output strings, structure types and a few other edge cases.

Of course not every function can be directly mapped onto a Windows implementation but there are large sections of the Windows API where this works really well.

GDI for example has many functions that can be directly mapped while Kernel has only a few — since most are implemented by Win3mu.

In cases where the automatic mapping doesn’t work it’s easy to fall back to a simpler type and handle it in the method implementation.

For example, the TextOut method has a string parameter whose length is passed in another parameter. I could add support to the type mapping for this, but it’s a rare case so I declared the string parameter as a uint (the same size as a 16-bit far string pointer) and wrote code to handle it specially:

[EntryPoint(0x0021)]
public bool TextOut(HDC hDC, nint x, nint y, uint pszString, nint cbString)
{
    // Get the buffer backing pszString
    int offset;
    var buf = _machine.GlobalHeap.GetBuffer(pszString, false, out offset);
    if (buf == null)
        return false;
    
    // Read the string from the buffer, using length param
    var str = Machine.AnsiEncoding.GetString(buf, offset, cbString);

    // Call real Windows API
    return TextOut(hDC, x, y, str, cbString);
}

What About Performance?

Reflection makes it really easy to setup these bindings between 16 and 32-bit code. The downside is that performance isn’t great:

  • MethodInfo.Invoke() is well known to be slower than calling a function directly from C# code.
  • Every time an API method is called the parameters need to be re-investigated, stack and type conversion look ups done, an array created for the parameters and then similar handling for the return value. There’s a lot of overhead on each and every call.

So far I’ve only noticed performance issues with one program. There’s a Checkers program (by Gregory Thatcher) makes about 50,000 calls to LocalAlloc and LocalFree when it’s planning it’s next move. That’s 100,000 API calls which takes less than a second under DosBox but about 6 or 7 seconds on Win3mu.

(I double checked — it’s not a problem with the local heap implementation. Making a similar number of calls from C# code took just a few milliseconds so this is probably related to reflection and/or perhaps just slower CPU execution — but that’s another area that hasn’t been performance tuned yet).

I’m not too worried about this because it can be fixed later by generating dynamic methods. Dynamic methods are a way to generate code (MSIL) at run-time that’s compiled to machine code when executed.

Rather than inspecting a method’s parameters each time the method is executed, eventually Win3mu will look at the parameters once and generate a dynamic method that knows how to do all the conversion. Not only will that code be compiled to machine code, it won’t have to re-investigate the parameter types each time since that information will be baked into the generated code.

I used this approach in PetaPoco (a tiny database ORM) with excellent results — it gives performance close to that of compiled code with all the advantages of reflection.

Writing dynamic methods is a little like writing assembly language (see here for an example). This is why Module32 doesn’t try to handle every different kind of parameter mapping. By keeping it simple, things will be easier when it comes time to dynamically generate code.

Read more