Returning C structs by value on different platforms. This was transcribed from the Factor FFI implementation and may be incorrect.

References:

• http://www.x86-64.org/documentation/abi.pdf - System V ABI for AMD64 (this is used on x86-64 Unix systems)
• http://msdn.microsoft.com/en-us/library/ms794533.aspx - Windows ABI for x86-64
• http://developer.apple.com/documentation/DeveloperTools/Conceptual/LowLevelABI/000-Introduction/introduction.html - Mac OS X calling conventions for x86-32, x86-64, PowerPC 32-bit and PowerPC 64-bit
• Calling conventions on other platforms were determined by trial and error

x86-32

On x86-32, we distinguish between "small structs" and "large structs".

On Linux, NetBSD and Solaris, all structs are large structs except C99 complex float values.

On Mac OS X, Windows, FreeBSD and OpenBSD, all structs are large structs except those that are precisely 1, 2, 4 or 8 bytes in size.

Note that a struct that's, for instance, 3 bytes in size is considered a large struct.

Small structs

Small structs are returned in EAX:EDX, with the first 4 bytes in EAX and the second 4 bytes in EDX. If the struct is smaller than 4 bytes in size, EDX is undefined.

Large structs

Large structs are handled as follows. The struct-returning function actually has a hidden first parameter which must be a pointer to a memory area large enough to store the returned struct. This memory area must be supplied by the caller. The EAX and EDX registers are unused in this case.

If the struct-returning function uses the stdcall calling convention (Windows only), then it behaves exactly like a void-returning function with an extra first parameter. However, on non-Windows platforms where the C calling convention is used, the struct-returning function returns with the stack pointer incremented by 4 bytes. So after calling such a function, if you care about the value of ESP, you must decrement ESP by 4 bytes, by pushing a dummy value, for instance.

x86-64

On x86-64, the situation is similar; there are small structs and large structs.

Large structs

Large structs are handed in a similar way to x86-32, where the caller passes a pointer to a memory area large enough to hold the struct as the first parameter. Unlike x86-32, there is no need to fix up RSP afterward.

Small structs

On Unix, a small struct is one that is 16 bytes in size or smaller. On Windows, a small struct is one that is precisely 1, 2, 4, 8 bytes in size. All other structs are large structs.

Small structs are "flattened" into a pair of C types, where each element of each pair is either void or double. That is, there are four combinations:

• void* void*
• void* double
• double void*
• double double

On Unix, structs are flattened by looking at the first and second 8 bytes of the struct. If a component consists entirely of floating point data (either a single double field or a pair of float fields) then that component flattens to double. Otherwise it flattens to void*.

Eg, the following struct,

struct foo { int x; float y; double z; }; 

flattens to void* double.

On Windows, all small structs flatten into void* void*.

Once the small struct has been flattened, we can compute which registers it will be returned in:

• void* void* => RAX RDX
• void* double => RAX XMM0
• double void* => XMM0 RAX
• double double => XMM0 XMM1

Note that if a component contains two float values, it is returned in the first two co-ordinates of the corresponding vector register.

So in the above foo example, the values x and y are returned in RAX, and z is returned in XMM0.

PowerPC 32-bit

On PowerPC 32-bit, all structs are returned by the caller passing a pointer to a memory area as the first parameter, except for C99 complex float and complex double values, which are returned in integer registers r3:r4:r5:r6.

PowerPC 64-bit

Factor does not run on PowerPC 64-bit.