Introduction

FFI and Memory Layout

Overview

libxev-go uses FFI (Foreign Function Interface) via the jupiterrider/ffi library to call into libxev (written in Zig) without requiring cgo. This approach provides several benefits:

• Pure Go build (no C compiler required)
• Better cross-compilation support
• Smaller binary size
• Better goroutine integration

However, FFI requires careful attention to memory layout and struct alignment between Go and Zig.

The Zig Field Reordering Issue

Problem

Zig automatically reorders struct fields by their alignment to minimize padding and ensure optimal memory access. This means fields are not necessarily laid out in memory in the order they are declared.

Example: xev.Options

In libxev’s Zig code, Options is declared as:

pub const Options = struct {
    entries: u32 = 256,
    thread_pool: ?*xev.ThreadPool = null,
};

You might expect the memory layout to be:

[entries: 4 bytes][padding: 4 bytes][thread_pool: 8 bytes]

But Zig reorders fields by alignment, resulting in:

[thread_pool: 8 bytes][entries: 4 bytes][padding: 4 bytes]

The 8-byte pointer comes first, followed by the 4-byte integer.

Go Struct Declaration

To match this layout in Go, you cannot declare fields in source order:

// WRONG - fields in source order
type LoopOptions struct {
    Entries    uint32      // offset 0
    _          uint32      // padding
    ThreadPool *ThreadPool // offset 8
}

Instead, you must declare them in alignment order to match Zig’s layout:

// CORRECT - fields in alignment order
type LoopOptions struct {
    ThreadPool *ThreadPool // offset 0 (8 bytes)
    Entries    uint32      // offset 8 (4 bytes)
    _          uint32      // padding to 16 bytes
}

Debugging Memory Layout Issues

Symptoms

When Go and Zig struct layouts don’t match, you’ll typically see:

1. SIGSEGV crashes with suspicious addresses like 0x100 (256) or other small offsets
2. Garbage values when printing struct fields (e.g., expecting 256 but seeing 1687552)
3. Crashes in memory allocators (libsystem_malloc.dylib)

Diagnostic Approach

1. Check field offsets in Zig:

   std.debug.print("offsetof(entries): {}\n", .{@offsetOf(xev.Options, "entries")});
   std.debug.print("offsetof(thread_pool): {}\n", .{@offsetOf(xev.Options, "thread_pool")});
   

2. Check field offsets in Go:

   import "unsafe"
   
   fmt.Printf("Entries offset: %d\n", unsafe.Offsetof(opts.Entries))
   fmt.Printf("ThreadPool offset: %d\n", unsafe.Offsetof(opts.ThreadPool))
   

3. Dump raw bytes:

   const bytes: [*]const u8 = @ptrCast(options);
   for (0..16) |i| {
       std.debug.print("{x:0>2} ", .{bytes[i]});
   }
   

4. Compare layouts: If offsets don’t match between Go and Zig, you need to reorder Go fields.

Best Practices

1. Order by Alignment

When creating Go structs that map to Zig structs:

1. List all fields with their sizes and alignments
2. Sort by alignment (descending): 8-byte pointers first, then 4-byte ints, etc.
3. Add explicit padding to match the total struct size

Example:

type MyStruct struct {
    // 8-byte aligned fields first
    Pointer1 *SomeType
    Pointer2 *AnotherType

    // 4-byte aligned fields
    Count  uint32
    Flags  uint32

    // 2-byte aligned fields
    ShortVal uint16

    // Explicit padding to match Zig struct size
    _ [6]byte
}

2. Document the Layout

Always add comments explaining:

• The Zig struct being mirrored
• Why fields are in a particular order
• The total size and padding

// LoopOptions matches xev.Options in libxev.
// IMPORTANT: Zig reorders struct fields by alignment!
// Actual memory layout:
//   thread_pool: ?*ThreadPool (8 bytes) at offset 0
//   entries: u32 (4 bytes) at offset 8
//   (4 bytes padding to 16 bytes total)
type LoopOptions struct {
    ThreadPool *ThreadPool
    Entries    uint32
    _          uint32
}

3. Verify with Tests

Create size verification tests:

func TestLayoutSizes(t *testing.T) {
    // Call Zig function that returns struct size
    zigSize := cxev.GetOptionsSize()
    goSize := unsafe.Sizeof(cxev.LoopOptions{})

    if zigSize != goSize {
        t.Errorf("size mismatch: Zig=%d Go=%d", zigSize, goSize)
    }
}

4. Export Size/Offset Functions from Zig

Add debug exports to verify layouts:

export fn xev_options_sizeof() usize {
    return @sizeOf(xev.Options);
}

export fn xev_options_field_offsets(entries_offset: *usize, tp_offset: *usize) void {
    entries_offset.* = @offsetOf(xev.Options, "entries");
    tp_offset.* = @offsetOf(xev.Options, "thread_pool");
}

Common Pitfalls

1. Assuming declaration order equals memory order - Zig reorders by alignment
2. Forgetting to add padding - Structs may have trailing padding
3. Not checking on both 32-bit and 64-bit - Pointer sizes differ
4. Ignoring warnings - If the code “works sometimes,” there’s likely a layout bug

Tools

• @offsetOf in Zig - Get field offset at compile time
• unsafe.Offsetof in Go - Get field offset
• @sizeOf / unsafe.Sizeof - Get struct sizes
• Debug prints with raw byte dumps - Visualize actual memory layout

Related Issues

For the specific issue that led to this documentation:

• Issue: SIGSEGV at addr=0x100 during File operations with thread pool
• Root cause: Go’s LoopOptions fields were in declaration order, but Zig’s Options had fields reordered by alignment
• Fix: Reordered Go struct fields to match Zig’s alignment-based layout
• Commit: f0e0ea3 “fix: add padding to LoopOptions for proper memory alignment”

Troubleshooting

SIGSEGV: Segmentation Violation

Symptom

SIGSEGV: segmentation violation
PC=0x187475464 m=9 sigcode=2 addr=0x100

The crash occurs when calling FFI functions, particularly LoopInitWithOptions.

Common Causes

1. Struct Layout Mismatch Between Go and Zig

Indicators:

• Crash address is a small offset like 0x100 (256), 0x80 (128)
• Crash happens in memory allocator (libsystem_malloc.dylib)
• Struct fields contain garbage values

Why it happens: Zig automatically reorders struct fields by alignment (pointers before integers), but Go keeps fields in declaration order. If your Go struct doesn’t match Zig’s actual memory layout, FFI passes incorrect data.

Solution:

1. Check field offsets in both languages
2. Reorder Go struct fields to match Zig’s alignment-based order
3. See FFI and Memory Layout for details

Example fix:

// Before (WRONG)
type LoopOptions struct {
    Entries    uint32      // offset 0
    _          uint32
    ThreadPool *ThreadPool // offset 8
}

// After (CORRECT)
type LoopOptions struct {
    ThreadPool *ThreadPool // offset 0 (Zig puts pointers first)
    Entries    uint32      // offset 8
    _          uint32
}

2. Incorrect Struct Sizes

Indicators:

• @sizeOf(Type) in Zig doesn’t match unsafe.Sizeof(Type{}) in Go
• Random crashes at different locations

Solution: Add size verification functions and tests:

// In Zig
export fn xev_loop_sizeof() usize {
    return @sizeOf(xev.Loop);
}

// In Go test
func TestSizes(t *testing.T) {
    zigSize := cxev.LoopSizeof()
    goSize := unsafe.Sizeof(cxev.Loop{})
    if zigSize != goSize {
        t.Errorf("size mismatch: zig=%d go=%d", zigSize, goSize)
    }
}

3. FFI Calling Convention Issues

Indicators:

• Crash before any code in the called function executes
• Works in some contexts but not others

Solution: Ensure FFI function signatures match exactly:

// C signature: int xev_loop_init_with_options(xev_loop* loop, xev_options* options)
fnLoopInitWithOptions, err = lib.Prep("xev_loop_init_with_options",
    &ffi.TypeSint32,    // return type: int -> sint32
    &ffi.TypePointer,   // arg1: xev_loop* -> pointer
    &ffi.TypePointer)   // arg2: xev_options* -> pointer

Extended Library Not Loaded

Symptom

Test skipped: extended library not loaded

Cause

The extended library (libxev_extended.dylib) is not found at runtime.

Solution

Set the LIBXEV_EXT_PATH environment variable:

# macOS
export LIBXEV_EXT_PATH=/path/to/libxev-go/zig/zig-out/lib/libxev_extended.dylib

# Linux
export LIBXEV_EXT_PATH=/path/to/libxev-go/zig/zig-out/lib/libxev_extended.so

# Run tests
go test ./...

Or use the justfile:

just test  # Automatically sets library paths

Thread Pool Operations Fail

Symptom

File operations don’t complete, or callbacks never fire.

Cause

The loop was initialized without a thread pool, but file operations require a thread pool on kqueue/epoll backends.

Solution

Use NewLoopWithThreadPool() instead of NewLoop():

// Wrong - no thread pool
loop, err := xev.NewLoop()

// Correct - with thread pool for file ops
loop, err := xev.NewLoopWithThreadPool()
if err != nil {
    return err
}
defer loop.Close()

Completion Pointer Issues (Historical)

Symptom (Before Fix)

SIGSEGV when file operation callbacks are invoked, particularly at addr=0x100 offset from NULL.

Historical Cause

libxev’s thread pool operations don’t preserve extended completion fields. The callback pointer stored in the completion struct was lost when operations went through the thread pool.

Solution (Implemented)

The file_api.zig now uses heap-allocated context:

const CallbackContext = extern struct {
    callback: *const anyopaque,
    userdata: ?*anyopaque,
};

// Allocate context on heap, not in completion
const ctx = std.heap.c_allocator.create(CallbackContext) catch @panic("alloc failed");
ctx.* = .{ .callback = @ptrCast(cb), .userdata = userdata };

// Pass context as userdata
f.write(loop, c, .{ .slice = buf[0..buf_len] }, CallbackContext, ctx, writeCallback);

This ensures the callback pointer survives the thread pool transition.

Debugging Tips

Enable Debug Output

Add debug prints in Zig code:

const std = @import("std");

export fn xev_loop_init_with_options(loop: *xev.Loop, options: *const xev.Options) c_int {
    std.debug.print("[DEBUG] entries: {}, thread_pool: {?}\n", .{options.entries, options.thread_pool});
    // ... rest of function
}

Rebuild and run tests to see debug output.

Check Raw Memory

Dump raw bytes to verify layout:

const bytes: [*]const u8 = @ptrCast(options);
std.debug.print("Raw bytes: ", .{});
for (0..16) |i| {
    std.debug.print("{x:0>2} ", .{bytes[i]});
}
std.debug.print("\n", .{});

Isolate the Issue

Create minimal test programs:

func TestMinimal(t *testing.T) {
    // Test just the failing component
    var opts cxev.LoopOptions
    opts.ThreadPool = &pool
    opts.Entries = 256

    fmt.Printf("Go layout: TP offset=%d, Entries offset=%d\n",
        unsafe.Offsetof(opts.ThreadPool),
        unsafe.Offsetof(opts.Entries))

    // Call and observe crash location
    err := cxev.LoopInitWithOptions(&loop, &opts)
    if err != nil {
        t.Fatal(err)
    }
}

Getting Help

If you encounter issues not covered here:

1. Check the FFI and Memory Layout guide
2. Look at recent commits for similar fixes
3. Create a minimal reproduction case
4. Open an issue with:
   • Go version
   • OS and architecture
   • Full error output including stack trace
   • Code snippet showing the problem