Some assembly still required

When I change an instruction, I don't want to remember the same edit in Go, amd64 and arm64. A comment saying “keep these in sync” is a request addressed to the version of me most likely to be in a hurry. I'm building a tool called asmgen inside Piko to help with that. The assembly implementations must agree on what an operation does, the constants and the field positions they use. Refactoring the Go shouldn't leave me hunting through assembly for numbers I've forgotten to change.

A text problem I can work with

Rob Pike's talk about the Go assembler helped me look at this differently. He describes replacing the old C assemblers with one written in Go. The new front end handles the assembly text, while an existing library called obj handles the machine details.

For testing, they gave the old and new assemblers identical input and worked towards matching every output bit, a comparison the slides explain too. Having an existing answer makes the replacement easier to check. My version is to compare the generated assembly text with the assembly I already have, which makes the work feel much more manageable to me. I can move a handler into the generator, compare its instructions with the handwritten version and check for changes before trying to improve anything else.

There are tools for writing assembly in Go already. Avo targets x86. I need both amd64 and arm64. The generator's job is to organise their shared work. A generator tied to one of them would leave me maintaining the other separately.

I'm treating asmgen as a text generator. It writes Plan 9 assembly for Go's assembler to turn into object code while I still choose the instructions and registers in each architecture's adapter. Ordinary Go functions, loops and interfaces give me plenty to organise that work with. Another language for the generator, another one to keep in my head.... I've got quite enough languages involved in the interpreter already.

One operation, two sets of instructions

An interpreter instruction says which operation to perform and which slots hold its values. For subtraction, the handler reads slots B and C, subtracts C from B, then writes the answer into slot A. The dispatch tail carries on to the next instruction.

What's shared? Here's a sketch of that order in a Go closure:

Emit: func(e *asmgen.Emitter, arch BytecodeArchitecturePort) {
    scratch := arch.ScratchRegisters()
    arch.ExtractA(e, scratch[0])
    arch.ExtractB(e, scratch[1])
    arch.ExtractC(e, scratch[2])
    arch.IntegerBinaryOperation(e, "SUB",
        scratch[0], scratch[1], scratch[2])
    arch.DispatchNext(e)
},

Those are interpreter slots. The adapter chooses the processor registers used to reach them, so I can share the order of work without making the two machines pretend to understand one instruction set.

With operands decoded, amd64 writes:

MOVQ (R8)(BX*8), SI
SUBQ (R8)(CX*8), SI
MOVQ SI, (R8)(AX*8)

R8 holds the integer-bank base (the address where the current call's integer slots begin). BX and CX hold B and C's slot indices, while AX holds A's. Multiplying an index by eight gets the byte position of an integer. SI temporarily holds the answer.

The arm64 adapter writes:

MOVD (R23)(R4<<3), R6
MOVD (R23)(R5<<3), R7
SUB  R7, R6, R6
MOVD R6, (R23)(R3<<3)

Here R23 holds the bank base. A's slot index is in R3, B's in R4 and C's in R5. Shifting an index left by three also multiplies it by eight. Both operands are loaded before SUB puts B minus C into R6.

Give B the value nineteen and C the value six and both versions store thirteen in A, but their instructions differ. The shared handler saves me repeating the surrounding decisions about operands and dispatch. Reading the instructions it leaves to me.

All those registers look rather serious, then I can follow them, see nineteen minus six, and there's a slightly childish pleasure in being able to read both versions.

I do enjoy that.

The other job is keeping assembly's numbers in step with the Go it reads, including each field's offset (its byte position in a structure). Did the field move? Assembly can keep reading a perfectly legal address after I've changed what lives there.

For a small example, I've added debug before the program counter, which records where execution should continue:

type dispatchContext struct {
    codeBase uintptr
    codeLen  int64
    debug    bool
    pc       int64
}

On amd64 and arm64, pc would begin at byte sixteen without debug. The int64 field needs an eight-byte boundary, so Go inserts padding after the boolean and pc starts at twenty-four. An assembly read from sixteen won't become a compiler error to remind me.

I can put the shared offset definitions in a generated header that both architecture files include. Constants and table positions belong in that maintenance job too. The aim is for a change to reach every place that needs it rather than to keep the old numbers fixed forever. Both files include that header now. Do they agree with Go? Some offsets are still literals in the generator, so I've made the assembly files agree with each other without checking the actual layout. A generator can produce yesterday's assumption very consistently.

That's why I also need layout assertions using unsafe.Offsetof. They ask where Go actually put the field and I can compare that answer with the generated number. An assertion expecting sixteen for this pc now gets twenty-four. The layout changed. The assertion and the header also have to agree on the number they're checking. Copying sixteen into a test leaves me another sixteen to maintain, which slightly spoils the exercise.

A fix made only in a generated file can work locally and disappear the next time the generator runs. Its warranty expires at regeneration.

The lasting change belongs in the shared handler or the adapter that produced the faulty instructions. I want the generated-file header to point back to that tool, so I can find the route again when I've forgotten it.

While moving a handler, comparing its generated instructions with the handwritten version gives me a migration check. After that, regenerating in memory and comparing with the committed files can catch output I've forgotten to update. A mismatch should point me to the file and first differing line.

The arithmetic and branch tests still have their own job. They check the behaviour I'm preserving. Pike's comparison lets me change how the code is maintained while holding its behaviour steady.

Give the generator another job

Vector maths can use this generator too. A dot product multiplies matching elements and adds their products, while a bytecode handler needs operand decoding and a way to continue execution. I don't want the file writer deciding how either one works.

The interpreter can define its BytecodeArchitecturePort, and the vector routines can define their own operations. Their adapters supply the assembly. The generation framework deals with file groups, target architectures and headers, while those handlers describe the work each consumer needs.

The adapters can use the processors' vector-instruction families, SSE and AVX2 on amd64 or NEON on arm64, and a routine can target its own architecture without needing a matching function in every output file.

An interpreter instruction and a vector dot product can then share the dull part of generating files. I can spend my time on the assembly inside them.