Adding two integers is a small job. Repeatedly deciding how to reach the addition can become a large part of running it.
In Piko's interpreter, I select an instruction family and an operation before choosing a route for its arguments. Switches inside switches. Those nested switches keep turning up when I profile a page. A lot of code is deciding how to reach an addition.
The interpreter runs bytecode, the instructions produced by its compiler. A handler is the piece of code that carries out one of those operations.
I expected faster operations from the assembly work, write the arithmetic closer to the machine, get the answer sooner, but the route between operations is taking more attention than I expected.
Another trip through Go can spend the gain I made a moment earlier. One trip returns to the Go loop after each handler. Another returns to Go for each function call. I'm working on removing both.
A page is going to ask for more than one lovely addition.
That changes which handler I'm interested in adding. Even an operation with little arithmetic to speed up can keep its neighbours in one assembly run. The Go implementation still handles the other operations and lets me compare the same bytecode. I'm choosing where to extend that run rather than collecting isolated assembly helpers.
Keep going from the handler
An interpreter loop reads an instruction and runs its handler before returning to the loop. For the assembly path, I'm changing that handover. Each handler can fetch the next instruction and jump straight to its handler, staying in assembly for a run of work.
That is threaded dispatch. The threading here is the route through handlers, nothing to do with concurrent goroutines.
Load six, add four and move the answer to its destination. Three operations. Calling three assembly helpers from Go still takes the trip three times. In threaded dispatch, each handler finishes with a common tail that checks whether another instruction remains, fetches it if there is one and advances the position before jumping through the handler table.
The operations haven't changed. I've moved the choice of handler into the assembly code that's already running. To put a cost on the change, I'd need to time both loops on the same bytecode. I'd also record where execution leaves assembly. Go-only operations interrupt that run. Native calls too. A page full of them won't look like a long run of integer maths.
The handlers need an agreement about where things live because an ordinary Go call isn't passing everything afresh.
The current instruction position, code, constants and register banks have to remain available as control passes between handlers, and I keep different kinds of value in separate arrays that I call register banks. A bank base points at the beginning of the current call's region. Each active call has a frame, a record of its execution state that remembers its code and the storage positions for its locals. On amd64 the integer-bank base lives in R8. A handler can reach its operands through that base without rebuilding a call interface. The addition may leave that register alone. It's still occupied. Borrowing it as scratch would leave another handler using whatever value I put there instead of the bank base it expects.
I'm having to read beyond the operation now and follow what it leaves for its neighbour.
Where does the run end? The dispatch tail needs a real end-of-code path. A loop that advances quickly past its final instruction has saved time in a rather unhelpful direction.
I'd write the tail once, jump to it from every handler, and.... Go's assembler scopes labels within a TEXT block, though, so a label in another function won't give me that shared destination. A DISPATCH_NEXT() macro can put the fetch, advance and jump inside each handler's own body instead. Getting the operations written hasn't saved me this small argument with the assembler.
Some operations need the Go side of the engine, so the saved state at that exit must say exactly which instruction remains to be done. If assembly has already fetched and advanced past an unsupported operation, handing Go that advanced position would skip the operation.
Coming back makes me less fond of those convenient cached pointers. Which frame is current now? The Go call may have changed the active function or its storage, so I reload the bank bases from the frame that is current when I return. Reusing the old base after that call could send a handler into somebody else's locals.
Leave a way home
A function call changes more than the next operation. Let's use this one:
func combine(a, b int) int {
return a*2 + b
}
result := combine(3, 4)
The answer is ten, so the arithmetic won't occupy us for long. Getting three and four into combine, then getting ten back without losing the caller's place gives me more to do.
The caller still has live values and an instruction to return to after the call. That saved position is its continuation.
The function being called is the callee. It needs separate locals. The arena's save points reserve that region while keeping the caller's storage intact.
The compiler already knows which caller value supplies a. It knows the result destination and the callee's storage requirements too. Making the handler work it out again would be irritating.
A prepared call record carries those answers into execution.
Whether a call can take the assembly route is still the handler's decision, so it checks frame and arena capacity and whether it supports the argument and result arrangements. Those guards run before changing anything the fast path can't finish setting up, so a failed guard hands Go an intact caller for handling. Once the guards pass, I save the caller's execution state, reserve the callee's locals and copy the supported arguments into its banks. Then the callee's code and banks become current. Three and four must be in the right slots before its first instruction runs. A good jump with the wrong banks is still a very bad call.
When combine finishes, ten goes to the agreed result destination. Safe callee storage can be released. The caller's code and constants become current together with its bank bases and continuation. Its instruction position alone wouldn't be enough.
Deferred calls, captured state and panics can leave more work before a frame disappears. Those exits return to the engine through the routes followed in Don't Panic.
For this supported call, both the handlers and the function return can stay in assembly. I also keep another caller value alive across the call, the ninety-nine in the diagram, and read it afterwards. Ten is welcome, but getting back with the caller's own values still there is what lets it carry on with the page.