Pipit

The page needed to run

In Piko, the Go code that produces a page lives beside the template that displays it. I change either part and want to see the result. Changing the Go means running a changed program, including the functions it calls, and I don't want to rebuild the whole application every time I tweak a page. I want the feedback I get from tweaking CSS. A smaller expression language would leave me translating things the application already knows. Go types, functions and dependencies are already on the other side of the template. Keeping Go in the page lets it use them.

I tried Yaegi. Gaps in language support kept getting in the way, because even a component a few lines long could bring in a dependency which needed much more of Go. I wanted to follow those problems into code I could change. So I started an interpreter.

Keep the answers Go already knows

Pipit uses Go's parser and type checker. It compiles the checked program into bytecode, instructions for its runtime, and then the engine runs those instructions without having to parse or check the program again. Reusing Go's frontend leaves me the execution work. Quite enough.

Reading Scriggo helped me see why different kinds of value might live in separate register banks. These are slots in the interpreter's storage (not necessarily registers on the processor). With integer inputs for a*2 + b, the type checker has already decided which arithmetic applies, so an integer instruction can work with integer slots without repacking everything into a general value. Why ask that question again?

When the engine finally worked it felt great. I could hardly believe it. Had I accidentally run a compiled version? After all that work I was suspicious of getting the answer back.

Preparation and execution have different lifetimes so the compiler can prepare a program once and the engine runs it with fresh data when a caller needs it. Programs can also be saved as bytecode, with compatible interpreter versions and host symbols.

Pipit now lives in a separate repository. It runs Go from the command line or inside another Go application. Pulling it out of Piko meant deciding which dependencies and agreements belonged to an interpreter. Moving a directory would have been considerably less work.

The name comes from one of Jersey's small birds. Quick and agile seemed right. I'd considered Puffin, but that felt a bit stereotypical. There are puffins, robins, gulls....

I liked giving a less celebrated bird a turn.

The application hosting Pipit chooses the packages and functions its scripts can call. A core interpreter starts without host packages and the standard-library provider is optional. Piko can supply the helpers its pages need, while another application supplies its own. Here is a complete embedding example using only the core library, with Go 1.27:

package main

import (
    "context"
    "fmt"
    "log"

    "pipit.sh/pipit"
)

func main() {
    interp := pipit.NewInterpreter()
    value, err := interp.Eval(context.Background(), "1 + 2 * 3")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%v (%T)\n", value, value)
}

It prints 7 (int). That ordinary integer result is useful when the caller is Go too.

The getting-started guide covers installation, embedding and running files from the command line. Pipit is alpha, with changing APIs and incomplete Go compatibility, so try the programs you intend to run.

What a script can reach

I'm interested in services with optional plugins for our internal tooling. I'd like a plugin to get a query or access to selected files, whatever its job needs, without handing it every function and object the service can reach. I choose what it receives. A native helper can reach resources beyond what its import name suggests.

Ordinary embedding runs in the host process. The restricted mode adds cooperative limits and capability checks with fresh state for each submission, but neither in-process mode provides a boundary against hostile code. There is also an experimental Linux worker which runs in a separate, confined process. Its security review is incomplete.

A timeout can stop interpreted execution but cannot forcibly interrupt an arbitrary native function already running, so that helper needs to cooperate too. The security documentation explains what the controls cover and what the experimental worker requires.

The playground lets you try code in the browser and step through it with a debugger. Pipit also supports a terminal debugger and the Debug Adapter Protocol for editors.

Step through a call, inspect a value, check the next instruction....

Or I can play Doom.

A Go port of Doom runs under Pipit. Slowly. I like being able to play it, see the engine keep a long-lived Go program running, and spend a little time doing something other than checking whether a tiny expression returned seven.