When a string outlives its arena

A page can build thousands of little strings and forget the lot once it has rendered. Giving every temporary string its own heap allocation leaves Go more to allocate and collect. I'm trying to give the interpreter a reusable scratchpad instead.

That's the appeal of an arena. I take a larger block of memory, a slab. A cursor marks the next free byte, and each allocation advances it by the space the new string needs. Once the work is finished, I can reset the scratchpad in one go. The strings don't each need their own allocation and cleanup. Resetting makes the space available again, rather than returning it to the operating system.

The operating system manages memory in pages. Using a fresh page can mean waiting for the OS to make it available, a page fault. Reusing one I've already touched can avoid that wait, and I'd rather borrow the same scratchpad for another evaluation than make the OS do that work again.

The slabs themselves still live on Go's heap. Their temporary contents give the collector fewer separate allocations to deal with.

Lovely, until something wants to keep a bit of the scratchpad. This label needs to survive after the interpreter has moved on to other work:

label := "item-" + strconv.Itoa(index)
labels[index] = label

I can make the first line cheap by borrowing space from the slab. But labels keeps it. If that collection survives the temporary storage being reused, somebody is keeping this text. When are those bytes mine to reuse?

I'd like to reset it. I'd also like my saved text to stay unchanged, which is where the two wishes start to argue.

In this Go implementation, I can reserve bytes by advancing the slab's cursor and use unsafe.String to make a string view whose header carries a data pointer and a length. Copying the header duplicates that pointer and length, but the text is still in the slab.

While the bytes remain untouched, the result behaves as intended. Printing item-12 straight away gives the allocator too easy a day. Construction worked. What happens when the allocator lends that space to item-34?

Nobody assigned to the saved value. The program kept item-12 and got item-34 back.

Keeping the slab alive is an easy answer to reach for. I've still got a valid pointer, so surely the string is safe? The arena can overwrite the bytes in a slab it still refers to, Go would be keeping that allocation alive quite correctly, and I'd still be the one lending those bytes out again. The unsafe.String contract requires them to stay unchanged while the string exists.

Give the escaping string suitable backing

In my mind, saving the label is a promise that I can read the same text later. The interpreter can copy the string into storage that lasts as long as its destination needs it. That's materialisation. The retained value gets independent backing, so reusing the temporary bytes won't change the saved label.

Before a value leaves temporary storage for a longer-lived global, map entry or captured variable, I call the copy helper.

I call this a store barrier. It's guarding transfers against my own storage reuse, a different job from Go's garbage-collector write barrier.

Before reuse, the saved header points to item-12 in the slab. After reuse without a copy, it points to item-34 instead. Copying the text before storing gives the saved header independent bytes containing item-12, while the slab holds item-34. — **Figure 1 - Keep the text after the slab moves on.** Compare the two ways of saving `item-12`. Arrows show where the saved header points.

A label used and forgotten inside the same bit of work belongs on the cheap temporary path. A label somebody keeps is asking for something else. The store is where I pay for its independent backing.

Storing a string directly is the friendly case. A general value can hold a struct containing a slice of strings. I can't stop at the outer value. I have to find the strings before its destination keeps it. Writing the copy helper is fine. Finding everywhere to call it is causing me a small amount of misery, especially when the string is buried inside that slice and I have to follow what's inside the general value. Direct assignment gives me somewhere obvious to look.

And the global stores? Getting direct assignments right leaves those to check on their own.

I can check a nested store by keeping the label, forcing reuse with unrelated work, then reading it again. It needs to say item-12. The allocator can borrow and reuse as much as it likes around it, provided the text I kept stays mine.

Then the evaluation doesn't end

After materialisation the saved label has independent backing, so it doesn't need the old slab, while strings still being used inside the evaluation can stay on the temporary path with their bytes in the arena.

Each call borrows register slots. Its strings can outlive it. Clearing the call's string headers is the easy part. Another part of the evaluation may still be reading the bytes behind them.

For now, a full byte slab stays with the arena. I allocate another slab for the new strings. Existing string views stay valid. When the whole evaluation finishes, I reset the lot.

I can also put an idle arena in a sync.Pool for another evaluation to borrow. The pool is free to throw an idle arena away, but one it hands back gives another render the scratchpad that's already there.

For a page render, I could wait until the evaluation ends, when the temporary values are gone and the scratchpad can go back in the pool. But I'm also trying to support scripts that keep running. When do I get my nice bulk reset then?

Imagine a script that keeps creating temporary strings while still using one small arena-backed string. This isn't the label I copied into labels. The bytes are borrowed. Repeat that work often enough and the arena retains a history the script has mostly forgotten.

The arena keeps references to old slabs A and B and current slab C. A working string still borrows bytes from A. No string uses B's bytes, but the arena keeps B reachable. New allocations continue in C. — **Figure 2 - What could be released?** Arrows show the references that keep the slabs reachable.

B is still reachable. The arena refers to B, so Go's collector keeps the allocation alive. I'd be happy to forget it, but the program keeps asking for it.

That's the label problem the other way round. Reachability didn't stop me overwriting item-12. Now, reachability alone keeps B around.

That working string makes “discard everything old” unusable, because age doesn't tell me whether anything can still read the bytes and a tiny survivor can keep a large slab around if its backing lives there.

What's still borrowed? I'd quite like to let B go and keep A, but first I need to know which values still borrow bytes from a slab and which slab that is. The stores that materialise a string remove one such dependency, but they don't account for all the temporary strings still being used.

Headers, values, strings inside slices....

I've built myself a reusable scratchpad and am now afraid to rub anything out.

I don't have a finished design for releasing these slabs yet. Copying the remaining live strings out would let me drop their old slabs, but now I'm paying for more copies. Tracking which slabs they still need is another job. Which costs me less in a script that keeps running?