Houses

• Ptmalloc2/glibc malloc is the default heap allocator for many GNU/Linux distributions. Stack exploitation has largely been mitigated by protections, e.g., ASLR and canaries. Heap corruption plays a central role in exploitation across modern native software.
• The CTF community has produced a large, high-quality corpus of heap-exploitation challenges driven by the competition selection, where the technique is most developed. It carries a dedicated vocabulary (heap feng shui) and a taxonomy of version-specific methods named House of Orange, House of Apple, and so on, which is where this benchmark gets its name. Allocator internals are documented exhaustively: unlinking during consolidation, _IO_FILE/vtable hijacking (FSOP) for control-flow capture.
• glibc heap exploitation is standardized and highly observable: allocator state, memory contents, and kernel activity can all be instrumented (e.g., via gdb). That is what lets us build a deterministic, dense reward that CVEs can't.
• The skill transfers: heap-exploitation competence generalizes to exploit development against harder targets and custom allocators like V8/Oilpan and jemalloc.

Most exploit-development RL environments grade out-of-band, and the problem gets worse as the task gets shorter. A reward defined over a checklist of named primitives cannot credit the intermediate steps a real chain is made of: an unlink or a partial overwrite scores nothing because it is not on the list. The checklist also perturbs the rollout: the policy stops halfway to satisfy the oracle instead of going for the flag, and the trajectory it leaves behind is contaminated with work that was never about exploitation.

We learned this the hard way. Our first ladder had fourteen rungs, banded L0 to L4 and named after the techniques: leak_libc, write_what_where, pc_control, overlapping_chunk. An LD_PRELOAD shim watched allocations, a signal handler watched faults, a socket tee watched I/O, and a warden negotiated a target address with the agent so we could tell a real primitive from a lucky crash. It broke in two ways.

The rungs were out-of-band. Naming them after playbook steps means grading imitation of known writeups. Negotiating a target address made it worse: it tells the agent the ladder exists, so the reward becomes a task specification. We watched a negotiated target hand out leak_libc for free.

The instrument perturbed the target. Every LD_PRELOAD, even a no-op DSO, adds a mapping and shifts ld.so, the stack and environ relative to one another. Our shim's mmap footprint made the mmap-adjacency houses unexploitable: we were losing challenges to our own instrument. And no writeup exploit could light the ladder without first being rewritten to take a --targets flag.

That last failure became the test we now hold ourselves to: an unmodified reference exploit has to light the ladder end to end. If the exploit has to know the oracle exists, the oracle is out-of-band.

glibc already ships a consistency model for the heap. do_check_chunk, do_check_free_chunk, do_check_inuse_chunk and do_check_malloc_state are a full arena-consistency walk, #defined to nothing unless you build with MALLOC_DEBUG, alongside the ~50 malloc_printerr sites cheap enough to keep in production builds. These checks state what must be true of the heap, and every named heap technique violates one of them. We lift the checker out of the process so the attacker cannot corrupt it, complete it, and extend it with an external ledger that decides what an in-process checker cannot — for instance, whether the pointer malloc just returned lands inside a live object.

Two ledgers live outside the guest. L_API records intent: (ptr, req, usable) at every allocator API return, dropped on free. It holds what the program believes it owns, and the attacker cannot touch it because it is not in the guest. L_MEM records what the allocator believes: chunk headers, tcache_perthread_struct, fastbinsY, bins[], top and the secondary arenas, parsed from the real in-guest structures with safe-linking decoded. A benign run keeps the two in bisimulation. Every exploitation step forces them to diverge, because lying to the allocator about memory the program does not own is the attacker's only leverage. The rungs classify those divergences, so no technique name appears in the set.

We check on entry to and exit from malloc and free, the same points where glibc calls check_malloc_state, because ptmalloc's invariants only have to hold when the API is quiescent. Checking there costs O(live chunks + bin entries) per call. Observation must not change what is observed, since heap layout, address-space layout and fault semantics are the object of the exploit; that rules out essentially the whole sanitiser literature. So we run a ptrace observer that breakpoints libc code, hardware watchpoints as a derived fast path, and a QEMU/TCG plugin that sees every load, store and syscall as a second, mechanically unrelated arm. With ASLR pinned, the service's entire output byte stream is byte-identical monitored and unmonitored (even on challenges whose output carries raw heap and libc pointers), and the two arms independently agree on the per-allocation chunk sequence.

The rungs sit on three tracks rather than a single chain. A single chain imposes one ordering of techniques, which is the technique-specific assumption we set out to remove.

A violation on its own is only an event; scoring it requires a consequence. An event scores only when its effect is later consumed: M2 when the divergent bytes are read back by allocator code, M4 when the escaped pointer is later handed out by malloc or dereferenced by the allocator, S1 when it is followed by S2. The implementation choice that matters: do not taint attacker input, taint the divergence. Full data-flow tracking over a process is expensive and imprecise, but the divergence set is already computed, small, and exactly the set that matters. A one-bit-per-byte shadow over the heap, seeded at each mismatch and cleared on consumption, is nearly free, and it does three things:

• No false positives. A benign write to a watched address produces no ledger divergence, so it taints nothing and fires nothing. The monitor derives which addresses matter from the ledger, at runtime, and never watches a statically chosen address.
• No reward hacking. Scribbling on metadata does not pay; only corruption the allocator actually consumes does.
• Ordering for free. Consumption is a causal edge, so the events form a DAG, and depth in that DAG is the ladder position. We recover that order from the trace instead of writing it down.

Because the tracks are orthogonal, a run's score is a vector, not a rung index. The three frontiers it reaches are its coordinates in a box: the near corner is an untouched process, the far vertex is total ownership of the allocator, the address-space secret and control flow at once. Two exploits that reach the same depth by different routes land at different coordinates. A scalar ladder discards that.

We mine a challenge's ladder by running the unmodified reference exploit once and taking the rungs it fires. That trace is a lower bound, never a required path: reward is the frontier reached, so an unseen route scores the same and a deeper one scores more. The shaping is potential-based, which leaves the optimal policy provably unchanged — it is still "get the flag". Spamming double-frees pays nothing, since a rung counts only once the allocator consumes the corruption; hardcoding an address pays nothing, since the identical exploit is re-run under a second ASLR seed and the leak witnesses have to track it.

The trajectories below are reference exploits, unedited, scored against one fixed lattice. They are nearly disjoint, and no rung was written for any of them. One makes the allocator write where it does not own but never makes it hand out foreign memory; another is the exact complement. A third has an empty S axis because it needs no leak at all, a route the leak→hook→shell shape cannot represent.

The agent is told none of this. It gets the binary, the libc and a connection; it writes exp.py; it receives the process's own output and, if it wins, the flag. If it knew the rung names, the reward would become a task specification, and it would optimise that instead of the exploit.

We group and hand-harness these challenges into "houses" — the heap-exploitation term for a class of primitive tied to a specific allocator version and its behaviour. Each house ships as a house.local / house.remote pair. house.local is the debug environment the model works in directly, with tooling such as pwndbg for developing the exploit; house.remote is an independent, deterministic grader that runs the payload against the challenge binary while the monitor watches allocator state. The two talk only over a socket, so nothing the model does to its own environment can reach the grader.

            <--

Throughout a rollout the model calls grade() freely; each attempt is scored across 16 independent episodes against the lattice, from the monitor's observations on the remote. A house's score is the deepest frontier reached within a 200-turn rollout, normalised by the frontier its own reference exploit mined. Every house is tested by hand first, and harnessed only once its reference exploit lights the lattice end to end, unedited.

As for what goes in them: CVEs make a poor training environment. Every target is one-off, environments drift, and there is no ground truth for how far a rollout got; you only learn whether it landed. ptmalloc has none of these problems. One allocator, one published mitigation timeline (tcache at 2.26, tcache_key at 2.29, safe-linking at 2.32, hooks gone by 2.34/2.35), so the glibc version becomes a variable we control instead of a confound we absorb. And the CTF community has spent a decade competing to make individual heap challenges hard, then proving each one solvable. The result is a labelled corpus nobody had to fund: adversarially generated, quality-forced by competition.

So we hand-collected and harnessed them: 135 challenges spanning glibc 2.19 to 2.42, filling 44 occupied cells of a six-era by ten-family matrix, drawn from the 3,315 binary-exploitation challenges listed on ctftime.org plus writeups and archives from across the internet. Each is pinned to an image digest, ships with a reference exploit, and is contamination-audited. The matrix doubles as a worklist: an empty cell is something to go find next, not padding for a count.