interlens¶
interlens
¶
APIParticipant
dataclass
¶
APIParticipant(
name: str = "",
model_id: str = "",
provider: Provider = "anthropic",
system_prompt: str | None = None,
private_context: tuple = (),
max_tokens: int = 512,
temperature: float = 1.0,
batch: bool = False,
client: object = None,
requires_alternating_roles: bool = True,
)
Bases: Functional, Participant
A participant backed by a hosted API — Claude via anthropic (provider="anthropic", the default) or
any model behind OpenRouter (provider="openrouter", OpenAI-compatible) — for use as a debate opponent,
moderator, or the classifier inside an analyze callback.
It is a full participant for conversation purposes but has no local model — so there is no device,
no activations, and no steering. Any interp request (capture/steering/patch/return_logprobs)
raises rather than silently no-op'ing: in a measurement harness, a steering sweep that quietly did
nothing on an API participant would produce a false "no effect" conclusion. Seeds don't bind hosted models,
so API turns are excluded from the identical-replay guarantee.
Concurrency is network-bound, so pure-API conversations run thread-pooled rather than process-per-GPU
(handled by the runner). The client callable is injectable for testing.
generate_batch
¶
generate_batch(
views: list[list[dict]],
*,
turn: int | None = None,
group_seed: int | None = None,
max_new_tokens: int | None = None
) -> list[Message]
Generate one turn for many independent conversations at once — the API analogue of
ModelParticipant.generate_batch, driven by the runner's co-stepper (rollout(..., batched=True)) to
make large API rollouts cheap and throughput-bound.
With batch=True every view is sent as one asynchronous provider batch (Anthropic Message Batches /
OpenAI Batch API) via client.submit_batch — ~50% cost and much higher throughput, at the price of
batch-window latency. If the provider has no batch API (e.g. OpenRouter) this raises rather than
silently degrading, so a requested batch is never quietly run as serial calls. With batch=False it
falls back to sequential per-view calls (correct, just no batch discount). Interp is unavailable here, as
for generate. turn/group_seed are accepted for co-stepper compatibility but unused (seeds do
not bind hosted models). metadata['batched'] marks these turns.
Source code in src/interlens/participant/participants/api_participant.py
ActivationCache
¶
A queryable store of captured activations, tagged by conversation structure.
Records know which participant/turn/layer/site they came from, so a downstream probe can ask for "bob's turn
3, layer 18, the answer span" rather than juggling anonymous tensors. This is the object every interp
consumer reads; the harness never puts activations in Message.metadata (that would blow up branch()'s
transcript copy), so the cache is the single home for heavy tensors.
Source code in src/interlens/interp/activation_cache.py
add_batch
¶
Add many records at once, offloading all their tensors in one batched pinned transfer (see
offload_to_cpu). Preferred over a loop of add when a single capture pass produces many records —
it turns N GPU->CPU copies into one per shape-group.
Source code in src/interlens/interp/activation_cache.py
at
¶
Return the single matching tensor, erroring if the filters aren't unique — the ergonomic accessor for "give me exactly this activation".
Source code in src/interlens/interp/activation_cache.py
AnyStopCondition
¶
Bases: StopCondition
Fires when any member condition fires. run(until=[...]) wraps a list in this.
Source code in src/interlens/stop/stop_condition.py
AutoModelParticipant
¶
Resolver for creating Participant instances automatically from HuggingFace model identifiers, local model paths, or already-loaded PreTrainedModels.
HF-style factory for family-correct local-model participants — the participant analog of
AutoModelForCausalLM. It resolves the concrete ModelParticipant subclass from the model's transformers
config.model_type via the class self-registry (ModelParticipant.for_model_type), then delegates the
actual build to that class's from_model / from_pretrained — so all the loading / tokenizer-inference /
chat-flag logic lives in one place (on ModelParticipant) and this class is only the family dispatcher.
from_dispatches on the argument type (str id →from_pretrained;PreTrainedModel→from_model).- To get a statically-known subclass, name it directly:
QwenModelParticipant.from_pretrained(...). This factory returns the (dynamically resolved) baseModelParticipanttype.
from_
staticmethod
¶
from_(
model: _QwenId,
*,
name: str,
tokenizer: PreTrainedTokenizerBase | None = ...,
device: str | device = ...,
load_kwargs: dict | None = ...,
**participant_kwargs: Any
) -> QwenModelParticipant
from_(
model: _GemmaId,
*,
name: str,
tokenizer: PreTrainedTokenizerBase | None = ...,
device: str | device = ...,
load_kwargs: dict | None = ...,
**participant_kwargs: Any
) -> GemmaModelParticipant
from_(
model: ModelLike,
*,
name: str,
tokenizer: PreTrainedTokenizerBase | None = None,
device: str | device = "cuda",
load_kwargs: dict | None = None,
**participant_kwargs: Any
) -> ModelParticipant
Build a participant from either an HF id (str) or an already-loaded PreTrainedModel, dispatching to
from_pretrained / from_model. tokenizer applies only to the loaded-model case (an id / path loads
its own matching tokenizer, so it is ignored there).
Source code in src/interlens/factories.py
from_model
staticmethod
¶
from_model(
model: PreTrainedModel,
tokenizer: PreTrainedTokenizerBase | None = None,
*,
name: str,
device: str | device | None = None,
**participant_kwargs: Any
) -> ModelParticipant
Build a family-correct participant from an already-loaded model — the class is resolved from
config.model_type (unknown types fall back to base ModelParticipant), then that class's
:meth:ModelParticipant.from_model does the tokenizer inference and chat-flag derivation.
Source code in src/interlens/factories.py
from_pretrained
staticmethod
¶
from_pretrained(
id_or_path: _QwenId,
*,
name: str,
device: str | device = ...,
load_kwargs: dict | None = ...,
**participant_kwargs: Any
) -> QwenModelParticipant
from_pretrained(
id_or_path: _GemmaId,
*,
name: str,
device: str | device = ...,
load_kwargs: dict | None = ...,
**participant_kwargs: Any
) -> GemmaModelParticipant
from_pretrained(
id_or_path: str | Path,
*,
name: str,
device: str | device = "cuda",
load_kwargs: dict | None = None,
**participant_kwargs: Any
) -> ModelParticipant
Return a family-correct participant for id_or_path (an HF id or local path) that will load its weights
lazily on first use. The concrete class is resolved from config.model_type by reading ONLY the
model's config (AutoConfig.from_pretrained — cheap, no weights); load_kwargs (dtype / attn /
quant / revision / weights_path) are recorded for that deferred load and participant_kwargs
go to the participant (see :meth:ModelParticipant.from_pretrained).
Source code in src/interlens/factories.py
CaptureSpec
dataclass
¶
CaptureSpec(
sites: tuple[Site, ...] = ("residual",),
layers: tuple[int, ...] | None = None,
offload: OffloadLocation = "cpu",
)
What to capture during a generation: which sites at which layers, and where to keep the tensors.
Capture defaults to a narrow set — you pass the layers/sites you actually want — because capturing all
layers × all tokens × many rollouts OOMs fast. offload='cpu' moves captured tensors off-GPU as they are
recorded (essential for large sweeps); offload=None keeps them on-device.
ContextItem
dataclass
¶
A single item of private asymmetric knowledge given to one participant (a briefing, a document, a fact).
Deliberately distinct from Message: a briefing is not a dialogue turn — it has no turn index and its
author is nominal — so overloading Message would give it misleading turn/authorship semantics.
Role semantics are pinned rather than role-swapped: a briefing renders by default as user-provided
context (role_hint=USER) — the participant reads it as information handed to it, not as its own prior
speech — with SYSTEM available for standing private instructions. Private context is injected only into
the owning participant's view and never enters the shared transcript.
ContextPolicy
¶
Bases: ABC
Decides how to fit a participant's view within its model's context window.
Crucially, fit runs on the typed ViewSegment list, before the family-specific finalize_view
folds/merges anything. Operating pre-finalize means the policy can reliably preserve the system block and
moderator seed (their origin is still intact) and trim only turn segments, instead of trying to
reverse-engineer meaning out of already-folded text.
to_dict
¶
Serialize as {"kind": ..., **params}. Subclasses with parameters extend the params; the default
covers parameterless policies.
Conversation
dataclass
¶
Conversation(
participants: tuple[Participant, ...],
transcript: Transcript = Transcript(),
shared_context: str | None = None,
shared_system_prompt: str | None = None,
moderator_name: str = "moderator",
context_policy: ContextPolicy = ErrorPolicy(),
context_limit: int | None = None,
reasoning_visibility: ReasoningVisibility = ReasoningVisibility.STRIP,
execution_mode: ExecutionMode = ExecutionMode.THROUGHPUT,
message_hooks: list = list(),
_turns: int | None = None,
_data: object = None,
_analyzer: object = None,
_name: str | None = None,
_seed: int = 0,
_run_until: object = None,
)
Bases: Functional
A multi-agent conversation: the recipe, the live dialogue, AND the benchmark-rollout driver, all in one lightweight object.
It orchestrates turn-taking between Participants over a shared, perspective-neutral Transcript, owning
who speaks when and the ordered view pipeline: for each speaker it assembles a structured, typed view
(system block → private context → transcript turns), fits it to the context window on those typed segments,
then lets the participant flatten it (family fold/merge) and generate. Fitting before the lossy family
flatten is what lets the context policy preserve the system/moderator framing reliably (see context/).
Scenario framing is split by ownership: shared framing (shared_context as a moderator seed turn,
shared_system_prompt) lives here; private framing (system_prompt, private_context) lives on each
participant, so it is naturally invisible to the others and to the transcript.
Copy-on-write + lazy. Participants load their weights lazily, so an unrun Conversation is a cheap
recipe. Build it up functionally — conv.turns(4).data(ds).analyzer(grade) — where each dot-modifier
returns a modified copy (via Functional.set), never mutating the original; call .set(field=value) for
fields without a dot-modifier. build-free: run it directly with run(), or expand it over data / N
samples with rollout() (which copies it per row — see there).
Also supports branching (branch() / branch_from()), in-place history editing (rewind(), edit(),
reset()), ephemeral sampling (sample() / sample_all()), activation capture (capture()), and
disk persistence (save() / load()).
context_limit
class-attribute
instance-attribute
¶
Token budget the context_policy fits to. None = use the tokenizer's own model_max_length.
context_policy
class-attribute
instance-attribute
¶
How each speaker's view is fit to the context window — run on the typed segments before the family
flatten so framing is preserved. Default ErrorPolicy raises on overflow rather than silently truncating;
see context/ for sliding-window / drop-oldest / summarize alternatives.
execution_mode
class-attribute
instance-attribute
¶
The determinism-vs-throughput tension the runner consults: THROUGHPUT (default) permits batched
co-stepping + KV reuse + flash-attn (only distributional reproducibility); DETERMINISTIC disables those
for token-identical replay and interp fidelity.
message_hooks
class-attribute
instance-attribute
¶
Runtime-only middleware (not persisted): each hook vets / edits / denies a freshly generated message
before it is committed. Applied in order; default empty = pass-through. See hooks/.
moderator_name
class-attribute
instance-attribute
¶
Author name used for the shared_context seed and any moderator turns. Must not collide with a
participant name (validated in __post_init__).
participants
instance-attribute
¶
The conversation's participants. Order is significant — it is the default speaking order. run
alternates through them in this order (turn k is participants[(start + k) % n]), and run(first=...)
only shifts the starting index; so the first element speaks first unless first overrides. Names must be
unique and none may equal moderator_name.
reasoning_visibility
class-attribute
instance-attribute
¶
Whether a prior turn's parsed <think> reasoning is re-injected into later views: STRIP (never),
SELF_RETAIN (a speaker sees only its own), SHARED (everyone sees everyone's). Reasoning is always
parsed into metadata['parsed_think'] regardless, so interp is unaffected.
row
property
¶
The dataset row that produced this conversation in a data-driven rollout (the same dict the
dataset_fields resolved against), or {} outside a rollout. Read it in an analyzer to reach per-row
side data — labels, gold answers, ids — WITHOUT templating it into the model's view (so it never leaks into
the conversation). Travels with the conversation across the spawn boundary.
shared_context
class-attribute
instance-attribute
¶
Scenario framing every participant sees, injected once as a leading moderator_name turn when the
transcript starts empty. None = no seed turn; "" still seeds a (blank) turn, so the first speaker has
a non-empty view.
shared_system_prompt
class-attribute
instance-attribute
¶
A system prompt merged into every participant's system block (after that participant's private
system_prompt). None = no shared system prompt.
transcript
class-attribute
instance-attribute
¶
The shared, perspective-neutral message history. Defaults to empty; when starting fresh, shared_context
is seeded as the first moderator_name turn (a branched/loaded transcript already contains it and is not
re-seeded).
branch
¶
Fork into a new Conversation that reuses the same participant objects (shared weights, zero GPU
cost) with a copied, independent transcript. The branch can diverge freely; the original is untouched. This
is exactly a no-op copy-on-write clone (self.set()). To fork from a specific point in the history rather
than the end, use branch_from.
Source code in src/interlens/conversation.py
branch_from
¶
Fork a new conversation whose history is this one's turns up to and including the turn ref — i.e.
branch as if the conversation had stopped right after ref, ready for a different continuation. ref is
a MessageRef: an int index (negatives count from the end) or the Message object itself (matched
by identity). The original is untouched; the fork shares participants (zero GPU cost).
Source code in src/interlens/conversation.py
capture
¶
Context manager that captures activations for every step inside the block into a fresh
ActivationCache, auto-tagged by the current speaker + turn::
with conv.capture(sites=["residual"], layers=[8, 12]) as cache:
conv.step(bob)
cache.at(participant="bob", layer=12)
Source code in src/interlens/conversation.py
edit
¶
edit(
ref: MessageRef,
content: str | None = None,
*,
author: str | None = None,
**metadata
) -> Message
Edit a committed past turn in place and return it (see Transcript.edit): ref is a MessageRef
(int index, negatives allowed, or a Message object matched by identity), and content / author /
**metadata are the fields to change. Editing the returned Message's fields directly works too, since
the transcript holds it by reference.
Source code in src/interlens/conversation.py
from_models
classmethod
¶
from_models(
models: tuple[ModelLike, ...],
names: tuple[str, ...] = ("a", "b"),
device: str | device = "cuda",
dtype: dtype = torch.bfloat16,
shared_context: str | None = None,
shared_system_prompt: str | None = None,
prompt: PromptLike = None,
**gen_kwargs
) -> "Conversation"
Scaffold a conversation directly from a tuple of models — each an HF id or an already-loaded
PreTrainedModel (see ModelLike). Convenience wrapper around
factories.conversation_from_models; each model becomes a family-correct participant and names gives
them identities. The order of models / names is the speaking order — the first speaks first
unless you pass first= to run. **gen_kwargs are forwarded to every participant.
Scenario framing is available here too: shared_system_prompt (instructions, system role) and
shared_context (a neutral moderator-voiced opening seen by everyone). prompt is a separate
convenience for a participant-voiced opener (a str is attributed to the last participant). See that
function for the details.
Source code in src/interlens/conversation.py
load
classmethod
¶
Rebuild a saved conversation: reconstruct (lazy) participants from the recipe and attach the saved transcript, resuming from that state (does NOT regenerate messages). Raises on an unsupported schema.
Source code in src/interlens/conversation.py
participant
¶
Return one of this conversation's participants, resolved from a Participant object, its name
(str), or its index (int). Raises (via _resolve_participant) if it isn't in this conversation.
Source code in src/interlens/conversation.py
render_templated
¶
render_templated(
pov: ParticipantLike,
*,
extra=(),
add_generation_prompt: bool = False,
tokenize: bool = False
)
The full view run through pov's tokenizer chat template — the exact prompt its model sees,
special/control tokens and all (a str, or token ids with tokenize=True). This is the truthful
counterpart to transcript.render_templated, which templates the transcript turns ONLY (no system /
private framing, no context-fit). Requires a local ModelParticipant (needs a tokenizer).
Source code in src/interlens/conversation.py
reset
¶
Return the conversation to its fresh, pre-run state: empty the transcript, then re-seed the
shared_context moderator turn (exactly as __post_init__ does on a fresh conversation). Use this
rather than transcript.clear() when you want to rerun the same scenario — the raw transcript.clear()
drops the shared_context framing too, whereas reset restores it.
Source code in src/interlens/conversation.py
rewind
¶
Rewind in place so the turn to becomes the new last turn, dropping everything after it (returns
self for chaining). to is a MessageRef (int index, negatives allowed, or a Message object).
Mutates this conversation — use branch_from instead to keep the original. See Transcript.rewind.
Source code in src/interlens/conversation.py
rollout
¶
rollout(
n: int | None = None,
*,
devices=None,
out_dir=None,
resume: bool = False,
batched: bool = True,
max_batch_size: int | None = None,
seed: int | None = None
) -> "RunReport"
Run many conversations from this one recipe and return a RunReport — the benchmark/scale entry point.
Rollout does NOT mutate this conversation. It makes an independent copy-on-write clone per job and runs
those; self stays the unrun recipe. Finished conversations live in report.results[job_id].conversation
(and their transcripts / analyses on the same RunResult) — that is where to sample() or inspect
afterwards, NOT on the original.
Two modes: with data() set, expands to ONE conversation per row (job ids row_00000…), resolving each
templated field (dataset_field) against the row; otherwise expands to n seeded copies of one scenario
(job ids rollout_0000…). n defaults to len(data). Sample i gets seed + i (seed defaults
to the conversation's seed field). Any ambient with StopCondition(...) blocks and the run_until
field are captured now and attached to every job (so they survive the spawn boundary).
Parallel by default on two axes (see :func:interlens.run): one worker process per device, and batched
co-stepping within each device. batched=False gives the token-identical DETERMINISTIC path.
Source code in src/interlens/conversation.py
run
¶
run(
turns: int | None = None,
until: StopCondition | list | None = None,
first: ParticipantLike | None = None,
prompt: PromptLike = None,
) -> Transcript
Alternate speakers until turns elapse and/or a StopCondition fires (whichever comes first).
Speakers are taken in participants order (that tuple's order IS the turn order); first sets who
starts (default: participants[0]) and the rest follow round-robin. first may be a Participant,
its name (str), or its index (int) — resolved via _resolve_participant (raises if it isn't in this
conversation).
Stop conditions are combined from three sources (any of which stops the run): until (this call), the
conversation's own run_until field, and any ambient with StopCondition(...): blocks. A condition may
also cap each turn's tokens (turn_cap) — e.g. a TokenBudget shrinks the final turn to land on budget.
Conditions are reset() at the start so a reused instance doesn't leak state. turns defaults to the
conversation's turns field; at least one of turns / a stop condition must be present.
prompt (optional) is appended to the transcript before the run: a str is attributed to the LAST
participant (so first replies to it), a Message is appended as-is, None appends nothing. It
appends to the current end and never resets/reseeds.
Source code in src/interlens/conversation.py
sample
¶
sample(
speaker,
message=None,
*,
as_author=None,
steering=None,
capture=None,
patch=None,
return_logprobs: bool = False,
max_new_tokens: int | None = None
)
Ephemerally sample a response without mutating the transcript — a pure read of current state, safe to
call repeatedly / in a loop. speaker is a ParticipantLike (name / index / Participant); pass a
list or tuple of them to sample each and get back {name: Message} instead of a single Message
(see also sample_all). message is an optional temporary incoming turn; it defaults to being
attributed to the moderator (a neutral, external voice — so "What do you think of Bob?" reads as an
interviewer asking, NOT as Bob speaking). Pass as_author="bob" to make it read as that participant's
turn instead (e.g. "what would you say if Bob had just said X?"). The same interp options as step are
honored on each ephemeral generation.
Source code in src/interlens/conversation.py
sample_all
¶
sample_all(
message: str | None = None,
*,
as_author: str | None = None,
steering=None,
capture=None,
patch=None,
return_logprobs: bool = False,
max_new_tokens: int | None = None
) -> "dict[str, Message]"
Ephemerally sample every participant's response — a convenience for
sample(list(self.participants), ...). Returns {name: Message}; the transcript is untouched. Handy
for "what does each model say to this right now?".
Source code in src/interlens/conversation.py
save
¶
Persist the conversation: the transcript (transcript.json) + the recipe (conversation.json — the
participants' own constructor kwargs + scenario framing/policies, no weights).
Source code in src/interlens/conversation.py
step
¶
step(
speaker: Participant,
*,
steering=None,
capture=None,
patch=None,
return_logprobs: bool = False,
max_new_tokens: int | None = None
) -> Message
Have speaker produce and commit one turn. Interp options flow to generate; if no capture is
passed but a conv.capture(...) block is active, its pending request is used (auto-tagged by turn).
Source code in src/interlens/conversation.py
view
¶
The full [{role, content}] view pov's model is conditioned on — the real generation input:
system block (its private system_prompt + shared_system_prompt) → private_context → the
transcript role-swapped to pov, then context-fit and family-flattened. This is exactly what
step / sample feed to generate. Unlike transcript.render_roles (transcript turns only, no
framing or fitting), it reflects everything the model actually sees. pov is a ParticipantLike (name
/ index / participant); extra renders temporary, uncommitted messages (as sample does).
Source code in src/interlens/conversation.py
DatasetField
dataclass
¶
A placeholder for row[name] in a templated field, created by :func:dataset_field. Frozen + tiny so it
serializes and pickles trivially across a spawn boundary.
DropOldestPolicy
¶
Bases: ContextPolicy
Drop the oldest turn segments (preserving system/moderator/private_context) until the view fits.
ElapsedTimeStopCondition
¶
Bases: StopCondition
Stop once seconds of wall-clock have elapsed since the run started (monotonic clock).
Source code in src/interlens/stop/conditions.py
ErrorPolicy
¶
Bases: ContextPolicy
The safe default: raise if the view exceeds the context window rather than silently dropping content.
Silent truncation reads as "everything fit" when it didn't, which is exactly the kind of quiet data loss this
harness avoids. Callers who want trimming opt into DropOldestPolicy/SlidingWindowPolicy explicitly.
ExecutionMode
¶
Bases: str, Enum
Names the deterministic-vs-throughput tension explicitly rather than pretending it doesn't exist.
Token-identical replay, performant defaults (flash-attn + KV reuse + batched generation), and interp measurement fidelity are NOT jointly satisfiable on CUDA, so callers pick a mode:
DETERMINISTIC: batching off, KV reuse off, sdpa/eager attention, deterministic algorithms. Slower, but token-identical on the same hardware and safe for capture/steering/probes. Backs the identical-replay guarantee.THROUGHPUT(default for large rollouts): flash-attn + batched generation + KV reuse. Guarantees only distributional reproducibility.
P1 defines the switch and threads it through; the optimizations it gates (batching, KV reuse) land in later phases and consult this flag.
Functional
¶
Mixin giving a dataclass copy-on-write updates via set(**changes).
set shallow-copies the instance (heavy refs like a loaded model are shared, not reloaded), applies the
changes, then calls _after_set to reset volatile per-instance state on the copy. Sugared field names
(declared via sugar_fields) are accepted under their public spelling and routed to their _-prefixed
storage. Unknown fields raise TypeError. The original is never mutated.
set
¶
Return a modified shallow copy: the same object with changes applied and volatile state reset. The
receiver is untouched (copy-on-write); heavy state (a loaded model, an API client) is shared by reference.
Unknown field names raise TypeError.
The copy is a raw instance-dict clone (NOT copy.copy), so it deliberately bypasses __getstate__ —
which exists to DROP the loaded model/client on pickle (spawn boundary). A .set() clone, by contrast,
is in-process and must KEEP those shared references (that is the whole point of copy-on-write here).
Source code in src/interlens/functional.py
GemmaModelParticipant
dataclass
¶
GemmaModelParticipant(
name: str = "",
hf_id: str | None = None,
weights_path: str | None = None,
dtype: str = "bfloat16",
attn: str = "flash_attention_2",
quant: str | None = None,
revision: str | None = None,
device: str | device | None = None,
max_new_tokens: int = 512,
temperature: float = 0.8,
top_p: float = 0.95,
seed: int | None = None,
thinking: bool | str = "auto",
system_prompt: str | None = None,
private_context: tuple = (),
tools: tuple = (),
max_tool_iters: int = 4,
kv_reuse: bool | str = "auto",
steering: object = None,
_model: "PreTrainedModel | None" = None,
_tokenizer: "PreTrainedTokenizerBase | None" = None,
)
Bases: ModelParticipant
A Gemma-family participant. The only thing that differs from base is the tool-call format
(```tool_code); the chat-template flags (Gemma 2 rejects a standalone system role and requires
strict user/model alternation, Gemma 3 accepts a system role) are auto-derived from the tokenizer's own
template, so both generations use this one class.
parse_tool_calls
¶
Parse Gemma's ```tool_code function-call blocks (best-effort).
Gemma emits calls like ```` ```tool_code
name(arg="x", n=2)
` ```` rather than Hermes JSON. We extract
the block, then the function name and simple keyword arguments. Falls back to[]`` (treat as final
message) on anything it can't parse, matching the base contract of never misfiring silently.
Source code in src/interlens/participant/participants/gemma.py
GradCaptureSpec
dataclass
¶
What grad-connected activations to pull from forward_with_grad (mirrors CaptureSpec minus offload).
sites is any of "residual"/"attn"/"mlp"; layers selects decoder-layer indices (None =
all). Unlike CaptureSpec there is no offload — the whole point is to keep tensors on-device and in the
autograd graph, so an intermediate-layer objective (e.g. project onto a concept direction) can be backpropagated.
GradForwardOutput
¶
Bases: NamedTuple
Result of forward_with_grad: grad-connected logits ([batch, seq, vocab]) and, if a
GradCaptureSpec was passed, hidden mapping (site, layer) -> tensor[batch, seq, d_model] (also
grad-connected). hidden is empty when no capture was requested.
LinearBridge
¶
Bases: Module
Learned linear map from model A's hidden width d_a to model B's embedding width d_b.
For heterogeneous (different-tokenizer) pairs where a vocab mixture is undefined: take A's last-layer hidden
states (via forward_with_grad(..., capture=GradCaptureSpec(sites=('residual',), layers=(last,)))), map them
into B's embedding space, and pass the result as inputs_embeds to B. Trained jointly with A against B's loss.
bias=False by default to match the (linear, origin-preserving) cross-model maps used in the Procrustes work.
Source code in src/interlens/interp/bridge.py
forward
¶
[batch, seq, d_a] -> [batch, seq, d_b] soft embeddings in B's input space.
LlamaModelParticipant
dataclass
¶
LlamaModelParticipant(
name: str = "",
hf_id: str | None = None,
weights_path: str | None = None,
dtype: str = "bfloat16",
attn: str = "flash_attention_2",
quant: str | None = None,
revision: str | None = None,
device: str | device | None = None,
max_new_tokens: int = 512,
temperature: float = 0.8,
top_p: float = 0.95,
seed: int | None = None,
thinking: bool | str = "auto",
system_prompt: str | None = None,
private_context: tuple = (),
tools: tuple = (),
max_tool_iters: int = 4,
kv_reuse: bool | str = "auto",
steering: object = None,
_model: "PreTrainedModel | None" = None,
_tokenizer: "PreTrainedTokenizerBase | None" = None,
)
Bases: ModelParticipant
A Llama-family participant. Chat-template flags are auto-derived from the tokenizer; only the tool-call
format differs from base: Llama 3 emits calls as <|python_tag|>{json} rather than Hermes/Qwen
<tool_call> blocks.
parse_tool_calls
¶
Parse Llama-3's <|python_tag|>{json} function-call format (best-effort).
Everything after <|python_tag|> is one or more JSON objects (separated by newlines or semicolons),
each {"name": ..., "arguments"/"parameters": {...}}. Anything unparseable is skipped, so a malformed
call yields [] (treated as a final message), matching the base contract of never misfiring.
Source code in src/interlens/participant/participants/llama.py
Message
dataclass
¶
A single committed turn in a conversation.
The transcript is stored canonically and author-centric: a message records who said what, and is
deliberately agnostic to the assistant/user role distinction. That role mapping is a per-participant
view concern (see Transcript.render_roles), not a property of the message itself — so one transcript can
be rendered from every participant's perspective without duplicating state.
author is the participant's name (a string), never the Participant object. This keeps transcripts
trivially JSON-serializable and valid even when no models are loaded (e.g. when scoring saved transcripts).
content is the committed, visible text — the only field that is authoritative for rendering history back
into a model. Anything else a generation produced (parsed <think> reasoning, the raw completion, tool
call/result trails, per-token logprobs) lives in metadata under neutral keys, so hidden generated text is
never silently promoted into what other participants see.
MessageHook
¶
Bases: ABC
Middleware that inspects each freshly generated message before it is committed to the transcript, and may approve / deny / edit it.
This is the seam for a future LLM-judge that vets or rewrites turns (e.g. safety filtering, format
enforcement). Hooks live on the live Conversation (conversation.message_hooks) and are NOT serialized
in the template — they're a runtime policy, not part of the scenario recipe. The default is an empty hook
list, i.e. today's pass-through behavior.
ModelParticipant
dataclass
¶
ModelParticipant(
name: str = "",
hf_id: str | None = None,
weights_path: str | None = None,
dtype: str = "bfloat16",
attn: str = "flash_attention_2",
quant: str | None = None,
revision: str | None = None,
device: str | device | None = None,
max_new_tokens: int = 512,
temperature: float = 0.8,
top_p: float = 0.95,
seed: int | None = None,
thinking: bool | str = "auto",
system_prompt: str | None = None,
private_context: tuple = (),
tools: tuple = (),
max_tool_iters: int = 4,
kv_reuse: bool | str = "auto",
steering: object = None,
_model: "PreTrainedModel | None" = None,
_tokenizer: "PreTrainedTokenizerBase | None" = None,
)
Bases: Functional, Participant
A conversation participant backed by a local HuggingFace causal LM.
Generation flow: the Conversation hands us a view — the transcript rendered from our perspective,
already context-fitted and flattened by finalize_view into [{role, content}]. We apply this model's
own chat template to it, generate, decode only the newly produced tokens, and split any <think>
reasoning out of the visible content. Only the visible answer becomes Message.content;
the parsed reasoning and raw completion live in metadata under neutral keys, so hidden generated text is
never fed back into other participants' views (it's stripped from history automatically because
render_roles uses content).
The participant is its own recipe. It stores what to load (hf_id + dtype/attn/quant/revision) and
loads the weights lazily on first use through the process-wide model cache — so an unrun participant is
cheap (KBs), pickles across a spawn boundary without shipping weights, and .set(...) copies share one
loaded model object by reference (the co-stepping batch win). model/tokenizer are properties that
trigger the load; the raw model is exposed deliberately (interp experiments register forward hooks on it).
Weight loads are keyed on (hf_id, device, dtype, attn, quant, revision) and cached, so all same-recipe
participants on one device share ONE object. Build via from_pretrained (lazy) or from_model (eager,
from an already-loaded model); .set() (from Functional) makes copy-on-write clones with fresh KV state.
model
property
¶
The loaded HF model, loading it on first access (cached process-wide). The raw model is exposed so interp experiments can register forward hooks directly.
tokenizer
property
¶
The loaded tokenizer, loading the model+tokenizer on first access (cached process-wide).
batch_signature
¶
Key identifying which model this participant would batch AS (used by the co-stepper). When loaded, the cached model object's identity is authoritative; when not yet loaded, the load recipe is (the cache guarantees same recipe on one device -> same object), so participants can be grouped WITHOUT forcing a load.
Source code in src/interlens/participant/participants/model_participant.py
for_model_type
classmethod
¶
The registered participant class for a transformers config.model_type (e.g. qwen2, gemma2),
falling back to base ModelParticipant for any family that declares no specialized subclass.
Source code in src/interlens/participant/participants/model_participant.py
from_model
classmethod
¶
from_model(
model: PreTrainedModel,
tokenizer: PreTrainedTokenizerBase | None = None,
*,
name: str,
device: str | device | None = None,
**participant_kwargs
) -> Self
Build a participant of THIS class from an already-loaded model (the eager path — e.g. to wrap a model
you already hold, or share weights between speakers). tokenizer is optional — when omitted it is inferred
from model.config._name_or_path. The chat-template flags (supports_system_role /
requires_alternating_roles) are derived from the tokenizer's own template. hf_id/dtype are read
back from the model so the participant can still be pickled + re-loaded on a spawn worker. Because it
constructs cls, calling it on a subclass returns that subclass — use AutoModelParticipant.from_model
to have the family resolved from config.model_type.
Source code in src/interlens/participant/participants/model_participant.py
from_pretrained
classmethod
¶
from_pretrained(
id_or_path: str | Path,
*,
name: str,
device: str | device = "cuda",
load_kwargs: dict | None = None,
**participant_kwargs
) -> Self
Build a participant of THIS class that will load id_or_path (an HF id or local path) lazily on first
use — no weights are touched here. load_kwargs (dtype / attn / quant / revision /
weights_path) are recorded for that deferred load; participant_kwargs (temperature,
max_new_tokens, system_prompt, tools, kv_reuse, …) go to the participant. When the load fires
it is process-cached, so all same-(id, device, dtype, …) participants share ONE model object. As with
from_model, the class is cls — use AutoModelParticipant.from_pretrained to resolve the family
from config.model_type (it reads only the config, still no weights).
Source code in src/interlens/participant/participants/model_participant.py
generate_batch
¶
generate_batch(
views: list[list[dict]],
*,
turn: int | None = None,
group_seed: int | None = None,
max_new_tokens: int | None = None
) -> list[Message]
Batched generation for many independent conversations that share THIS model (throughput mode).
Renders each view with this model's chat template and runs one model.generate over the
left-padded batch, returning one Message per view — the co-stepping throughput win (5-20x on a
rollout). max_new_tokens overrides this participant's per-turn cap for the batch (the co-stepper passes a
turn_cap here so a TokenBudget can shrink the final round). No tools/steering/capture/logprobs
here: callers needing those fall back to the per-conversation generate. Tokens are not guaranteed
identical to unbatched — batch composition and the single global RNG perturb rows (see PLAN §Execution
modes); only distributional reproducibility holds. metadata['batched'] marks these turns;
metadata['shared_prefill'] marks the fast path.
Source code in src/interlens/participant/participants/model_participant.py
parse_tool_calls
¶
Parse tool calls out of a generation. Base handles the common Hermes/Qwen <tool_call>{json}</tool_call>
format; families with other formats (Gemma's ```tool_code, Llama's <|python_tag|>) override
this. An unrecognized/absent call yields [] so the loop treats the output as a final message.
Source code in src/interlens/participant/participants/model_participant.py
render_tool_result
¶
Render a tool result as the standard structured tool message; the tokenizer's own template turns it
into the family-native format.
Source code in src/interlens/participant/participants/model_participant.py
split_reasoning
¶
Split a raw completion into (visible_content, parsed_think). Base handles the <think>...</think>
convention; families with other delimiters override this.
Source code in src/interlens/participant/participants/model_participant.py
Participant
¶
Bases: ABC
A participant in a conversation, either a model or a person.
A participant owns three things: an identity within the conversation (name + self_role/
others_role), its private framing (system_prompt + private_context — instructions/knowledge
only it sees), and the ability to turn a rendered view into its next message (generate). The
Conversation assembles the structured view from the shared transcript; the participant flattens that view
to what its chat template expects via finalize_view and generates.
name
instance-attribute
¶
A name or identifier to uniquely identify this participant within a conversation.
finalize_view
¶
Flatten the structured, context-fitted view into the [{role, content}] list the chat template
consumes. Applies family-specific repairs driven by the capability flags:
supports_system_role=False→ fold the leading system content into the first user turn (Gemma's template errors on a standalonesystemrole).requires_alternating_roles=True→ merge consecutive same-role segments (Gemma requires strict user/model alternation; the moderator seed + another speaker + private context can otherwise produce consecutiveuserturns that the template rejects). Merged turns keep author labels so speaker identity isn't lost in the concatenation.
Source code in src/interlens/participant/participant.py
generate
abstractmethod
¶
generate(
view: list[dict],
*,
steering=None,
capture=None,
patch=None,
return_logprobs: bool = False,
turn: int | None = None,
max_new_tokens: int | None = None
) -> Message
Produce this participant's next message given view — the conversation flattened to
[{"role", "content"}] from this participant's perspective. Returns a Message it authored.
Interp options apply to local-model participants: steering (a SteeringSpec), capture (a
CaptureRequest), patch (a Patch), and return_logprobs; turn is the message index used
to tag captured activations. Participants that can't honor an interp request (e.g. API-backed) must raise
rather than silently ignore it — a failed capture/steer must fail loudly.
Source code in src/interlens/participant/participant.py
Patch
dataclass
¶
Activation patching: overwrite a decoder layer's residual at specific token positions with saved
activations (captured from another run/branch).
This is the cross-branch causal-tracing primitive: capture activations at turn N in one branch (via
ActivationCache), then inject them at the aligned positions of another branch's forward. The harness owns
this because only it knows the turn/position correspondence between branches.
P2 applies the patch on the (single) prompt forward — positions index into the prompt sequence. Aligning
positions across branches is the caller's responsibility; Patch just performs the overwrite.
QwenModelParticipant
dataclass
¶
QwenModelParticipant(
name: str = "",
hf_id: str | None = None,
weights_path: str | None = None,
dtype: str = "bfloat16",
attn: str = "flash_attention_2",
quant: str | None = None,
revision: str | None = None,
device: str | device | None = None,
max_new_tokens: int = 512,
temperature: float = 0.8,
top_p: float = 0.95,
seed: int | None = None,
thinking: bool | str = "auto",
system_prompt: str | None = None,
private_context: tuple = (),
tools: tuple = (),
max_tool_iters: int = 4,
kv_reuse: bool | str = "auto",
steering: object = None,
_model: "PreTrainedModel | None" = None,
_tokenizer: "PreTrainedTokenizerBase | None" = None,
)
Bases: ModelParticipant
A participant in a conversation that is a Qwen language model. Its tool-call format is the Hermes/base
<tool_call> JSON already handled by ModelParticipant, so this class adds no behavior — it exists so
Qwen models resolve to a distinct, statically-typed participant class.
ReasoningVisibility
¶
Bases: str, Enum
Controls whether a participant's prior <think> reasoning is re-injected into views on later turns.
Reasoning is always parsed out of Message.content into metadata['parsed_think'] (so it is available
for interpretability regardless of this setting). This enum only governs history re-injection:
STRIP(default): prior reasoning is dropped from all views — matches R1/Qwen3 chat templates, keeps reasoning a genuinely private scratchpad the other participant never sees, and is always template-safe.SELF_RETAIN: a participant sees its own prior reasoning re-injected into its view (visible-scratchpad self-continuation experiments).SHARED: all participants see all prior reasoning (shared-CoT collaboration/debate experiments).
Non-STRIP modes re-inject reasoning as tagged text at render time so templates that reject a native prior
<think> turn don't error.
RunReport
dataclass
¶
Aggregate outcome of a run. Failures are isolated, not fatal: results holds every attempted job,
failed lists the ones that errored, skipped lists resume-skipped (already-checkpointed) ids.
RunResult
dataclass
¶
RunResult(
job_id: str,
conversation: object = None,
transcript: object = None,
analysis: object = None,
error: str | None = None,
device: str | None = None,
)
Outcome of one job: the finished conversation (weightless participants + completed transcript), its
transcript and (serializable) analysis, or an error string if it failed. conversation is the
object to sample()/inspect after a rollout (the source recipe is never mutated).
tokens_generated
property
¶
Total generated tokens in this conversation (summed from each turn's metadata['n_tokens']) — the
realized compute, for verifying matched-compute comparisons. 0 if the job failed.
SlidingWindowPolicy
¶
Bases: ContextPolicy
Keep the preserved framing plus the most recent keep_last turns; drop older turns.
keep_system=True (the default) preserves the system/moderator/private_context framing regardless of the
window; set it False to also let framing fall outside the window (rarely wanted). Unlike DropOldestPolicy
this is a fixed-size window rather than a fit-to-budget trim, so it's predictable turn-to-turn.
Source code in src/interlens/context/sliding_window_policy.py
SteeringSpec
dataclass
¶
A residual-stream intervention applied during generation via forward hooks on decoder layers.
mode='add' adds coef * direction to the residual at layers; mode='ablate' projects the
direction component out of the residual (directional ablation). The same mechanism covers both because
ablation is just the projection-removal variant of an additive hook.
A summary (mode, layers, coef, direction norm) is recorded into Message.metadata['steering'] by the
participant so a steered/ablated turn is reproducible.
difference_of_means
classmethod
¶
difference_of_means(
pos_acts,
neg_acts,
layers,
coef: float = 1.0,
mode: Mode = "add",
) -> "SteeringSpec"
Build a spec whose direction is the unit difference-of-means of two activation populations,
normalize(mean(pos) − mean(neg)) — the classic contrastive/concept-direction recipe, pointing TOWARD
the positive class. pos_acts/neg_acts are [n, d_model] (a stack of per-example residuals) or a
pre-pooled [d_model] vector; anything torch.as_tensor accepts works. Steer toward the positive
concept with coef>0 and away from it (suppress) with coef<0. layers is an int or an
iterable of decoder-layer indices. This lives here (not re-hand-rolled per experiment) per the
contribution convention.
Source code in src/interlens/interp/steering.py
register
¶
Register the steering hooks on model and return the handles (caller removes them after generate).
Source code in src/interlens/interp/steering.py
StopCondition
¶
Bases: ABC
A stateful predicate that ends a Conversation.run early.
Each condition is stateful (tracks its own counters) and is checked after every committed turn via
should_stop(conversation, last_message). reset() clears state and is called at the start of each
run (and branches get fresh copies), so one instance can be reused across runs without leaking state.
A condition may also cap the next generation via turn_cap (e.g. a token budget shrinks the last turn so
it lands exactly on budget) and may be installed ambiently as a context manager (with TokenBudget(...):
conv.rollout(...) applies it to every conversation in the block). New conditions subclass this without any
change to run.
reset
¶
turn_cap
¶
An upper bound on the NEXT turn's generated tokens, or None for no cap. The run loop passes it as
max_new_tokens (bounded by the speaker's own cap), so a budget condition can prevent both overshoot and a
single turn from consuming the whole allowance. Default: no cap.
Source code in src/interlens/stop/stop_condition.py
StopStringCondition
¶
Bases: StopCondition
Stop when a committed message's visible content contains any of the given strings (a done-signal).
Source code in src/interlens/stop/conditions.py
SummarizePolicy
¶
Bases: ContextPolicy
Compress older turns into a single summary segment instead of dropping them outright (the heaviest policy).
Keeps the preserved framing (system / moderator / private_context) and the most recent keep_last turns
verbatim, and replaces the older middle turns with one summary segment produced by summarizer — a
callable list[str] -> str over the dropped turns' contents. With no summarizer it inserts a neutral
placeholder, so it degrades to a labelled drop rather than silently losing content.
summarizer is a live callable and so is not serialized; a loaded template gets summarizer=None and
the caller re-injects one if needed.
Source code in src/interlens/context/summarize_policy.py
TokenBudget
¶
Bases: StopCondition
A per-conversation compute budget — the matched-compute primitive for fair solo-vs-pair comparisons.
per_conversation stops a conversation once ITS OWN cumulative generated tokens reach the budget, and
per_turn caps each individual turn so the allowance is spread across real conversation turns rather than
consumed by one monologue. Both are enforced via turn_cap too: the run loop shrinks the next generation to
min(speaker cap, per_turn, per_conversation - spent), so the budget is respected without overshoot.
The budget is per-conversation, not a shared pool. Spend is read from the conversation's own transcript
(metadata['n_tokens']), so the condition is stateless — in a rollout of N copies, each copy independently
gets the full budget. Cheap to count (never re-tokenizes) and trivially picklable, so it works installed
directly (run_until=TokenBudget(...)) or ambiently (with TokenBudget(per_conversation=200): conv.rollout()).
Source code in src/interlens/stop/conditions.py
TokenStopCondition
¶
Bases: StopCondition
Stop once the total generated tokens across turns reach max_tokens.
The per-turn count comes from Message.metadata['n_tokens'], which ModelParticipant.generate records —
so the source of truth is defined, not guessed. Turns without a count (e.g. seeded messages) contribute 0.
Source code in src/interlens/stop/conditions.py
Tool
¶
Bases: ABC
A capability a participant can invoke during its turn.
A tool has a name, a JSON schema (in the function-calling format chat templates render via their
tools= argument), and is callable with keyword arguments to produce a string result. Tools contain live
callables and so are NOT serializable — templates store tool names and resolve them against a
ToolRegistry at build time, mirroring how models are resolved from ids.
schema
abstractmethod
property
¶
The tool's JSON function schema, e.g.
{"type": "function", "function": {"name", "description", "parameters": {...}}}, passed straight to
apply_chat_template(tools=...) so each family renders it in its native format.
ToolCall
dataclass
¶
A parsed request from the model to invoke a tool: the tool name and its arguments.
raw keeps the exact text the model emitted (useful for debugging a family parser). Tool calls are parsed
out of the generation by a per-family parse_tool_calls — the format differs across model families, so the
parsed structure is uniform even though the surface syntax isn't.
ToolRegistry
¶
Resolves tool names (which serialize) to live Tool instances (which don't).
This is the tools analogue of the model registry: a template stores tool_names and, at build time on
each worker, the registry turns them into callables. Because spawned worker processes inherit no parent
state, tools must be registered at import time (or via a worker-init hook), not imperatively in the parent.
Source code in src/interlens/tools/registry.py
ToolResult
dataclass
¶
The outcome of executing a ToolCall: the tool name and its string output (or error text).
Transcript
dataclass
¶
The canonical, perspective-neutral record of a conversation, plus the logic to render it from a given participant's perspective.
Design: the transcript never commits to assistant/user roles. Each message just knows its author.
When it's a participant's turn, render_roles maps that neutral record into the chat-template [{role,
content}] shape from that participant's point of view — its own turns become self_role (normally
assistant), everyone else's become others_role (normally user). This single trick is why two
different models with different tokenizers can share one transcript: each gets the view its own template
expects, and the stored record stays singular.
P0 scope: this holds only the message list + rendering + list-like ergonomics. Serialization, context policies, author-labelling for N-party, and reasoning re-injection are layered on in later phases.
append
¶
Append a committed turn and return it. metadata captures anything non-authoritative (parsed
reasoning, logprobs, tool trails) without polluting content.
Source code in src/interlens/transcript.py
clear
¶
Drop every turn, returning self.
⚠️ This wipes the whole record, including any leading seed turn — the shared_context scenario framing and
any moderator/initial-instruction turns injected at construction. Nothing about the conversation's opening
setup survives. If you want to clear the dialogue but keep that framing (e.g. to rerun the same scenario),
call Conversation.reset instead, which empties and then re-seeds the shared_context turn.
Source code in src/interlens/transcript.py
copy
¶
Deep-ish copy of the message list for branching. Messages are small string records, so this is cheap;
the invariant that keeps it cheap is that heavy tensors never live in Message.metadata.
Source code in src/interlens/transcript.py
edit
¶
edit(
ref: "MessageRef",
content: str | None = None,
*,
author: str | None = None,
**metadata
) -> Message
Edit a committed past turn in place and return it. ref is a MessageRef (int index, negatives
allowed, or a Message object matched by identity). Only the arguments you pass are changed: content
and/or author are replaced when given; metadata keys are merged over the existing metadata (pass an
explicit key to overwrite it — untouched keys are left alone). Editing the Message object's fields
directly does the same thing, since the transcript holds it by reference.
Source code in src/interlens/transcript.py
load
classmethod
¶
Load a transcript from disk. Works with no models present (for scoring/analysis of saved runs).
pretty
¶
A human-readable dump for debugging: one [i] author: content block per turn. With metadata=True,
also list each turn's non-empty metadata (parsed reasoning, tool trail, token counts, …). Also what
str(transcript) / print(transcript) uses (without metadata).
Source code in src/interlens/transcript.py
render_roles
¶
Render the transcript turns only as [{"role", "content"}] from pov's perspective (pov is
keyword-only: render_roles(pov=alice)). Its own turns become self_role (normally assistant),
everyone else's become others_role (normally user). For a "what if X had just been said" render,
extend first with with_extra and render the result.
Limitation — this is NOT what the model actually sees. The Transcript is the perspective-neutral
record; it knows nothing about a participant's system_prompt / private_context, the conversation's
shared_system_prompt / shared_context framing, the context_policy fitting, or the family-specific
flatten (Gemma system-folding etc.). Those are added by the Conversation pipeline. For the real
generation input use Conversation.view(pov) (or Conversation.render_templated(pov) for the templated
string); use this method for lower-level inspection of the record itself.
Source code in src/interlens/transcript.py
render_templated
¶
render_templated(
*,
pov: "ModelParticipant",
add_generation_prompt: bool = False,
tokenize: bool = False
)
Template the transcript turns only from pov's point of view — i.e. render_roles role-swapped,
then run through pov's tokenizer chat template so the special / control tokens (<|im_start|>assistant
etc.) are included. pov is keyword-only: call render_templated(pov=alice). Returns a str
(tokenize=False, default) or token ids (tokenize=True); add_generation_prompt appends the
assistant open tag.
Prefer Conversation.render_templated if you have access to a conversation: it adds the system prompt, private context, context-fitting, and family-specific flattening, printing the exact input models see.
Limitations¶
NOT the exact model input. This omits the system prompt, private context, context-fitting,
and the family-specific flatten that a real turn also gets (see render_roles); it also renders the raw
role-swapped turns as-is, which can even raise for families needing strict alternation / system-folding
(e.g. Gemma) since the merge isn't applied. For the exact, family-correct prompt the model actually sees,
use Conversation.render_templated(pov). This method is a lower-level inspection of the record. Requires a
local model participant (with a tokenizer); an API participant has none and raises.
Source code in src/interlens/transcript.py
resolve_index
¶
Normalize a message reference to a concrete [0, len) position. ref is either an int index
(Python semantics — negatives count from the end, -1 = last turn) or a Message object, located
by identity (is, not equality — two turns with the same text are still distinct). Raises IndexError
for an out-of-range int and ValueError for a message that isn't in this transcript.
Source code in src/interlens/transcript.py
rewind
¶
Rewind so the turn referenced by to becomes the new last turn — i.e. drop everything after it,
keeping to itself (mutates in place, returns self). to is a MessageRef: an int index
(negatives count from the end — rewind(to=-2) takes back the single last turn) or a Message object
matched by identity. To drop all turns use clear.
Source code in src/interlens/transcript.py
truncate
¶
Keep only the first length turns, dropping everything after (mutates in place, returns self).
length is a count, clamped to [0, len]: over-long values are a no-op and negatives count from the
end (-1 keeps all but the last turn). To cut relative to a specific message, use resolve_index /
rewind(to=…) instead.
Source code in src/interlens/transcript.py
with_extra
¶
Return a lightweight, ephemeral read-only Transcript = this one's messages followed by extra,
backed by a lazy concatenation (_ConcatMessages): the base message list is not copied — only
extra is materialized, so this is O(len(extra)), not O(len(transcript)), and Message objects are
shared by reference. The original is untouched.
This is the data-layer primitive behind ephemeral sampling ("what would a participant say if extra had
just been said?"): build the extended transcript, render it, discard it —
transcript.with_extra(Message("bob", "…")).render_roles(pov=alice). The result is read-only (rendering,
len, indexing, iteration all work; appending raises — copy() first if you need a mutable one).
(A one-shot generator was rejected: Transcript needs len / indexing / repeated iteration, which a
generator can't provide.)
Source code in src/interlens/transcript.py
TurnStopCondition
¶
Bases: StopCondition
Stop after max_turns committed turns.
Source code in src/interlens/stop/conditions.py
available_devices
¶
List the devices to spread conversations across: every CUDA GPU, else a single mps/cpu fallback.
Multi-GPU parallelism lives across conversations (they're independent); within one conversation turns are sequential, so more GPUs never speed up a single conversation — only throughput over many.
Source code in src/interlens/runner/devices.py
continuation_logprob
¶
continuation_logprob(
model: "PreTrainedModel",
*,
target_ids: Tensor,
prefix_ids: Tensor | None = None,
prefix_embeds: Tensor | None = None,
attention_mask: Tensor | None = None,
reduction: str = "mean",
checkpoint: bool = False
) -> torch.Tensor
Differentiable teacher-forced logprob of target_ids continuing a prefix, under model.
The grad-enabled analog of train_to_steer/rl_elicit.py::cont_logprob (which is @torch.no_grad and returns
a float). Supply the prefix as prefix_ids ([batch, P] long) OR prefix_embeds ([batch, P, d_model]
float — e.g. a learnable soft prompt, or soft embeddings bridged from model A). target_ids is [batch, T]
long (or [T] broadcast to batch 1). The prefix and target are embedded via model.get_input_embeddings()
and run in one forward; the returned scalar (or per-example vector, see reduction) is the logprob the model
assigns to the exact target tokens, still attached to the graph.
reduction: "mean" (mean over target tokens, mean over batch -> scalar; the reward used by rl_elicit),
"sum" (sum over target tokens, mean over batch), or "none" ([batch, T] per-token logprobs). Use
"mean" as a length-normalized elicitation reward, "sum" when total sequence likelihood matters, "none"
for custom weighting. checkpoint is forwarded to save memory when backpropagating through a large frozen B.
Source code in src/interlens/interp/grad.py
conversation_from_ids
¶
Deprecated thin alias for :func:conversation_from_models / :meth:Conversation.from_models, kept for
back-compat. Prefer Conversation.from_models(models=..., ...).
Source code in src/interlens/factories.py
conversation_from_models
¶
conversation_from_models(
models: tuple[ModelLike, ...],
names: tuple[str, ...] = ("a", "b"),
device: str | device = "cuda",
dtype: dtype = torch.bfloat16,
shared_context: str | None = None,
shared_system_prompt: str | None = None,
prompt: PromptLike = None,
**gen_kwargs: Any
) -> Conversation
Scaffold a conversation from a tuple of models — each an HF id (str) or an already-loaded
PreTrainedModel (see ModelLike). Each becomes a family-correct participant via
AutoModelParticipant.from_; names gives them their identities. The order of models / names
is the speaking order — the first speaks first unless you pass first= to run — and **gen_kwargs
are forwarded to every participant.
This is the implementation behind :meth:Conversation.from_models (which just wraps it). If two ids resolve
to the same HF model the weights are loaded once and shared (via load_model's process cache).
Two ways to seed the opening, without touching the transcript by hand:
shared_context— a neutral,moderator-voiced turn everyone sees (scenario/topic framing); pair withshared_system_promptfor system-role instructions. These are the principled framing knobs (serialized into a template).prompt— a participant-voiced opener: astris attributed to the LAST participant so the first speaker replies to it, aMessagesets the author explicitly. Use this when the opener should read as something a speaker said rather than moderator framing.
Source code in src/interlens/factories.py
dataset_field
¶
Reference a dataset column in a templated field: shared_context=("Solve:\n\n", dataset_field("question"))
resolves row["question"] for each row at rollout expansion. See the module docstring for all template forms.
Source code in src/interlens/templating.py
decoder_layers
¶
Return the list of transformer decoder layer modules, across common HF architectures.
Steering/patching hooks and per-layer capture all need the ordered layer stack. Qwen and Gemma both expose
it at model.model.layers; a few fallbacks cover other families so the interp layer isn't Qwen/Gemma-only.
Multimodal image-text-to-text wrappers (Qwen 3.5, Gemma 4, …) nest the text decoder one level deeper — its
layers live at model.language_model.layers (or .model.language_model.model.layers) — so those paths are
included too. PEFT/adapter wrappers (PeftModelForCausalLM) are unwrapped first so hooks land on the real
decoder layers.
Source code in src/interlens/interp/layers.py
forward_with_grad
¶
forward_with_grad(
model: "PreTrainedModel",
*,
input_ids: Tensor | None = None,
inputs_embeds: Tensor | None = None,
attention_mask: Tensor | None = None,
capture: GradCaptureSpec | None = None,
checkpoint: bool = False
) -> GradForwardOutput
One grad-connected forward pass over input_ids XOR inputs_embeds.
Exactly one of input_ids ([batch, seq] long) or inputs_embeds ([batch, seq, d_model] float,
typically the output of bridge.soft_embed or a learnable soft prompt) must be given. attention_mask is
optional ([batch, seq]). Returns logits and any requested hidden activations, all still attached to the
graph so .backward() flows back to inputs_embeds / soft-prompt params / an upstream model A.
checkpoint=True enables HF gradient checkpointing for this call (use_reentrant=False) and restores the
prior setting afterwards; it recomputes layer activations on the backward pass to trade compute for memory, and
produces identical gradients. Residual sites come from output_hidden_states (grad-connected); attn/mlp
sites come from non-detaching forward hooks on the submodules.
Source code in src/interlens/interp/grad.py
gumbel_softmax_tokens
¶
Relaxed (Gumbel-softmax) sample over the last (vocab) dim of logits, differentiable in logits.
tau is the temperature: high -> smooth mixture (biased but low-variance gradients), low -> near-one-hot
(faithful to discrete sampling but higher-variance). Anneal tau down over training to move from an easy soft
optimization toward the discrete tokens B will actually see. hard=True uses the straight-through estimator: a
true one-hot on the forward pass, softmax gradient on the backward pass — so the value B consumes is a real token
while gradients still flow. Returns [..., V] to feed straight into soft_embed.
Source code in src/interlens/interp/bridge.py
register_worker_init
¶
Register a zero-arg callable to run once at worker startup (e.g. to populate the tool/analyzer registries).
run
¶
run(
conversations,
devices=None,
out_dir=None,
resume=False,
batched=True,
max_batch_size=None,
) -> RunReport
Run several conversation lineups in ONE pool — the multi-lineup entry point (e.g. a ladder of model pairs × conditions in one overnight job).
Each conversation is expanded to its jobs (one per data() row if it has data, else a single conversation),
with job ids namespaced by that conversation's name (default conv{i}) so ids stay unique and resumable.
All jobs go into one pool, so GPUs stay packed across lineups (no idle tail between sequential rollouts) under a
single out_dir/resume namespace, returning one merged RunReport. Mixing lineups is safe: batched
co-stepping groups by schedule signature, so each distinct lineup forms its own batch group.
Conversation.rollout is the single-lineup sugar for this (run([conv], ...)-equivalent via run_jobs).
Source code in src/interlens/runner/pool.py
run_jobs
¶
run_jobs(
jobs,
devices=None,
out_dir=None,
resume=False,
batched=True,
max_batch_size=None,
) -> RunReport
Run (job_id, Conversation) jobs across devices, with checkpointing, resume, and per-job failure isolation.
Parallel by default on two axes: one worker process per device (jobs round-robined; multi-GPU spawns via
torch.multiprocessing since fork+CUDA is broken — a single device runs in-process), and within each device
batched co-stepping (batched=True). Jobs are grouped by co-step schedule signature so same-schedule jobs
batch into one model.generate — correct for ANY mix. batched=False gives the DETERMINISTIC path.
Source code in src/interlens/runner/pool.py
soft_embed
¶
Mix model's input-embedding rows by a per-position distribution: [..., V] @ E[V, d] -> [..., d].
probs is a (relaxed or one-hot) distribution over model's vocab for each position — e.g. a softmax /
gumbel_softmax_tokens output of an upstream model. Returns grad-connected soft embeddings suitable to pass as
inputs_embeds to forward_with_grad / continuation_logprob. This is exact for hard one-hots: feeding
one_hot(ids) reproduces model.get_input_embeddings()(ids).
Source code in src/interlens/interp/bridge.py
token_logprobs
¶
Compute per-token logprobs / surprisal / entropy for a generation.
scores is the tuple of per-step logit tensors from model.generate(..., output_scores=True,
return_dict_in_generate=True) (one [vocab] per generated token); generated_ids are the sampled
token ids. Returns lists suitable to drop into Message.metadata — scalar-per-token, so they stay small
and don't violate the "no heavy tensors in metadata" invariant.
Surprisal = -logprob (nats); entropy is the full next-token distribution entropy at each step (a readout of the model's uncertainty, distinct from the surprisal of the token it actually emitted).