Skip to content

interlens

interlens

APIParticipant dataclass

APIParticipant(
    name: str = "",
    model_id: str = "",
    provider: Provider = "anthropic",
    system_prompt: str | None = None,
    private_context: tuple = (),
    max_tokens: int = 512,
    temperature: float = 1.0,
    batch: bool = False,
    client: object = None,
    requires_alternating_roles: bool = True,
)

Bases: Functional, Participant

A participant backed by a hosted API — Claude via anthropic (provider="anthropic", the default) or any model behind OpenRouter (provider="openrouter", OpenAI-compatible) — for use as a debate opponent, moderator, or the classifier inside an analyze callback.

It is a full participant for conversation purposes but has no local model — so there is no device, no activations, and no steering. Any interp request (capture/steering/patch/return_logprobs) raises rather than silently no-op'ing: in a measurement harness, a steering sweep that quietly did nothing on an API participant would produce a false "no effect" conclusion. Seeds don't bind hosted models, so API turns are excluded from the identical-replay guarantee.

Concurrency is network-bound, so pure-API conversations run thread-pooled rather than process-per-GPU (handled by the runner). The client callable is injectable for testing.

generate_batch

generate_batch(
    views: list[list[dict]],
    *,
    turn: int | None = None,
    group_seed: int | None = None,
    max_new_tokens: int | None = None
) -> list[Message]

Generate one turn for many independent conversations at once — the API analogue of ModelParticipant.generate_batch, driven by the runner's co-stepper (rollout(..., batched=True)) to make large API rollouts cheap and throughput-bound.

With batch=True every view is sent as one asynchronous provider batch (Anthropic Message Batches / OpenAI Batch API) via client.submit_batch — ~50% cost and much higher throughput, at the price of batch-window latency. If the provider has no batch API (e.g. OpenRouter) this raises rather than silently degrading, so a requested batch is never quietly run as serial calls. With batch=False it falls back to sequential per-view calls (correct, just no batch discount). Interp is unavailable here, as for generate. turn/group_seed are accepted for co-stepper compatibility but unused (seeds do not bind hosted models). metadata['batched'] marks these turns.

Source code in src/interlens/participant/participants/api_participant.py
def generate_batch(self, views: list[list[dict]], *, turn: int | None = None,
                   group_seed: int | None = None, max_new_tokens: int | None = None) -> list[Message]:
	"""Generate one turn for many independent conversations at once — the API analogue of
	``ModelParticipant.generate_batch``, driven by the runner's co-stepper (``rollout(..., batched=True)``) to
	make large API rollouts cheap and throughput-bound.

	With ``batch=True`` every view is sent as one **asynchronous provider batch** (Anthropic Message Batches /
	OpenAI Batch API) via ``client.submit_batch`` — ~50% cost and much higher throughput, at the price of
	batch-window latency. **If the provider has no batch API (e.g. OpenRouter) this raises** rather than
	silently degrading, so a requested batch is never quietly run as serial calls. With ``batch=False`` it
	falls back to sequential per-view calls (correct, just no batch discount). Interp is unavailable here, as
	for ``generate``. ``turn``/``group_seed`` are accepted for co-stepper compatibility but unused (seeds do
	not bind hosted models). ``metadata['batched']`` marks these turns."""
	if not views:
		return []
	client = self.client or _default_client(self.provider)
	max_tokens = max_new_tokens if max_new_tokens is not None else self.max_tokens
	requests = []
	for view in views:
		system, messages = self._split_view(view)
		requests.append(dict(system=system, messages=messages, model=self.model_id,
		                     max_tokens=max_tokens, temperature=self.temperature))
	if self.batch:
		if not hasattr(client, "submit_batch"):
			raise NotImplementedError(
				f"APIParticipant {self.name!r} has batch=True but its client {type(client).__name__} exposes "
				f"no submit_batch; batch mode is unavailable for provider {self.provider!r}.")
		texts = client.submit_batch(requests)
	else:
		texts = [client(**r) for r in requests]
	return [Message(author=self.name, content=t,
	                metadata={"provider": self.provider, "model": self.model_id, "batched": True})
	        for t in texts]

ActivationCache

ActivationCache(offload: OffloadLocation = 'cpu')

A queryable store of captured activations, tagged by conversation structure.

Records know which participant/turn/layer/site they came from, so a downstream probe can ask for "bob's turn 3, layer 18, the answer span" rather than juggling anonymous tensors. This is the object every interp consumer reads; the harness never puts activations in Message.metadata (that would blow up branch()'s transcript copy), so the cache is the single home for heavy tensors.

Source code in src/interlens/interp/activation_cache.py
def __init__(self, offload: OffloadLocation = "cpu"):
	self.offload = offload
	self.records: list[ActivationRecord] = []

add_batch

add_batch(records: list[ActivationRecord]) -> None

Add many records at once, offloading all their tensors in one batched pinned transfer (see offload_to_cpu). Preferred over a loop of add when a single capture pass produces many records — it turns N GPU->CPU copies into one per shape-group.

Source code in src/interlens/interp/activation_cache.py
def add_batch(self, records: list[ActivationRecord]) -> None:
	"""Add many records at once, offloading all their tensors in one batched pinned transfer (see
	``offload_to_cpu``). Preferred over a loop of ``add`` when a single capture pass produces many records —
	it turns N GPU->CPU copies into one per shape-group."""
	if not records:
		return
	if self.offload == "cpu":
		offloaded = offload_to_cpu([r.tensor for r in records])
		for r, t in zip(records, offloaded):
			r.tensor = t
	else:
		for r in records:
			r.tensor = r.tensor.detach()
	self.records.extend(records)

at

at(
    *,
    participant=None,
    message_idx=None,
    layer=None,
    site="residual"
) -> torch.Tensor

Return the single matching tensor, erroring if the filters aren't unique — the ergonomic accessor for "give me exactly this activation".

Source code in src/interlens/interp/activation_cache.py
def at(self, *, participant=None, message_idx=None, layer=None, site="residual") -> torch.Tensor:
	"""Return the single matching tensor, erroring if the filters aren't unique — the ergonomic accessor for
	"give me exactly this activation"."""
	matches = self.query(participant=participant, message_idx=message_idx, layer=layer, site=site)
	if len(matches) != 1:
		raise KeyError(f"expected exactly one record, got {len(matches)} for "
		               f"participant={participant} message_idx={message_idx} layer={layer} site={site}")
	return matches[0].tensor

AnyStopCondition

AnyStopCondition(conditions: list[StopCondition])

Bases: StopCondition

Fires when any member condition fires. run(until=[...]) wraps a list in this.

Source code in src/interlens/stop/stop_condition.py
def __init__(self, conditions: list[StopCondition]):
	self.conditions = list(conditions)

AutoModelParticipant

Resolver for creating Participant instances automatically from HuggingFace model identifiers, local model paths, or already-loaded PreTrainedModels.

HF-style factory for family-correct local-model participants — the participant analog of AutoModelForCausalLM. It resolves the concrete ModelParticipant subclass from the model's transformers config.model_type via the class self-registry (ModelParticipant.for_model_type), then delegates the actual build to that class's from_model / from_pretrained — so all the loading / tokenizer-inference / chat-flag logic lives in one place (on ModelParticipant) and this class is only the family dispatcher.

  • from_ dispatches on the argument type (str id → from_pretrained; PreTrainedModelfrom_model).
  • To get a statically-known subclass, name it directly: QwenModelParticipant.from_pretrained(...). This factory returns the (dynamically resolved) base ModelParticipant type.

from_ staticmethod

from_(
    model: _QwenId,
    *,
    name: str,
    tokenizer: PreTrainedTokenizerBase | None = ...,
    device: str | device = ...,
    load_kwargs: dict | None = ...,
    **participant_kwargs: Any
) -> QwenModelParticipant
from_(
    model: _GemmaId,
    *,
    name: str,
    tokenizer: PreTrainedTokenizerBase | None = ...,
    device: str | device = ...,
    load_kwargs: dict | None = ...,
    **participant_kwargs: Any
) -> GemmaModelParticipant
from_(
    model: _LlamaId,
    *,
    name: str,
    tokenizer: PreTrainedTokenizerBase | None = ...,
    device: str | device = ...,
    load_kwargs: dict | None = ...,
    **participant_kwargs: Any
) -> LlamaModelParticipant
from_(
    model: ModelLike,
    *,
    name: str,
    tokenizer: PreTrainedTokenizerBase | None = ...,
    device: str | device = ...,
    load_kwargs: dict | None = ...,
    **participant_kwargs: Any
) -> ModelParticipant
from_(
    model: ModelLike,
    *,
    name: str,
    tokenizer: PreTrainedTokenizerBase | None = None,
    device: str | device = "cuda",
    load_kwargs: dict | None = None,
    **participant_kwargs: Any
) -> ModelParticipant

Build a participant from either an HF id (str) or an already-loaded PreTrainedModel, dispatching to from_pretrained / from_model. tokenizer applies only to the loaded-model case (an id / path loads its own matching tokenizer, so it is ignored there).

Source code in src/interlens/factories.py
@staticmethod
def from_(model: ModelLike, *, name: str, tokenizer: PreTrainedTokenizerBase | None = None,
          device: str | torch.device = "cuda", load_kwargs: dict | None = None,
          **participant_kwargs) -> ModelParticipant:
	"""Build a participant from either an HF id (str) or an already-loaded ``PreTrainedModel``, dispatching to
	``from_pretrained`` / ``from_model``. ``tokenizer`` applies only to the loaded-model case (an id / path loads
	its own matching tokenizer, so it is ignored there)."""
	if isinstance(model, (str, Path)):
		return AutoModelParticipant.from_pretrained(model, name=name, device=device,
		                                            load_kwargs=load_kwargs, **participant_kwargs)
	return AutoModelParticipant.from_model(model, tokenizer, name=name, device=device, **participant_kwargs)

from_model staticmethod

from_model(
    model: PreTrainedModel,
    tokenizer: PreTrainedTokenizerBase | None = None,
    *,
    name: str,
    device: str | device | None = None,
    **participant_kwargs: Any
) -> ModelParticipant

Build a family-correct participant from an already-loaded model — the class is resolved from config.model_type (unknown types fall back to base ModelParticipant), then that class's :meth:ModelParticipant.from_model does the tokenizer inference and chat-flag derivation.

Source code in src/interlens/factories.py
@staticmethod
def from_model(model: PreTrainedModel, tokenizer: PreTrainedTokenizerBase | None = None, *, name: str,
               device: str | torch.device | None = None, **participant_kwargs) -> ModelParticipant:
	"""Build a family-correct participant from an already-loaded ``model`` — the class is resolved from
	``config.model_type`` (unknown types fall back to base ``ModelParticipant``), then that class's
	:meth:`ModelParticipant.from_model` does the tokenizer inference and chat-flag derivation."""
	cls = ModelParticipant.for_model_type(getattr(model.config, "model_type", None))
	return cls.from_model(model, tokenizer, name=name, device=device, **participant_kwargs)

from_pretrained staticmethod

from_pretrained(
    id_or_path: _QwenId,
    *,
    name: str,
    device: str | device = ...,
    load_kwargs: dict | None = ...,
    **participant_kwargs: Any
) -> QwenModelParticipant
from_pretrained(
    id_or_path: _GemmaId,
    *,
    name: str,
    device: str | device = ...,
    load_kwargs: dict | None = ...,
    **participant_kwargs: Any
) -> GemmaModelParticipant
from_pretrained(
    id_or_path: _LlamaId,
    *,
    name: str,
    device: str | device = ...,
    load_kwargs: dict | None = ...,
    **participant_kwargs: Any
) -> LlamaModelParticipant
from_pretrained(
    id_or_path: str | Path,
    *,
    name: str,
    device: str | device = ...,
    load_kwargs: dict | None = ...,
    **participant_kwargs: Any
) -> ModelParticipant
from_pretrained(
    id_or_path: str | Path,
    *,
    name: str,
    device: str | device = "cuda",
    load_kwargs: dict | None = None,
    **participant_kwargs: Any
) -> ModelParticipant

Return a family-correct participant for id_or_path (an HF id or local path) that will load its weights lazily on first use. The concrete class is resolved from config.model_type by reading ONLY the model's config (AutoConfig.from_pretrained — cheap, no weights); load_kwargs (dtype / attn / quant / revision / weights_path) are recorded for that deferred load and participant_kwargs go to the participant (see :meth:ModelParticipant.from_pretrained).

Source code in src/interlens/factories.py
@staticmethod
def from_pretrained(id_or_path: str | Path, *, name: str, device: str | torch.device = "cuda",
                    load_kwargs: dict | None = None, **participant_kwargs) -> ModelParticipant:
	"""Return a family-correct participant for ``id_or_path`` (an HF id or local path) that will load its weights
	**lazily on first use**. The concrete class is resolved from ``config.model_type`` by reading ONLY the
	model's config (``AutoConfig.from_pretrained`` — cheap, no weights); ``load_kwargs`` (``dtype`` / ``attn`` /
	``quant`` / ``revision`` / ``weights_path``) are recorded for that deferred load and ``participant_kwargs``
	go to the participant (see :meth:`ModelParticipant.from_pretrained`)."""
	from transformers import AutoConfig  # lazy: avoids importing transformers at module import
	lk = load_kwargs or {}
	cfg = AutoConfig.from_pretrained(id_or_path, revision=lk.get("revision"))
	cls = ModelParticipant.for_model_type(getattr(cfg, "model_type", None))
	return cls.from_pretrained(id_or_path, name=name, device=device, load_kwargs=load_kwargs,
	                           **participant_kwargs)

CaptureSpec dataclass

CaptureSpec(
    sites: tuple[Site, ...] = ("residual",),
    layers: tuple[int, ...] | None = None,
    offload: OffloadLocation = "cpu",
)

What to capture during a generation: which sites at which layers, and where to keep the tensors.

Capture defaults to a narrow set — you pass the layers/sites you actually want — because capturing all layers × all tokens × many rollouts OOMs fast. offload='cpu' moves captured tensors off-GPU as they are recorded (essential for large sweeps); offload=None keeps them on-device.

ContextItem dataclass

ContextItem(
    content: str,
    role_hint: Role = "user",
    author: str = "moderator",
)

A single item of private asymmetric knowledge given to one participant (a briefing, a document, a fact).

Deliberately distinct from Message: a briefing is not a dialogue turn — it has no turn index and its author is nominal — so overloading Message would give it misleading turn/authorship semantics.

Role semantics are pinned rather than role-swapped: a briefing renders by default as user-provided context (role_hint=USER) — the participant reads it as information handed to it, not as its own prior speech — with SYSTEM available for standing private instructions. Private context is injected only into the owning participant's view and never enters the shared transcript.

ContextPolicy

Bases: ABC

Decides how to fit a participant's view within its model's context window.

Crucially, fit runs on the typed ViewSegment list, before the family-specific finalize_view folds/merges anything. Operating pre-finalize means the policy can reliably preserve the system block and moderator seed (their origin is still intact) and trim only turn segments, instead of trying to reverse-engineer meaning out of already-folded text.

to_dict

to_dict() -> dict

Serialize as {"kind": ..., **params}. Subclasses with parameters extend the params; the default covers parameterless policies.

Source code in src/interlens/context/context_policy.py
def to_dict(self) -> dict:
	"""Serialize as ``{"kind": ..., **params}``. Subclasses with parameters extend the params; the default
	covers parameterless policies."""
	return {"kind": type(self).__name__}

Conversation dataclass

Conversation(
    participants: tuple[Participant, ...],
    transcript: Transcript = Transcript(),
    shared_context: str | None = None,
    shared_system_prompt: str | None = None,
    moderator_name: str = "moderator",
    context_policy: ContextPolicy = ErrorPolicy(),
    context_limit: int | None = None,
    reasoning_visibility: ReasoningVisibility = ReasoningVisibility.STRIP,
    execution_mode: ExecutionMode = ExecutionMode.THROUGHPUT,
    message_hooks: list = list(),
    _turns: int | None = None,
    _data: object = None,
    _analyzer: object = None,
    _name: str | None = None,
    _seed: int = 0,
    _run_until: object = None,
)

Bases: Functional

A multi-agent conversation: the recipe, the live dialogue, AND the benchmark-rollout driver, all in one lightweight object.

It orchestrates turn-taking between Participants over a shared, perspective-neutral Transcript, owning who speaks when and the ordered view pipeline: for each speaker it assembles a structured, typed view (system block → private context → transcript turns), fits it to the context window on those typed segments, then lets the participant flatten it (family fold/merge) and generate. Fitting before the lossy family flatten is what lets the context policy preserve the system/moderator framing reliably (see context/).

Scenario framing is split by ownership: shared framing (shared_context as a moderator seed turn, shared_system_prompt) lives here; private framing (system_prompt, private_context) lives on each participant, so it is naturally invisible to the others and to the transcript.

Copy-on-write + lazy. Participants load their weights lazily, so an unrun Conversation is a cheap recipe. Build it up functionally — conv.turns(4).data(ds).analyzer(grade) — where each dot-modifier returns a modified copy (via Functional.set), never mutating the original; call .set(field=value) for fields without a dot-modifier. build-free: run it directly with run(), or expand it over data / N samples with rollout() (which copies it per row — see there).

Also supports branching (branch() / branch_from()), in-place history editing (rewind(), edit(), reset()), ephemeral sampling (sample() / sample_all()), activation capture (capture()), and disk persistence (save() / load()).

context_limit class-attribute instance-attribute

context_limit: int | None = None

Token budget the context_policy fits to. None = use the tokenizer's own model_max_length.

context_policy class-attribute instance-attribute

context_policy: ContextPolicy = field(
    default_factory=ErrorPolicy
)

How each speaker's view is fit to the context window — run on the typed segments before the family flatten so framing is preserved. Default ErrorPolicy raises on overflow rather than silently truncating; see context/ for sliding-window / drop-oldest / summarize alternatives.

execution_mode class-attribute instance-attribute

execution_mode: ExecutionMode = ExecutionMode.THROUGHPUT

The determinism-vs-throughput tension the runner consults: THROUGHPUT (default) permits batched co-stepping + KV reuse + flash-attn (only distributional reproducibility); DETERMINISTIC disables those for token-identical replay and interp fidelity.

message_hooks class-attribute instance-attribute

message_hooks: list = field(default_factory=list)

Runtime-only middleware (not persisted): each hook vets / edits / denies a freshly generated message before it is committed. Applied in order; default empty = pass-through. See hooks/.

moderator_name class-attribute instance-attribute

moderator_name: str = 'moderator'

Author name used for the shared_context seed and any moderator turns. Must not collide with a participant name (validated in __post_init__).

participants instance-attribute

participants: tuple[Participant, ...]

The conversation's participants. Order is significant — it is the default speaking order. run alternates through them in this order (turn k is participants[(start + k) % n]), and run(first=...) only shifts the starting index; so the first element speaks first unless first overrides. Names must be unique and none may equal moderator_name.

reasoning_visibility class-attribute instance-attribute

reasoning_visibility: ReasoningVisibility = (
    ReasoningVisibility.STRIP
)

Whether a prior turn's parsed <think> reasoning is re-injected into later views: STRIP (never), SELF_RETAIN (a speaker sees only its own), SHARED (everyone sees everyone's). Reasoning is always parsed into metadata['parsed_think'] regardless, so interp is unaffected.

row property

row: dict

The dataset row that produced this conversation in a data-driven rollout (the same dict the dataset_fields resolved against), or {} outside a rollout. Read it in an analyzer to reach per-row side data — labels, gold answers, ids — WITHOUT templating it into the model's view (so it never leaks into the conversation). Travels with the conversation across the spawn boundary.

shared_context class-attribute instance-attribute

shared_context: str | None = None

Scenario framing every participant sees, injected once as a leading moderator_name turn when the transcript starts empty. None = no seed turn; "" still seeds a (blank) turn, so the first speaker has a non-empty view.

shared_system_prompt class-attribute instance-attribute

shared_system_prompt: str | None = None

A system prompt merged into every participant's system block (after that participant's private system_prompt). None = no shared system prompt.

transcript class-attribute instance-attribute

transcript: Transcript = field(default_factory=Transcript)

The shared, perspective-neutral message history. Defaults to empty; when starting fresh, shared_context is seeded as the first moderator_name turn (a branched/loaded transcript already contains it and is not re-seeded).

branch

branch() -> 'Conversation'

Fork into a new Conversation that reuses the same participant objects (shared weights, zero GPU cost) with a copied, independent transcript. The branch can diverge freely; the original is untouched. This is exactly a no-op copy-on-write clone (self.set()). To fork from a specific point in the history rather than the end, use branch_from.

Source code in src/interlens/conversation.py
def branch(self) -> "Conversation":
	"""Fork into a new ``Conversation`` that **reuses the same participant objects** (shared weights, zero GPU
	cost) with a copied, independent transcript. The branch can diverge freely; the original is untouched. This
	is exactly a no-op copy-on-write clone (``self.set()``). To fork from a specific point in the history rather
	than the end, use ``branch_from``."""
	return self.set()

branch_from

branch_from(ref: MessageRef) -> 'Conversation'

Fork a new conversation whose history is this one's turns up to and including the turn ref — i.e. branch as if the conversation had stopped right after ref, ready for a different continuation. ref is a MessageRef: an int index (negatives count from the end) or the Message object itself (matched by identity). The original is untouched; the fork shares participants (zero GPU cost).

Source code in src/interlens/conversation.py
def branch_from(self, ref: MessageRef) -> "Conversation":
	"""Fork a new conversation whose history is this one's turns **up to and including** the turn ``ref`` — i.e.
	branch as if the conversation had stopped right after ``ref``, ready for a different continuation. ``ref`` is
	a ``MessageRef``: an ``int`` index (negatives count from the end) or the ``Message`` object itself (matched
	by identity). The original is untouched; the fork shares participants (zero GPU cost)."""
	cut = self.transcript.resolve_index(ref)  # resolve against the original (the fork holds copies, new identities)
	fork = self.branch()
	fork.transcript.rewind(to=cut)
	return fork

capture

capture(
    sites=("residual",),
    layers=None,
    offload: OffloadLocation = "cpu",
)

Context manager that captures activations for every step inside the block into a fresh ActivationCache, auto-tagged by the current speaker + turn::

    with conv.capture(sites=["residual"], layers=[8, 12]) as cache:
            conv.step(bob)
    cache.at(participant="bob", layer=12)
Source code in src/interlens/conversation.py
@contextmanager
def capture(self, sites=("residual",), layers=None, offload: OffloadLocation = "cpu"):
	"""Context manager that captures activations for every ``step`` inside the block into a fresh
	``ActivationCache``, auto-tagged by the current speaker + turn::

		with conv.capture(sites=["residual"], layers=[8, 12]) as cache:
			conv.step(bob)
		cache.at(participant="bob", layer=12)
	"""
	cache = ActivationCache(offload=offload)
	spec = CaptureSpec(sites=tuple(sites), layers=tuple(layers) if layers is not None else None, offload=offload)
	self._pending_capture = CaptureRequest(cache=cache, spec=spec)
	try:
		yield cache
	finally:
		self._pending_capture = None

edit

edit(
    ref: MessageRef,
    content: str | None = None,
    *,
    author: str | None = None,
    **metadata
) -> Message

Edit a committed past turn in place and return it (see Transcript.edit): ref is a MessageRef (int index, negatives allowed, or a Message object matched by identity), and content / author / **metadata are the fields to change. Editing the returned Message's fields directly works too, since the transcript holds it by reference.

Source code in src/interlens/conversation.py
def edit(self, ref: MessageRef, content: str | None = None, *, author: str | None = None, **metadata) -> Message:
	"""Edit a committed past turn in place and return it (see ``Transcript.edit``): ``ref`` is a ``MessageRef``
	(int index, negatives allowed, or a ``Message`` object matched by identity), and ``content`` / ``author`` /
	``**metadata`` are the fields to change. Editing the returned ``Message``'s fields directly works too, since
	the transcript holds it by reference."""
	return self.transcript.edit(ref, content, author=author, **metadata)

from_models classmethod

from_models(
    models: tuple[ModelLike, ...],
    names: tuple[str, ...] = ("a", "b"),
    device: str | device = "cuda",
    dtype: dtype = torch.bfloat16,
    shared_context: str | None = None,
    shared_system_prompt: str | None = None,
    prompt: PromptLike = None,
    **gen_kwargs
) -> "Conversation"

Scaffold a conversation directly from a tuple of models — each an HF id or an already-loaded PreTrainedModel (see ModelLike). Convenience wrapper around factories.conversation_from_models; each model becomes a family-correct participant and names gives them identities. The order of models / names is the speaking order — the first speaks first unless you pass first= to run. **gen_kwargs are forwarded to every participant.

Scenario framing is available here too: shared_system_prompt (instructions, system role) and shared_context (a neutral moderator-voiced opening seen by everyone). prompt is a separate convenience for a participant-voiced opener (a str is attributed to the last participant). See that function for the details.

Source code in src/interlens/conversation.py
@classmethod
def from_models(cls, models: tuple[ModelLike, ...], names: tuple[str, ...] = ("a", "b"),
                device: str | torch.device = "cuda", dtype: torch.dtype = torch.bfloat16,
                shared_context: str | None = None, shared_system_prompt: str | None = None,
                prompt: PromptLike = None, **gen_kwargs) -> "Conversation":
	"""Scaffold a conversation directly from a tuple of ``models`` — each an HF id or an already-loaded
	``PreTrainedModel`` (see ``ModelLike``). Convenience wrapper around
	``factories.conversation_from_models``; each model becomes a family-correct participant and ``names`` gives
	them identities. **The order of ``models`` / ``names`` is the speaking order** — the first speaks first
	unless you pass ``first=`` to ``run``. ``**gen_kwargs`` are forwarded to every participant.

	Scenario framing is available here too: ``shared_system_prompt`` (instructions, system role) and
	``shared_context`` (a neutral ``moderator``-voiced opening seen by everyone). ``prompt`` is a separate
	convenience for a *participant*-voiced opener (a ``str`` is attributed to the last participant). See that
	function for the details."""
	from .factories import conversation_from_models  # lazy: factories imports this module

	return conversation_from_models(models, names=names, device=device, dtype=dtype,
	                                shared_context=shared_context, shared_system_prompt=shared_system_prompt,
	                                prompt=prompt, **gen_kwargs)

load classmethod

load(directory, devices='cuda') -> 'Conversation'

Rebuild a saved conversation: reconstruct (lazy) participants from the recipe and attach the saved transcript, resuming from that state (does NOT regenerate messages). Raises on an unsupported schema.

Source code in src/interlens/conversation.py
@classmethod
def load(cls, directory, devices="cuda") -> "Conversation":
	"""Rebuild a saved conversation: reconstruct (lazy) participants from the recipe and attach the saved
	transcript, resuming from that state (does NOT regenerate messages). Raises on an unsupported schema."""
	import json
	from pathlib import Path
	from .participant.serialize import participant_from_dict
	from .transcript import Transcript, SCHEMA_VERSION

	directory = Path(directory)
	recipe = json.loads((directory / "conversation.json").read_text())
	version = recipe.get("schema_version")
	if version != SCHEMA_VERSION:
		raise ValueError(f"checkpoint schema {version!r} is no longer supported (current {SCHEMA_VERSION}); "
		                 f"re-generate the run with the current interlens.")
	device = devices[0] if isinstance(devices, (list, tuple)) else devices
	participants = tuple(participant_from_dict(p, device=device) for p in recipe["participants"])
	return cls(
		participants=participants,
		transcript=Transcript.load(directory / "transcript.json"),
		shared_context=recipe.get("shared_context"),
		shared_system_prompt=recipe.get("shared_system_prompt"),
		moderator_name=recipe.get("moderator_name", "moderator"),
		context_limit=recipe.get("context_limit"),
		reasoning_visibility=ReasoningVisibility(recipe.get("reasoning_visibility", "strip")),
		execution_mode=ExecutionMode(recipe.get("execution_mode", "throughput")),
	)

participant

participant(which: ParticipantLike) -> Participant

Return one of this conversation's participants, resolved from a Participant object, its name (str), or its index (int). Raises (via _resolve_participant) if it isn't in this conversation.

Source code in src/interlens/conversation.py
def participant(self, which: ParticipantLike) -> Participant:
	"""Return one of this conversation's participants, resolved from a ``Participant`` object, its ``name``
	(str), or its index (int). Raises (via ``_resolve_participant``) if it isn't in this conversation."""
	return self.participants[self._resolve_participant(which)]

render_templated

render_templated(
    pov: ParticipantLike,
    *,
    extra=(),
    add_generation_prompt: bool = False,
    tokenize: bool = False
)

The full view run through pov's tokenizer chat template — the exact prompt its model sees, special/control tokens and all (a str, or token ids with tokenize=True). This is the truthful counterpart to transcript.render_templated, which templates the transcript turns ONLY (no system / private framing, no context-fit). Requires a local ModelParticipant (needs a tokenizer).

Source code in src/interlens/conversation.py
def render_templated(self, pov: ParticipantLike, *, extra=(), add_generation_prompt: bool = False,
                     tokenize: bool = False):
	"""The full ``view`` run through ``pov``'s tokenizer chat template — the **exact prompt its model sees**,
	special/control tokens and all (a str, or token ids with ``tokenize=True``). This is the truthful
	counterpart to ``transcript.render_templated``, which templates the transcript turns ONLY (no system /
	private framing, no context-fit). Requires a local ``ModelParticipant`` (needs a tokenizer)."""
	participant = self.participant(pov)
	tokenizer = getattr(participant, "tokenizer", None)
	if tokenizer is None:
		raise TypeError(f"{type(participant).__name__} {participant.name!r} has no tokenizer; "
		                f"render_templated needs a local ModelParticipant")
	return tokenizer.apply_chat_template(self.view(pov, extra=extra), tokenize=tokenize,
	                                     add_generation_prompt=add_generation_prompt)

reset

reset() -> 'Conversation'

Return the conversation to its fresh, pre-run state: empty the transcript, then re-seed the shared_context moderator turn (exactly as __post_init__ does on a fresh conversation). Use this rather than transcript.clear() when you want to rerun the same scenario — the raw transcript.clear() drops the shared_context framing too, whereas reset restores it.

Source code in src/interlens/conversation.py
def reset(self) -> "Conversation":
	"""Return the conversation to its fresh, pre-``run`` state: empty the transcript, then **re-seed** the
	``shared_context`` moderator turn (exactly as ``__post_init__`` does on a fresh conversation). Use this
	rather than ``transcript.clear()`` when you want to rerun the same scenario — the raw ``transcript.clear()``
	drops the ``shared_context`` framing too, whereas ``reset`` restores it."""
	self.transcript.clear()
	if self.shared_context is not None:
		self.transcript.append(self.moderator_name, self.shared_context)
	return self

rewind

rewind(*, to: MessageRef) -> 'Conversation'

Rewind in place so the turn to becomes the new last turn, dropping everything after it (returns self for chaining). to is a MessageRef (int index, negatives allowed, or a Message object). Mutates this conversation — use branch_from instead to keep the original. See Transcript.rewind.

Source code in src/interlens/conversation.py
def rewind(self, *, to: MessageRef) -> "Conversation":
	"""Rewind in place so the turn ``to`` becomes the new last turn, dropping everything after it (returns
	``self`` for chaining). ``to`` is a ``MessageRef`` (int index, negatives allowed, or a ``Message`` object).
	Mutates this conversation — use ``branch_from`` instead to keep the original. See ``Transcript.rewind``."""
	self.transcript.rewind(to=to)
	return self

rollout

rollout(
    n: int | None = None,
    *,
    devices=None,
    out_dir=None,
    resume: bool = False,
    batched: bool = True,
    max_batch_size: int | None = None,
    seed: int | None = None
) -> "RunReport"

Run many conversations from this one recipe and return a RunReport — the benchmark/scale entry point.

Rollout does NOT mutate this conversation. It makes an independent copy-on-write clone per job and runs those; self stays the unrun recipe. Finished conversations live in report.results[job_id].conversation (and their transcripts / analyses on the same RunResult) — that is where to sample() or inspect afterwards, NOT on the original.

Two modes: with data() set, expands to ONE conversation per row (job ids row_00000…), resolving each templated field (dataset_field) against the row; otherwise expands to n seeded copies of one scenario (job ids rollout_0000…). n defaults to len(data). Sample i gets seed + i (seed defaults to the conversation's seed field). Any ambient with StopCondition(...) blocks and the run_until field are captured now and attached to every job (so they survive the spawn boundary).

Parallel by default on two axes (see :func:interlens.run): one worker process per device, and batched co-stepping within each device. batched=False gives the token-identical DETERMINISTIC path.

Source code in src/interlens/conversation.py
def rollout(self, n: int | None = None, *, devices=None, out_dir=None, resume: bool = False,
            batched: bool = True, max_batch_size: int | None = None, seed: int | None = None) -> "RunReport":
	"""Run many conversations from this one recipe and return a ``RunReport`` — the benchmark/scale entry point.

	**Rollout does NOT mutate this conversation.** It makes an independent copy-on-write clone per job and runs
	*those*; ``self`` stays the unrun recipe. Finished conversations live in ``report.results[job_id].conversation``
	(and their transcripts / analyses on the same ``RunResult``) — that is where to ``sample()`` or inspect
	afterwards, NOT on the original.

	Two modes: with ``data()`` set, expands to ONE conversation per row (job ids ``row_00000``…), resolving each
	templated field (``dataset_field``) against the row; otherwise expands to ``n`` seeded copies of one scenario
	(job ids ``rollout_0000``…). ``n`` defaults to ``len(data)``. Sample *i* gets ``seed + i`` (``seed`` defaults
	to the conversation's ``seed`` field). Any ambient ``with StopCondition(...)`` blocks and the ``run_until``
	field are captured now and attached to every job (so they survive the spawn boundary).

	Parallel by default on two axes (see :func:`interlens.run`): one worker process per device, and batched
	co-stepping within each device. ``batched=False`` gives the token-identical DETERMINISTIC path.
	"""
	from .runner.pool import run_jobs
	seed = self._seed if seed is None else seed
	stop = self._combined_stop_for_jobs()
	jobs = self._expand(n, seed, stop)
	return run_jobs(jobs, devices=devices, out_dir=out_dir, resume=resume, batched=batched,
	                max_batch_size=max_batch_size)

run

run(
    turns: int | None = None,
    until: StopCondition | list | None = None,
    first: ParticipantLike | None = None,
    prompt: PromptLike = None,
) -> Transcript

Alternate speakers until turns elapse and/or a StopCondition fires (whichever comes first).

Speakers are taken in participants order (that tuple's order IS the turn order); first sets who starts (default: participants[0]) and the rest follow round-robin. first may be a Participant, its name (str), or its index (int) — resolved via _resolve_participant (raises if it isn't in this conversation).

Stop conditions are combined from three sources (any of which stops the run): until (this call), the conversation's own run_until field, and any ambient with StopCondition(...): blocks. A condition may also cap each turn's tokens (turn_cap) — e.g. a TokenBudget shrinks the final turn to land on budget. Conditions are reset() at the start so a reused instance doesn't leak state. turns defaults to the conversation's turns field; at least one of turns / a stop condition must be present.

prompt (optional) is appended to the transcript before the run: a str is attributed to the LAST participant (so first replies to it), a Message is appended as-is, None appends nothing. It appends to the current end and never resets/reseeds.

Source code in src/interlens/conversation.py
def run(self, turns: int | None = None, until: StopCondition | list | None = None,
        first: ParticipantLike | None = None, prompt: PromptLike = None) -> Transcript:
	"""Alternate speakers until ``turns`` elapse and/or a ``StopCondition`` fires (whichever comes first).

	Speakers are taken in ``participants`` order (that tuple's order IS the turn order); ``first`` sets who
	starts (default: ``participants[0]``) and the rest follow round-robin. ``first`` may be a ``Participant``,
	its ``name`` (str), or its index (int) — resolved via ``_resolve_participant`` (raises if it isn't in this
	conversation).

	Stop conditions are combined from three sources (any of which stops the run): ``until`` (this call), the
	conversation's own ``run_until`` field, and any ambient ``with StopCondition(...):`` blocks. A condition may
	also cap each turn's tokens (``turn_cap``) — e.g. a ``TokenBudget`` shrinks the final turn to land on budget.
	Conditions are ``reset()`` at the start so a reused instance doesn't leak state. ``turns`` defaults to the
	conversation's ``turns`` field; at least one of turns / a stop condition must be present.

	``prompt`` (optional) is appended to the transcript **before** the run: a ``str`` is attributed to the LAST
	participant (so ``first`` replies to it), a ``Message`` is appended as-is, ``None`` appends nothing. It
	appends to the current end and never resets/reseeds.
	"""
	self._check_resolved("run")
	if turns is None:
		turns = self._turns
	stop = self._resolve_stop(until)
	if turns is None and stop is None:
		raise ValueError("run requires at least one of `turns`, a `run_until`/`until` stop condition, or an "
		                 "ambient `with StopCondition(...)` block")
	self._append_prompt(prompt)
	if stop is not None:
		stop.reset()

	start = self._resolve_participant(first) if first is not None else 0
	n = len(self.participants)
	i = 0
	while turns is None or i < turns:
		cap = stop.turn_cap(self) if stop is not None else None
		message = self.step(self.participants[(start + i) % n], max_new_tokens=self._turn_budget(start, i, cap))
		i += 1
		# A hook may have denied the turn (message is None); only a committed message is checked for stopping.
		if message is not None and stop is not None and stop.should_stop(self, message):
			break
	return self.transcript

sample

sample(
    speaker: ParticipantLike,
    message: str | None = None,
    *,
    as_author: str | None = None,
    steering=None,
    capture=None,
    patch=None,
    return_logprobs: bool = False,
    max_new_tokens: int | None = None
) -> Message
sample(
    speaker: "list[ParticipantLike] | tuple[ParticipantLike, ...]",
    message: str | None = None,
    *,
    as_author: str | None = None,
    steering=None,
    capture=None,
    patch=None,
    return_logprobs: bool = False,
    max_new_tokens: int | None = None
) -> "dict[str, Message]"
sample(
    speaker,
    message=None,
    *,
    as_author=None,
    steering=None,
    capture=None,
    patch=None,
    return_logprobs: bool = False,
    max_new_tokens: int | None = None
)

Ephemerally sample a response without mutating the transcript — a pure read of current state, safe to call repeatedly / in a loop. speaker is a ParticipantLike (name / index / Participant); pass a list or tuple of them to sample each and get back {name: Message} instead of a single Message (see also sample_all). message is an optional temporary incoming turn; it defaults to being attributed to the moderator (a neutral, external voice — so "What do you think of Bob?" reads as an interviewer asking, NOT as Bob speaking). Pass as_author="bob" to make it read as that participant's turn instead (e.g. "what would you say if Bob had just said X?"). The same interp options as step are honored on each ephemeral generation.

Source code in src/interlens/conversation.py
def sample(self, speaker, message=None, *, as_author=None, steering=None, capture=None, patch=None,
           return_logprobs: bool = False, max_new_tokens: int | None = None):
	"""Ephemerally sample a response **without mutating the transcript** — a pure read of current state, safe to
	call repeatedly / in a loop. ``speaker`` is a ``ParticipantLike`` (name / index / ``Participant``); pass a
	**list or tuple** of them to sample each and get back ``{name: Message}`` instead of a single ``Message``
	(see also ``sample_all``). ``message`` is an optional temporary incoming turn; it defaults to being
	attributed to the **moderator** (a neutral, external voice — so "What do you think of Bob?" reads as an
	interviewer asking, NOT as Bob speaking). Pass ``as_author="bob"`` to make it read as that participant's
	turn instead (e.g. "what would you say if Bob had just said X?"). The same interp options as ``step`` are
	honored on each ephemeral generation."""
	self._check_resolved("sample")
	if isinstance(speaker, (list, tuple)):
		return {self.participant(s).name: self.sample(s, message, as_author=as_author, steering=steering,
		                                              capture=capture, patch=patch, return_logprobs=return_logprobs,
		                                              max_new_tokens=max_new_tokens) for s in speaker}
	speaker = self.participant(speaker)
	extra = []
	if message is not None:
		extra = [Message(author=as_author or self.moderator_name, content=message)]
	return speaker.generate(self._view(speaker, extra=extra), steering=steering, capture=capture,
	                        patch=patch, return_logprobs=return_logprobs, turn=len(self.transcript),
	                        max_new_tokens=max_new_tokens)

sample_all

sample_all(
    message: str | None = None,
    *,
    as_author: str | None = None,
    steering=None,
    capture=None,
    patch=None,
    return_logprobs: bool = False,
    max_new_tokens: int | None = None
) -> "dict[str, Message]"

Ephemerally sample every participant's response — a convenience for sample(list(self.participants), ...). Returns {name: Message}; the transcript is untouched. Handy for "what does each model say to this right now?".

Source code in src/interlens/conversation.py
def sample_all(self, message: str | None = None, *, as_author: str | None = None, steering=None,
               capture=None, patch=None, return_logprobs: bool = False,
               max_new_tokens: int | None = None) -> "dict[str, Message]":
	"""Ephemerally sample **every** participant's response — a convenience for
	``sample(list(self.participants), ...)``. Returns ``{name: Message}``; the transcript is untouched. Handy
	for "what does each model say to this right now?"."""
	return self.sample(list(self.participants), message, as_author=as_author, steering=steering,
	                   capture=capture, patch=patch, return_logprobs=return_logprobs, max_new_tokens=max_new_tokens)

save

save(directory) -> None

Persist the conversation: the transcript (transcript.json) + the recipe (conversation.json — the participants' own constructor kwargs + scenario framing/policies, no weights).

Source code in src/interlens/conversation.py
def save(self, directory) -> None:
	"""Persist the conversation: the transcript (``transcript.json``) + the recipe (``conversation.json`` — the
	participants' own constructor kwargs + scenario framing/policies, no weights)."""
	import json
	from pathlib import Path
	from .participant.serialize import participant_to_dict
	from .transcript import SCHEMA_VERSION

	directory = Path(directory)
	directory.mkdir(parents=True, exist_ok=True)
	recipe = {
		"schema_version": SCHEMA_VERSION,
		"participants": [participant_to_dict(p) for p in self.participants],
		"shared_context": self.shared_context if not has_fields(self.shared_context) else None,
		"shared_system_prompt": self.shared_system_prompt if not has_fields(self.shared_system_prompt) else None,
		"moderator_name": self.moderator_name,
		"context_limit": self.context_limit,
		"reasoning_visibility": self.reasoning_visibility.value,
		"execution_mode": self.execution_mode.value,
	}
	(directory / "conversation.json").write_text(json.dumps(recipe, indent=2))
	self.transcript.save(directory / "transcript.json")

step

step(
    speaker: Participant,
    *,
    steering=None,
    capture=None,
    patch=None,
    return_logprobs: bool = False,
    max_new_tokens: int | None = None
) -> Message

Have speaker produce and commit one turn. Interp options flow to generate; if no capture is passed but a conv.capture(...) block is active, its pending request is used (auto-tagged by turn).

Source code in src/interlens/conversation.py
def step(self, speaker: Participant, *, steering=None, capture=None, patch=None,
         return_logprobs: bool = False, max_new_tokens: int | None = None) -> Message:
	"""Have ``speaker`` produce and commit one turn. Interp options flow to ``generate``; if no ``capture`` is
	passed but a ``conv.capture(...)`` block is active, its pending request is used (auto-tagged by turn)."""
	if capture is None:
		capture = self._pending_capture
	turn = len(self.transcript)
	message = speaker.generate(self._view(speaker), steering=steering, capture=capture,
	                           patch=patch, return_logprobs=return_logprobs, turn=turn,
	                           max_new_tokens=max_new_tokens)
	message = self._apply_hooks(message)
	if message is None:
		return None  # a hook denied this turn; nothing is committed
	self.transcript.messages.append(message)
	return message

view

view(pov: ParticipantLike, extra=()) -> list[dict]

The full [{role, content}] view pov's model is conditioned on — the real generation input: system block (its private system_prompt + shared_system_prompt) → private_context → the transcript role-swapped to pov, then context-fit and family-flattened. This is exactly what step / sample feed to generate. Unlike transcript.render_roles (transcript turns only, no framing or fitting), it reflects everything the model actually sees. pov is a ParticipantLike (name / index / participant); extra renders temporary, uncommitted messages (as sample does).

Source code in src/interlens/conversation.py
def view(self, pov: ParticipantLike, extra=()) -> list[dict]:
	"""The full ``[{role, content}]`` view ``pov``'s model is conditioned on — the **real generation input**:
	system block (its private ``system_prompt`` + ``shared_system_prompt``) → ``private_context`` → the
	transcript role-swapped to ``pov``, then **context-fit** and **family-flattened**. This is exactly what
	``step`` / ``sample`` feed to ``generate``. Unlike ``transcript.render_roles`` (transcript turns only, no
	framing or fitting), it reflects everything the model actually sees. ``pov`` is a ``ParticipantLike`` (name
	/ index / participant); ``extra`` renders temporary, uncommitted messages (as ``sample`` does)."""
	return self._view(self.participant(pov), extra=extra)

DatasetField dataclass

DatasetField(name: str)

A placeholder for row[name] in a templated field, created by :func:dataset_field. Frozen + tiny so it serializes and pickles trivially across a spawn boundary.

DropOldestPolicy

Bases: ContextPolicy

Drop the oldest turn segments (preserving system/moderator/private_context) until the view fits.

ElapsedTimeStopCondition

ElapsedTimeStopCondition(seconds: float)

Bases: StopCondition

Stop once seconds of wall-clock have elapsed since the run started (monotonic clock).

Source code in src/interlens/stop/conditions.py
def __init__(self, seconds: float):
	self.seconds = seconds
	self.start = None

ErrorPolicy

Bases: ContextPolicy

The safe default: raise if the view exceeds the context window rather than silently dropping content.

Silent truncation reads as "everything fit" when it didn't, which is exactly the kind of quiet data loss this harness avoids. Callers who want trimming opt into DropOldestPolicy/SlidingWindowPolicy explicitly.

ExecutionMode

Bases: str, Enum

Names the deterministic-vs-throughput tension explicitly rather than pretending it doesn't exist.

Token-identical replay, performant defaults (flash-attn + KV reuse + batched generation), and interp measurement fidelity are NOT jointly satisfiable on CUDA, so callers pick a mode:

  • DETERMINISTIC: batching off, KV reuse off, sdpa/eager attention, deterministic algorithms. Slower, but token-identical on the same hardware and safe for capture/steering/probes. Backs the identical-replay guarantee.
  • THROUGHPUT (default for large rollouts): flash-attn + batched generation + KV reuse. Guarantees only distributional reproducibility.

P1 defines the switch and threads it through; the optimizations it gates (batching, KV reuse) land in later phases and consult this flag.

Functional

Mixin giving a dataclass copy-on-write updates via set(**changes).

set shallow-copies the instance (heavy refs like a loaded model are shared, not reloaded), applies the changes, then calls _after_set to reset volatile per-instance state on the copy. Sugared field names (declared via sugar_fields) are accepted under their public spelling and routed to their _-prefixed storage. Unknown fields raise TypeError. The original is never mutated.

set

set(**changes)

Return a modified shallow copy: the same object with changes applied and volatile state reset. The receiver is untouched (copy-on-write); heavy state (a loaded model, an API client) is shared by reference. Unknown field names raise TypeError.

The copy is a raw instance-dict clone (NOT copy.copy), so it deliberately bypasses __getstate__ — which exists to DROP the loaded model/client on pickle (spawn boundary). A .set() clone, by contrast, is in-process and must KEEP those shared references (that is the whole point of copy-on-write here).

Source code in src/interlens/functional.py
def set(self, **changes):
	"""Return a modified shallow copy: the same object with ``changes`` applied and volatile state reset. The
	receiver is untouched (copy-on-write); heavy state (a loaded model, an API client) is shared by reference.
	Unknown field names raise ``TypeError``.

	The copy is a raw instance-dict clone (NOT ``copy.copy``), so it deliberately bypasses ``__getstate__`` —
	which exists to DROP the loaded model/client on *pickle* (spawn boundary). A ``.set()`` clone, by contrast,
	is in-process and must KEEP those shared references (that is the whole point of copy-on-write here)."""
	allowed = self._settable_names()
	new = self.__class__.__new__(self.__class__)
	new.__dict__.update(self.__dict__)
	for key, value in changes.items():
		if key not in allowed:
			raise TypeError(f"{type(self).__name__}.set() got an unexpected field {key!r}; "
			                f"settable fields are {sorted(allowed)}")
		setattr(new, f"_{key}" if key in self._SUGARED else key, value)
	new._after_set(self)
	return new

GemmaModelParticipant dataclass

GemmaModelParticipant(
    name: str = "",
    hf_id: str | None = None,
    weights_path: str | None = None,
    dtype: str = "bfloat16",
    attn: str = "flash_attention_2",
    quant: str | None = None,
    revision: str | None = None,
    device: str | device | None = None,
    max_new_tokens: int = 512,
    temperature: float = 0.8,
    top_p: float = 0.95,
    seed: int | None = None,
    thinking: bool | str = "auto",
    system_prompt: str | None = None,
    private_context: tuple = (),
    tools: tuple = (),
    max_tool_iters: int = 4,
    kv_reuse: bool | str = "auto",
    steering: object = None,
    _model: "PreTrainedModel | None" = None,
    _tokenizer: "PreTrainedTokenizerBase | None" = None,
)

Bases: ModelParticipant

A Gemma-family participant. The only thing that differs from base is the tool-call format (```tool_code); the chat-template flags (Gemma 2 rejects a standalone system role and requires strict user/model alternation, Gemma 3 accepts a system role) are auto-derived from the tokenizer's own template, so both generations use this one class.

parse_tool_calls

parse_tool_calls(text: str) -> list

Parse Gemma's ```tool_code function-call blocks (best-effort).

            Gemma emits calls like ```` ```tool_code

name(arg="x", n=2) ` ```` rather than Hermes JSON. We extract the block, then the function name and simple keyword arguments. Falls back to[]`` (treat as final message) on anything it can't parse, matching the base contract of never misfiring silently.

Source code in src/interlens/participant/participants/gemma.py
def parse_tool_calls(self, text: str) -> list:
	"""Parse Gemma's ```` ```tool_code ```` function-call blocks (best-effort).

	Gemma emits calls like ```` ```tool_code\nname(arg="x", n=2)\n``` ```` rather than Hermes JSON. We extract
	the block, then the function name and simple keyword arguments. Falls back to ``[]`` (treat as final
	message) on anything it can't parse, matching the base contract of never misfiring silently.
	"""
	import ast
	import re
	from ...tools.tool_call import ToolCall

	calls = []
	for block in re.findall(r"```tool_code\s*(.*?)```", text, re.DOTALL):
		call_match = re.match(r"\s*([A-Za-z_][\w.]*)\s*\((.*)\)\s*$", block.strip(), re.DOTALL)
		if not call_match:
			continue
		name, arg_src = call_match.group(1), call_match.group(2)
		arguments = {}
		try:
			# Parse ``k=v`` pairs via a synthetic call node, so values are real Python literals.
			parsed = ast.parse(f"f({arg_src})", mode="eval")
			for kw in parsed.body.keywords:
				arguments[kw.arg] = ast.literal_eval(kw.value)
		except (SyntaxError, ValueError) as exc:
			logger.debug("dropping unparseable tool_code block %r: %s", block, exc)
			continue
		calls.append(ToolCall(name=name.split(".")[-1], arguments=arguments, raw=block))
	return calls

GradCaptureSpec dataclass

GradCaptureSpec(
    sites: tuple[Site, ...] = ("residual",),
    layers: tuple[int, ...] | None = None,
)

What grad-connected activations to pull from forward_with_grad (mirrors CaptureSpec minus offload).

sites is any of "residual"/"attn"/"mlp"; layers selects decoder-layer indices (None = all). Unlike CaptureSpec there is no offload — the whole point is to keep tensors on-device and in the autograd graph, so an intermediate-layer objective (e.g. project onto a concept direction) can be backpropagated.

GradForwardOutput

Bases: NamedTuple

Result of forward_with_grad: grad-connected logits ([batch, seq, vocab]) and, if a GradCaptureSpec was passed, hidden mapping (site, layer) -> tensor[batch, seq, d_model] (also grad-connected). hidden is empty when no capture was requested.

LinearBridge

LinearBridge(d_a: int, d_b: int, bias: bool = False)

Bases: Module

Learned linear map from model A's hidden width d_a to model B's embedding width d_b.

For heterogeneous (different-tokenizer) pairs where a vocab mixture is undefined: take A's last-layer hidden states (via forward_with_grad(..., capture=GradCaptureSpec(sites=('residual',), layers=(last,)))), map them into B's embedding space, and pass the result as inputs_embeds to B. Trained jointly with A against B's loss. bias=False by default to match the (linear, origin-preserving) cross-model maps used in the Procrustes work.

Source code in src/interlens/interp/bridge.py
def __init__(self, d_a: int, d_b: int, bias: bool = False):
	super().__init__()
	self.proj = nn.Linear(d_a, d_b, bias=bias)

forward

forward(hidden: Tensor) -> torch.Tensor

[batch, seq, d_a] -> [batch, seq, d_b] soft embeddings in B's input space.

Source code in src/interlens/interp/bridge.py
def forward(self, hidden: torch.Tensor) -> torch.Tensor:
	"""``[batch, seq, d_a] -> [batch, seq, d_b]`` soft embeddings in B's input space."""
	w = self.proj.weight
	return self.proj(hidden.to(dtype=w.dtype, device=w.device))

LlamaModelParticipant dataclass

LlamaModelParticipant(
    name: str = "",
    hf_id: str | None = None,
    weights_path: str | None = None,
    dtype: str = "bfloat16",
    attn: str = "flash_attention_2",
    quant: str | None = None,
    revision: str | None = None,
    device: str | device | None = None,
    max_new_tokens: int = 512,
    temperature: float = 0.8,
    top_p: float = 0.95,
    seed: int | None = None,
    thinking: bool | str = "auto",
    system_prompt: str | None = None,
    private_context: tuple = (),
    tools: tuple = (),
    max_tool_iters: int = 4,
    kv_reuse: bool | str = "auto",
    steering: object = None,
    _model: "PreTrainedModel | None" = None,
    _tokenizer: "PreTrainedTokenizerBase | None" = None,
)

Bases: ModelParticipant

A Llama-family participant. Chat-template flags are auto-derived from the tokenizer; only the tool-call format differs from base: Llama 3 emits calls as <|python_tag|>{json} rather than Hermes/Qwen <tool_call> blocks.

parse_tool_calls

parse_tool_calls(text: str) -> list

Parse Llama-3's <|python_tag|>{json} function-call format (best-effort).

Everything after <|python_tag|> is one or more JSON objects (separated by newlines or semicolons), each {"name": ..., "arguments"/"parameters": {...}}. Anything unparseable is skipped, so a malformed call yields [] (treated as a final message), matching the base contract of never misfiring.

Source code in src/interlens/participant/participants/llama.py
def parse_tool_calls(self, text: str) -> list:
	"""Parse Llama-3's ``<|python_tag|>{json}`` function-call format (best-effort).

	Everything after ``<|python_tag|>`` is one or more JSON objects (separated by newlines or semicolons),
	each ``{"name": ..., "arguments"/"parameters": {...}}``. Anything unparseable is skipped, so a malformed
	call yields ``[]`` (treated as a final message), matching the base contract of never misfiring."""
	import json
	from ...tools.tool_call import ToolCall

	tag = "<|python_tag|>"
	idx = text.find(tag)
	if idx < 0:
		return []
	body = text[idx + len(tag):]
	calls = []
	for chunk in body.replace(";", "\n").splitlines():
		chunk = chunk.strip()
		if not chunk:
			continue
		try:
			data = json.loads(chunk)
			name = data["name"]
			arguments = data.get("arguments") or data.get("parameters") or {}
		except (json.JSONDecodeError, KeyError) as exc:
			logger.debug("dropping unparseable python_tag call %r: %s", chunk, exc)
			continue
		calls.append(ToolCall(name=name, arguments=arguments, raw=chunk))
	return calls

Message dataclass

Message(author: str, content: str, metadata: dict = dict())

A single committed turn in a conversation.

The transcript is stored canonically and author-centric: a message records who said what, and is deliberately agnostic to the assistant/user role distinction. That role mapping is a per-participant view concern (see Transcript.render_roles), not a property of the message itself — so one transcript can be rendered from every participant's perspective without duplicating state.

author is the participant's name (a string), never the Participant object. This keeps transcripts trivially JSON-serializable and valid even when no models are loaded (e.g. when scoring saved transcripts).

content is the committed, visible text — the only field that is authoritative for rendering history back into a model. Anything else a generation produced (parsed <think> reasoning, the raw completion, tool call/result trails, per-token logprobs) lives in metadata under neutral keys, so hidden generated text is never silently promoted into what other participants see.

MessageHook

Bases: ABC

Middleware that inspects each freshly generated message before it is committed to the transcript, and may approve / deny / edit it.

This is the seam for a future LLM-judge that vets or rewrites turns (e.g. safety filtering, format enforcement). Hooks live on the live Conversation (conversation.message_hooks) and are NOT serialized in the template — they're a runtime policy, not part of the scenario recipe. The default is an empty hook list, i.e. today's pass-through behavior.

ModelParticipant dataclass

ModelParticipant(
    name: str = "",
    hf_id: str | None = None,
    weights_path: str | None = None,
    dtype: str = "bfloat16",
    attn: str = "flash_attention_2",
    quant: str | None = None,
    revision: str | None = None,
    device: str | device | None = None,
    max_new_tokens: int = 512,
    temperature: float = 0.8,
    top_p: float = 0.95,
    seed: int | None = None,
    thinking: bool | str = "auto",
    system_prompt: str | None = None,
    private_context: tuple = (),
    tools: tuple = (),
    max_tool_iters: int = 4,
    kv_reuse: bool | str = "auto",
    steering: object = None,
    _model: "PreTrainedModel | None" = None,
    _tokenizer: "PreTrainedTokenizerBase | None" = None,
)

Bases: Functional, Participant

A conversation participant backed by a local HuggingFace causal LM.

Generation flow: the Conversation hands us a view — the transcript rendered from our perspective, already context-fitted and flattened by finalize_view into [{role, content}]. We apply this model's own chat template to it, generate, decode only the newly produced tokens, and split any <think> reasoning out of the visible content. Only the visible answer becomes Message.content; the parsed reasoning and raw completion live in metadata under neutral keys, so hidden generated text is never fed back into other participants' views (it's stripped from history automatically because render_roles uses content).

The participant is its own recipe. It stores what to load (hf_id + dtype/attn/quant/revision) and loads the weights lazily on first use through the process-wide model cache — so an unrun participant is cheap (KBs), pickles across a spawn boundary without shipping weights, and .set(...) copies share one loaded model object by reference (the co-stepping batch win). model/tokenizer are properties that trigger the load; the raw model is exposed deliberately (interp experiments register forward hooks on it). Weight loads are keyed on (hf_id, device, dtype, attn, quant, revision) and cached, so all same-recipe participants on one device share ONE object. Build via from_pretrained (lazy) or from_model (eager, from an already-loaded model); .set() (from Functional) makes copy-on-write clones with fresh KV state.

model property

model: 'PreTrainedModel'

The loaded HF model, loading it on first access (cached process-wide). The raw model is exposed so interp experiments can register forward hooks directly.

tokenizer property

tokenizer: 'PreTrainedTokenizerBase'

The loaded tokenizer, loading the model+tokenizer on first access (cached process-wide).

batch_signature

batch_signature() -> tuple

Key identifying which model this participant would batch AS (used by the co-stepper). When loaded, the cached model object's identity is authoritative; when not yet loaded, the load recipe is (the cache guarantees same recipe on one device -> same object), so participants can be grouped WITHOUT forcing a load.

Source code in src/interlens/participant/participants/model_participant.py
def batch_signature(self) -> tuple:
	"""Key identifying which model this participant would batch AS (used by the co-stepper). When loaded, the
	cached model object's identity is authoritative; when not yet loaded, the load recipe is (the cache
	guarantees same recipe on one device -> same object), so participants can be grouped WITHOUT forcing a
	load."""
	if self._model is not None:
		return ("model", id(self._model))
	return ("model", self.weights_path or self.hf_id, str(self.device), self.dtype, self.attn, self.quant,
	        self.revision)

for_model_type classmethod

for_model_type(
    model_type: str | None,
) -> type["ModelParticipant"]

The registered participant class for a transformers config.model_type (e.g. qwen2, gemma2), falling back to base ModelParticipant for any family that declares no specialized subclass.

Source code in src/interlens/participant/participants/model_participant.py
@classmethod
def for_model_type(cls, model_type: str | None) -> type["ModelParticipant"]:
	"""The registered participant class for a transformers ``config.model_type`` (e.g. ``qwen2``, ``gemma2``),
	falling back to base ``ModelParticipant`` for any family that declares no specialized subclass."""
	return cls._REGISTRY.get(model_type or "", ModelParticipant)

from_model classmethod

from_model(
    model: PreTrainedModel,
    tokenizer: PreTrainedTokenizerBase | None = None,
    *,
    name: str,
    device: str | device | None = None,
    **participant_kwargs
) -> Self

Build a participant of THIS class from an already-loaded model (the eager path — e.g. to wrap a model you already hold, or share weights between speakers). tokenizer is optional — when omitted it is inferred from model.config._name_or_path. The chat-template flags (supports_system_role / requires_alternating_roles) are derived from the tokenizer's own template. hf_id/dtype are read back from the model so the participant can still be pickled + re-loaded on a spawn worker. Because it constructs cls, calling it on a subclass returns that subclass — use AutoModelParticipant.from_model to have the family resolved from config.model_type.

Source code in src/interlens/participant/participants/model_participant.py
@classmethod
def from_model(cls, model: PreTrainedModel, tokenizer: PreTrainedTokenizerBase | None = None, *, name: str,
               device: str | torch.device | None = None, **participant_kwargs) -> Self:
	"""Build a participant of THIS class from an already-loaded ``model`` (the eager path — e.g. to wrap a model
	you already hold, or share weights between speakers). ``tokenizer`` is optional — when omitted it is inferred
	from ``model.config._name_or_path``. The chat-template flags (``supports_system_role`` /
	``requires_alternating_roles``) are derived from the tokenizer's own template. ``hf_id``/``dtype`` are read
	back from the model so the participant can still be pickled + re-loaded on a spawn worker. Because it
	constructs ``cls``, calling it on a subclass returns that subclass — use ``AutoModelParticipant.from_model``
	to have the family resolved from ``config.model_type``."""
	from ...loading import load_tokenizer, derive_chat_flags  # lazy: loading imports this module
	hf_id = getattr(model.config, "_name_or_path", None) or None
	if tokenizer is None:
		if not hf_id:
			raise ValueError("cannot infer a tokenizer: model.config._name_or_path is empty; pass tokenizer=")
		tokenizer = load_tokenizer(hf_id)
	supports_system, requires_alt = derive_chat_flags(tokenizer)
	p = cls(name=name, device=device, hf_id=hf_id, dtype=dtype_to_str(model.dtype),
	        attn=getattr(model, "_resolved_attn", "flash_attention_2"), **participant_kwargs)
	p._model = model
	p._tokenizer = tokenizer
	p.supports_system_role = supports_system
	p.requires_alternating_roles = requires_alt
	if p.device is None:
		p.device = model.device
	return p

from_pretrained classmethod

from_pretrained(
    id_or_path: str | Path,
    *,
    name: str,
    device: str | device = "cuda",
    load_kwargs: dict | None = None,
    **participant_kwargs
) -> Self

Build a participant of THIS class that will load id_or_path (an HF id or local path) lazily on first use — no weights are touched here. load_kwargs (dtype / attn / quant / revision / weights_path) are recorded for that deferred load; participant_kwargs (temperature, max_new_tokens, system_prompt, tools, kv_reuse, …) go to the participant. When the load fires it is process-cached, so all same-(id, device, dtype, …) participants share ONE model object. As with from_model, the class is cls — use AutoModelParticipant.from_pretrained to resolve the family from config.model_type (it reads only the config, still no weights).

Source code in src/interlens/participant/participants/model_participant.py
@classmethod
def from_pretrained(cls, id_or_path: str | Path, *, name: str, device: str | torch.device = "cuda",
                    load_kwargs: dict | None = None, **participant_kwargs) -> Self:
	"""Build a participant of THIS class that will load ``id_or_path`` (an HF id or local path) **lazily on first
	use** — no weights are touched here. ``load_kwargs`` (``dtype`` / ``attn`` / ``quant`` / ``revision`` /
	``weights_path``) are recorded for that deferred load; ``participant_kwargs`` (``temperature``,
	``max_new_tokens``, ``system_prompt``, ``tools``, ``kv_reuse``, …) go to the participant. When the load fires
	it is process-cached, so all same-(id, device, dtype, …) participants share ONE model object. As with
	``from_model``, the class is ``cls`` — use ``AutoModelParticipant.from_pretrained`` to resolve the family
	from ``config.model_type`` (it reads only the config, still no weights)."""
	lk = dict(load_kwargs or {})
	return cls(name=name, device=device, hf_id=str(id_or_path),
	           weights_path=lk.get("weights_path"), dtype=dtype_to_str(lk.get("dtype", torch.bfloat16)),
	           attn=lk.get("attn", "flash_attention_2"), quant=lk.get("quant"), revision=lk.get("revision"),
	           **participant_kwargs)

generate_batch

generate_batch(
    views: list[list[dict]],
    *,
    turn: int | None = None,
    group_seed: int | None = None,
    max_new_tokens: int | None = None
) -> list[Message]

Batched generation for many independent conversations that share THIS model (throughput mode).

Renders each view with this model's chat template and runs one model.generate over the left-padded batch, returning one Message per view — the co-stepping throughput win (5-20x on a rollout). max_new_tokens overrides this participant's per-turn cap for the batch (the co-stepper passes a turn_cap here so a TokenBudget can shrink the final round). No tools/steering/capture/logprobs here: callers needing those fall back to the per-conversation generate. Tokens are not guaranteed identical to unbatched — batch composition and the single global RNG perturb rows (see PLAN §Execution modes); only distributional reproducibility holds. metadata['batched'] marks these turns; metadata['shared_prefill'] marks the fast path.

Source code in src/interlens/participant/participants/model_participant.py
def generate_batch(self, views: list[list[dict]], *, turn: int | None = None,
                   group_seed: int | None = None, max_new_tokens: int | None = None) -> list[Message]:
	"""Batched generation for many independent conversations that share THIS model (``throughput`` mode).

	Renders each ``view`` with this model's chat template and runs **one** ``model.generate`` over the
	left-padded batch, returning one ``Message`` per view — the co-stepping throughput win (5-20x on a
	rollout). ``max_new_tokens`` overrides this participant's per-turn cap for the batch (the co-stepper passes a
	``turn_cap`` here so a ``TokenBudget`` can shrink the final round). **No tools/steering/capture/logprobs**
	here: callers needing those fall back to the per-conversation ``generate``. Tokens are **not** guaranteed
	identical to unbatched — batch composition and the single global RNG perturb rows (see PLAN §Execution
	modes); only distributional reproducibility holds. ``metadata['batched']`` marks these turns;
	``metadata['shared_prefill']`` marks the fast path.
	"""
	if not views:
		return []
	seed = group_seed if group_seed is not None else self.seed
	if seed is not None:
		torch.manual_seed(seed)

	template_kwargs = {}
	if isinstance(self.thinking, bool):
		template_kwargs["enable_thinking"] = self.thinking
	prompts = [self.tokenizer.apply_chat_template(v, tokenize=False, add_generation_prompt=True,
	                                              **template_kwargs) for v in views]

	do_sample = bool(self.temperature and self.temperature > 0)
	gen = dict(max_new_tokens=max_new_tokens if max_new_tokens is not None else self.max_new_tokens,
	           do_sample=do_sample, pad_token_id=self.tokenizer.pad_token_id)
	if do_sample:
		gen.update(temperature=self.temperature, top_p=self.top_p)

	# Shared-scenario fast path: when every rendered prompt is token-identical (turn 1 of a rollout off one
	# scenario), prefill the shared prefix ONCE and fork to N samples via ``num_return_sequences`` instead of
	# N redundant prefills — the "prefill the shared prefix once per participant" win (PLAN §Shared-scenario
	# reuse). Only valid when sampling (identical greedy rows would be identical, so batching is pointless).
	shared_prefill = do_sample and len(views) > 1 and len(set(prompts)) == 1
	if shared_prefill:
		enc = self.tokenizer(prompts[0], return_tensors="pt", add_special_tokens=False).to(self.device)
		prompt_len = enc["input_ids"].shape[1]
		with torch.inference_mode():
			out = self.model.generate(**enc, num_return_sequences=len(views), **gen)
		new = out[:, prompt_len:]
	else:
		prev_side = self.tokenizer.padding_side
		self.tokenizer.padding_side = "left"
		try:
			enc = self.tokenizer(prompts, return_tensors="pt", padding=True,
			                     add_special_tokens=False).to(self.device)
		finally:
			self.tokenizer.padding_side = prev_side
		prompt_len = enc["input_ids"].shape[1]
		with torch.inference_mode():
			out = self.model.generate(**enc, **gen)
		new = out[:, prompt_len:]

	pad_id = self.tokenizer.pad_token_id
	messages = []
	for row in new:
		# Trailing pad ids appear once a row hits EOS before its peers; strip them for an honest token count.
		keep = row[row != pad_id] if pad_id is not None else row
		raw = self.tokenizer.decode(keep, skip_special_tokens=True)
		content, parsed_think = self.split_reasoning(raw)
		messages.append(Message(author=self.name, content=content, metadata={
			"raw_completion": raw, "parsed_think": parsed_think, "n_tokens": int(keep.shape[0]),
			"batched": True, "shared_prefill": shared_prefill,
		}))
	return messages

parse_tool_calls

parse_tool_calls(text: str) -> list

Parse tool calls out of a generation. Base handles the common Hermes/Qwen <tool_call>{json}</tool_call> format; families with other formats (Gemma's ```tool_code, Llama's <|python_tag|>) override this. An unrecognized/absent call yields [] so the loop treats the output as a final message.

Source code in src/interlens/participant/participants/model_participant.py
def parse_tool_calls(self, text: str) -> list:
	"""Parse tool calls out of a generation. Base handles the common Hermes/Qwen ``<tool_call>{json}</tool_call>``
	format; families with other formats (Gemma's ```` ```tool_code ````, Llama's ``<|python_tag|>``) override
	this. An unrecognized/absent call yields ``[]`` so the loop treats the output as a final message."""
	from ...tools.tool_call import ToolCall

	calls = []
	for match in _TOOL_CALL_RE.finditer(text):
		try:
			data = json.loads(match.group(1))
			calls.append(ToolCall(name=data["name"], arguments=data.get("arguments", {}), raw=match.group(0)))
		except (json.JSONDecodeError, KeyError) as exc:
			logger.debug("dropping malformed tool call %r: %s", match.group(0), exc)
			continue
	return calls

render_tool_result

render_tool_result(call, result) -> dict

Render a tool result as the standard structured tool message; the tokenizer's own template turns it into the family-native format.

Source code in src/interlens/participant/participants/model_participant.py
def render_tool_result(self, call, result) -> dict:
	"""Render a tool result as the standard structured ``tool`` message; the tokenizer's own template turns it
	into the family-native format."""
	return {"role": "tool", "name": result.name, "content": result.output}

split_reasoning

split_reasoning(text: str) -> tuple[str, str | None]

Split a raw completion into (visible_content, parsed_think). Base handles the <think>...</think> convention; families with other delimiters override this.

Source code in src/interlens/participant/participants/model_participant.py
def split_reasoning(self, text: str) -> tuple[str, str | None]:
	"""Split a raw completion into ``(visible_content, parsed_think)``. Base handles the ``<think>...</think>``
	convention; families with other delimiters override this."""
	match = _THINK_RE.match(text)
	if not match:
		return text.strip(), None
	return text[match.end():].strip(), match.group(1).strip()

Participant

Bases: ABC

A participant in a conversation, either a model or a person.

A participant owns three things: an identity within the conversation (name + self_role/ others_role), its private framing (system_prompt + private_context — instructions/knowledge only it sees), and the ability to turn a rendered view into its next message (generate). The Conversation assembles the structured view from the shared transcript; the participant flattens that view to what its chat template expects via finalize_view and generates.

name instance-attribute

name: str

A name or identifier to uniquely identify this participant within a conversation.

finalize_view

finalize_view(segments: list[ViewSegment]) -> list[dict]

Flatten the structured, context-fitted view into the [{role, content}] list the chat template consumes. Applies family-specific repairs driven by the capability flags:

  • supports_system_role=False → fold the leading system content into the first user turn (Gemma's template errors on a standalone system role).
  • requires_alternating_roles=True → merge consecutive same-role segments (Gemma requires strict user/model alternation; the moderator seed + another speaker + private context can otherwise produce consecutive user turns that the template rejects). Merged turns keep author labels so speaker identity isn't lost in the concatenation.
Source code in src/interlens/participant/participant.py
def finalize_view(self, segments: list[ViewSegment]) -> list[dict]:
	"""Flatten the structured, context-fitted view into the ``[{role, content}]`` list the chat template
	consumes. Applies family-specific repairs driven by the capability flags:

	- ``supports_system_role=False`` → fold the leading system content into the first user turn (Gemma's
	  template errors on a standalone ``system`` role).
	- ``requires_alternating_roles=True`` → merge consecutive same-role segments (Gemma requires strict
	  user/model alternation; the moderator seed + another speaker + private context can otherwise produce
	  consecutive ``user`` turns that the template rejects). Merged turns keep author labels so speaker
	  identity isn't lost in the concatenation.
	"""
	segments = list(segments)
	if not self.supports_system_role:
		segments = self._fold_system_into_first_user(segments)
	if self.requires_alternating_roles:
		return self._merge_consecutive_same_role(segments)
	return [s.as_message() for s in segments]

generate abstractmethod

generate(
    view: list[dict],
    *,
    steering=None,
    capture=None,
    patch=None,
    return_logprobs: bool = False,
    turn: int | None = None,
    max_new_tokens: int | None = None
) -> Message

Produce this participant's next message given view — the conversation flattened to [{"role", "content"}] from this participant's perspective. Returns a Message it authored.

Interp options apply to local-model participants: steering (a SteeringSpec), capture (a CaptureRequest), patch (a Patch), and return_logprobs; turn is the message index used to tag captured activations. Participants that can't honor an interp request (e.g. API-backed) must raise rather than silently ignore it — a failed capture/steer must fail loudly.

Source code in src/interlens/participant/participant.py
@abstractmethod
def generate(self, view: list[dict], *, steering=None, capture=None, patch=None,
             return_logprobs: bool = False, turn: int | None = None,
             max_new_tokens: int | None = None) -> Message:
	"""Produce this participant's next message given ``view`` — the conversation flattened to
	``[{"role", "content"}]`` from this participant's perspective. Returns a ``Message`` it authored.

	Interp options apply to local-model participants: ``steering`` (a ``SteeringSpec``), ``capture`` (a
	``CaptureRequest``), ``patch`` (a ``Patch``), and ``return_logprobs``; ``turn`` is the message index used
	to tag captured activations. Participants that can't honor an interp request (e.g. API-backed) must raise
	rather than silently ignore it — a failed capture/steer must fail loudly."""
	...

Patch dataclass

Patch(
    activations: Tensor,
    layer: int,
    positions: tuple[int, ...],
)

Activation patching: overwrite a decoder layer's residual at specific token positions with saved activations (captured from another run/branch).

This is the cross-branch causal-tracing primitive: capture activations at turn N in one branch (via ActivationCache), then inject them at the aligned positions of another branch's forward. The harness owns this because only it knows the turn/position correspondence between branches.

P2 applies the patch on the (single) prompt forward — positions index into the prompt sequence. Aligning positions across branches is the caller's responsibility; Patch just performs the overwrite.

QwenModelParticipant dataclass

QwenModelParticipant(
    name: str = "",
    hf_id: str | None = None,
    weights_path: str | None = None,
    dtype: str = "bfloat16",
    attn: str = "flash_attention_2",
    quant: str | None = None,
    revision: str | None = None,
    device: str | device | None = None,
    max_new_tokens: int = 512,
    temperature: float = 0.8,
    top_p: float = 0.95,
    seed: int | None = None,
    thinking: bool | str = "auto",
    system_prompt: str | None = None,
    private_context: tuple = (),
    tools: tuple = (),
    max_tool_iters: int = 4,
    kv_reuse: bool | str = "auto",
    steering: object = None,
    _model: "PreTrainedModel | None" = None,
    _tokenizer: "PreTrainedTokenizerBase | None" = None,
)

Bases: ModelParticipant

A participant in a conversation that is a Qwen language model. Its tool-call format is the Hermes/base <tool_call> JSON already handled by ModelParticipant, so this class adds no behavior — it exists so Qwen models resolve to a distinct, statically-typed participant class.

ReasoningVisibility

Bases: str, Enum

Controls whether a participant's prior <think> reasoning is re-injected into views on later turns.

Reasoning is always parsed out of Message.content into metadata['parsed_think'] (so it is available for interpretability regardless of this setting). This enum only governs history re-injection:

  • STRIP (default): prior reasoning is dropped from all views — matches R1/Qwen3 chat templates, keeps reasoning a genuinely private scratchpad the other participant never sees, and is always template-safe.
  • SELF_RETAIN: a participant sees its own prior reasoning re-injected into its view (visible-scratchpad self-continuation experiments).
  • SHARED: all participants see all prior reasoning (shared-CoT collaboration/debate experiments).

Non-STRIP modes re-inject reasoning as tagged text at render time so templates that reject a native prior <think> turn don't error.

RunReport dataclass

RunReport(results: dict = dict(), skipped: list = list())

Aggregate outcome of a run. Failures are isolated, not fatal: results holds every attempted job, failed lists the ones that errored, skipped lists resume-skipped (already-checkpointed) ids.

RunResult dataclass

RunResult(
    job_id: str,
    conversation: object = None,
    transcript: object = None,
    analysis: object = None,
    error: str | None = None,
    device: str | None = None,
)

Outcome of one job: the finished conversation (weightless participants + completed transcript), its transcript and (serializable) analysis, or an error string if it failed. conversation is the object to sample()/inspect after a rollout (the source recipe is never mutated).

tokens_generated property

tokens_generated: int

Total generated tokens in this conversation (summed from each turn's metadata['n_tokens']) — the realized compute, for verifying matched-compute comparisons. 0 if the job failed.

SlidingWindowPolicy

SlidingWindowPolicy(
    keep_last: int, keep_system: bool = True
)

Bases: ContextPolicy

Keep the preserved framing plus the most recent keep_last turns; drop older turns.

keep_system=True (the default) preserves the system/moderator/private_context framing regardless of the window; set it False to also let framing fall outside the window (rarely wanted). Unlike DropOldestPolicy this is a fixed-size window rather than a fit-to-budget trim, so it's predictable turn-to-turn.

Source code in src/interlens/context/sliding_window_policy.py
def __init__(self, keep_last: int, keep_system: bool = True):
	self.keep_last = keep_last
	self.keep_system = keep_system

SteeringSpec dataclass

SteeringSpec(
    direction: Tensor,
    layers: tuple[int, ...],
    coef: float = 1.0,
    mode: Mode = "add",
)

A residual-stream intervention applied during generation via forward hooks on decoder layers.

mode='add' adds coef * direction to the residual at layers; mode='ablate' projects the direction component out of the residual (directional ablation). The same mechanism covers both because ablation is just the projection-removal variant of an additive hook.

A summary (mode, layers, coef, direction norm) is recorded into Message.metadata['steering'] by the participant so a steered/ablated turn is reproducible.

difference_of_means classmethod

difference_of_means(
    pos_acts,
    neg_acts,
    layers,
    coef: float = 1.0,
    mode: Mode = "add",
) -> "SteeringSpec"

Build a spec whose direction is the unit difference-of-means of two activation populations, normalize(mean(pos) − mean(neg)) — the classic contrastive/concept-direction recipe, pointing TOWARD the positive class. pos_acts/neg_acts are [n, d_model] (a stack of per-example residuals) or a pre-pooled [d_model] vector; anything torch.as_tensor accepts works. Steer toward the positive concept with coef>0 and away from it (suppress) with coef<0. layers is an int or an iterable of decoder-layer indices. This lives here (not re-hand-rolled per experiment) per the contribution convention.

Source code in src/interlens/interp/steering.py
@classmethod
def difference_of_means(cls, pos_acts, neg_acts, layers, coef: float = 1.0, mode: Mode = "add") -> "SteeringSpec":
	"""Build a spec whose ``direction`` is the **unit difference-of-means** of two activation populations,
	``normalize(mean(pos) − mean(neg))`` — the classic contrastive/concept-direction recipe, pointing TOWARD
	the positive class. ``pos_acts``/``neg_acts`` are ``[n, d_model]`` (a stack of per-example residuals) or a
	pre-pooled ``[d_model]`` vector; anything ``torch.as_tensor`` accepts works. Steer toward the positive
	concept with ``coef>0`` and **away** from it (suppress) with ``coef<0``. ``layers`` is an int or an
	iterable of decoder-layer indices. This lives here (not re-hand-rolled per experiment) per the
	contribution convention."""
	pos = torch.as_tensor(pos_acts, dtype=torch.float32)
	neg = torch.as_tensor(neg_acts, dtype=torch.float32)
	pm = pos if pos.ndim == 1 else pos.mean(0)
	nm = neg if neg.ndim == 1 else neg.mean(0)
	d = pm - nm
	d = d / (d.norm() + 1e-8)
	layers = (layers,) if isinstance(layers, int) else tuple(layers)
	return cls(direction=d, layers=layers, coef=coef, mode=mode)

register

register(model: 'PreTrainedModel') -> list

Register the steering hooks on model and return the handles (caller removes them after generate).

Source code in src/interlens/interp/steering.py
def register(self, model: "PreTrainedModel") -> list:
	"""Register the steering hooks on ``model`` and return the handles (caller removes them after generate)."""
	layers = decoder_layers(model)
	handles = []
	for li in self.layers:
		handles.append(layers[li].register_forward_hook(self._hook()))
	return handles

StopCondition

Bases: ABC

A stateful predicate that ends a Conversation.run early.

Each condition is stateful (tracks its own counters) and is checked after every committed turn via should_stop(conversation, last_message). reset() clears state and is called at the start of each run (and branches get fresh copies), so one instance can be reused across runs without leaking state.

A condition may also cap the next generation via turn_cap (e.g. a token budget shrinks the last turn so it lands exactly on budget) and may be installed ambiently as a context manager (with TokenBudget(...): conv.rollout(...) applies it to every conversation in the block). New conditions subclass this without any change to run.

reset

reset() -> None

Clear any accumulated state. Default no-op for stateless conditions.

Source code in src/interlens/stop/stop_condition.py
def reset(self) -> None:
	"""Clear any accumulated state. Default no-op for stateless conditions."""

turn_cap

turn_cap(conversation: 'Conversation') -> int | None

An upper bound on the NEXT turn's generated tokens, or None for no cap. The run loop passes it as max_new_tokens (bounded by the speaker's own cap), so a budget condition can prevent both overshoot and a single turn from consuming the whole allowance. Default: no cap.

Source code in src/interlens/stop/stop_condition.py
def turn_cap(self, conversation: "Conversation") -> int | None:
	"""An upper bound on the NEXT turn's generated tokens, or ``None`` for no cap. The run loop passes it as
	``max_new_tokens`` (bounded by the speaker's own cap), so a budget condition can prevent both overshoot and a
	single turn from consuming the whole allowance. Default: no cap."""
	return None

StopStringCondition

StopStringCondition(strings)

Bases: StopCondition

Stop when a committed message's visible content contains any of the given strings (a done-signal).

Source code in src/interlens/stop/conditions.py
def __init__(self, strings):
	self.strings = [strings] if isinstance(strings, str) else list(strings)

SummarizePolicy

SummarizePolicy(keep_last: int = 4, summarizer=None)

Bases: ContextPolicy

Compress older turns into a single summary segment instead of dropping them outright (the heaviest policy).

Keeps the preserved framing (system / moderator / private_context) and the most recent keep_last turns verbatim, and replaces the older middle turns with one summary segment produced by summarizer — a callable list[str] -> str over the dropped turns' contents. With no summarizer it inserts a neutral placeholder, so it degrades to a labelled drop rather than silently losing content.

summarizer is a live callable and so is not serialized; a loaded template gets summarizer=None and the caller re-injects one if needed.

Source code in src/interlens/context/summarize_policy.py
def __init__(self, keep_last: int = 4, summarizer=None):
	self.keep_last = keep_last
	self.summarizer = summarizer

TokenBudget

TokenBudget(
    per_conversation: int | None = None,
    per_turn: int | None = None,
)

Bases: StopCondition

A per-conversation compute budget — the matched-compute primitive for fair solo-vs-pair comparisons.

per_conversation stops a conversation once ITS OWN cumulative generated tokens reach the budget, and per_turn caps each individual turn so the allowance is spread across real conversation turns rather than consumed by one monologue. Both are enforced via turn_cap too: the run loop shrinks the next generation to min(speaker cap, per_turn, per_conversation - spent), so the budget is respected without overshoot.

The budget is per-conversation, not a shared pool. Spend is read from the conversation's own transcript (metadata['n_tokens']), so the condition is stateless — in a rollout of N copies, each copy independently gets the full budget. Cheap to count (never re-tokenizes) and trivially picklable, so it works installed directly (run_until=TokenBudget(...)) or ambiently (with TokenBudget(per_conversation=200): conv.rollout()).

Source code in src/interlens/stop/conditions.py
def __init__(self, per_conversation: int | None = None, per_turn: int | None = None):
	if per_conversation is None and per_turn is None:
		raise ValueError("TokenBudget needs at least one of per_conversation= or per_turn=")
	self.per_conversation = per_conversation
	self.per_turn = per_turn

TokenStopCondition

TokenStopCondition(max_tokens: int)

Bases: StopCondition

Stop once the total generated tokens across turns reach max_tokens.

The per-turn count comes from Message.metadata['n_tokens'], which ModelParticipant.generate records — so the source of truth is defined, not guessed. Turns without a count (e.g. seeded messages) contribute 0.

Source code in src/interlens/stop/conditions.py
def __init__(self, max_tokens: int):
	self.max_tokens = max_tokens
	self.total = 0

Tool

Bases: ABC

A capability a participant can invoke during its turn.

A tool has a name, a JSON schema (in the function-calling format chat templates render via their tools= argument), and is callable with keyword arguments to produce a string result. Tools contain live callables and so are NOT serializable — templates store tool names and resolve them against a ToolRegistry at build time, mirroring how models are resolved from ids.

schema abstractmethod property

schema: dict

The tool's JSON function schema, e.g. {"type": "function", "function": {"name", "description", "parameters": {...}}}, passed straight to apply_chat_template(tools=...) so each family renders it in its native format.

ToolCall dataclass

ToolCall(
    name: str, arguments: dict = dict(), raw: str = ""
)

A parsed request from the model to invoke a tool: the tool name and its arguments.

raw keeps the exact text the model emitted (useful for debugging a family parser). Tool calls are parsed out of the generation by a per-family parse_tool_calls — the format differs across model families, so the parsed structure is uniform even though the surface syntax isn't.

ToolRegistry

ToolRegistry()

Resolves tool names (which serialize) to live Tool instances (which don't).

This is the tools analogue of the model registry: a template stores tool_names and, at build time on each worker, the registry turns them into callables. Because spawned worker processes inherit no parent state, tools must be registered at import time (or via a worker-init hook), not imperatively in the parent.

Source code in src/interlens/tools/registry.py
def __init__(self):
	self._tools: dict[str, Tool] = {}

ToolResult dataclass

ToolResult(name: str, output: str, error: bool = False)

The outcome of executing a ToolCall: the tool name and its string output (or error text).

Transcript dataclass

Transcript(messages: list[Message] = list())

The canonical, perspective-neutral record of a conversation, plus the logic to render it from a given participant's perspective.

Design: the transcript never commits to assistant/user roles. Each message just knows its author. When it's a participant's turn, render_roles maps that neutral record into the chat-template [{role, content}] shape from that participant's point of view — its own turns become self_role (normally assistant), everyone else's become others_role (normally user). This single trick is why two different models with different tokenizers can share one transcript: each gets the view its own template expects, and the stored record stays singular.

P0 scope: this holds only the message list + rendering + list-like ergonomics. Serialization, context policies, author-labelling for N-party, and reasoning re-injection are layered on in later phases.

append

append(author: str, content: str, **metadata) -> Message

Append a committed turn and return it. metadata captures anything non-authoritative (parsed reasoning, logprobs, tool trails) without polluting content.

Source code in src/interlens/transcript.py
def append(self, author: str, content: str, **metadata) -> Message:
	"""Append a committed turn and return it. ``metadata`` captures anything non-authoritative (parsed
	reasoning, logprobs, tool trails) without polluting ``content``."""
	message = Message(author=author, content=content, metadata=dict(metadata))
	self.messages.append(message)
	return message

clear

clear() -> 'Transcript'

Drop every turn, returning self.

⚠️ This wipes the whole record, including any leading seed turn — the shared_context scenario framing and any moderator/initial-instruction turns injected at construction. Nothing about the conversation's opening setup survives. If you want to clear the dialogue but keep that framing (e.g. to rerun the same scenario), call Conversation.reset instead, which empties and then re-seeds the shared_context turn.

Source code in src/interlens/transcript.py
def clear(self) -> "Transcript":
	"""Drop **every** turn, returning ``self``.

	⚠️ This wipes the whole record, including any leading seed turn — the ``shared_context`` scenario framing and
	any moderator/initial-instruction turns injected at construction. Nothing about the conversation's opening
	setup survives. If you want to clear the *dialogue* but keep that framing (e.g. to rerun the same scenario),
	call ``Conversation.reset`` instead, which empties and then re-seeds the ``shared_context`` turn."""
	self.messages.clear()
	return self

copy

copy() -> 'Transcript'

Deep-ish copy of the message list for branching. Messages are small string records, so this is cheap; the invariant that keeps it cheap is that heavy tensors never live in Message.metadata.

Source code in src/interlens/transcript.py
def copy(self) -> "Transcript":
	"""Deep-ish copy of the message list for branching. Messages are small string records, so this is cheap;
	the invariant that keeps it cheap is that heavy tensors never live in ``Message.metadata``."""
	return Transcript([Message(m.author, m.content, dict(m.metadata)) for m in self.messages])

edit

edit(
    ref: "MessageRef",
    content: str | None = None,
    *,
    author: str | None = None,
    **metadata
) -> Message

Edit a committed past turn in place and return it. ref is a MessageRef (int index, negatives allowed, or a Message object matched by identity). Only the arguments you pass are changed: content and/or author are replaced when given; metadata keys are merged over the existing metadata (pass an explicit key to overwrite it — untouched keys are left alone). Editing the Message object's fields directly does the same thing, since the transcript holds it by reference.

Source code in src/interlens/transcript.py
def edit(self, ref: "MessageRef", content: str | None = None, *, author: str | None = None, **metadata) -> Message:
	"""Edit a committed past turn in place and return it. ``ref`` is a ``MessageRef`` (int index, negatives
	allowed, or a ``Message`` object matched by identity). Only the arguments you pass are changed: ``content``
	and/or ``author`` are replaced when given; ``metadata`` keys are merged over the existing metadata (pass an
	explicit key to overwrite it — untouched keys are left alone). Editing the ``Message`` object's fields
	directly does the same thing, since the transcript holds it by reference."""
	message = self.messages[self.resolve_index(ref)]
	if content is not None:
		message.content = content
	if author is not None:
		message.author = author
	if metadata:
		message.metadata.update(metadata)
	return message

load classmethod

load(path: str | Path) -> 'Transcript'

Load a transcript from disk. Works with no models present (for scoring/analysis of saved runs).

Source code in src/interlens/transcript.py
@classmethod
def load(cls, path: str | Path) -> "Transcript":
	"""Load a transcript from disk. Works with no models present (for scoring/analysis of saved runs)."""
	return cls.from_dict(json.loads(Path(path).read_text()))

pretty

pretty(metadata: bool = False) -> str

A human-readable dump for debugging: one [i] author: content block per turn. With metadata=True, also list each turn's non-empty metadata (parsed reasoning, tool trail, token counts, …). Also what str(transcript) / print(transcript) uses (without metadata).

Source code in src/interlens/transcript.py
def pretty(self, metadata: bool = False) -> str:
	"""A human-readable dump for debugging: one ``[i] author: content`` block per turn. With ``metadata=True``,
	also list each turn's non-empty metadata (parsed reasoning, tool trail, token counts, …). Also what
	``str(transcript)`` / ``print(transcript)`` uses (without metadata)."""
	lines = []
	for i, m in enumerate(self.messages):
		lines.append(f"[{i}] {m.author}: {m.content}")
		if metadata and m.metadata:
			for k, v in m.metadata.items():
				lines.append(f"      · {k}: {v}")
	return "\n".join(lines)

render_roles

render_roles(*, pov: 'Participant') -> list[dict]

Render the transcript turns only as [{"role", "content"}] from pov's perspective (pov is keyword-only: render_roles(pov=alice)). Its own turns become self_role (normally assistant), everyone else's become others_role (normally user). For a "what if X had just been said" render, extend first with with_extra and render the result.

Limitation — this is NOT what the model actually sees. The Transcript is the perspective-neutral record; it knows nothing about a participant's system_prompt / private_context, the conversation's shared_system_prompt / shared_context framing, the context_policy fitting, or the family-specific flatten (Gemma system-folding etc.). Those are added by the Conversation pipeline. For the real generation input use Conversation.view(pov) (or Conversation.render_templated(pov) for the templated string); use this method for lower-level inspection of the record itself.

Source code in src/interlens/transcript.py
def render_roles(self, *, pov: "Participant") -> list[dict]:
	"""Render **the transcript turns only** as ``[{"role", "content"}]`` from ``pov``'s perspective (``pov`` is
	keyword-only: ``render_roles(pov=alice)``). Its own turns become ``self_role`` (normally ``assistant``),
	everyone else's become ``others_role`` (normally ``user``). For a "what if X had just been said" render,
	extend first with ``with_extra`` and render the result.

	**Limitation — this is NOT what the model actually sees.** The ``Transcript`` is the perspective-neutral
	record; it knows nothing about a participant's ``system_prompt`` / ``private_context``, the conversation's
	``shared_system_prompt`` / ``shared_context`` framing, the ``context_policy`` fitting, or the family-specific
	flatten (Gemma system-folding etc.). Those are added by the ``Conversation`` pipeline. For the **real**
	generation input use ``Conversation.view(pov)`` (or ``Conversation.render_templated(pov)`` for the templated
	string); use this method for lower-level inspection of the record itself."""
	rendered = []
	for message in self.messages:
		role = pov.self_role if message.author == pov.name else pov.others_role
		rendered.append({"role": role, "content": message.content})
	return rendered

render_templated

render_templated(
    *,
    pov: "ModelParticipant",
    add_generation_prompt: bool = False,
    tokenize: bool = False
)

Template the transcript turns only from pov's point of view — i.e. render_roles role-swapped, then run through pov's tokenizer chat template so the special / control tokens (<|im_start|>assistant etc.) are included. pov is keyword-only: call render_templated(pov=alice). Returns a str (tokenize=False, default) or token ids (tokenize=True); add_generation_prompt appends the assistant open tag.

Prefer Conversation.render_templated if you have access to a conversation: it adds the system prompt, private context, context-fitting, and family-specific flattening, printing the exact input models see.

Limitations

NOT the exact model input. This omits the system prompt, private context, context-fitting, and the family-specific flatten that a real turn also gets (see render_roles); it also renders the raw role-swapped turns as-is, which can even raise for families needing strict alternation / system-folding (e.g. Gemma) since the merge isn't applied. For the exact, family-correct prompt the model actually sees, use Conversation.render_templated(pov). This method is a lower-level inspection of the record. Requires a local model participant (with a tokenizer); an API participant has none and raises.

Source code in src/interlens/transcript.py
def render_templated(self, *, pov: "ModelParticipant",
                     add_generation_prompt: bool = False, tokenize: bool = False):
	"""
	Template **the transcript turns only** from ``pov``'s point of view — i.e. ``render_roles`` role-swapped,
	then run through ``pov``'s tokenizer chat template so the special / control tokens (``<|im_start|>assistant``
	etc.) are included. ``pov`` is keyword-only: call ``render_templated(pov=alice)``. Returns a str
	(``tokenize=False``, default) or token ids (``tokenize=True``); ``add_generation_prompt`` appends the
	assistant open tag.

	**Prefer ``Conversation.render_templated`` if you have access to a conversation**: it adds the system prompt, private context, context-fitting, and family-specific flattening, printing the exact input models see.

	### Limitations
	**NOT the exact model input.** This omits the system prompt, private context, context-fitting,
	and the family-specific flatten that a real turn also gets (see ``render_roles``); it also renders the raw
	role-swapped turns as-is, which can even *raise* for families needing strict alternation / system-folding
	(e.g. Gemma) since the merge isn't applied. For the **exact, family-correct prompt the model actually sees**,
	use ``Conversation.render_templated(pov)``. This method is a lower-level inspection of the record. Requires a
	local model participant (with a tokenizer); an API participant has none and raises."""
	tokenizer = getattr(pov, "tokenizer", None)
	if tokenizer is None:
		raise TypeError(f"{type(pov).__name__} {getattr(pov, 'name', '')!r} has no tokenizer; "
		                f"render_templated needs a local ModelParticipant. If you have a conversation, use `conversation.render_templated(pov='participant_name')` or `transcript.render_templated(pov=conversation.participant('participant_name'))` instead.")
	view = self.render_roles(pov=pov)
	return tokenizer.apply_chat_template(view, tokenize=tokenize, add_generation_prompt=add_generation_prompt)

resolve_index

resolve_index(ref: 'MessageRef') -> int

Normalize a message reference to a concrete [0, len) position. ref is either an int index (Python semantics — negatives count from the end, -1 = last turn) or a Message object, located by identity (is, not equality — two turns with the same text are still distinct). Raises IndexError for an out-of-range int and ValueError for a message that isn't in this transcript.

Source code in src/interlens/transcript.py
def resolve_index(self, ref: "MessageRef") -> int:
	"""Normalize a message reference to a concrete ``[0, len)`` position. ``ref`` is either an ``int`` index
	(Python semantics — negatives count from the end, ``-1`` = last turn) or a ``Message`` **object**, located
	by identity (``is``, not equality — two turns with the same text are still distinct). Raises ``IndexError``
	for an out-of-range int and ``ValueError`` for a message that isn't in this transcript."""
	if isinstance(ref, Message):
		for i, m in enumerate(self.messages):
			if m is ref:
				return i
		raise ValueError("message is not in this transcript")
	n = len(self.messages)
	i = ref + n if ref < 0 else ref
	if not 0 <= i < n:
		raise IndexError(f"message index {ref} out of range for transcript of length {n}")
	return i

rewind

rewind(*, to: 'MessageRef') -> 'Transcript'

Rewind so the turn referenced by to becomes the new last turn — i.e. drop everything after it, keeping to itself (mutates in place, returns self). to is a MessageRef: an int index (negatives count from the end — rewind(to=-2) takes back the single last turn) or a Message object matched by identity. To drop all turns use clear.

Source code in src/interlens/transcript.py
def rewind(self, *, to: "MessageRef") -> "Transcript":
	"""Rewind so the turn referenced by ``to`` becomes the new last turn — i.e. drop everything **after** it,
	keeping ``to`` itself (mutates in place, returns ``self``). ``to`` is a ``MessageRef``: an ``int`` index
	(negatives count from the end — ``rewind(to=-2)`` takes back the single last turn) or a ``Message`` object
	matched by identity. To drop all turns use ``clear``."""
	return self.truncate(self.resolve_index(to) + 1)

truncate

truncate(length: int) -> 'Transcript'

Keep only the first length turns, dropping everything after (mutates in place, returns self). length is a count, clamped to [0, len]: over-long values are a no-op and negatives count from the end (-1 keeps all but the last turn). To cut relative to a specific message, use resolve_index / rewind(to=…) instead.

Source code in src/interlens/transcript.py
def truncate(self, length: int) -> "Transcript":
	"""Keep only the first ``length`` turns, dropping everything after (mutates in place, returns ``self``).
	``length`` is a *count*, clamped to ``[0, len]``: over-long values are a no-op and negatives count from the
	end (``-1`` keeps all but the last turn). To cut relative to a specific message, use ``resolve_index`` /
	``rewind(to=…)`` instead."""
	if length < 0:
		length = max(0, len(self.messages) + length)
	del self.messages[length:]
	return self

with_extra

with_extra(*extra: Message) -> 'Transcript'

Return a lightweight, ephemeral read-only Transcript = this one's messages followed by extra, backed by a lazy concatenation (_ConcatMessages): the base message list is not copied — only extra is materialized, so this is O(len(extra)), not O(len(transcript)), and Message objects are shared by reference. The original is untouched.

This is the data-layer primitive behind ephemeral sampling ("what would a participant say if extra had just been said?"): build the extended transcript, render it, discard it — transcript.with_extra(Message("bob", "…")).render_roles(pov=alice). The result is read-only (rendering, len, indexing, iteration all work; appending raises — copy() first if you need a mutable one).

(A one-shot generator was rejected: Transcript needs len / indexing / repeated iteration, which a generator can't provide.)

Source code in src/interlens/transcript.py
def with_extra(self, *extra: Message) -> "Transcript":
	"""Return a lightweight, ephemeral **read-only** ``Transcript`` = this one's messages followed by ``extra``,
	backed by a lazy concatenation (``_ConcatMessages``): the base message list is **not copied** — only
	``extra`` is materialized, so this is O(len(extra)), not O(len(transcript)), and ``Message`` objects are
	shared by reference. The original is untouched.

	This is the data-layer primitive behind ephemeral sampling ("what would a participant say if ``extra`` had
	just been said?"): build the extended transcript, render it, discard it —
	``transcript.with_extra(Message("bob", "…")).render_roles(pov=alice)``. The result is read-only (rendering,
	``len``, indexing, iteration all work; appending raises — ``copy()`` first if you need a mutable one).

	(A one-shot generator was rejected: ``Transcript`` needs ``len`` / indexing / repeated iteration, which a
	generator can't provide.)"""
	return Transcript(_ConcatMessages(self.messages, extra))

TurnStopCondition

TurnStopCondition(max_turns: int)

Bases: StopCondition

Stop after max_turns committed turns.

Source code in src/interlens/stop/conditions.py
def __init__(self, max_turns: int):
	self.max_turns = max_turns
	self.count = 0

available_devices

available_devices() -> list[str]

List the devices to spread conversations across: every CUDA GPU, else a single mps/cpu fallback.

Multi-GPU parallelism lives across conversations (they're independent); within one conversation turns are sequential, so more GPUs never speed up a single conversation — only throughput over many.

Source code in src/interlens/runner/devices.py
def available_devices() -> list[str]:
	"""List the devices to spread conversations across: every CUDA GPU, else a single mps/cpu fallback.

	Multi-GPU parallelism lives *across* conversations (they're independent); within one conversation turns are
	sequential, so more GPUs never speed up a single conversation — only throughput over many."""
	if torch.cuda.is_available():
		return [f"cuda:{i}" for i in range(torch.cuda.device_count())]
	if torch.backends.mps.is_available():
		return ["mps"]
	return ["cpu"]

continuation_logprob

continuation_logprob(
    model: "PreTrainedModel",
    *,
    target_ids: Tensor,
    prefix_ids: Tensor | None = None,
    prefix_embeds: Tensor | None = None,
    attention_mask: Tensor | None = None,
    reduction: str = "mean",
    checkpoint: bool = False
) -> torch.Tensor

Differentiable teacher-forced logprob of target_ids continuing a prefix, under model.

The grad-enabled analog of train_to_steer/rl_elicit.py::cont_logprob (which is @torch.no_grad and returns a float). Supply the prefix as prefix_ids ([batch, P] long) OR prefix_embeds ([batch, P, d_model] float — e.g. a learnable soft prompt, or soft embeddings bridged from model A). target_ids is [batch, T] long (or [T] broadcast to batch 1). The prefix and target are embedded via model.get_input_embeddings() and run in one forward; the returned scalar (or per-example vector, see reduction) is the logprob the model assigns to the exact target tokens, still attached to the graph.

reduction: "mean" (mean over target tokens, mean over batch -> scalar; the reward used by rl_elicit), "sum" (sum over target tokens, mean over batch), or "none" ([batch, T] per-token logprobs). Use "mean" as a length-normalized elicitation reward, "sum" when total sequence likelihood matters, "none" for custom weighting. checkpoint is forwarded to save memory when backpropagating through a large frozen B.

Source code in src/interlens/interp/grad.py
def continuation_logprob(
	model: "PreTrainedModel",
	*,
	target_ids: torch.Tensor,
	prefix_ids: torch.Tensor | None = None,
	prefix_embeds: torch.Tensor | None = None,
	attention_mask: torch.Tensor | None = None,
	reduction: str = "mean",
	checkpoint: bool = False,
) -> torch.Tensor:
	"""Differentiable teacher-forced logprob of ``target_ids`` continuing a prefix, under ``model``.

	The grad-enabled analog of ``train_to_steer/rl_elicit.py::cont_logprob`` (which is ``@torch.no_grad`` and returns
	a float). Supply the prefix as ``prefix_ids`` (``[batch, P]`` long) OR ``prefix_embeds`` (``[batch, P, d_model]``
	float — e.g. a learnable soft prompt, or soft embeddings bridged from model A). ``target_ids`` is ``[batch, T]``
	long (or ``[T]`` broadcast to batch 1). The prefix and target are embedded via ``model.get_input_embeddings()``
	and run in one forward; the returned scalar (or per-example vector, see ``reduction``) is the logprob the model
	assigns to the exact target tokens, still attached to the graph.

	``reduction``: ``"mean"`` (mean over target tokens, mean over batch -> scalar; the reward used by rl_elicit),
	``"sum"`` (sum over target tokens, mean over batch), or ``"none"`` (``[batch, T]`` per-token logprobs). Use
	``"mean"`` as a length-normalized elicitation reward, ``"sum"`` when total sequence likelihood matters, ``"none"``
	for custom weighting. ``checkpoint`` is forwarded to save memory when backpropagating through a large frozen B.
	"""
	if (prefix_ids is None) == (prefix_embeds is None):
		raise ValueError("pass exactly one of prefix_ids or prefix_embeds")
	embed = model.get_input_embeddings()
	device = embed.weight.device
	if target_ids.dim() == 1:
		target_ids = target_ids.unsqueeze(0)
	target_ids = target_ids.to(device)

	if prefix_embeds is None:
		if prefix_ids.dim() == 1:
			prefix_ids = prefix_ids.unsqueeze(0)
		prefix_embeds = embed(prefix_ids.to(device))
	prefix_embeds = prefix_embeds.to(device)
	if prefix_embeds.shape[0] == 1 and target_ids.shape[0] > 1:
		prefix_embeds = prefix_embeds.expand(target_ids.shape[0], -1, -1)

	target_embeds = embed(target_ids)
	full = torch.cat([prefix_embeds, target_embeds], dim=1)
	fout = forward_with_grad(model, inputs_embeds=full, attention_mask=attention_mask, checkpoint=checkpoint)

	P = prefix_embeds.shape[1]
	T = target_ids.shape[1]
	# Logits at position i predict token i+1; the T target tokens sit at positions P..P+T-1, so the logits that
	# predict them are at positions P-1..P+T-2.
	pred_logits = fout.logits[:, P - 1: P + T - 1, :].float()
	logp = torch.log_softmax(pred_logits, dim=-1)
	tok_logp = logp.gather(2, target_ids.unsqueeze(2)).squeeze(2)  # [batch, T]

	if reduction == "none":
		return tok_logp
	if reduction == "sum":
		return tok_logp.sum(dim=1).mean()
	if reduction == "mean":
		return tok_logp.mean()
	raise ValueError(f"unknown reduction {reduction!r}")

conversation_from_ids

conversation_from_ids(
    ids: tuple[ModelLike, ...], **kwargs: Any
) -> Conversation

Deprecated thin alias for :func:conversation_from_models / :meth:Conversation.from_models, kept for back-compat. Prefer Conversation.from_models(models=..., ...).

Source code in src/interlens/factories.py
def conversation_from_ids(ids: tuple[ModelLike, ...], **kwargs) -> Conversation:
	"""Deprecated thin alias for :func:`conversation_from_models` / :meth:`Conversation.from_models`, kept for
	back-compat. Prefer ``Conversation.from_models(models=..., ...)``."""
	return conversation_from_models(ids, **kwargs)

conversation_from_models

conversation_from_models(
    models: tuple[ModelLike, ...],
    names: tuple[str, ...] = ("a", "b"),
    device: str | device = "cuda",
    dtype: dtype = torch.bfloat16,
    shared_context: str | None = None,
    shared_system_prompt: str | None = None,
    prompt: PromptLike = None,
    **gen_kwargs: Any
) -> Conversation

Scaffold a conversation from a tuple of models — each an HF id (str) or an already-loaded PreTrainedModel (see ModelLike). Each becomes a family-correct participant via AutoModelParticipant.from_; names gives them their identities. The order of models / names is the speaking order — the first speaks first unless you pass first= to run — and **gen_kwargs are forwarded to every participant.

This is the implementation behind :meth:Conversation.from_models (which just wraps it). If two ids resolve to the same HF model the weights are loaded once and shared (via load_model's process cache).

Two ways to seed the opening, without touching the transcript by hand:

  • shared_context — a neutral, moderator-voiced turn everyone sees (scenario/topic framing); pair with shared_system_prompt for system-role instructions. These are the principled framing knobs (serialized into a template).
  • prompt — a participant-voiced opener: a str is attributed to the LAST participant so the first speaker replies to it, a Message sets the author explicitly. Use this when the opener should read as something a speaker said rather than moderator framing.
Source code in src/interlens/factories.py
def conversation_from_models(
	models: tuple[ModelLike, ...],
	names: tuple[str, ...] = ("a", "b"),
	device: str | torch.device = "cuda",
	dtype: torch.dtype = torch.bfloat16,
	shared_context: str | None = None,
	shared_system_prompt: str | None = None,
	prompt: PromptLike = None,
	**gen_kwargs,
) -> Conversation:
	"""Scaffold a conversation from a tuple of ``models`` — each an HF id (str) or an already-loaded
	``PreTrainedModel`` (see ``ModelLike``). Each becomes a family-correct participant via
	``AutoModelParticipant.from_``; ``names`` gives them their identities. **The order of ``models`` / ``names``
	is the speaking order** — the first speaks first unless you pass ``first=`` to ``run`` — and ``**gen_kwargs``
	are forwarded to every participant.

	This is the implementation behind :meth:`Conversation.from_models` (which just wraps it). If two ids resolve
	to the same HF model the weights are loaded **once** and shared (via ``load_model``'s process cache).

	Two ways to seed the opening, without touching the transcript by hand:

	- ``shared_context`` — a neutral, ``moderator``-voiced turn everyone sees (scenario/topic framing); pair with
	  ``shared_system_prompt`` for system-role instructions. These are the principled framing knobs (serialized
	  into a template).
	- ``prompt`` — a *participant*-voiced opener: a ``str`` is attributed to the LAST participant so the first
	  speaker replies to it, a ``Message`` sets the author explicitly. Use this when the opener should read as
	  something a speaker said rather than moderator framing.
	"""
	participants = tuple(
		AutoModelParticipant.from_(m, name=n, device=device, load_kwargs={"dtype": dtype}, **gen_kwargs)
		for m, n in zip(models, names)
	)
	conv = Conversation(participants=participants, shared_context=shared_context,
	                    shared_system_prompt=shared_system_prompt)
	conv._append_prompt(prompt)
	return conv

dataset_field

dataset_field(name: str) -> DatasetField

Reference a dataset column in a templated field: shared_context=("Solve:\n\n", dataset_field("question")) resolves row["question"] for each row at rollout expansion. See the module docstring for all template forms.

Source code in src/interlens/templating.py
def dataset_field(name: str) -> DatasetField:
	"""Reference a dataset column in a templated field: ``shared_context=("Solve:\\n\\n", dataset_field("question"))``
	resolves ``row["question"]`` for each row at rollout expansion. See the module docstring for all template forms."""
	return DatasetField(name)

decoder_layers

decoder_layers(model: 'PreTrainedModel')

Return the list of transformer decoder layer modules, across common HF architectures.

Steering/patching hooks and per-layer capture all need the ordered layer stack. Qwen and Gemma both expose it at model.model.layers; a few fallbacks cover other families so the interp layer isn't Qwen/Gemma-only. Multimodal image-text-to-text wrappers (Qwen 3.5, Gemma 4, …) nest the text decoder one level deeper — its layers live at model.language_model.layers (or .model.language_model.model.layers) — so those paths are included too. PEFT/adapter wrappers (PeftModelForCausalLM) are unwrapped first so hooks land on the real decoder layers.

Source code in src/interlens/interp/layers.py
def decoder_layers(model: "PreTrainedModel"):
	"""Return the list of transformer decoder layer modules, across common HF architectures.

	Steering/patching hooks and per-layer capture all need the ordered layer stack. Qwen and Gemma both expose
	it at ``model.model.layers``; a few fallbacks cover other families so the interp layer isn't Qwen/Gemma-only.
	Multimodal image-text-to-text wrappers (Qwen 3.5, Gemma 4, …) nest the text decoder one level deeper — its
	layers live at ``model.language_model.layers`` (or ``.model.language_model.model.layers``) — so those paths are
	included too. PEFT/adapter wrappers (``PeftModelForCausalLM``) are unwrapped first so hooks land on the real
	decoder layers.
	"""
	if hasattr(model, "get_base_model"):  # peft wrapper -> underlying transformers model
		try:
			model = model.get_base_model()
		except Exception:
			pass
	for path in ("model.layers", "transformer.h", "gpt_neox.layers", "model.decoder.layers",
	             "model.language_model.layers", "language_model.layers", "model.language_model.model.layers"):
		obj = model
		try:
			for attr in path.split("."):
				obj = getattr(obj, attr)
			return obj
		except AttributeError:
			continue
	raise AttributeError(f"could not locate decoder layers on {type(model).__name__}")

forward_with_grad

forward_with_grad(
    model: "PreTrainedModel",
    *,
    input_ids: Tensor | None = None,
    inputs_embeds: Tensor | None = None,
    attention_mask: Tensor | None = None,
    capture: GradCaptureSpec | None = None,
    checkpoint: bool = False
) -> GradForwardOutput

One grad-connected forward pass over input_ids XOR inputs_embeds.

Exactly one of input_ids ([batch, seq] long) or inputs_embeds ([batch, seq, d_model] float, typically the output of bridge.soft_embed or a learnable soft prompt) must be given. attention_mask is optional ([batch, seq]). Returns logits and any requested hidden activations, all still attached to the graph so .backward() flows back to inputs_embeds / soft-prompt params / an upstream model A.

checkpoint=True enables HF gradient checkpointing for this call (use_reentrant=False) and restores the prior setting afterwards; it recomputes layer activations on the backward pass to trade compute for memory, and produces identical gradients. Residual sites come from output_hidden_states (grad-connected); attn/mlp sites come from non-detaching forward hooks on the submodules.

Source code in src/interlens/interp/grad.py
def forward_with_grad(
	model: "PreTrainedModel",
	*,
	input_ids: torch.Tensor | None = None,
	inputs_embeds: torch.Tensor | None = None,
	attention_mask: torch.Tensor | None = None,
	capture: GradCaptureSpec | None = None,
	checkpoint: bool = False,
) -> GradForwardOutput:
	"""One grad-connected forward pass over ``input_ids`` XOR ``inputs_embeds``.

	Exactly one of ``input_ids`` (``[batch, seq]`` long) or ``inputs_embeds`` (``[batch, seq, d_model]`` float,
	typically the output of ``bridge.soft_embed`` or a learnable soft prompt) must be given. ``attention_mask`` is
	optional (``[batch, seq]``). Returns ``logits`` and any requested hidden activations, all still attached to the
	graph so ``.backward()`` flows back to ``inputs_embeds`` / soft-prompt params / an upstream model A.

	``checkpoint=True`` enables HF gradient checkpointing for this call (``use_reentrant=False``) and restores the
	prior setting afterwards; it recomputes layer activations on the backward pass to trade compute for memory, and
	produces identical gradients. Residual sites come from ``output_hidden_states`` (grad-connected); ``attn``/``mlp``
	sites come from non-detaching forward hooks on the submodules.
	"""
	if (input_ids is None) == (inputs_embeds is None):
		raise ValueError("pass exactly one of input_ids or inputs_embeds")

	sites: set[Site] = set(capture.sites) if capture is not None else set()
	need_hidden = "residual" in sites
	need_hooks = bool(sites - {"residual"})  # attn/mlp need submodule hooks; residual comes from hidden_states

	handles = []
	hook_store: dict[tuple[Site, int], torch.Tensor] = {}

	def make_hook(layer_idx: int, site: Site):
		def hook(module, inputs, output):
			hs = output[0] if isinstance(output, tuple) else output
			hook_store[(site, layer_idx)] = hs  # keep grad — do NOT detach
		return hook

	if need_hooks:  # only touch the layer stack when a submodule hook is actually required
		layers = decoder_layers(model)
		hook_layers = capture.layers if capture.layers is not None else tuple(range(len(layers)))
		for li in hook_layers:
			if "attn" in sites:
				handles.append(layers[li].self_attn.register_forward_hook(make_hook(li, "attn")))
			if "mlp" in sites:
				handles.append(layers[li].mlp.register_forward_hook(make_hook(li, "mlp")))

	was_checkpointing = getattr(model, "is_gradient_checkpointing", False)
	try:
		if checkpoint and not was_checkpointing:
			model.gradient_checkpointing_enable(gradient_checkpointing_kwargs={"use_reentrant": False})
		out = model(
			input_ids=input_ids,
			inputs_embeds=inputs_embeds,
			attention_mask=attention_mask,
			output_hidden_states=need_hidden,
			use_cache=False,
		)
	finally:
		for h in handles:
			h.remove()
		if checkpoint and not was_checkpointing:
			model.gradient_checkpointing_disable()

	hidden: dict[tuple[Site, int], torch.Tensor] = {}
	if need_hidden:
		# hidden_states = (embeddings, layer_0_out, ..., layer_{L-1}_out); layer li output is index li+1.
		hs = out.hidden_states
		want = capture.layers if capture.layers is not None else tuple(range(len(hs) - 1))
		for li in want:
			hidden[("residual", li)] = hs[li + 1]
	hidden.update(hook_store)
	return GradForwardOutput(logits=out.logits, hidden=hidden)

gumbel_softmax_tokens

gumbel_softmax_tokens(
    logits: Tensor, tau: float = 1.0, hard: bool = False
) -> torch.Tensor

Relaxed (Gumbel-softmax) sample over the last (vocab) dim of logits, differentiable in logits.

tau is the temperature: high -> smooth mixture (biased but low-variance gradients), low -> near-one-hot (faithful to discrete sampling but higher-variance). Anneal tau down over training to move from an easy soft optimization toward the discrete tokens B will actually see. hard=True uses the straight-through estimator: a true one-hot on the forward pass, softmax gradient on the backward pass — so the value B consumes is a real token while gradients still flow. Returns [..., V] to feed straight into soft_embed.

Source code in src/interlens/interp/bridge.py
def gumbel_softmax_tokens(logits: torch.Tensor, tau: float = 1.0, hard: bool = False) -> torch.Tensor:
	"""Relaxed (Gumbel-softmax) sample over the last (vocab) dim of ``logits``, differentiable in ``logits``.

	``tau`` is the temperature: high -> smooth mixture (biased but low-variance gradients), low -> near-one-hot
	(faithful to discrete sampling but higher-variance). Anneal ``tau`` down over training to move from an easy soft
	optimization toward the discrete tokens B will actually see. ``hard=True`` uses the straight-through estimator: a
	true one-hot on the forward pass, softmax gradient on the backward pass — so the value B consumes is a real token
	while gradients still flow. Returns ``[..., V]`` to feed straight into ``soft_embed``.
	"""
	return F.gumbel_softmax(logits, tau=tau, hard=hard, dim=-1)

register_worker_init

register_worker_init(fn) -> object

Register a zero-arg callable to run once at worker startup (e.g. to populate the tool/analyzer registries).

Source code in src/interlens/runner/worker_init.py
def register_worker_init(fn) -> object:
	"""Register a zero-arg callable to run once at worker startup (e.g. to populate the tool/analyzer registries)."""
	_WORKER_INIT_HOOKS.append(fn)
	return fn

run

run(
    conversations,
    devices=None,
    out_dir=None,
    resume=False,
    batched=True,
    max_batch_size=None,
) -> RunReport

Run several conversation lineups in ONE pool — the multi-lineup entry point (e.g. a ladder of model pairs × conditions in one overnight job).

Each conversation is expanded to its jobs (one per data() row if it has data, else a single conversation), with job ids namespaced by that conversation's name (default conv{i}) so ids stay unique and resumable. All jobs go into one pool, so GPUs stay packed across lineups (no idle tail between sequential rollouts) under a single out_dir/resume namespace, returning one merged RunReport. Mixing lineups is safe: batched co-stepping groups by schedule signature, so each distinct lineup forms its own batch group.

Conversation.rollout is the single-lineup sugar for this (run([conv], ...)-equivalent via run_jobs).

Source code in src/interlens/runner/pool.py
def run(conversations, devices=None, out_dir=None, resume=False, batched=True, max_batch_size=None) -> RunReport:
	"""Run several conversation lineups in ONE pool — the multi-lineup entry point (e.g. a ladder of model pairs ×
	conditions in one overnight job).

	Each conversation is expanded to its jobs (one per ``data()`` row if it has data, else a single conversation),
	with job ids namespaced by that conversation's ``name`` (default ``conv{i}``) so ids stay unique and resumable.
	All jobs go into one pool, so GPUs stay packed across lineups (no idle tail between sequential rollouts) under a
	single ``out_dir``/resume namespace, returning one merged ``RunReport``. Mixing lineups is safe: batched
	co-stepping groups by schedule signature, so each distinct lineup forms its own batch group.

	``Conversation.rollout`` is the single-lineup sugar for this (``run([conv], ...)``-equivalent via ``run_jobs``).
	"""
	jobs = []
	for i, conv in enumerate(conversations):
		jobs.extend(conv._jobs_for_run(i))
	return run_jobs(jobs, devices=devices, out_dir=out_dir, resume=resume, batched=batched,
	                max_batch_size=max_batch_size)

run_jobs

run_jobs(
    jobs,
    devices=None,
    out_dir=None,
    resume=False,
    batched=True,
    max_batch_size=None,
) -> RunReport

Run (job_id, Conversation) jobs across devices, with checkpointing, resume, and per-job failure isolation.

Parallel by default on two axes: one worker process per device (jobs round-robined; multi-GPU spawns via torch.multiprocessing since fork+CUDA is broken — a single device runs in-process), and within each device batched co-stepping (batched=True). Jobs are grouped by co-step schedule signature so same-schedule jobs batch into one model.generate — correct for ANY mix. batched=False gives the DETERMINISTIC path.

Source code in src/interlens/runner/pool.py
def run_jobs(jobs, devices=None, out_dir=None, resume=False, batched=True, max_batch_size=None) -> RunReport:
	"""Run ``(job_id, Conversation)`` jobs across devices, with checkpointing, resume, and per-job failure isolation.

	Parallel by default on **two** axes: one worker **process per device** (jobs round-robined; multi-GPU spawns via
	``torch.multiprocessing`` since fork+CUDA is broken — a single device runs in-process), and within each device
	**batched co-stepping** (``batched=True``). Jobs are grouped by co-step schedule signature so same-schedule jobs
	batch into one ``model.generate`` — correct for ANY mix. ``batched=False`` gives the DETERMINISTIC path.
	"""
	devices = devices or available_devices()
	pending = [(jid, conv) for jid, conv in jobs if not (resume and _already_done(out_dir, jid))]
	skipped = [jid for jid, conv in jobs if resume and _already_done(out_dir, jid)]

	if len(devices) > 1:
		results = _run_spawn(pending, devices, out_dir, batched, max_batch_size)
	elif batched:
		results = _run_group(pending, devices[0], out_dir, max_batch_size)
	else:
		results = {}
		for i, (jid, conv) in enumerate(pending):
			results[jid] = _run_one(jid, conv, devices[i % len(devices)], out_dir)
	return RunReport(results=results, skipped=skipped)

soft_embed

soft_embed(
    model: "PreTrainedModel", probs: Tensor
) -> torch.Tensor

Mix model's input-embedding rows by a per-position distribution: [..., V] @ E[V, d] -> [..., d].

probs is a (relaxed or one-hot) distribution over model's vocab for each position — e.g. a softmax / gumbel_softmax_tokens output of an upstream model. Returns grad-connected soft embeddings suitable to pass as inputs_embeds to forward_with_grad / continuation_logprob. This is exact for hard one-hots: feeding one_hot(ids) reproduces model.get_input_embeddings()(ids).

Source code in src/interlens/interp/bridge.py
def soft_embed(model: "PreTrainedModel", probs: torch.Tensor) -> torch.Tensor:
	"""Mix ``model``'s input-embedding rows by a per-position distribution: ``[..., V] @ E[V, d] -> [..., d]``.

	``probs`` is a (relaxed or one-hot) distribution over ``model``'s vocab for each position — e.g. a softmax /
	``gumbel_softmax_tokens`` output of an upstream model. Returns grad-connected soft embeddings suitable to pass as
	``inputs_embeds`` to ``forward_with_grad`` / ``continuation_logprob``. This is exact for hard one-hots: feeding
	``one_hot(ids)`` reproduces ``model.get_input_embeddings()(ids)``.
	"""
	weight = model.get_input_embeddings().weight  # [V, d]
	return probs.to(dtype=weight.dtype, device=weight.device) @ weight

token_logprobs

token_logprobs(
    scores: tuple[Tensor, ...], generated_ids: Tensor
) -> dict

Compute per-token logprobs / surprisal / entropy for a generation.

scores is the tuple of per-step logit tensors from model.generate(..., output_scores=True, return_dict_in_generate=True) (one [vocab] per generated token); generated_ids are the sampled token ids. Returns lists suitable to drop into Message.metadata — scalar-per-token, so they stay small and don't violate the "no heavy tensors in metadata" invariant.

Surprisal = -logprob (nats); entropy is the full next-token distribution entropy at each step (a readout of the model's uncertainty, distinct from the surprisal of the token it actually emitted).

Source code in src/interlens/interp/logprobs.py
def token_logprobs(scores: tuple[torch.Tensor, ...], generated_ids: torch.Tensor) -> dict:
	"""Compute per-token logprobs / surprisal / entropy for a generation.

	``scores`` is the tuple of per-step logit tensors from ``model.generate(..., output_scores=True,
	return_dict_in_generate=True)`` (one ``[vocab]`` per generated token); ``generated_ids`` are the sampled
	token ids. Returns lists suitable to drop into ``Message.metadata`` — scalar-per-token, so they stay small
	and don't violate the "no heavy tensors in metadata" invariant.

	Surprisal = -logprob (nats); entropy is the full next-token distribution entropy at each step (a readout of
	the model's uncertainty, distinct from the surprisal of the token it actually emitted).
	"""
	if len(scores) == 0:
		return {"logprobs": [], "surprisal": [], "entropy": []}
	# Stack once -> [steps, vocab]; batch all ops so there is a single GPU->CPU sync at the end.
	logits = torch.stack([s[0] for s in scores]).float()
	logp = torch.log_softmax(logits, dim=-1)
	toks = torch.as_tensor(generated_ids, device=logp.device, dtype=torch.long)[: logp.shape[0]]
	lp = logp.gather(1, toks.unsqueeze(1)).squeeze(1)
	ent = -(logp.exp() * logp).sum(dim=-1)
	logprobs = lp.tolist()
	return {
		"logprobs": logprobs,
		"surprisal": (-lp).tolist(),
		"entropy": ent.tolist(),
	}