`interlens.participant.participants.api_participant`¶

interlens.participant.participants.api_participant ¶

APIParticipant `dataclass` ¶

APIParticipant(
    name: str = "",
    model_id: str = "",
    provider: Provider = "anthropic",
    system_prompt: str | None = None,
    private_context: tuple = (),
    max_tokens: int = 512,
    temperature: float = 1.0,
    batch: bool = False,
    client: object = None,
    requires_alternating_roles: bool = True,
)

Bases: Functional, Participant

A participant backed by a hosted API — Claude via anthropic (provider="anthropic", the default) or any model behind OpenRouter (provider="openrouter", OpenAI-compatible) — for use as a debate opponent, moderator, or the classifier inside an analyze callback.

It is a full participant for conversation purposes but has no local model — so there is no device, no activations, and no steering. Any interp request (capture/steering/patch/return_logprobs) raises rather than silently no-op'ing: in a measurement harness, a steering sweep that quietly did nothing on an API participant would produce a false "no effect" conclusion. Seeds don't bind hosted models, so API turns are excluded from the identical-replay guarantee.

Concurrency is network-bound, so pure-API conversations run thread-pooled rather than process-per-GPU (handled by the runner). The client callable is injectable for testing.

generate_batch ¶

generate_batch(
    views: list[list[dict]],
    *,
    turn: int | None = None,
    group_seed: int | None = None,
    max_new_tokens: int | None = None
) -> list[Message]

Generate one turn for many independent conversations at once — the API analogue of ModelParticipant.generate_batch, driven by the runner's co-stepper (rollout(..., batched=True)) to make large API rollouts cheap and throughput-bound.

With batch=True every view is sent as one asynchronous provider batch (Anthropic Message Batches / OpenAI Batch API) via client.submit_batch — ~50% cost and much higher throughput, at the price of batch-window latency. If the provider has no batch API (e.g. OpenRouter) this raises rather than silently degrading, so a requested batch is never quietly run as serial calls. With batch=False it falls back to sequential per-view calls (correct, just no batch discount). Interp is unavailable here, as for generate. turn/group_seed are accepted for co-stepper compatibility but unused (seeds do not bind hosted models). metadata['batched'] marks these turns.

Source code in src/interlens/participant/participants/api_participant.py

def generate_batch(self, views: list[list[dict]], *, turn: int | None = None,
                   group_seed: int | None = None, max_new_tokens: int | None = None) -> list[Message]:
	"""Generate one turn for many independent conversations at once — the API analogue of
	``ModelParticipant.generate_batch``, driven by the runner's co-stepper (``rollout(..., batched=True)``) to
	make large API rollouts cheap and throughput-bound.

	With ``batch=True`` every view is sent as one **asynchronous provider batch** (Anthropic Message Batches /
	OpenAI Batch API) via ``client.submit_batch`` — ~50% cost and much higher throughput, at the price of
	batch-window latency. **If the provider has no batch API (e.g. OpenRouter) this raises** rather than
	silently degrading, so a requested batch is never quietly run as serial calls. With ``batch=False`` it
	falls back to sequential per-view calls (correct, just no batch discount). Interp is unavailable here, as
	for ``generate``. ``turn``/``group_seed`` are accepted for co-stepper compatibility but unused (seeds do
	not bind hosted models). ``metadata['batched']`` marks these turns."""
	if not views:
		return []
	client = self.client or _default_client(self.provider)
	max_tokens = max_new_tokens if max_new_tokens is not None else self.max_tokens
	requests = []
	for view in views:
		system, messages = self._split_view(view)
		requests.append(dict(system=system, messages=messages, model=self.model_id,
		                     max_tokens=max_tokens, temperature=self.temperature))
	if self.batch:
		if not hasattr(client, "submit_batch"):
			raise NotImplementedError(
				f"APIParticipant {self.name!r} has batch=True but its client {type(client).__name__} exposes "
				f"no submit_batch; batch mode is unavailable for provider {self.provider!r}.")
		texts = client.submit_batch(requests)
	else:
		texts = [client(**r) for r in requests]
	return [Message(author=self.name, content=t,
	                metadata={"provider": self.provider, "model": self.model_id, "batched": True})
	        for t in texts]

interlens.participant.participants.api_participant¶