Skip to content

interlens.participant.participants.llama

interlens.participant.participants.llama

LlamaModelParticipant dataclass 

LlamaModelParticipant(
    name: str = "",
    hf_id: str | None = None,
    weights_path: str | None = None,
    dtype: str = "bfloat16",
    attn: str = "flash_attention_2",
    quant: str | None = None,
    revision: str | None = None,
    device: str | device | None = None,
    max_new_tokens: int = 512,
    temperature: float = 0.8,
    top_p: float = 0.95,
    seed: int | None = None,
    thinking: bool | str = "auto",
    system_prompt: str | None = None,
    private_context: tuple = (),
    tools: tuple = (),
    max_tool_iters: int = 4,
    kv_reuse: bool | str = "auto",
    steering: object = None,
    _model: "PreTrainedModel | None" = None,
    _tokenizer: "PreTrainedTokenizerBase | None" = None,
)

Bases: ModelParticipant

A Llama-family participant. Chat-template flags are auto-derived from the tokenizer; only the tool-call format differs from base: Llama 3 emits calls as <|python_tag|>{json} rather than Hermes/Qwen <tool_call> blocks.

parse_tool_calls 

parse_tool_calls(text: str) -> list

Parse Llama-3's <|python_tag|>{json} function-call format (best-effort).

Everything after <|python_tag|> is one or more JSON objects (separated by newlines or semicolons), each {"name": ..., "arguments"/"parameters": {...}}. Anything unparseable is skipped, so a malformed call yields [] (treated as a final message), matching the base contract of never misfiring.

Source code in src/interlens/participant/participants/llama.py 
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
def parse_tool_calls(self, text: str) -> list:
	"""Parse Llama-3's ``<|python_tag|>{json}`` function-call format (best-effort).

	Everything after ``<|python_tag|>`` is one or more JSON objects (separated by newlines or semicolons),
	each ``{"name": ..., "arguments"/"parameters": {...}}``. Anything unparseable is skipped, so a malformed
	call yields ``[]`` (treated as a final message), matching the base contract of never misfiring."""
	import json
	from ...tools.tool_call import ToolCall

	tag = "<|python_tag|>"
	idx = text.find(tag)
	if idx < 0:
		return []
	body = text[idx + len(tag):]
	calls = []
	for chunk in body.replace(";", "\n").splitlines():
		chunk = chunk.strip()
		if not chunk:
			continue
		try:
			data = json.loads(chunk)
			name = data["name"]
			arguments = data.get("arguments") or data.get("parameters") or {}
		except (json.JSONDecodeError, KeyError) as exc:
			logger.debug("dropping unparseable python_tag call %r: %s", chunk, exc)
			continue
		calls.append(ToolCall(name=name, arguments=arguments, raw=chunk))
	return calls