Ajustes conforme relatorio de testes 2026-08-27
This commit is contained in:
@@ -2171,71 +2171,30 @@ trace_id
|
||||
|
||||
#### 5.1.1.21.3. Instrumentação automática do cliente OpenAI pelo Langfuse
|
||||
|
||||
O padrão oficial do framework é:
|
||||
|
||||
```python
|
||||
ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=true
|
||||
```
|
||||
|
||||
habilita a instrumentação automática do cliente OpenAI pelo Langfuse.
|
||||
|
||||
Quando habilitada, todas as chamadas realizadas através do cliente OpenAI instrumentado passam a gerar automaticamente spans e generations detalhadas no Langfuse.
|
||||
|
||||
Benefícios
|
||||
|
||||
Com a instrumentação automática ativada, o Langfuse passa a registrar informações como:
|
||||
|
||||
* OpenAI-generation
|
||||
* Prompt enviado ao modelo
|
||||
* Resposta retornada pelo modelo
|
||||
* Modelo utilizado
|
||||
* Quantidade de tokens
|
||||
* Custos estimados
|
||||
* Latência da chamada
|
||||
* Erros de execução
|
||||
|
||||
Essas informações ficam associadas ao trace principal da conversa, facilitando análise, troubleshooting e auditoria.
|
||||
|
||||
Comportamento quando desabilitado
|
||||
|
||||
Quando:
|
||||
|
||||
```python
|
||||
```env
|
||||
ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=false
|
||||
```
|
||||
|
||||
ou a variável não está definida:
|
||||
O framework já instrumenta as chamadas LLM por meio de `Telemetry.generation(...)`, preservando `trace_id`, `session_id`, `user_id`, metadados, tokens, custos, latência e o relacionamento pai/filho dentro do trace de negócio. Por esse motivo, a auto-instrumentação do cliente OpenAI não é necessária no fluxo normal do framework.
|
||||
|
||||
* As chamadas LLM continuam funcionando normalmente.
|
||||
* Os spans customizados do framework continuam sendo emitidos.
|
||||
* O Langfuse deixa de criar automaticamente as entradas OpenAI-generation.
|
||||
* Menos detalhes ficam disponíveis para análise das chamadas ao modelo.
|
||||
Quando `false`:
|
||||
|
||||
Quando utilizar
|
||||
* as chamadas LLM continuam funcionando normalmente;
|
||||
* prompts, respostas, modelo, tokens, custos e latência continuam disponíveis pela telemetria explícita do framework;
|
||||
* as generations permanecem correlacionadas ao trace principal da requisição;
|
||||
* evita-se dupla instrumentação e `OpenAI-generation` como trace raiz separado.
|
||||
|
||||
Recomenda-se habilitar em:
|
||||
A opção `true` existe apenas para compatibilidade ou diagnóstico de código que chama diretamente o SDK OpenAI/OpenAI-compatible fora da camada de `Telemetry` do framework. Nesses casos, o wrapper `langfuse.openai` pode capturar automaticamente essas chamadas. Entretanto, em uma aplicação que já usa a instrumentação nativa do framework, mantê-la habilitada pode gerar duplicidade de observations, contagem duplicada de tokens/custos ou traces independentes quando não houver um parent Langfuse ativo.
|
||||
|
||||
* Ambientes de desenvolvimento.
|
||||
* Ambientes de homologação.
|
||||
* Ambientes de produção que necessitem observabilidade detalhada das chamadas LLM.
|
||||
* Cenários de troubleshooting, tuning de prompts e análise de custos.
|
||||
```env
|
||||
# Padrão recomendado para todos os templates e ambientes do framework
|
||||
ENABLE_LANGFUSE=true
|
||||
ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=false
|
||||
```
|
||||
|
||||
Observação
|
||||
|
||||
Esta configuração afeta apenas a telemetria automática do Langfuse.
|
||||
|
||||
Ela não altera:
|
||||
|
||||
* O comportamento dos agentes.
|
||||
* O roteamento do Supervisor.
|
||||
* Guardrails.
|
||||
* Judges.
|
||||
* MCP Tool Router.
|
||||
* Fluxos LangGraph.
|
||||
|
||||
Seu único objetivo é enriquecer a observabilidade das chamadas realizadas ao modelo de linguagem.
|
||||
---
|
||||
|
||||
### 5.1.1.22. Recomendações de arquitetura
|
||||
Todos os arquivos `.env.example` distribuídos pelo projeto mantêm essa opção explicitamente em `false`. Se um componente externo precisar de captura automática, habilite-a somente naquele deployment e valide a árvore de traces no Langfuse.
|
||||
|
||||
#### 5.1.1.22.1. Para demos e desenvolvimento
|
||||
|
||||
|
||||
@@ -2167,71 +2167,30 @@ trace_id
|
||||
|
||||
#### 5.1.1.21.3. Automatic Langfuse instrumentation for the OpenAI client
|
||||
|
||||
```python
|
||||
ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=true
|
||||
```
|
||||
The framework's official default is:
|
||||
|
||||
enables automatic Langfuse instrumentation for the OpenAI client.
|
||||
|
||||
When enabled, every request executed through the Langfuse-instrumented OpenAI client automatically generates detailed spans and generations within Langfuse.
|
||||
|
||||
Benefits
|
||||
|
||||
With automatic instrumentation enabled, Langfuse can automatically capture and display information such as:
|
||||
|
||||
* OpenAI-generation
|
||||
* Prompt sent to the model
|
||||
* Model response
|
||||
* Model name used
|
||||
* Token consumption
|
||||
* Estimated costs
|
||||
* Request latency
|
||||
* Execution errors
|
||||
|
||||
All of this information is linked to the main conversation trace, making troubleshooting, auditing, and performance analysis significantly easier.
|
||||
|
||||
Behavior When Disabled
|
||||
|
||||
When:
|
||||
|
||||
```python
|
||||
```env
|
||||
ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=false
|
||||
```
|
||||
|
||||
or when the variable is not defined:
|
||||
The framework already instruments LLM calls through `Telemetry.generation(...)`, preserving `trace_id`, `session_id`, `user_id`, metadata, token usage, cost, latency, and the parent/child relationship inside the business trace. Therefore, OpenAI client auto-instrumentation is not required in the normal framework path.
|
||||
|
||||
* LLM calls continue to function normally.
|
||||
* Custom framework spans are still emitted.
|
||||
* Langfuse no longer automatically creates OpenAI-generation entries.
|
||||
* Less detailed information is available for analyzing model interactions.
|
||||
When set to `false`:
|
||||
|
||||
Recommended Usage
|
||||
* LLM calls continue to work normally;
|
||||
* prompts, responses, model, tokens, costs, and latency remain available through the framework's explicit telemetry;
|
||||
* generations remain correlated with the main request trace;
|
||||
* duplicate instrumentation and standalone `OpenAI-generation` root traces are avoided.
|
||||
|
||||
It is recommended to enable this setting in:
|
||||
The `true` option exists only for compatibility or diagnostics for code that calls the OpenAI/OpenAI-compatible SDK directly outside the framework `Telemetry` layer. In such cases, the `langfuse.openai` wrapper can automatically capture those calls. In an application already using the framework's native instrumentation, keeping it enabled may create duplicate observations, duplicate token/cost accounting, or independent traces when no active Langfuse parent exists.
|
||||
|
||||
* Development environments
|
||||
* Testing and staging environments
|
||||
* Production environments that require detailed LLM observability
|
||||
* Prompt engineering, troubleshooting, and cost analysis scenarios
|
||||
```env
|
||||
# Recommended default for every framework template and environment
|
||||
ENABLE_LANGFUSE=true
|
||||
ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=false
|
||||
```
|
||||
|
||||
Important Note
|
||||
|
||||
This setting only affects Langfuse automatic telemetry and observability.
|
||||
|
||||
It does not change:
|
||||
|
||||
* Agent behavior
|
||||
* Supervisor routing
|
||||
* Guardrails
|
||||
* Judges
|
||||
* MCP Tool Router
|
||||
* LangGraph workflows
|
||||
|
||||
Its sole purpose is to enrich the observability of language model interactions and provide more detailed execution insights within Langfuse.
|
||||
|
||||
---
|
||||
|
||||
### 5.1.1.22. Architecture recommendations
|
||||
Every `.env.example` distributed with the project explicitly keeps this option set to `false`. If an external component requires automatic capture, enable it only for that deployment and validate the trace tree in Langfuse.
|
||||
|
||||
#### 5.1.1.22.1. For demos and development
|
||||
|
||||
|
||||
@@ -200,6 +200,19 @@ request_id → tenant_id → agent_id → session_id → user_id → channel →
|
||||
|
||||
The context uses `ContextVar`, so it works across async calls, FastAPI, LangGraph, and LLM providers.
|
||||
|
||||
### Langfuse OpenAI auto-instrumentation policy
|
||||
|
||||
The official framework template configuration is:
|
||||
|
||||
```env
|
||||
ENABLE_LANGFUSE=true
|
||||
ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=false
|
||||
```
|
||||
|
||||
The `false` value is intentional. The framework already records model calls through `Telemetry.generation(...)` and keeps each generation inside the request trace. Enabling `langfuse.openai` at the same time adds a second instrumentation layer and may produce standalone `OpenAI-generation` root traces, duplicate observations, and duplicate token/cost accounting.
|
||||
|
||||
Use `true` only to capture direct OpenAI/OpenAI-compatible SDK calls that occur outside the framework telemetry layer. This is a compatibility/diagnostic mode, not the operational default. Every `.env.example` in the repository must explicitly keep the value set to `false`.
|
||||
|
||||
### Langfuse
|
||||
|
||||
Enable in `.env`:
|
||||
|
||||
@@ -201,6 +201,19 @@ request_id → tenant_id → agent_id → session_id → user_id → channel →
|
||||
|
||||
O contexto usa `ContextVar`, portanto funciona em chamadas assíncronas, FastAPI, LangGraph e providers LLM.
|
||||
|
||||
### Política de auto-instrumentação OpenAI no Langfuse
|
||||
|
||||
A configuração oficial dos templates do framework é:
|
||||
|
||||
```env
|
||||
ENABLE_LANGFUSE=true
|
||||
ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=false
|
||||
```
|
||||
|
||||
O `false` é intencional. O framework já registra as chamadas ao modelo usando `Telemetry.generation(...)` e mantém a generation dentro do trace da requisição. Habilitar simultaneamente `langfuse.openai` cria uma segunda camada de instrumentação e pode resultar em `OpenAI-generation` como trace raiz, observations duplicadas e dupla contabilização de tokens/custos.
|
||||
|
||||
Use `true` somente para capturar chamadas diretas ao SDK OpenAI/OpenAI-compatible que ocorram fora da telemetria do framework. Esse é um modo de compatibilidade/diagnóstico, não o padrão operacional. Todos os `.env.example` do repositório devem permanecer explicitamente com o valor `false`.
|
||||
|
||||
### Langfuse
|
||||
|
||||
Ative no `.env`:
|
||||
|
||||
@@ -145,8 +145,17 @@ class GuardrailLLMClient:
|
||||
except RuntimeError:
|
||||
return asyncio.run(_call())
|
||||
|
||||
# ``ContextVar`` values do not cross ThreadPoolExecutor boundaries by
|
||||
# default. Preserve the framework request/trace/parent observation when
|
||||
# this legacy sync bridge needs a worker thread; otherwise a provider
|
||||
# created inside the worker sees no active correlation context and the
|
||||
# optional langfuse.openai wrapper may emit a standalone OpenAI-generation
|
||||
# trace.
|
||||
from contextvars import copy_context
|
||||
|
||||
context = copy_context()
|
||||
with ThreadPoolExecutor(max_workers=1, thread_name_prefix="guardrail-compat") as executor:
|
||||
return executor.submit(lambda: asyncio.run(_call())).result()
|
||||
return executor.submit(context.run, lambda: asyncio.run(_call())).result()
|
||||
|
||||
def classify(
|
||||
self,
|
||||
|
||||
@@ -58,6 +58,55 @@ def _coerce_reasoning_text(value: Any) -> str | None:
|
||||
return text or None
|
||||
|
||||
|
||||
def _coerce_message_content(value: Any) -> str:
|
||||
"""Normalize OpenAI-compatible message content without using reasoning as answer.
|
||||
|
||||
OpenAI-compatible implementations may expose ``message.content`` as a plain
|
||||
string, a list of content parts, or SDK objects/dicts containing ``text``.
|
||||
Unknown shapes fail closed to an empty string instead of serializing the raw
|
||||
response object into the assistant answer.
|
||||
"""
|
||||
if value is None:
|
||||
return ""
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
if isinstance(value, (list, tuple)):
|
||||
chunks: list[str] = []
|
||||
for item in value:
|
||||
if isinstance(item, str):
|
||||
chunks.append(item)
|
||||
continue
|
||||
if isinstance(item, dict):
|
||||
text = item.get("text")
|
||||
if isinstance(text, str):
|
||||
chunks.append(text)
|
||||
continue
|
||||
text = getattr(item, "text", None)
|
||||
if isinstance(text, str):
|
||||
chunks.append(text)
|
||||
return "".join(chunks)
|
||||
if isinstance(value, dict):
|
||||
text = value.get("text")
|
||||
return text if isinstance(text, str) else ""
|
||||
text = getattr(value, "text", None)
|
||||
return text if isinstance(text, str) else ""
|
||||
|
||||
|
||||
def _extract_openai_message_content(message: Any) -> str:
|
||||
if message is None:
|
||||
return ""
|
||||
if isinstance(message, dict):
|
||||
return _coerce_message_content(message.get("content"))
|
||||
return _coerce_message_content(getattr(message, "content", None))
|
||||
|
||||
|
||||
def _extract_finish_reason(choice: Any) -> str | None:
|
||||
if choice is None:
|
||||
return None
|
||||
value = choice.get("finish_reason") if isinstance(choice, dict) else getattr(choice, "finish_reason", None)
|
||||
return str(value) if value is not None else None
|
||||
|
||||
|
||||
def _extract_reasoning_content(obj: Any) -> str | None:
|
||||
"""Best-effort extraction across OpenAI-compatible and OCI response shapes."""
|
||||
if obj is None:
|
||||
@@ -272,10 +321,27 @@ class OCICompatibleOpenAIProvider(LLMProvider):
|
||||
getattr(settings, "ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION", None)
|
||||
or os.getenv("ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION", "false")
|
||||
).strip().lower() in {"1", "true", "yes", "on", "y"}
|
||||
if self.telemetry is not None and use_langfuse_wrapper:
|
||||
|
||||
# The framework owns Langfuse correlation. Even compatibility paths may
|
||||
# instantiate a provider without passing ``Telemetry`` explicitly while a
|
||||
# request is already active (for example GuardrailLLMClient running in its
|
||||
# sync bridge). In that situation langfuse.openai would auto-create an
|
||||
# ``OpenAI-generation`` root trace instead of attaching to the business
|
||||
# request. Treat an active framework observability context exactly like
|
||||
# an injected Telemetry instance and keep the standard OpenAI client.
|
||||
active_framework_trace = False
|
||||
try:
|
||||
from agent_framework.observability.context import get_observability_context
|
||||
|
||||
obs_ctx = get_observability_context()
|
||||
active_framework_trace = bool(obs_ctx.trace_id or obs_ctx.request_id)
|
||||
except Exception:
|
||||
active_framework_trace = False
|
||||
|
||||
if use_langfuse_wrapper and (self.telemetry is not None or active_framework_trace):
|
||||
logger.warning(
|
||||
"ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=true ignorado porque o provider já recebeu "
|
||||
"Telemetry do framework; instrumentação dupla pode criar observations fora do contrato de mapping."
|
||||
"ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=true ignorado durante execução correlacionada "
|
||||
"do framework; langfuse.openai pode criar OpenAI-generation como trace raiz separado."
|
||||
)
|
||||
use_langfuse_wrapper = False
|
||||
if getattr(settings, "ENABLE_LANGFUSE", False) and use_langfuse_wrapper:
|
||||
@@ -432,9 +498,29 @@ class OCICompatibleOpenAIProvider(LLMProvider):
|
||||
model_parameters=model_parameters,
|
||||
) as generation:
|
||||
resp = await client.chat.completions.create(**request_kwargs)
|
||||
message = resp.choices[0].message
|
||||
answer = message.content or ""
|
||||
reasoning_content = _extract_reasoning_content(message)
|
||||
choices = getattr(resp, "choices", None) or []
|
||||
if not choices:
|
||||
message = None
|
||||
answer = ""
|
||||
reasoning_content = None
|
||||
finish_reason = None
|
||||
logger.warning(
|
||||
"OpenAI-compatible LLM returned no choices provider=%s model=%s profile=%s component=%s",
|
||||
provider, model, resolved_profile_name, component_name,
|
||||
)
|
||||
else:
|
||||
choice = choices[0]
|
||||
message = choice.get("message") if isinstance(choice, dict) else getattr(choice, "message", None)
|
||||
answer = _extract_openai_message_content(message)
|
||||
reasoning_content = _extract_reasoning_content(message)
|
||||
finish_reason = _extract_finish_reason(choice)
|
||||
|
||||
logger.info(
|
||||
"OpenAI-compatible LLM response provider=%s model=%s profile=%s component=%s "
|
||||
"finish_reason=%s content_len=%d reasoning_len=%d",
|
||||
provider, model, resolved_profile_name, component_name,
|
||||
finish_reason, len(answer), len(reasoning_content or ""),
|
||||
)
|
||||
|
||||
usage_metadata = self.token_collector.enrich(model, getattr(resp, "usage", None))
|
||||
usage_metadata.update({
|
||||
@@ -445,8 +531,16 @@ class OCICompatibleOpenAIProvider(LLMProvider):
|
||||
"component": component_name,
|
||||
"model": model,
|
||||
"provider": provider,
|
||||
"finish_reason": finish_reason,
|
||||
"content_length": len(answer),
|
||||
"reasoning_content_length": len(reasoning_content or ""),
|
||||
**model_parameters,
|
||||
})
|
||||
llm_metadata.update({
|
||||
"finish_reason": finish_reason,
|
||||
"content_length": len(answer),
|
||||
"reasoning_content_length": len(reasoning_content or ""),
|
||||
})
|
||||
generation.set_output(answer)
|
||||
generation.set_usage(usage_metadata)
|
||||
generation.set_metadata(**usage_metadata)
|
||||
|
||||
@@ -47,13 +47,13 @@ internal steps = observations/spans/generations inside that trace
|
||||
|
||||
- Langfuse OpenAI auto-instrumentation is now opt-in.
|
||||
- Default behavior uses the standard `openai.AsyncOpenAI` client and relies on the framework's own `Telemetry.generation(...)` to create correlated Langfuse generations.
|
||||
- To re-enable wrapper-based auto-instrumentation, set:
|
||||
- The supported framework default is:
|
||||
|
||||
```env
|
||||
ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=true
|
||||
ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=false
|
||||
```
|
||||
|
||||
For this framework, the recommended default is to keep it disabled.
|
||||
All `.env.example` files in the repository declare this value explicitly. Set it to `true` only for isolated compatibility/diagnostic deployments that intentionally need to capture OpenAI SDK calls outside the framework telemetry path.
|
||||
|
||||
## Expected result
|
||||
|
||||
@@ -86,7 +86,7 @@ framework_judges
|
||||
Run the backend and execute one request. Then verify:
|
||||
|
||||
1. The `Traces` screen has one trace row for the request, not one row per node.
|
||||
2. `OpenAI-generation` no longer appears as a separate top-level trace unless `ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=true`.
|
||||
2. `OpenAI-generation` no longer appears as a separate top-level trace with the supported default `ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=false`.
|
||||
3. LangGraph node events and IC/NOC/GRL events appear under the same request trace.
|
||||
|
||||
|
||||
|
||||
@@ -145,8 +145,17 @@ class GuardrailLLMClient:
|
||||
except RuntimeError:
|
||||
return asyncio.run(_call())
|
||||
|
||||
# ``ContextVar`` values do not cross ThreadPoolExecutor boundaries by
|
||||
# default. Preserve the framework request/trace/parent observation when
|
||||
# this legacy sync bridge needs a worker thread; otherwise a provider
|
||||
# created inside the worker sees no active correlation context and the
|
||||
# optional langfuse.openai wrapper may emit a standalone OpenAI-generation
|
||||
# trace.
|
||||
from contextvars import copy_context
|
||||
|
||||
context = copy_context()
|
||||
with ThreadPoolExecutor(max_workers=1, thread_name_prefix="guardrail-compat") as executor:
|
||||
return executor.submit(lambda: asyncio.run(_call())).result()
|
||||
return executor.submit(context.run, lambda: asyncio.run(_call())).result()
|
||||
|
||||
def classify(
|
||||
self,
|
||||
|
||||
@@ -321,10 +321,27 @@ class OCICompatibleOpenAIProvider(LLMProvider):
|
||||
getattr(settings, "ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION", None)
|
||||
or os.getenv("ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION", "false")
|
||||
).strip().lower() in {"1", "true", "yes", "on", "y"}
|
||||
if self.telemetry is not None and use_langfuse_wrapper:
|
||||
|
||||
# The framework owns Langfuse correlation. Even compatibility paths may
|
||||
# instantiate a provider without passing ``Telemetry`` explicitly while a
|
||||
# request is already active (for example GuardrailLLMClient running in its
|
||||
# sync bridge). In that situation langfuse.openai would auto-create an
|
||||
# ``OpenAI-generation`` root trace instead of attaching to the business
|
||||
# request. Treat an active framework observability context exactly like
|
||||
# an injected Telemetry instance and keep the standard OpenAI client.
|
||||
active_framework_trace = False
|
||||
try:
|
||||
from agent_framework.observability.context import get_observability_context
|
||||
|
||||
obs_ctx = get_observability_context()
|
||||
active_framework_trace = bool(obs_ctx.trace_id or obs_ctx.request_id)
|
||||
except Exception:
|
||||
active_framework_trace = False
|
||||
|
||||
if use_langfuse_wrapper and (self.telemetry is not None or active_framework_trace):
|
||||
logger.warning(
|
||||
"ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=true ignorado porque o provider já recebeu "
|
||||
"Telemetry do framework; instrumentação dupla pode criar observations fora do contrato de mapping."
|
||||
"ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=true ignorado durante execução correlacionada "
|
||||
"do framework; langfuse.openai pode criar OpenAI-generation como trace raiz separado."
|
||||
)
|
||||
use_langfuse_wrapper = False
|
||||
if getattr(settings, "ENABLE_LANGFUSE", False) and use_langfuse_wrapper:
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from types import ModuleType, SimpleNamespace
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_framework.guardrails.calibrated.llm_client import GuardrailLLMClient
|
||||
from agent_framework.llm.providers import OCICompatibleOpenAIProvider
|
||||
from agent_framework.observability.context import (
|
||||
clear_observability_context,
|
||||
get_observability_context,
|
||||
set_observability_context,
|
||||
)
|
||||
|
||||
|
||||
def test_openai_langfuse_wrapper_is_disabled_inside_active_framework_trace(monkeypatch):
|
||||
"""A correlated request must never create a standalone OpenAI-generation trace."""
|
||||
clear_observability_context()
|
||||
set_observability_context(request_id="req-123", trace_id="trace-123")
|
||||
monkeypatch.setenv("ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION", "true")
|
||||
|
||||
provider = OCICompatibleOpenAIProvider.__new__(OCICompatibleOpenAIProvider)
|
||||
provider.telemetry = None # compatibility path: no Telemetry explicitly injected
|
||||
settings = SimpleNamespace(
|
||||
ENABLE_LANGFUSE=True,
|
||||
ENABLE_LANGFUSE_OPENAI_AUTO_INSTRUMENTATION=True,
|
||||
)
|
||||
|
||||
fake_openai = ModuleType("openai")
|
||||
class AsyncOpenAI:
|
||||
pass
|
||||
AsyncOpenAI.__module__ = "openai"
|
||||
fake_openai.AsyncOpenAI = AsyncOpenAI
|
||||
monkeypatch.setitem(sys.modules, "openai", fake_openai)
|
||||
|
||||
client_cls = provider._resolve_async_openai(settings)
|
||||
|
||||
# Standard OpenAI client = framework owns observability/correlation.
|
||||
assert client_cls.__module__.startswith("openai")
|
||||
assert "langfuse" not in client_cls.__module__
|
||||
clear_observability_context()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_guardrail_sync_bridge_preserves_observability_context_in_worker(monkeypatch):
|
||||
"""Legacy sync guardrail bridge must carry request/trace ContextVars to its worker."""
|
||||
clear_observability_context()
|
||||
set_observability_context(request_id="req-guardrail", trace_id="trace-guardrail")
|
||||
|
||||
import agent_framework.guardrails.framework_llm_client as framework_client
|
||||
|
||||
async def fake_classifier(llm, task, payload, **kwargs):
|
||||
ctx = get_observability_context()
|
||||
return {"request_id": ctx.request_id, "trace_id": ctx.trace_id, "task": task}
|
||||
|
||||
monkeypatch.setattr(framework_client, "classify_with_framework_llm", fake_classifier)
|
||||
|
||||
result = GuardrailLLMClient._run_framework_classifier("TOX", {"text": "ok"})
|
||||
|
||||
assert result["request_id"] == "req-guardrail"
|
||||
assert result["trace_id"] == "trace-guardrail"
|
||||
assert result["task"] == "TOX"
|
||||
clear_observability_context()
|
||||
Reference in New Issue
Block a user