OPENAI_API_KEY/ANTHROPIC_API_KEY/GEMINI_API_KEY) set on the engine, since it makes a real judge call server-side. If you don’t have one configured - or you’re already working in Claude Code anyway - there’s a second way in, the same idea as Langfuse’s own Claude-skill prompt-improvement workflow: Claude’s own reasoning stands in for the judge call, so no engine-side LLM key is needed at all.
Setup
Copy the skill fromAgentX-trace-eval/skills/improve-prompt into .claude/skills/improve-prompt/ in whatever project you run Claude Code from (your own agent’s repo works fine - the skill only talks to your self-host engine over HTTP, it doesn’t need to be in this repo). Then, with your engine running:
What it does
Claude reads the prompt’s real worst-rated eval results straight from your engine (GET /prompts/:id/examples - the same evidence Suggest improvement uses, just without a judge call in front of it), drafts a full rewrite and explains what changed and why, and shows you both versions side by side. It never publishes on its own - only once you say something like “publish it” does it call the same POST /prompts/:id/versions the dashboard button uses. It finds your engine automatically: the API key comes straight from ~/.agentx/config.json (written the first time you start the engine), and the base URL defaults to AGENTX_API_BASE_URL, the same variable the SDK itself reads.