> ## Documentation Index > Fetch the complete documentation index at: https://docs.cloudeval.ai/llms.txt > Use this file to discover all available pages before exploring further. # Agent behavior and automation safety > Build reliable Cloudeval integrations with the CLI and machine-readable context files. This page is for developers building scripts, internal tools, or agents on top of Cloudeval. ## Start with safe defaults Use these defaults unless you have a good reason not to: * `cloudeval capabilities --format json` * `cloudeval doctor --format json` * `cloudeval doctor --mcp --format json` before MCP client setup * `--format json` * `--non-interactive` * `--profile ` * `--print-url --no-open` * stored `cloudeval login --headless` auth, or `--machine` when service-principal credentials are configured * `--output ` when the result must be persisted These defaults reduce ambiguity and make the CLI easier to compose with other systems. ## Stdout and stderr contract For machine-readable commands: * stdout is the data channel * stderr is for prompts, warnings, auth flow text, and browser-open messages Do not parse terminal UI output. If you need automation, use explicit subcommands such as `setup`, `config`, `doctor`, `status`, `models`, `sessions`, `projects`, `reports`, `ask`, `agents`, `connections`, `billing`, `validate`, `rules`, or `open`. ## CLI profiles for agents Use named profiles when multiple agents, environments, or workspaces share the same host. A profile can hold default Cloudeval service URL, app URL, project, model, and output preferences. ```bash theme={null} cloudeval setup \ --non-interactive \ --profile codex \ --project \ --model gpt-5-nano \ --format json ``` Then pass the same profile to automation commands: ```bash theme={null} cloudeval ask "Summarize project risk" \ --profile codex \ --format json \ --non-interactive ``` Explicit flags still override profile defaults, so scripts can pin a project or model for one run without changing the stored profile. ## Agent Profiles Agent Profiles are Cloudeval-owned reviewer roles. They help the same project evidence produce different kinds of answers without every client rebuilding its own prompt logic. Use Agent Profiles when the question is not just "what did Cloudeval find?" but "who is reading this, and what decision are they trying to make?" * `Architecture` reviews topology, dependencies, blast radius, availability, and Well-Architected tradeoffs. * `Cost` reviews spend drivers, waste signals, savings confidence, and cost validation checks. * `Triage` reviews likely failure domains, impact, containment, and rollback signals. * `Remediation` turns findings into ordered fixes with owners, dependencies, rollout cautions, and validation checks. * `Visual Explainer` explains topology, dependencies, ownership boundaries, blast radius, and evidence gaps visually when graph evidence supports it. * `Scripter` turns evidence-backed recommendations into copyable Bash, PowerShell, CLI, YAML, or runbook snippets with validation and rollback steps. * `Change Reviewer` reviews PRs and infrastructure changes as deltas across current/base source, proposed changes, and live Azure state when sync evidence exists. * `Evidence Auditor` checks whether answers, reports, findings, and review decisions are trustworthy enough for the claim being made. * `Security Reviewer` reviews exposure, identity, network boundaries, secrets, policy controls, and attack paths from supplied evidence. The public profile ids are `architecture`, `cost`, `triage`, `remediation`, `visual-explainer`, `scripter`, `change-reviewer`, `evidence-auditor`, and `security-reviewer`. Display names may contain spaces. Architecture includes the Well-Architected review lens, so there is no separate Well-Architected Agent Profile. Agent Profiles work across the web app, CLI, and MCP. Cloudeval applies the profile on the backend, so clients do not need to maintain separate prompt templates or local fallback logic. ```bash theme={null} cloudeval agents list --format json cloudeval agents show cost --format json cloudeval agents run cost \ --project \ --format json \ --non-interactive ``` Expected output from `agents run`: ```json theme={null} { "ok": true, "command": "agents run", "data": { "profile": { "id": "cost", "display_name": "Cost" }, "prompt": "Review this live Azure sync for cost risk...", "response": "Top cost signal: ...", "threadId": "8d5d...", "project": { "id": "project-123", "name": "Production" }, "frontendUrl": "https://app.cloudeval.com/app/chat?thread=8d5d..." } } ``` Agent Profiles are distinct from local CLI config profiles: * An Agent Profile is the reviewer role sent to Cloudeval: `architecture`, `cost`, `triage`, `remediation`, `visual-explainer`, `scripter`, `change-reviewer`, `evidence-auditor`, or `security-reviewer`. * A CLI config profile is local machine configuration: Cloudeval service URL, app URL, default project, model, output format, and local hooks. ### How Agent Profiles behave Cloudeval owns the profile catalog. Chat, CLI, and MCP all use the same public profile ids, labels, starter prompts, capability hints, and server-side answer contracts. | Field | What it controls | | ------------------------------- | ------------------------------------------------------------------------------------------------- | | `id` | Stable machine id used by Chat, CLI, and MCP. | | `display_name` | Public label shown in UI and command output. Some labels contain spaces. | | `description` and `personality` | Catalog copy and launch surfaces. | | `starter_prompts.template` | Default prompt for ARM/Bicep template projects. | | `starter_prompts.sync` | Default prompt for live sync projects. | | `required_capabilities` | Capabilities the client can show before running the profile. | | `default_settings` | Backend-owned run defaults such as agent mode, response depth, reasoning effort, and tool limits. | | `output_contract` | The shape of the answer Cloudeval asks the model to produce for that role. | ```mermaid theme={null} flowchart LR Choose["Choose a profile
Chat, CLI, or MCP"] --> Project["Select project context"] Project --> Prompt["Use custom prompt
or profile starter"] Prompt --> Lens["Cloudeval applies
the profile lens"] Lens --> Tools["Bias evidence collection
toward profile tools"] Tools --> Review["Review project evidence"] Review --> Answer["Return profile-shaped
recommendations"] ``` ### How profiles shape a run Agent Profiles do not create a separate product workflow. They shape the normal Cloudeval review path: 1. The user chooses one of the public profile ids listed above. 2. Cloudeval keeps the selected profile separate from selected resources. 3. Cloudeval applies the profile's focus, starter prompt, response style, output contract, and tool priorities. 4. The run uses the same project evidence, grounding rules, and safety checks as normal chat. 5. The final answer is shaped by the selected role without overriding user intent, accuracy, or citation expectations. ```mermaid theme={null} flowchart TD Start["User asks Cloudeval"] --> Profile{"Profile selected?"} Profile -- "No" --> Default["General project-aware answer"] Profile -- "Yes" --> Focus["Apply role focus,
settings, and answer contract"] Default --> Evidence["Read available project evidence"] Focus --> ToolBias["Prefer profile-specific
evidence tools"] ToolBias --> Evidence Evidence --> Safety["Apply grounding and safety checks"] Safety --> Response["Return answer with
next actions"] ``` No selected Agent Profile means the normal chat path remains in control. Cloudeval does not treat `Default` as a profile id. ### Same question, different lens Use the same plain-English question and choose a different profile when the decision context changes. ```bash theme={null} cloudeval agents run architecture "What needs attention first?" --project cloudeval agents run cost "What needs attention first?" --project cloudeval agents run triage "What needs attention first?" --project cloudeval agents run remediation "What needs attention first?" --project cloudeval agents run change-reviewer "Review this pull request" --project cloudeval agents run security-reviewer "Review exposure and identity risk" --project ``` Expected answer shape: | Profile | What the answer should emphasize | Typical next action | | ------------------- | --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | | `architecture` | Topology risk, dependency path, blast radius, availability, or Well-Architected tradeoff. | Validate the design boundary or dependency path before changing architecture. | | `cost` | Spend exposure, waste, idle capacity, expensive configuration, savings confidence, and period or unit when available. | Check the top spend driver and validate the highest-confidence savings opportunity. | | `triage` | Likely failure domain, affected path, containment, verification, and monitor or rollback signal. | Contain impact, verify the failure domain, then monitor or roll back. | | `remediation` | Ordered fix, owner expectation, dependency, rollout caution, and validation check. | Turn the highest-confidence finding into a sequenced fix plan. | | `visual-explainer` | Compact diagrams, dependency maps, before/after sketches, and evidence gaps. | Verify graph or report evidence before treating a diagram as confirmed topology. | | `scripter` | Copyable commands, scripts, workflow snippets, dry-runs, validation, and rollback. | Run a dry-run or validation command before applying a generated script. | | `change-reviewer` | PASS/WARN/BLOCK decision, current/base state, proposed change, live Azure evidence, and issue classification. | Resolve new regressions or evidence gaps before merging. | | `evidence-auditor` | Source coverage, freshness, scanner provenance, report timestamps, confidence, and Not assessed gaps. | Refresh stale evidence or weaken unsupported claims. | | `security-reviewer` | Public exposure, identity, network boundaries, secrets, encryption, policy gates, and attack paths. | Validate confirmed risks separately from hardening advice and missing evidence. | For example, a generic question such as "Can we trust this readout?" should not produce nine copies of the same response: * `architecture` should explain whether the topology and dependency evidence is enough to trust the architecture conclusion. * `cost` should explain whether the spend or savings evidence is strong enough to trust the financial conclusion. * `triage` should explain whether the evidence is enough to act during an incident or investigation. * `remediation` should explain whether the evidence is strong enough to assign work, sequence fixes, and validate completion. * `visual-explainer` should explain trust through graph-backed diagrams and explicit visual evidence gaps. * `scripter` should explain trust through safe commands, dry-runs, and verification checkpoints. * `change-reviewer` should separate current/base, proposed, and live Azure evidence before making a PASS, WARN, or BLOCK decision. * `evidence-auditor` should focus on source coverage, freshness, provenance, unsupported claims, and Not assessed areas. * `security-reviewer` should separate confirmed security risks from missing evidence and operational hardening advice. ### Web app report prompts Report prompt chips in the web app can launch chat with a profile already selected. The visible chat message should stay short, such as: ```text theme={null} Can we trust this readout? ``` Report details, evidence references, tab state, and prompt metadata should travel as chat context, not as a large visible message. That keeps the conversation readable while preserving grounding. When a prompt comes from a report, the selected profile should match the report area: | Report area | Recommended profile | Why | | ----------------------------------------------------------- | ------------------- | ---------------------------------------------------------------------------------------- | | Architecture, dependencies, or Well-Architected questions | `architecture` | The answer needs topology, dependency, blast-radius, and Well-Architected context. | | Visual walkthroughs, diagrams, or before/after explanations | `visual-explainer` | The answer should prioritize graph-backed visual explanation and evidence gaps. | | Cost and FinOps | `cost` | The answer needs spend, waste, rightsizing, and savings-confidence context. | | Active issue investigation or "what broke?" questions | `triage` | The answer needs failure-domain and containment framing. | | Action plan or fix sequencing | `remediation` | The answer needs owners, dependencies, rollout cautions, and validation checks. | | PR deltas, GitHub Actions failures, or merge readiness | `change-reviewer` | The answer must compare current/base, proposed, and live Azure evidence where available. | | Evidence trust, coverage, freshness, or unsupported claims | `evidence-auditor` | The answer should distinguish observed evidence from inference. | | Security exposure, identity, secrets, and attack paths | `security-reviewer` | The answer needs confirmed security risk separated from evidence gaps. | | Script, YAML, CLI, or runbook generation | `scripter` | The answer should produce copyable snippets with validation and rollback. | For report-triggered prompts, prefer one-shot controls: apply the profile, mode, project, and hidden context to that request, then return the chat input to the user's previous selection. ### Starter prompts by project source and mode When a user launches a profile from the Developer workspace or runs `cloudeval agents run ` without a prompt, Cloudeval picks a starter prompt based on the selected project source and selected mode. | Profile | Template ask starter | Template agent starter | Live sync ask starter | Live sync agent starter | | ------------------- | ------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | ------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | `architecture` | One architecture risk and first validation check. | Topology, dependency evidence, Well-Architected tradeoff, design action, and checkpoint. | One live architecture risk and first validation check. | Topology, dependencies, blast radius, Well-Architected evidence, and validation checkpoints. | | `cost` | One template cost risk and first savings validation check. | Cost evidence, expensive SKUs, waste risk, savings confidence, and validation checks. | One live cost risk and first savings validation check. | Current spend signals, idle resources, rightsizing actions, savings confidence, and checkpoint. | | `triage` | One likely deployment/runtime failure domain and first check. | Failure domain, blast radius, contain step, verify check, and rollback signal. | One likely active failure domain and first check. | Failure path, affected resources, contain step, verify check, and monitor/rollback signal. | | `remediation` | One template fix and first validation check. | Ordered edits, prerequisites, owners, dependencies, rollout cautions, and verification. | One live fix and first validation check. | Ordered actions, owners, dependencies, rollout cautions, and validation checks. | | `visual-explainer` | One graph-backed visual explanation and first evidence gap. | Topology, dependency map, blast radius, and recovery check for missing relationships. | One live topology explanation and first evidence gap. | Live graph-backed visual explanation, dependency map, and recovery check. | | `scripter` | One safe template command and validation check. | Dry-run commands, parameterized script, rollback, and verification. | One safe live command and validation check. | Read-only checks, dry-run commands, rollback, and verification. | | `change-reviewer` | One proposed-change risk and evidence gap. | Current/base, proposed change, changed resources, and review decision. | One source-to-live drift risk and evidence gap. | Current/base, proposed change, live Azure state, child-job freshness, and PASS/WARN/BLOCK decision. | | `evidence-auditor` | One evidence trust concern and recovery check. | Source coverage, freshness, provenance, unsupported claims, and Not assessed gaps. | One live evidence trust concern and recovery check. | Sync freshness, report timestamps, source coverage, and unsupported claims. | | `security-reviewer` | One security exposure risk and first verification check. | Exposure, identity scope, network boundaries, secrets, encryption, and policy gates. | One live security exposure risk and first verification check. | Public ingress, identity boundaries, sensitive paths, and policy failures. | Cloudeval can expose multiple starter variants for each source/mode pair. The Developer workspace randomizes the visible launch prompt when opening a profile. The CLI stays deterministic so scripts are repeatable and uses the first matching variant when no explicit prompt is passed. Pass an explicit prompt when you want the profile behavior but not the default starter text: ```bash theme={null} cloudeval agents run architecture \ "Review only public ingress, private networking, and dependency blast radius." \ --project \ --format json ``` ### MCP tools MCP-compatible agents can use the same catalog without shelling out to the CLI. The public tools are: * `agent_profiles_list` * `agent_profiles_get` * `agent_profiles_run` Use MCP when the agent already has a tool-calling loop and needs Cloudeval as a live evidence source. Use the CLI when the workflow is a script, CI job, or terminal session. ### Common mistakes * Do not send `default` as an Agent Profile id. Leave the profile selector empty for the normal chat flow. * Do not create a separate Well-Architected profile in integrations. Use `architecture`. * Do not treat stale or missing Cloudeval evidence as a passing `change-reviewer` result. Return an evidence gap instead. * Do not confuse `--profile codex` with `agents run cost`. The first selects local CLI config. The second selects Cloudeval reviewer behavior. * Do not paste long report contracts into visible chat messages. Send the short question as the message and attach report details as context. * Do not assume local hooks are Cloudeval-hosted automation. Hooks run on the CLI host only and can be bypassed with `--no-hooks`. ## Local CLI hooks Local hooks are opt-in commands stored in the active CLI config profile. They run on the local machine only; Cloudeval does not store, execute, or report hook runs in v1. Supported hook events: * `cli.command.before` * `cli.command.after` * `cli.command.error` * `agent_profile.run.before` * `agent_profile.run.after` * `agent_profile.run.error` Use `--no-hooks` on supported commands to bypass local hook execution for one run. Hook output goes to stderr so JSON and NDJSON stdout remain parseable. ## MCP server for agents Use `cloudeval mcp serve` when your agent framework already supports MCP and you want Cloudeval as a live tool server instead of a shell command wrapper. Check local MCP discovery first: ```bash theme={null} cloudeval mcp status --format json cloudeval doctor --mcp --format json ``` Example client configuration: ```json theme={null} { "mcpServers": { "cloudeval": { "command": "cloudeval", "args": ["mcp", "serve"] } } } ``` Cloudeval can also generate setup guidance for common clients: ```bash theme={null} cloudeval mcp setup codex --dry-run cloudeval mcp setup claude --dry-run cloudeval mcp setup cursor --dry-run cloudeval mcp setup vscode --dry-run cloudeval mcp setup generic --dry-run --toolset readonly --format json ``` `mcp setup --toolset` accepts `all`, `readonly`, `projects`, `reports`, or `billing`. Use `mcp serve --toolset graph` or `--toolset validation` when the agent needs those surfaces. Use `generic` for MCP-compatible clients that expect an `mcpServers` JSON entry. For Ollama-powered agents, configure the MCP host launched by Ollama with that generated Cloudeval stdio entry. Use focused toolsets when an agent only needs part of the Cloudeval surface: ```bash theme={null} cloudeval mcp serve --toolset readonly cloudeval mcp serve --toolset projects cloudeval mcp serve --toolset reports cloudeval mcp serve --toolset billing cloudeval mcp serve --toolset graph cloudeval mcp serve --toolset validation ``` Important rules: * The server uses `stdio`. * Authenticate with stored `cloudeval login` credentials, stored `cloudeval login --headless` credentials, or `--machine`. * Run login before starting `mcp serve`; stdin is reserved for MCP protocol messages. * Treat MCP tool results as the same Cloudeval data contract you would expect from the CLI: stable envelopes, returned IDs, and explicit errors. * Prefer focused MCP toolsets for assistants that should only inspect projects, reports, billing, or read-only data. * MCP clients that support resources and prompts can discover Cloudeval capabilities, project context, billing summaries, latest reports, and review-oriented prompt templates. * Agent Profile MCP tools are `agent_profiles_list`, `agent_profiles_get`, and `agent_profiles_run`; they use the same `architecture`, `cost`, `triage`, `remediation`, `visual-explainer`, `scripter`, `change-reviewer`, `evidence-auditor`, and `security-reviewer` ids as the CLI. ## Graph and validation automation Use graph commands for project intelligence that should not require opening the workspace UI: ```bash theme={null} cloudeval projects graph --format json --non-interactive cloudeval projects graph sync-runs --format json --non-interactive cloudeval projects graph insights --focus overview --format json --non-interactive ``` Use template validation commands before deployment-oriented automation: ```bash theme={null} cloudeval validate template \ --template-file ./azuredeploy.json \ --parameters-file ./azuredeploy.parameters.json \ --rule \ --details \ --wait \ --progress stderr \ --wait-timeout 600000 \ --format json \ --non-interactive cloudeval validate tests \ --template-file ./azuredeploy.json \ --parameters-file ./azuredeploy.parameters.json \ --wait \ --progress stderr \ --wait-timeout 600000 \ --format json \ --non-interactive cloudeval validate parse \ --template-file ./azuredeploy.json \ --parameters-file ./azuredeploy.parameters.json \ --format json \ --non-interactive cloudeval rules search "public network" --format json --non-interactive ``` `--parameters-file` is optional for both validation and parsing. Agents should pass it when a parameter file is present and omit it when defaults are enough. Use `rules search` or `rules show` to resolve check ids, then pass repeatable `--rule` values when an automation should run only specific checks. Use `--wait` for deployment gates that need final validation results instead of only a queued job id, and include `--wait-timeout` so the gate fails instead of hanging indefinitely. Add `--progress stderr` when a human should see queued/running/completed status while the command waits. Progress is written to stderr; final JSON remains on stdout for automation. Completed progress includes failing check/test details when the backend returns message, recommendation, severity, and location fields. Worker-local temp file paths are replaced with the submitted template filename so automation logs do not point at inaccessible backend files. Do not block automation solely because a parameters file is missing. ## Stable JSON envelope Cloudeval uses a stable JSON envelope for machine-readable success and error responses: ```json theme={null} { "ok": true, "command": "projects create", "data": {}, "frontendUrl": "https://cloudeval.ai/app/projects/..." } ``` ```json theme={null} { "ok": false, "command": "reports run", "error": { "message": "Authentication required" } } ``` Some commands also include fields such as: * `warnings` * `filesWritten` * `traceId` When you use `ndjson`, arrays are emitted one JSON object per line instead of one wrapped array payload. ## Ask mode vs agent mode Cloudeval supports two practical usage patterns: * `ASK` mode is best for one grounded answer, usually through `cloudeval ask`. * `AGENT` mode is best for multi-step workflows that may inspect projects, run reports, open deeplinks, or create Cloudeval artifacts when explicitly requested. Important guardrails: * ASK flows should stay read-first and should not silently create or change Cloudeval artifacts. * AGENT workflows can be broader, but they still need explicit intent before taking write actions. * Do not claim Cloudeval mutates customer cloud infrastructure unless a separately verified feature explicitly supports that behavior. ## Session continuity for agents Successful `ask` runs create local, profile-scoped session history. Use it when an agent needs to find or continue recent Cloudeval work on the same machine. ```bash theme={null} cloudeval sessions search "cost review" --profile codex --format json cloudeval sessions rename "Cost review" --profile codex --format json cloudeval chat --resume "Cost review" --profile codex cloudeval ask "Continue the same investigation" --thread --profile codex --format json --non-interactive ``` Rules for session use: * Session history is local to the machine and scoped by profile. * Use `sessions search` before assuming a thread ID. * Use `sessions rename` to make important threads easy to find later. * Use `ask --thread` only when a one-shot follow-up should stay attached to an existing conversation. ## Grounding model Cloudeval answers are expected to be grounded in the data the product actually has access to. That can include: * project metadata * connection metadata * ARM or Bicep-derived ARM template content * resource graph and diagram relationships * saved cost reports * saved architecture or Well-Architected reports * report history and trend data where available * pricing and product metadata * chat thread history * local CLI session history for one-shot `ask` runs If the evidence is missing, the right behavior is to say what is missing and suggest the next useful command. ## Authentication and permissions * `cloudeval login` uses a browser-based login flow. * `cloudeval login --headless` uses a device-code flow for headless sessions. * Browser-based CLI login is restricted to loopback redirect targets on the local machine. * Use `cloudeval login --headless` for SSH, containers, or remote terminals. * Use stored login state or `--machine` when you run `cloudeval mcp serve`. * Always use IDs returned by Cloudeval responses. Do not guess project, report, connection, or thread IDs. ## Practical limits * Azure is the primary supported provider today. * ARM JSON is the strongest current IaC path. * AWS and GCP should not be treated as full-parity live sync or reporting paths unless current capabilities confirm it. * Diagram freshness depends on the latest successful import or sync. * Cost outputs can be estimates, not final billing truth. * Architecture and security findings are evaluations, not compliance attestations. * Some browser workflows are still easier or only available in the web app. ## Verification before production use Before shipping a new integration: 1. Run `cloudeval capabilities --format json`. 2. Run `cloudeval doctor --format json` for the profile or environment you will use. 3. Run `cloudeval doctor --mcp --format json` if an MCP client is part of the workflow. 4. Test the exact commands you plan to automate with `--format json --non-interactive`. 5. Confirm the target project, report, connection, or thread IDs come from Cloudeval output. 6. Check that your workflow handles auth-required, service-unavailable, and not-found failures cleanly. ## Next step Use [llms.txt and llms-full.txt](/agents/context-files) for the public context files, or [CLI command reference](/cli/commands) for the exact command groups and flags.