Agent 安全不是加規則

Cold open / the case thread

它沒有叛變，它只是照做了

It did not rebel. It followed the path we gave it.

我們用同一個 customer-success agent 貫穿全場：它讀 CRM、整理客戶脈絡、草擬回覆。demo 很漂亮，但真正問題不是它聰不聰明，而是組織能否證明它受控。

Bad answer vs bad outcome

安全邊界不是它說了什麼，而是它做了什麼

The boundary moves from what it said to what it did

• 答錯摘要，是內容品質問題。A bad summary is a content-quality issue.
• 寄錯客戶、改錯權限、付款錯誤，是營運事故。Wrong sends, permission changes or payments become operational incidents.
• Agent 安全要同時處理 intent、permission、execution、rollback。Agent security covers intent, permission, execution and rollback.

Checkpoint 1 / classify the system

先分清楚：它到底是不是 Agent？

Before governance, classify the thing you built

同一個客戶跟進場景，如果只是固定步驟，它是 workflow；如果它要判斷資料、選工具、處理例外，才進入 agent security model。

Agent worthiness test

四個問題決定要不要升級

Four questions before autonomy

• Job 是否清楚，不是「萬能助手」？Is the job clear, not an all-purpose assistant?
• Context 是否會變，而且需要判斷？Does context vary enough to require judgement?
• Tool risk 是否可限制與觀察？Can tool risk be constrained and observed?
• Eval 是否能在上線前先寫出來？Can eval cases be written before launch?

Best practice

先做小，再做穩，再擴權

Start small, make it stable, then grant authority

• 第一版只處理一個高頻、低後果的任務。V1 handles one frequent, low-consequence job.
• 先建立 context pack，再接工具。Define the context pack before tools.
• 先測拒絕與追問，再測成功。Test refusal and clarification, not only success.

Identity model

這個 Agent 到底代表誰？

Who does this agent represent?

• 代表用戶本人？代表團隊？代表服務帳號？User identity, team identity, or service identity?
• 使用短期 token，而不是長期萬能 key。Use short-lived tokens, not permanent broad keys.
• 不同 agent 應有不同權限，不靠「信任模型」解決。Different agents need different scopes; do not rely on trusting the model.

Permission ladder

權限不是開關，是樓梯

Permission is a ladder, not a switch

• Read：查詢、摘要、比對。Read: lookup, summarise, compare.
• Draft：準備 email、ticket、calendar、PR。Draft: prepare email, ticket, calendar or PR.
• Write：內部、可復原、有限額。Write: internal, reversible, thresholded.
• External/Admin：發送、付款、刪除、deploy、改權限。External/Admin: send, pay, delete, deploy, change permissions.

Checkpoint 2 / permission matrix

第一版權限要細到可以辯護

V1 authority must be small enough to defend

故事中的 agent 想讀 CRM、草擬 email、更新 ticket、建立 meeting。不是問「可不可以」，而是問：每一格權限有沒有清楚理由、上限、證據與復原路徑？

Case branch / Bad tool design

工具開太大，事故半徑也會變大

Broad tools create broad accidents

• Full Gmail access 讓讀取與發送混在一起。Full Gmail access mixes reading with sending.
• Raw SQL 讓查詢、修改、刪除共用同一把刀。Raw SQL merges lookup, mutation and deletion.
• Shell access 讓 package、file、network、secret 風險一起放大。Shell access expands package, file, network and secret risk at once.

Good tool contract

工具介面要窄、可 dry-run、可追蹤

Narrow, dry-run capable, traceable

• 用 create_email_draft，不用 full_email_access。Use create_email_draft, not full_email_access.
• 驗證 arguments、destination、scope、policy version。Validate arguments, destination, scope and policy version.
• 每次 tool call 都留下 reason、source、result、rollback path。Every tool call records reason, source, result and rollback path.

Checkpoint 3 / contract rewrite

把一個大工具拆成可審計的小工具

Rewrite broad access into narrow, reviewable tools

把「讓 agent 用 Gmail」改成一組有邊界的 tool contract。目標不是令 agent 更方便，而是令每一次行動都可被檢查。

Approval UX

不要只問「Are you sure?」

Approval must show the actual side effect

• 誰在批准？agent 代表誰？Who approves, and who does the agent represent?
• 將要對誰做什麼？內容與來源是什麼？What action, target, content and source?
• 影響範圍、可否復原、policy reason 是什麼？Impact, recoverability and policy reason?

Bypass rules

可以略過批准，但不能略過紀錄

Approval can be skipped; logging cannot

• 低風險、read-only、可復原、範圍清楚，可以自動。Low-risk, read-only, reversible and bounded actions can be autonomous.
• 涉及客戶、金錢、敏感資料、外部發送、不可逆，必須批准。Customers, money, sensitive data, external sends and irreversible actions require approval.
• 略過批准也要留下 threshold、policy、trace。Skipped approval still needs threshold, policy and trace.

Checkpoint 4 / approval evidence

重寫這個批准畫面

Redesign the approval prompt into evidence

壞例子：「Agent wants to send an email. Are you sure?」好例子要顯示 recipient、source、claim、impact、rollback。

Case branch / Hidden instruction

客戶 email 不是你的系統指令

Customer content is not system authority

• Email、網頁、文件、ticket 都是不可信輸入。Email, webpages, documents and tickets are untrusted input.
• 內容可用來理解事實，不可改變 policy。Content may inform facts, not change policy.
• 防線在 tool policy，不只是在 prompt wording。The defense is tool policy, not prompt wording alone.

Instruction hierarchy

先分清楚：誰有權下指令？

Separate instruction from content

• System / developer policy：不可被外部內容覆寫。System and developer policy cannot be overridden by external content.
• User intent：由真人目標定義。User intent comes from the human goal.
• Tool result / document：是資料，不是命令。Tool results and documents are data, not commands.

Checkpoint 5 / authority boundary

找出哪一句只是資料，哪一句在偷授權

Separate facts from attempted authority

給 audience 一段混合 email：客戶需求、真實資料、隱藏指令、外部連結。請標記哪些是 facts，哪些是 attempted authority。

Sensitive context

不是 agent 看得到，就代表它需要看

Visibility is not necessity

• API key、password、token、cookie 不進 prompt。API keys, passwords, tokens and cookies do not belong in prompts.
• PII、財務、HR、客戶資料要分級與遮罩。PII, finance, HR and customer data need classification and redaction.
• Logs 要可調查，但不能變成秘密外洩副本。Logs must support investigation without becoming secret copies.

Case branch / Exfiltration path

外洩通常不是單一工具造成

Leaks often come from tool combinations

• 讀 CRM + 發 email + call URL，就可能形成外洩路徑。CRM read + email send + URL call can form an exfiltration path.
• 同一個 run 不應任意混合 sensitive read 與 external write。A run should not freely combine sensitive reads with external writes.
• 工具之間需要 sink restrictions。Tools need sink restrictions between data sources and destinations.

Containment defaults

Browser、files、code、network 要分層隔離

Separate browser, files, code and network

• Filesystem：只 mount 任務需要的資料夾。Mount only task-required folders.
• Browser：隔離 session，不碰內網與 localhost。Isolate sessions; avoid internal networks and localhost.
• Network：egress allowlist，擋 metadata/internal CIDR。Use egress allowlists; block metadata and internal CIDRs.
• Code：shell、package、admin action 要隔離與批准。Shell, packages and admin actions need isolation and approval.

Recovery mode

Kill switch 不能靠 prompt

A kill switch lives outside the model

• Stop = revoke sessions, tokens, queued actions, tool access。Stop means revoking sessions, tokens, queued actions and tool access.
• Circuit breaker：read-only、approval-only、full stop。Circuit breakers degrade to read-only, approval-only or full stop.
• 先保留 evidence，再清 memory 或 rotate credentials。Preserve evidence before clearing memory or rotating credentials.

Checkpoint 6 / blast-radius design

替故事中的 agent 設 sandbox default

Choose the default-deny boundaries

它要讀 CRM、草擬回覆、安排 follow-up。哪些 filesystem、browser、network、production access 應該一開始就關掉？

Trace anatomy

一條可審計的 action trail

An action trail that can be investigated

• 記錄 user intent、sources、plan、tool calls。Record user intent, sources, plan and tool calls.
• 記錄 policy version、approval screen、approver、final action。Record policy version, approval screen, approver and final action.
• 不要讓 agent 修改自己的 audit trail。Do not let the agent modify its own audit trail.

Eval set

Eval 不是只測成功

Evals must include refusal, attack and recovery

• 成功案例：它能完成正確任務。Success: it completes the intended job.
• 拒絕案例：它知道哪些要求不可做。Refusal: it knows what not to do.
• 攻擊案例：它不會把不可信內容當成指令。Attack: it does not treat untrusted content as authority.
• 復原案例：出錯後能停止、回復、調查。Recovery: it can stop, recover and support investigation.

Checkpoint 7 / eval evidence

替它寫三個能說服風控的 eval

Write evals that prove control, not just usefulness

一個成功、一個拒絕、一個攻擊。每個 eval 都要有 input、expected behavior、trace signal。

Board questions

上線前，管理層要問同一條主問題

The boardroom asks one repeated question

• 它代表誰？用什麼 token？Who does it represent and which token does it use?
• 能碰哪些資料、工具、客戶、金錢、權限？Which data, tools, customers, money and permissions can it touch?
• 如果它做錯，我們能否停止、復原、解釋？If it acts wrongly, can we stop, recover and explain?

Final board gate

你會批准上線嗎？

Ship, delay, or block?

回到故事中的 customer-success agent。你現在不是憑感覺判斷；你手上有 permission matrix、tool contract、approval threshold、injection tests、audit schema。

Evidence kit

這堂課的交付物是一份上線證據包

The takeaway is a production evidence package

• Agent permission matrix
• Tool contract checklist
• Approval threshold table
• Prompt-injection test checklist
• Audit log schema
• Incident mini-runbook

Closing principle

Agent-ready = 組織能證明它受控

Agent-ready means the organization can prove control

Build 的核心不是「安全清單背熟了」，而是能把一個會行動的 agent 逐層證明：誰授權、做什麼、邊界在哪、證據在哪、出事如何停。