From incident diagnosis to action: what an SRE-agent FDE must control

Read the actual work and seniority

PagerDuty's listing is titled Forward Deployed Engineer (SDEV). Its body describes a Staff Engineer working on the SRE Agent and assesses experience against Staff Engineer / SDE V expectations. It asks for eight or more years shipping production software, production LLM and agent experience, evaluation pipelines, guardrail design and customer-facing engineering. This is an experienced-engineer reference for preparation, rather than an entry-level route created by completing a small project.^[1]

The work includes connecting customer observability, infrastructure-as-code, GitOps and CI/CD systems, then helping teams move from AI-assisted incident response to supervised and autonomous actions. The posting names canary, blue/green and soak delivery among basic qualifications. Policy-as-code and fine-grained permissioning appear under preferred qualifications. It is a San Francisco role with office or client-site attendance and travel requirements; company-wide Canadian location text does not establish Canadian eligibility for this vacancy.^[1]

Separate diagnosis from authority to change state

For a proposed local exercise, use fabricated service telemetry and a simulated executor. Give the diagnostic tool permission to read that telemetry without changing service state. Let the agent prepare a remediation proposal containing an action, target, parameters, expected effect and evidence. A plausible diagnosis should leave the simulated service unchanged until a separate authorization check permits execution. This design translates the posting's permissions and guardrail responsibilities into a learning task; it is not PagerDuty's published internal design.^[1]

Write an action-permission matrix before adding tools. As an illustrative policy, reading a synthetic error counter is read-only; restarting one named mock worker requires approval; deleting data or changing identity permissions is excluded from automatic execution. Choose the classes for your own simulation and explain the reason for each. Do not use customer systems, real credentials or a live cloud account to make the exercise appear more realistic.^[1]

Bind approval to the action that was reviewed

An approval record in this proposed exercise should identify the exact target, action parameters, plan version and validity period. If the agent changes the target or the proposed command, require a new decision. Treat missing approval, expired approval and insufficient telemetry as reasons to leave state unchanged. Show the executor refusing an unauthorized state change as well as permitting an authorized one. These are editorial test cases derived from the role's guardrail work, not disclosed employer assessment questions.^[1]

Include a repeated-request case. After one permitted simulated action, send the same request again and check whether the executor prevents an accidental second execution. Record the actual behavior once you build it, including any failure. Until then, present the test matrix as planned work. A diagram, generated transcript or expected-result table cannot establish that the permission boundary works.^[1]

Specify the rollout and abort decision

Use one bounded, reversible simulated change to practise progressive delivery. Define the small initial target, the telemetry required before starting, the observation interval, the condition for continuing and the condition for aborting. Make those choices explicit rather than inventing a universal safe error threshold. PagerDuty names progressive delivery as a requested capability; the correct thresholds for a real service would depend on that service and its owners.^[1]

Plan a failing-canary case in which simulated health worsens. Check that the system stops further rollout, records the reason and follows the mock recovery policy. Also plan a missing-telemetry case: an absent measurement should not silently count as a healthy result. Keep the agent's recommendation, the authorization decision, the execution result and the later state observation as separate records so a reviewer can locate a disagreement.^[1]

Present the evidence and its limits

If you complete the exercise, share the action matrix, state diagram, real pass/fail results and sanitized traces, with a short explanation of the rollout and abort decisions. Label every fake connector and every untested production assumption. Use the work to discuss how you constrain mutations and communicate operational risk. One job posting supports this task choice; it establishes neither market-wide demand nor a hiring benefit. No Jobby sandbox execution, employer evaluation or production safety result is reported here.^[1]