LLaManchaAI Enablement
Checking access…
← All modules
Advanced Module 9Optional advanced track

Advanced AI-assisted software delivery

Optional technical track for coding, testing, review, CI/CD, documentation, runbooks, and incident support with validation gates.

Outcomes

What you will be able to do

  • Use agentic coding assistants inside engineering controls.
  • Drive AI with tests, not just accept AI-written code.
  • Account for supply-chain, security, and cost risk in AI-assisted dev.
  • Document validation before accepting output.
Completion check

How this module is approved

Complete one sanitized technical lab; submit the prompt, output, the controls it sat inside, and validation notes.

Pass criteria

  • Uses sanitized technical example
  • Names the tool and the controls it operated within
  • Includes AI output or summary
  • Includes validation notes appropriate to the risk
  • Identifies human review gate
  • Does not include sensitive/proprietary details
Loading personalization context…
Lesson50–65 minutes self-guided, or 80–95 minutes with technical lab discussion

What you should take away

By the end of this advanced optional module, technical participants should be able to use AI tools for software delivery acceleration while preserving engineering discipline: small scope, sanitized context, tests, code review, security checks, and documented validation before anything ships.

Part 1

AI can accelerate delivery, but it does not lower the bar

AI-assisted software work is useful because it can reduce blank-page time, suggest alternatives, explain unfamiliar code, draft tests, summarize logs, and help produce documentation. It is dangerous when teams treat polished output as verified work. The delivery bar stays the same: correctness, security, maintainability, accessibility, performance, observability, and review.

  • Use AI to accelerate thinking and drafting, not to bypass engineering judgment.
  • Prefer small, reviewable tasks over broad prompts like 'fix this app.'
  • Keep humans responsible for requirements, architecture decisions, test coverage, security, and production readiness.
  • If the AI output cannot be tested or reviewed, it is not ready to trust.

Quick check · <30 sec

AI accelerates delivery by lowering which of these?

  • A. The correctness/security/review bar
  • B. Blank-page and drafting time, not the delivery bar
  • C. The need for tests
  • D. Architectural accountability
Show answer
It compresses time-to-first-draft. Correctness, security, tests, and review stay exactly where they were.

Part 2

AI in your IDE: what changed in the last 12 months

The 2024 mental model — autocomplete that suggests the next line — undersells where tools are in 2026. Agentic coding assistants can read a repo, plan a change, edit multiple files, run tests, and iterate, with a human approving steps. This is more capable and more dangerous; the controls matter more, not less.

  • Inline-suggestion assistants (e.g., GitHub Copilot completions) — fast, low blast radius, still needs review.
  • Agentic assistants (e.g., Claude Code, Cursor, Aider, Codex-style CLIs, Copilot agent mode, Windsurf, Devin-style agents) — can take multi-step actions across a codebase; verify current capabilities in your approved tool rather than assuming.
  • The rule is unchanged: an agent operates inside your existing controls — branch protection, review, CI gates, least-privilege credentials — never around them.
  • Smallest-safe-workflow (Module 6) applies directly: scope the task, keep the human on merge and deploy.

Quick check · <30 sec

An agentic coding assistant should operate…

  • A. With production credentials so it can self-deploy
  • B. Inside existing controls — branch protection, review, CI — with a human on merge/deploy
  • C. Outside CI to move faster
  • D. With PR-approval rights
Show answer
More capability raises the stakes of the gates. The agent works within them; humans keep merge and deploy.

Part 3

Good technical use cases

AI is strongest in software delivery when the problem can be framed clearly and the result can be checked. It is especially helpful for scaffolding, translation between formats, test generation, documentation, refactoring options, and troubleshooting plans.

  • Generate first-pass unit tests from a described function contract.
  • Explain a stack trace or log excerpt after secrets and identifiers are removed.
  • Draft a pull request summary, release note, runbook, or incident follow-up from sanitized notes.
  • Refactor a small function for readability while preserving behavior.
  • Compare implementation options and list tradeoffs, risks, and validation steps.
  • Create a checklist for code review, migration, deployment, or rollback planning.

Quick check · <30 sec

Which framing is most likely to produce checkable AI output?

  • A. "Fix this app"
  • B. "Propose unit tests for this described, sanitized function contract"
  • C. "Make the code better"
  • D. "Rewrite the service"
Show answer
A narrow, framed task with a checkable result. Broad prompts produce broad, unverifiable output.

Part 4

Tests with AI, not just code from AI

The highest-leverage technical AI pattern is not 'AI writes the code.' It is 'AI helps write the tests, then code is judged against them.' Test-first keeps the human owning the specification and turns the AI's output into something automatically falsifiable.

  • Describe the behavior contract (sanitized), ask AI for tests and edge cases first — you review the tests as the spec.
  • Then have AI (or you) implement; the tests, not the prose, decide acceptance.
  • AI is good at enumerating edge cases humans skip; it is bad at knowing which matter — you still curate.
  • A generated implementation with no failing-then-passing test is unproven, not done.
Activity · ~10 minbuild artifact

Test-first on a sanitized contract: a function returns true only if a string has exactly one '@', non-empty text on both sides, and ends in an approved domain. Do NOT ask for the implementation yet.

  • Contract: one '@', text before and after, ends with an approved domain (domain list is a parameter)
  • Rule: Tests and edge cases first; you curate before any code exists

Your task

Write the prompt that asks only for unit tests + edge cases (no implementation), then list which generated cases you would keep, add, or cut and why.

Show a hint
Expect valid, missing-@, multiple-@, empty-side, wrong-domain, uppercase-domain, whitespace, and subdomain cases — decide which are in-contract.
Compare with a strong answer
Prompt: 'Given this behavior contract, propose unit tests and edge cases only — no implementation. Do not invent APIs. List assumptions.' Keep: valid, missing @, multiple @, empty local/domain part, wrong domain, whitespace. Add: subdomain of an approved domain (under-specified — I must clarify the contract). Cut: 'internationalized domain' as out of scope for now. The act of curating tests forced the contract ambiguity into the open before any code was written.

Why this matters: Driving with tests is the discipline that makes AI-assisted coding safe; writing code-first and testing later is where the failures cluster.

Quick check · <30 sec

In test-first AI development, what decides acceptance?

  • A. The AI saying it works
  • B. The curated tests passing (after first failing)
  • C. The code reading well
  • D. It compiles
Show answer
The human-curated tests are the spec; red-then-green against them is acceptance, not fluent-looking code.

Part 5

High-risk technical patterns

Some AI-assisted software patterns require extra caution or should be prohibited unless the company has explicitly approved the tool, data flow, and review process. Technical teams need to be especially careful because code, logs, tickets, and infrastructure details often contain sensitive information.

  • Do not paste secrets, tokens, credentials, private keys, customer data, proprietary source code, incident details, or infrastructure diagrams into unapproved tools.
  • Do not accept AI-generated code without running tests and reviewing for security, licensing, performance, and maintainability.
  • Do not ask AI to make unsupervised production changes, approve pull requests, alter access controls, or bypass CI/CD gates.
  • Do not trust generated package names, commands, APIs, or configuration snippets without checking official docs or the actual repo.
  • Do not use AI output to conceal authorship, evade review, or create misleading technical evidence.

Quick check · <30 sec

AI suggests installing a package you have never heard of. First move?

  • A. Install it; the AI is usually right
  • B. Verify the package exists, is the real maintainer, and is license/security-acceptable before adding it
  • C. Install and run tests
  • D. Ask AI if it's safe
Show answer
Hallucinated or typo-squatted package names are a real supply-chain risk. Verify existence, provenance, and license before adding a dependency.

Part 6

A practical AI-assisted delivery loop

The safest pattern is to keep AI inside an explicit delivery loop: frame the task, provide sanitized context, request a narrow output, inspect the result, run validation, revise, and document what was checked. This turns AI from a shortcut into a disciplined assistant.

  • Frame: describe the goal, constraints, language/framework, and acceptance criteria.
  • Sanitize: remove secrets, customer data, proprietary code, private URLs, and internal identifiers unless using an approved protected tool.
  • Generate: ask for a small artifact: test cases, review checklist, refactor suggestion, troubleshooting plan, or draft documentation.
  • Inspect: compare the output against requirements, repo conventions, security expectations, and existing patterns.
  • Validate: run tests, linters, type checks, build checks, static analysis, or manual reproduction steps as appropriate.
  • Document: note what AI helped with and what human validation was performed.

Quick check · <30 sec

Which loop step is most often skipped under time pressure — and most dangerous to skip?

Show answer
Validate (and the Document of what was validated). Skipping it is exactly how unverified AI output reaches production looking finished.

Part 7

Prompt patterns for technical work

Technical prompts should be narrower and more explicit than general business prompts. Ask for assumptions, risks, and validation gates every time. When you need code, request small diffs or isolated examples instead of a wholesale rewrite.

  • Test generation: 'Given this sanitized behavior description, propose unit tests, edge cases, and mocks. Do not invent external APIs.'
  • Code review: 'Review this sanitized snippet for readability, error handling, security concerns, and missing tests. Return findings by severity.'
  • Troubleshooting: 'Given these redacted logs and symptoms, list likely causes, checks to run, and what evidence would confirm or rule out each cause.'
  • Refactoring: 'Suggest a minimal refactor that preserves behavior. Explain tradeoffs and tests I should run.'
  • Documentation: 'Turn these sanitized implementation notes into a runbook with prerequisites, steps, rollback, and validation.'

Quick check · <30 sec

Why request a small diff instead of a wholesale rewrite?

Show answer
A small diff is reviewable and testable in isolation; a wholesale rewrite hides behavior changes and defeats the validation step.

Part 8

AI in CI and the software supply chain

AI in delivery is not only in the editor; it increasingly appears in the pipeline (review bots, doc generation, incident summarization) and it changes your dependency risk. Two disciplines keep this safe: treat pipeline AI as advisory until validated, and treat AI-suggested dependencies as untrusted until vetted. Cost is the third axis — bigger models and agentic loops are not free.

  • Pipeline AI (review-comment bots, generated changelogs, incident summaries) is advisory input to a human gate, never an approver.
  • Supply chain: AI may hallucinate or typo-squat package names; verify existence, maintainer, license compatibility, and that it appears in your SBOM/scan before adoption.
  • Security: generated code can carry injection, secret-handling, or auth flaws that read as clean — security review scales with risk, and security-relevant changes escalate.
  • Cost discipline: prefer the smallest model that passes; cap agentic iteration; batch where possible. 'It worked' and 'it was worth the spend' are different acceptance questions.
Activity · ~8 minrole play review

A teammate's PR was produced with an agentic assistant. You are the reviewer. The diff adds a new dependency and a function with no test. Apply the controls from this module.

  • PR contents: new third-party package 'fastvalidate-pro'; a validation function; updated lockfile; no new tests; green CI (no tests cover the new code)
  • Your gates: tests, security, supply chain, review, cost

Your task

List the specific objections you raise before approving, mapped to the controls. State what would have to be true to approve.

Show a hint
Green CI with no tests covering new code is a false-completeness signal, not a pass. The unknown package is a supply-chain stop.
Compare with a strong answer
Objections: (1) Supply chain — 'fastvalidate-pro' unverified: confirm it exists, maintainer/provenance, license compatibility, SBOM/scan entry; possible typo-squat. (2) Tests — new code has none; green CI proves nothing here; require failing-then-passing tests. (3) Security — review the validation logic for bypass/injection. (4) Authorship/validation — PR must state what was AI-generated and what was validated. Approve only when the dependency is vetted, tests cover the behavior, security signs off if risk warrants, and validation is documented.

Why this matters: Agentic output looks finished and green. Reviewing one adversarially is the skill that keeps the controls real instead of ceremonial.

Quick check · <30 sec

Green CI on an AI-generated PR with no tests covering the new code means…

  • A. It's verified — approve
  • B. Nothing about the new code — it is unproven, not validated
  • C. The tests passed
  • D. CI is broken
Show answer
Passing CI that does not exercise the new code is false completeness. Unproven is not validated.

Part 9

Validation before acceptance

The submission standard for AI-assisted technical work should be simple: no validation, no acceptance. Validation does not need to be elaborate for every task, but it must fit the risk of the change. A generated release note needs human review. A generated function needs tests. A generated deployment plan needs operational review and rollback thinking.

  • For code: run relevant tests, type checks, linting, and peer review.
  • For troubleshooting: verify against logs, metrics, source systems, or reproduction steps.
  • For documentation: confirm commands, paths, owners, dates, URLs, and rollback steps.
  • For security-sensitive work: involve security/architecture review and avoid unapproved data exposure.
  • For production changes: keep normal approvals, CI/CD gates, and change-management expectations intact.

Quick check · <30 sec

The submission standard for AI-assisted technical work in one phrase:

Show answer
No validation, no acceptance — and the validation must fit the risk of the change.

End-of-module quick check

Five short retrieval questions. Answer from memory first, then reveal each explanation.

  1. 1. An agentic coding assistant should…

    • A. Hold PR-approval and deploy rights to move fast
    • B. Operate inside branch protection, review, and CI with a human on merge/deploy
    • C. Run with production credentials
    • D. Bypass CI when confident
    Show answer
    More capability raises the stakes of the gates; the agent works within them.
  2. 2. In test-first AI development, acceptance is decided by…

    • A. The code reading well
    • B. Human-curated tests failing then passing
    • C. The AI's confidence
    • D. Successful compilation
    Show answer
    Curated tests are the spec; red-then-green is acceptance.
  3. 3. AI suggests an unfamiliar dependency. The supply-chain-safe response is…

    • A. Install it
    • B. Verify existence, maintainer, license, and SBOM/scan before adding it
    • C. Install and run tests
    • D. Trust it if CI is green
    Show answer
    Hallucinated/typo-squatted packages are a real risk; vet provenance and license before adoption.
  4. 4. True or false: green CI on an AI-generated PR with no tests on the new code means it's validated.

    • A. True
    • B. False
    Show answer
    False. CI that does not exercise the new code is false completeness — unproven, not validated.
  5. 5. State the submission standard for AI-assisted technical work.

    Show answer
    No validation, no acceptance — and the validation must fit the risk of the change.

Further reading

Worked examples by role

Software developer

Agentic assistant, inside controls

Used an approved agentic assistant on a feature branch to scaffold a parser change. Drove it test-first: curated the generated tests as the spec, let it implement, ran the suite + linters locally. It is on a protected branch; peer review and CI gate the merge; the assistant has no PR-approval or deploy rights. PR notes what was AI-generated and what was validated.

Operations manager

Incident summary from sanitized signals

Ops-engineering use: AI drafts an incident timeline and follow-up actions from redacted logs and metrics (secrets/identifiers removed). Treated as advisory — the on-call lead verifies each claim against the source systems before it enters the postmortem. AI never decides severity or closes the incident.

Training manager

Engineering-onboarding runbook

Eng-enablement use: turned sanitized setup notes into a runbook (prereqs, steps, rollback, validation). Validation: every command and path confirmed against a real environment, accessibility/reading-level pass, a senior engineer reviews before it goes to new hires. No internal URLs or credentials in the draft.

Completion artifact

Complete one sanitized technical lab. Submit the prompt, the AI output or summary of output, and validation notes. The example may involve tests, review, documentation, troubleshooting, or refactoring, but it must not include proprietary code, secrets, customer data, credentials, internal URLs, or sensitive operational details.

ExercisePilot-ready artifact

Complete one sanitized technical lab. Use AI for a narrow software-delivery task, then document the output and validation path before accepting it.

Participant template

  • Technical task:
  • Sanitized context:
  • Tool used (inline / agentic) and the controls it sits inside:
  • Prompt used:
  • AI output or summary:
  • Validation performed or planned (tests / security / supply chain):
  • Human review gate:
  • Cost/scope note (why this scope/model was appropriate):
  • Sensitive details intentionally excluded:

Example submission

Technical task: propose tests for a utility that validates work email strings (test-first). Sanitized context: behavior rules only; no proprietary code. Tool: approved inline assistant; change on a protected branch, CI + peer review gate merge. Prompt: propose edge-case unit tests only, no implementation, include assumptions. Output: valid, blank, missing @, multiple @, wrong domain, uppercase domain, whitespace. Validation: curate cases, add subdomain case (clarified contract), run suite red-then-green before accepting any implementation. Cost note: trivial task, smallest model. Excluded: real domains, repo code, credentials, internal URLs.

Role-flavored variants

Same exercise, framed for different roles. Use the one closest to your work.

Software developer

Use a test-first task with an agentic assistant. The controls line must name branch protection / review / CI; the validation line must show red-then-green.

See a sample submission
Developer lab (excerpt): Task: add input validation via test-first. Tool: approved agentic assistant on a protected feature branch; CI + peer review gate merge; no PR-approval rights. Prompt: tests + edge cases only, then implement to pass. Validation: tests failed then passed, lint/type clean, security skim of the validation path. Cost note: small model sufficed; capped to 2 agent iterations. Excluded: repo code, secrets, internal URLs.

Operations manager

Use a sanitized incident-support task. The validation line must show each AI claim checked against a source system; AI stays advisory.

See a sample submission
Ops-eng lab (excerpt): Task: draft an incident timeline from redacted logs/metrics. Tool: approved chat tier (no system access). Prompt: build a timeline + candidate causes, list evidence to confirm each. Validation: every claim verified against the real dashboards before the postmortem; AI did not set severity. Review gate: on-call lead. Excluded: secrets, identifiers, internal hostnames.

Learner checklist

Use this as a final check before submitting. Program leads use a separate review guide when they approve or coach submissions.

  • Uses sanitized technical example
  • Names the tool and the controls it operated within
  • Includes AI output or summary
  • Includes validation notes appropriate to the risk
  • Identifies human review gate
  • Does not include sensitive/proprietary details
Loading your module status…

Previous module

Business AI policy path

Review the prior step in the path.