AI Usage &
Capability Framework
for Engineers
A model for how engineers work with AI effectively, repeatably, and safely — supporting assessment, coaching, calibration, and adoption planning.
Executive Summary
This framework answers one specific question: how well does an engineer work with AI? It does not re-label the software development lifecycle as separate AI dimensions. Coding, testing, design, documentation, and operations are evidence areas where AI capability shows up — they are not the dimensions themselves.
A person should not be considered high-maturity if they delegate heavily but cannot properly verify results or manage risk. Evaluation and Governance act as guardrail dimensions in the scoring model.
Proposed Dimensions
- Framing — turn ambiguous work into AI-suitable tasks
- Delegation — safely hand work to AI
- Evaluation — verify correctness, fit, and risk
- Operationalization — make useful AI behavior repeatable
- Governance — work within policy, privacy, security, IP
Proposed Levels
- Level 0 — Ineffective or unsafe use
- Level 1 — Ad hoc assistant use
- Level 2 — Structured individual use
- Level 3 — Repeatable high-leverage use
- Level 4 — System builder / standard setter
What This Framework Is and Isn't
This framework is
- A way to assess AI usage maturity for engineers
- A coaching tool for growth plans
- A calibration tool for managers
- A structure for team expectations and enablement
- A way to separate "uses AI a lot" from "uses AI well"
This framework is not
- A replacement for the engineering career ladder
- A measure of general engineering seniority
- A count of how many AI tools someone has tried
- A reward for prompt or code-generation volume
- A substitute for domain expertise or product judgment
An engineer can be highly capable in their core discipline and still be early in AI maturity. The reverse is also true.
Why This Model Changed
Traditional matrices break skill into coding, testing, system design, and tooling. That structure works for general engineering, but creates overlap for AI usage.
Instead of scoring where in the SDLC AI is used, this framework scores how the engineer works with AI.
Grounded in four ideas
Cumulative levels
Higher levels build on lower ones.
Responsibility ≠ skill
SFIA separates generic responsibility attributes (autonomy, influence, complexity) from the underlying professional skill.
Governance is cross-cutting
NIST's AI RMF treats governance as an ongoing function that spans the lifecycle.
Modern AI needs new capabilities
Framing, delegation, verification, and repeatability are distinct skills in practice.
Design Principles
Cumulative levels
A higher level implies the engineer also demonstrates the behaviors below that level.
Distinct dimensions
Each dimension answers a different question — framing, delegation, evaluation, operationalization, governance.
SDLC outputs are evidence, not dimensions
Coding, testing, design, docs, and incident work remain useful inputs — as examples in context, not scoring axes.
Evidence over self-description
Assignment comes from observed artifacts and repeated behavior, not confidence or fluency in AI language.
Guardrails matter
High output without good judgment is not maturity. Evaluation and governance are gating competencies.
Outcomes matter more than novelty
Useful AI maturity increases quality, speed, consistency, and clarity without creating hidden risk.
Framework Overview
5.1 Level Definitions
Ineffective or unsafe use
Rare, unreliable, or risky use of AI. Little evidence of good judgment or working method.
Ad hoc assistant use
Uses AI as a helper for bounded tasks. Gains some productivity but remains inconsistent.
Structured individual use
Uses AI deliberately with context, constraints, and verification. Repeatable for personal work.
Repeatable high-leverage use
A durable force multiplier. Scales workflows, coaches others, improves team outcomes.
System builder / standard setter
Creates systems, standards, tooling, or operating models that others use.
5.2 Dimension Definitions
| Dimension | What it measures | What it is not |
|---|---|---|
| Framing | Defining the problem, providing context, setting constraints, and specifying success criteria for AI work | Prompt cleverness or phrasing tricks |
| Delegation | The size and complexity of work safely handed to AI | Raw tool usage or request volume |
| Evaluation | Verifying correctness, quality, fitness, and risk in AI outputs | Unit testing alone |
| Operationalization | Making AI use repeatable for self, team, or org | Personal improvisation that others cannot reuse |
| Governance | Using AI within policy, security, privacy, IP, and accountability boundaries | A separate legal / compliance function only |
Detailed Rubric by Dimension
6.1 Framing
Turn an ambiguous engineering problem into an AI-suitable task by providing the right context, constraints, examples, and definition of done.
- Vague or underspecified prompts
- Little context; expects AI to infer intent
- Rarely states constraints or definition of done
- Plausible but misaligned output
- Asks AI for small, localized help
- Provides just-enough immediate context
- Mentions language, file, function
- Success depends on trial and error
- Consistently provides context, constraints, examples, criteria
- Breaks ambiguous work into AI-suitable tasks
- States what must not change and sources of truth
- Narrows or broadens scope intentionally
- Designs multi-step AI workflows via strong framing
- Uses docs, prior PRs, architecture refs to improve later steps
- Frames differently for research vs code vs review
- Coaches others on task preparation
- Establishes team/org patterns for framing
- Defines standards, brief formats, reusable prompt patterns
- Shapes repos and docs to improve AI performance
- Builds a shared language for "well-scoped for AI"
6.2 Delegation
The amount and type of work safely handed to AI, and how well the handoff is managed.
- Does not delegate meaningfully, or delegates with no control
- Uses AI as a search box / copy-paste
- Cannot distinguish safe vs risky handoff
- Delegates bounded tasks: snippets, tests, boilerplate, doc drafts
- Keeps AI work local and low-risk
- AI as helper inside their own workflow
- Multi-step or multi-file tasks with checkpoints
- Chooses chat, IDE, review, or agentic mode deliberately
- Breaks larger work into chunks AI can handle safely
- Uses feedback loops
- Ticket-sized work or bounded feature increments
- Manages iterative PR-comment loops
- Knows when to pull work back to a human
- Delegates strategically for throughput
- Designs agentic operating models for the team
- Defines routing, approvals, permissions, escalation
- Creates reusable delegation patterns
- Builds or selects AI systems for shared use
6.3 Evaluation
Verify whether AI output is correct, appropriate, maintainable, and safe enough to use.
- Accepts AI output at face value
- Misses hallucinations or mismatches with requirements
- Confuses plausibility with correctness
- Basic spot checks or manual review
- Catches obvious syntax / logic issues
- Inconsistent validation — relies on intuition
- Systematic validation with tests, linters, static analysis
- Checks alignment with architecture and conventions
- Probes assumptions, edge cases, failure modes
- Scales review depth to risk
- Anticipates performance, security, dependency, licensing issues
- Uses structured review heuristics
- Calibrates trust by task and risk
- Helps others improve validation
- Builds evaluation systems, not just reviews
- Checklists, benchmarks, CI gates, red-team patterns
- Defines "good enough to merge" for AI-assisted work
- Detects low-quality / unsafe AI output early
6.4 Operationalization
Turn AI from ad hoc usage into a repeatable workflow that improves consistency and leverage.
- Inconsistent use; cannot reproduce prior success
- No saved workflows, standards, or artifacts
- Depends on mood, memory, novelty
- A few personal habits, saved prompts, favorite workflows
- Knows which tools help for some tasks
- Still mostly an individual improviser
- Repeatable personal workflows
- Maintains reusable prompts, templates, checklists, context files
- Reproduces outcomes across tasks
- Tracks where AI helps or hurts
- Codifies team practice in repo instructions, PR review, CI checks, prompt libs
- Standardizes AI use for common work types
- Improves onboarding via teachable patterns
- Creates measurable team uplift
- Org-level systems, tooling, templates, enablement
- Operating norms, measurement frameworks, shared assets
- Connects AI use to quality, delivery, risk metrics
- Makes effective AI use how the org works
6.5 Governance
Use AI within approved boundaries for privacy, security, IP, compliance, data handling, auditability, and human accountability.
- Unaware of policy; treats AI as an individual choice
- May paste sensitive, proprietary, or regulated data
- Does not consider auditability or accountability
- Basic awareness of obvious rules when reminded
- Knows some work shouldn't go to AI, but can't explain why
- Depends on managers for judgment
- Consistently works within policy and tool boundaries
- Redacts, summarizes, or avoids sensitive inputs
- Preserves traceability; explains where AI was used
- Anticipates governance risks before they become incidents
- Considers privacy, security, licensing, IP, regulation
- Helps define review and approval boundaries
- Coaches others on safe usage
- Shapes standards, controls, enablement, incident response
- Influences vendor choices, access models, review requirements
- Builds guidance teams can actually follow
- Strengthens AI risk posture while preserving adoption
SDLC Outputs as Evidence Areas
The old categories are still useful — but they belong in the evidence model, not as dimensions.
Coding & Pull Requests
- Framing scoping the task before code
- Delegation change size safely handed off
- Evaluation review of code, tests, deps, fit
- Operationalization repo instructions, PR patterns
- Governance secrets, proprietary code, permissions
Testing & QA
- Framing asking for edge cases & scenarios
- Delegation suites, fixtures, test data
- Evaluation interpreting test and analysis output
- Operationalization standardized AI test generation
- Governance fabricated tests, unsafe data use
Architecture & Design
- Framing constraints, tradeoffs, criteria
- Delegation options, ADR drafts, critique
- Evaluation fit against system constraints
- Operationalization reusable design prompts
- Governance security, compliance, vendor
Documentation
- Framing context for accurate docs
- Delegation drafts, onboarding, API explainers
- Evaluation fact-checking against code
- Operationalization reusable doc workflows
- Governance avoid exposing sensitive material
Ops, Incidents & Postmortems
- Framing framing the operational question
- Delegation timelines, runbooks, summaries
- Evaluation verify against logs and evidence
- Operationalization reusable runbooks
- Governance production data handling
Scoring & Assignment Model
8.1 Scoring Rules
Score each engineer from 0 to 4 in all five dimensions based on observed behavior in the most recent review period. Use typical demonstrated level, not the single best example.
8.2 Evidence Thresholds
For Level 1
- At least one credible example in real work
- Behavior still narrow, local, or inconsistent
For Level 2
- Repeated evidence in normal work, not just experiments
- The engineer can explain their method
- Evidence in more than one task or context
For Level 3
- Repeated evidence across multiple contexts
- Clear impact beyond the individual
- Artifacts / workflows reused by others
For Level 4
- Has built something durable that scales
- Practice / tool / standard adopted by others
- Organization-level leverage or influence
8.3 Overall Level Calculation
Assign a score from 0 – 4 for each of the five dimensions.
Compute the provisional overall level as the median of the five scores.
Apply guardrails:
- Overall cannot exceed Evaluation + 1
- Overall cannot exceed Governance + 1
- If Evaluation < 2 or Governance < 2, overall ≤ 2
Managers may lower the final score if evidence is outdated, role-constrained, or inconsistent.
A person who delegates aggressively but cannot verify or safely govern that work is not genuinely high-maturity. They are high-volume.
8.4 Example Scoring Sheet
| Engineer | Fram | Deleg | Eval | Ops | Gov | Provisional | Final | Notes |
|---|---|---|---|---|---|---|---|---|
| Example engineer | 3 | 2 | 3 | 1 | 2 | 2 | 2 | Strong personal use; limited team standardization |
Manager Assessment Guide
Use evidence from real work in the last 60–90 days where possible.
Framing — 9.1 Questions to ask
- Show me a task or prompt you prepared for AI. What context did you provide?
- What acceptance criteria or definition of done did you give?
- What sources of truth did you tell AI to use or ignore?
- How did you change your framing when the first attempt was misaligned?
Delegation — 9.2 Questions to ask
- What is the largest unit of work you routinely hand to AI?
- How do you choose between chat, IDE, code review, or an agent?
- What checkpoints or approvals do you put around delegated work?
- Where do you deliberately stop delegating and take work back?
Evaluation — 9.3 Questions to ask
- Walk me through how you verified the last significant AI-generated output you used.
- What tests, linters, analyzers, or review steps did you apply?
- What assumptions or edge cases did you challenge?
- Give an example where AI output looked good but was actually wrong.
Operationalization — 9.4 Questions to ask
- What reusable artifacts or workflows have you created for yourself or your team?
- Do you maintain instructions, templates, prompt files, or review guides?
- How is successful AI usage made discoverable by others?
- What evidence shows your approach improves speed, quality, or consistency?
Governance — 9.5 Questions to ask
- What types of information do you avoid putting into AI tools?
- How do you handle sensitive code, customer data, regulated material?
- What permissions or tool boundaries matter in your workflow?
- When have you decided not to use AI because risk or policy was wrong?
Example Profiles for Calibration
Productive Individual User
Strong personal AI user, real leverage, not yet creating team standards.
Team Multiplier
Uses AI well personally and creates durable workflows for others.
High Delegation, Weak Judgment
Uses agents aggressively, high output — but weak verification and governance.
System Builder / Standard Setter
Engineer or AI platform lead who shapes how many others work.
Suggested Target Ranges by Role
Starting points only — adjust to your org, tech stack, risk model, and enablement maturity.
Do not force title and AI maturity to match. A senior engineer may still be early in AI maturity if their workflow has only recently adopted AI tools.
Rollout Recommendations
Pilot first
Assess a small group before making this broadly consequential.
Self + manager
Self-awareness matters, but evidence should win.
Calibrate together
Review artifacts as a leadership group to reduce scoring drift.
Build an evidence library
Keep anonymized examples of Level 1, 2, 3, and 4 behaviors.
Decouple from comp (at first)
Ensure the model is stable, understandable, and fair first.
Reassess often
AI workflows change fast — semiannual review is reasonable.
Invest in enablement
If you assess maturity, provide the tools, examples, and training to improve it.
Lightweight Assessment Template
Copy/paste for each engineer.
# Engineer AI Usage Assessment
## Summary
- Engineer:
- Reviewer:
- Review period:
- Team / area:
## Scores
- Framing:
- Delegation:
- Evaluation:
- Operationalization:
- Governance:
## Provisional overall (median):
## Final overall:
## Evidence
### Framing
-
### Delegation
-
### Evaluation
-
### Operationalization
-
### Governance
-
## Growth priorities for next cycle
1.
2.
3.Final Recommendation
Adopt the framework with these core choices:
Keep the 0–4 cumulative levels
Use Framing, Delegation, Evaluation, Operationalization, Governance as the primary dimensions
Treat coding, testing, design, docs, and ops as evidence areas, not dimensions
Apply Evaluation and Governance as guardrails when assigning overall level
That structure preserves the clarity of a classic competency matrix while fitting the reality of modern AI-enabled engineering work.
References
- Sijin Joseph, Programmer Competency Matrix. Cumulative, progressive levels. sijinjoseph.com
- SFIA — How SFIA Works. Separating responsibility attributes (autonomy, influence, complexity) from underlying skills. sfia-online.org
- NIST AI RMF Core. Governance as a cross-cutting function; context-setting in risk management. airc.nist.gov
- GitHub — Responsible use of Copilot coding agent. Delegation, scoped tasks, iterative PR work. docs.github.com
- GitHub — Review AI-generated code. Evaluation practices: tests, static analysis, architecture fit, review. docs.github.com
- GitHub — Adding repository custom instructions. Operationalization via persistent repo-level context. docs.github.com
- GitHub — Best practices for Copilot CLI. Team conventions, instruction precedence, allowed-tool boundaries, agentic workflow controls. docs.github.com