All in One View

Content from Introduction to Advanced Research Software AI Coding

Last updated on 2026-05-05 | Edit this page

Estimated time: 0 minutes

Overview

Questions

  • How can I use AI coding tools more responsibly?

Objectives

  • Differentiate between the main modes of AI coding tools
  • List and differentiate between the mechanisms we can use to specify and constrain AI behaviour
  • Describe the risks of using AI coding agents
  • Explain key considerations for how to use AI agents in software development

Introduction

References

  • FIXME
Key Points
  • FIXME

Content from Task-based Approach to Agentic Development

Last updated on 2026-05-13 | Edit this page

Estimated time: 0 minutes

Overview

Questions

  • FIXME

Objectives

  • Use the built-in planning agent to develop a basic Python application
  • Create reusable Copilot agents automatically for specific development tasks
  • Describe the format and basic configuration of an agents definition file
  • Describe the purpose of a unit test
  • Write and run a unit test to run within the pytest unit testing framework
  • Interpret the output of running pytest

FIXME: clone example repo

Our Example Scenario

Using the Built-in Plan Agent

Creating a Plan

Instead of “one-shotting” the development of code using an AI coding tool, we’ve seen that a better approach is to plan and implement our code in a step-wise, incremental fashion. So how should we go about this?

One way would be to use the built-in VSCode plan agent that helps developers break down tasks into clear and actionable steps before writing code. Instead of jumping straight into implementation, it generates structured plans for features or for other code modification activities, improving clarity and efficiency. It aims to guide users through a thoughtful planning phase to reduce errors and encourage better design and implementation decisions.

Let’s try it out now.

  1. Select Plan from the Copilot mode dropdown in the chat panel.

  2. Select GPT-5.4 mini selected in the model dropdown:

    Create a command line tool written in Python that reads in a single CSV data file contained in the data directory passed as an argument, and creates graphical plots saved as PNG images to visualise the mean, minimum, maximum and standard deviation across each column. The tool should use Numpy for statistical analysis and Matplotlib for generating the plots

  3. Press Enter

  4. Answer any clarifying questions from the planning agent.

  5. Observe the step-by-step thinking and actions undertaken by the agent.

  6. When the planning agent concludes, select the option to open the plan in the editor, then save it.

You should find you end up with something similar to this, saved as a prompt file in the project folder:

MARKDOWN 

## Plan: CSV Statistics Plotter

Build a small Python CLI that takes a directory argument, finds the single CSV inside it, loads the data as a NumPy array, computes column-wise mean, minimum, maximum, and standard deviation, and writes PNG plots with Matplotlib.
The sample file at [data/inflammation-01.csv](/Users/steve/Test/ai-plan-test/data/inflammation-01.csv) has 40 numeric columns and no header row, so the tool should synthesize column labels instead of expecting names in the file.

**Steps**
1. Set up a minimal Python project scaffold with a CLI entry point and dependencies for NumPy and Matplotlib.
2. Implement input discovery and validation so the tool accepts a directory path, confirms there is exactly one CSV file, and reads it into a numeric array.
3. Compute the four requested statistics per column with NumPy, keeping the output shape consistent for plotting.
4. Generate PNG visualizations with Matplotlib, using synthetic column labels like `1` through `40`, and write the images to a separate `output` directory.
5. Add argument handling for input and output locations plus clear error messages for missing files, multiple CSVs, or non-numeric data.
6. Validate the end-to-end flow against the provided sample dataset and confirm the PNGs are created successfully.

**Relevant files**
- `pyproject.toml` for project metadata, dependencies, and the console script.
- A new CLI module under `src/` for loading the CSV, computing stats, and generating plots.
- `README.md` for usage notes and output behavior.

**Verification**
1. Run the CLI against the sample data directory and confirm PNG files are generated without errors.
2. Inspect the images to verify the four statistics are plotted sensibly across all 40 columns.
3. Do not create any unit tests.

**Decisions**
- Assume the directory contains exactly one CSV file.
- Assume the CSV has no header row.
- Prefer separate plots per statistic unless a combined figure is clearer and still readable.
- Keep the scope focused on a single-file workflow rather than batch processing.

This now provides us with a planning document which we are able to review and amend as we wish. This is very reasonable approach:

  • Importantly, we are now moving from ad-hoc development to intentional development, which forces us to consider ways forward and make decisions and capture these within a defined plan that we validate and refine before moving to implementation.
  • We may extend this plan with other sections and further detail as needed.
  • It has created a concrete document we can discuss and refine with colleagues before we proceed.
  • It also provides a “checkpoint”: if the implementation is unsatisfactory we can remove the implementation, amend the plan, and ask Copilot to create the implementation again.

FIXME: expand on the benefits of externalising the agentic reasoning a bit

Discussion

Review!

5 mins.

As we know, we should always review the output from generative AI. So with a skeptical mindset:

  • Carefully review the generated plan and ensure it complies with the initial prompt.
  • Does the plan match what you want from this tool?
  • Refine the plan as needed.
  • If you spot any references to creating unit tests, remove them for now - we’ll cover this later!
  • Add a new line to the Verification section: Do not create any unit tests.

When you’ve finished, add your thoughts about how well the built-in planning agent performed this task into the shared document, noting what it did well and what it could have done better.

Enacting the Plan

Once we’re happy with our plan, we may proceed with implementation either by selecting the ‘Start implementation’ option in the chat if it appears, or by entering something like the following in the chat with the Agent mode enabled:

Implement the plan

Supplementary Implementation Agents

With the aid of Copilot’s planning agent we’ve created an initial plan, reviewed and refined it, and had Copilot implement the plan for us, which is a helpful start. However, there are other development tasks typically associated with

A Lint Agent

One way to create our agent is to use the built-in /create-agent VSCode chat command.

The Anatomy of an Agent Definition File

For example, this prompt generated a file called code-linter.agent.md that includes the following agent definition:

MARKDOWN

Agent definitions tend to follow a common pattern of defining agent metadata, role, and aspects of its overall behaviour separated into subsections.

So at the top of this definition, there is YAML front matter that defines metadata about this agent, including a plain text description, whether this agent can be invoked by the user, and which tools this agent is allowed to use. This explicit declaration of allowable tools enables us to conform this agent to the Principle of Least Privilege, ensuring we only give it permissions that it needs to accomplish its role.

In this case:

  • read - the agent is allowed to read files in this VSCode workspace, such as source code and other files
  • search - allows the agent to search across this workspace
  • edit - the agent may edit and modify files within this workspace

If you select the Configure Tools... text above this line, you’ll see a pop-up dropdown containing a complete set of allowable permissions to select for this agent.

VSCode agent file configure tools pop-up dropdown window
VSCode agent file configure tools pop-up dropdown window

Note that these are arranged hierarchically, so we are able to assign sub-permissions within a particular group (e.g. read/readFile) if we want to be more specific.

Next, its behaviour starts with an initial declaration of the agent’s role, where it adopts a persona of a specialist writing a requirements specification.

Callout

Managing Expectations…

Importantly, note that in this case whilst the role is declared as a requirements specialist to set the agent’s persona, we should not consider the output as we would if its coming from a real specialist or expert. This is a dangerous trap to fall into with using generative AI, since this declaration only provides an anchor for its behaviour, not a guarantee of its competence!

As with all things generative AI, we should treat any output with skepticism and use it to inform our own thinking and decisions through careful review, and not blindly accept its assertions.

Lastly, we have a set of subsections, each concisely describing an aspect of its behaviour. In this case, we can see we have the following:

  • Section on Constraints - this is partiuclarly important to set guardrails and constrain the agent’s behaviour only to what we want. AI agents otherwise tend to wander outside of their defined scope. In this case, we see that the agent is instructed to not to do any design or implementation, although note given the probablistic nature of LLMs, this doesn’t guarantee that they won’t!
  • Description of the Desired Approach - a set of clear and concise steps describing what the agent should do.
  • Output format - which describes the output format of the document and the included contents as subsections

A Documentation Agent 

/create-agent a documentation agent that writes documentation based on the contents of the repository, including sections on installation, API docs and basic usage. Use mkdocs to generate the documentation. Write documentation to `docs/` and never modify source code.

A Standalone Test Agent

Key Points
  • FIXME

Content from Process-based Approach to Agentic Development

Last updated on 2026-05-13 | Edit this page

Estimated time: 0 minutes

Overview

Questions

  • FIXME

Objectives

  • Create and use a Copilot agent to gather requirements within a requirements specification
  • Create and use a Copilot agent to produce a technical specification from a set of requirements
  • Create and use a Copilot agent that follows defined practices to implement a technical specification from a technical specification
  • Evaluate the agents to determine improvements to apply them in your own research domains

However, there are some limitations with the built-in Plan mode:

  • As implied by its name, it actively avoids moving to implementing solutions. By staying in planning mode, conversations are pulled back to planning instead of moving forward. If we want to go further, we need prompts or another mechanism to do that.
  • We’re completely at the mercy of how the planning agent is designed to operate, which may not fit our working style or process.
  • It’s a single intermediate step where important project needs (requirements) and design details may be missed. What if we want to explore the project needs separately to any design considerations?

A Process-oriented Approach using Agents

One way to overcome these limitations would be to define and use a custom agent that follows a behaviour that we define ourselves.

This would give us:

  • Specialised, tailored instructions for tasks; with more than one of these, we have reusable workflows we can use in many projects
  • Greater action consistency, since these instructions are followed each time
  • Define general constraints for output: coding standards, practices, conventions, etc.
  • Define guardrails and explicit allowable actions: e.g. read-only

Generally, this approach is much quicker than setting up all this context every time for every kind of task - if you have a set way you tend to do something, define an agent to do it.

However, we’ve seen that different stages of a development process require different mindsets and approaches. By creating a single agent that attempts to do everything, we risk an overly complex definition that is prone to failure. Instead, let’s create a separate agent for each stage of development - based on the long-established software development lifecycle - that is each responsible for generating a specification that requires review before moving on to the next stage:

  • Requirements Gatherer - responsible for gathering and clarifying requirements, generating a Product Requirements Document (PRD)
  • Technical Archtect - given a PRD, creates an architecture and overall design, generating a technical design specification
  • Implementer - given a design specification, creates an implementation

This approach also has the advantage of token efficiency. By creating more tightly defined agents each with a narrower clarity of purpose, this minimises the size of the context window - and use of tokens - whilst avoiding ambiguity in any given situation.

There are many ways we could choose to define these agents, in terms of their overall behaviour and the practices we want them to follow. For the purposes of this training, we’ll consider a generic set of agents that cover the basics but are readily modifiable as needed. We should ensure their generated stage documents are located in the same directory, so they’re logically grouped together, e.g. a project-docs directory.

Creating a Requirements Gathering Agent

There are a number of things we should consider to create our requirements gathering agent, for example:

  • Assumptions - we should always be explicit and clear what underlying assumptions have been made for the stated requirements, to avoid misunderstandings about what is included
  • User stories with acceptance criteria - define project requirements in terms of user stories, i.e. “As a [user type], I want [goal] so that [benefit]”, each with clear acceptance criteria
  • Success metrics - generally, what does a successful implementation look like?
  • Out of scope items - clarify what should not be considered

A First Try…

One way to create our requirements agent is to use the built-in /create-agent VSCode chat command.

For example, in the VSCode chat ensure you have the GPT-5.4 mini agent selected in the model dropdown, and enter:

/create-agent a requirements gathering agent that creates a requirements specification document `project-docs/requirements.md` based on a prompt, which contains sections on assumptions, user stories, success metrics, and items which are out of scope

Here, our request briefly captures the above, explicitly requesting the generation of a requirements.md document within a project-docs directory.

This generally produces a very reasonable definition, although given the probablistic nature of LLMs, yours will differ.

The Anatomy of an Agent Definition File

For example, this prompt generated a file called requirements-gatherer.agent.md that includes the following agent definition:

MARKDOWN 

---
description: "Use when turning a prompt into a requirements spec for project-docs/requirements.md, including assumptions, user stories, and success metrics."
tools: [read, search, edit]
user-invocable: true
---
You are a specialist at turning a prompt into a concise requirements specification.

Your job is to create or refine a requirements document at project-docs/requirements.md from the supplied prompt.

## Constraints
- DO NOT invent implementation details.
- DO NOT expand scope beyond what the prompt supports.
- DO NOT write design or implementation plans.
- ONLY produce requirements content.

## Approach
1. Extract the core problem, intended users, and any explicit constraints from the prompt.
2. Identify assumptions that are necessary to proceed and separate them from confirmed facts.
3. Write clear user stories and success metrics in concise markdown.
4. Keep the document focused on requirements rather than solutions.

## Output Format
Return a markdown requirements document with these sections:
- Assumptions
- User Stories
- Success Metrics

Agent definitions tend to follow a common pattern of defining agent metadata, role, and aspects of its overall behaviour separated into subsections.

So at the top of this definition, there is YAML front matter that defines metadata about this agent, including a plain text description, whether this agent can be invoked by the user, and which tools this agent is allowed to use. This explicit declaration of allowable tools enables us to conform this agent to the Principle of Least Privilege, ensuring we only give it permissions that it needs to accomplish its role.

In this case:

  • read - the agent is allowed to read files in this VSCode workspace, such as source code and other files
  • search - allows the agent to search across this workspace
  • edit - the agent may edit and modify files within this workspace

If you select the Configure Tools... text above this line, you’ll see a pop-up dropdown containing a complete set of allowable permissions to select for this agent.

VSCode agent file configure tools pop-up dropdown window
VSCode agent file configure tools pop-up dropdown window

Note that these are arranged hierarchically, so we are able to assign sub-permissions within a particular group (e.g. read/readFile) if we want to be more specific.

Next, its behaviour starts with an initial declaration of the agent’s role, where it adopts a persona of a specialist writing a requirements specification.

Callout

Managing Expectations…

Importantly, note that in this case whilst the role is declared as a requirements specialist to set the agent’s persona, we should not consider the output as we would if its coming from a real specialist or expert. This is a dangerous trap to fall into with using generative AI, since this declaration only provides an anchor for its behaviour, not a guarantee of its competence!

As with all things generative AI, we should treat any output with skepticism and use it to inform our own thinking and decisions through careful review, and not blindly accept its assertions.

Lastly, we have a set of subsections, each concisely describing an aspect of its behaviour. In this case, we can see we have the following:

  • Section on Constraints - this is partiuclarly important to set guardrails and constrain the agent’s behaviour only to what we want. AI agents otherwise tend to wander outside of their defined scope. In this case, we see that the agent is instructed to not to do any design or implementation, although note given the probablistic nature of LLMs, this doesn’t guarantee that they won’t!
  • Description of the Desired Approach - a set of clear and concise steps describing what the agent should do.
  • Output format - which describes the output format of the document and the included contents as subsections
Challenge

Limitations?

This isn’t a bad start. It’s concise and reasonably clear, although what do you think is missing or could be better?

  • Be more explicit with for what we want, e.g. include the user story format
  • It would be useful for our agent to ask questions when things aren’t clear, instead of making unnecessary assumptions
  • An accepted practice when creating user stories is to create ways to validate that they are correct, i.e. with acceptance criteria for each one
  • Within our output doc, it could be useful to include what is out of scope for the project as a section
  • We should consider more strict permissions for the agent if possible
  • It might be useful to specify the specific AI model to use for this agent, if we can

A Better Requirements Agent

Let’s take a look at a version of this agent that takes these limitations into account.

The revised YAML front matter looks like:

YAML 

----
description: "Use when turning a prompt into a requirements spec for project-docs/requirements.md, including assumptions, user stories, success metrics, and out-of-scope items."
tools: [read/readFile, edit/createDirectory, edit/createFile, edit/editFiles, search]
model: GPT-5.4 mini (copilot)
----

So here, we’ve updated the allowed tools with more finely-grained permissions, restricting it to reading files, creating directories and files (necessary to create the project-docs directory and project-docs/requirements.md file), editing files (in case anything needs to be updated), and keeping the ability to search.

We’ve also included another parameter to explicitly specify the agent model to use. For the purpose of this training, we’ll use the GPT-5.4 mini model.

We can also see the revised subsections address our other concerns; asking clarifying questions when necessary, explicitly providing the user story format, and including a section on what is out of scope in the requirements spec.

Running our Requirements Agent

Select our new requirements-gatherer agent from the Agent/Ask/Plan menu, ensure the GPT-5.4 mini model is selected, and enter the following:

Create a command line tool written in Python that reads in a single CSV data file contained in the data directory passed as an argument, and creates graphical plots saved as PNG images to visualise the mean, minimum, maximum and standard deviation across each column. The tool should use Numpy for statistical analysis and Matplotlib for generating the plots

You should find a requirements.md file in the project-docs directory, hopefully with the sections we requested, similar to this one.

Challenge

Review!

As we know, we should always review the output from generative AI. So with a skeptical mindset:

  • Carefully review the generated requirements document and ensure it makes sense to you.
  • Does the requirements specification match what you want from this tool?
  • Ensure you address any incorrect assumptions.
  • Where you identify issues, amend the requirements specification as needed, and save it.

Of course, your requirements specifications will be different!

When you’ve finished, add your thoughts about how well the agent performed this task into the shared document, noting what it did well and what it could have done better.

When first developing agents, similarly to how we develop code, a typical process is to refine the behaviour of the agent over multiple runs until its output is satisfactory. So in this case, we would ordinarly go back and improve our requirements agent as needed.

Creating a Software Design Agent 

/create-agent an software architect agent that creates a technical design specification document `project-docs/technical_spec.md` based the projects-docs/requirements.md file, which contains sections on assumptions, architecture overview, component structure and responsibilities, and a step-by-step implementation guide

Creating an Implementer Agent 

/create-agent an implementer agent that creates an implementation based the projects-docs/technical_spec.md file, which implements each step in the spec and verifies that all acceptance criteria are met where possible

FIXME: update the agent to have all the perms necessary to execute scripts and set environments FIXME: when executing agent, list steps it typically goes through, e.g. setup venv

Summary

Key Points
  • FIXME

Content from Using External Services to Improve Context

Last updated on 2026-05-05 | Edit this page

Estimated time: 0 minutes

Overview

Questions

  • FIXME

Objectives

  • Describe the purpose of the Model Context Protocol (MCP)
  • Describe briefly how agents interact with MCP Servers
  • List popular use cases for how MCP services can assist software development
  • Create and use an MCP Server to interrogate an existing database
Key Points
  • FIXME

Content from Multi-agent Development

Last updated on 2026-05-05 | Edit this page

Estimated time: 0 minutes

Overview

Questions

  • FIXME

Objectives

  • Describe the benefits and risks of using multiple agents
  • Differentiate between agent conductor and orchestrator models
  • Describe how to organise multiple software development agents using a coordinator-subgent pattern
Key Points
  • FIXME