Good practices in using code assistant AI tools

Last updated on 2026-10-28 | Edit this page

Overview

Questions

  • What practices can ensure that a code assistant AI tool is used properly in research code development?

Objectives

  • Apply good practices to mitigate ethical, security, and skills-related risks when using AI in research.
  • Develop personal or team-level guidelines for using a code assistant AI tool.

In the previous episode, we learned about the general issues and potential risks of using AI tools in developing codes in research. Here, we learn about a few practices that are particularly important for reproducibility, provenance, licensing, research integrity, and data protection:

  1. AI platforms differ in how they handle submitted code and data. Check and configure the AI tool before use:
    • Check the terms of use, privacy settings, and code/data handling policies of the AI platform. Look out specifically for “Zero Data Retention (ZDR)” and “Data Processing Agreements (DPA)”. These can differ by provider or model, especially when using a routing provider that routes requests to models hosted by other providers.
    • Ensure that your use of the tool is consistent with your institution’s, funder’s, and research project’s requirements.
    • Where relevant, check whether your content (prompts and code) are used by the provider for model training.
    • Keep in mind that not using your content for training does not necessarily mean that the platform does not store it. Never share sensitive and private information. This includes exposing sensitive data stored in files, e.g. .env files, and SSH keys.
    • Where relevant, check whether the “memory” feature of the AI tool is enabled and how it is set up to avoid the risk of cross-project contamination. Note that some platforms provide settings that separate personal memory from project-specific memory.
  2. AI-generated code can contain bugs, incorrect assumptions, insecure practices, or scientifically inappropriate implementations. Treat AI-generated code as a draft:
    • Review, understand, test, and validate every generated code before using it. In your prompts, ask the tool to make small, well-contained, testable changes, for example, implementing one well-defined feature or fixing one bug at a time. Review and understand each change before proceeding to the next one.
    • Do not assume that code is correct simply because it looks plausible or runs successfully. Use unit tests and integration tests where appropriate. See this Carpentries testing tutorial.
    • Check scientific assumptions, algorithms, numerical methods, edge cases, and error handling. Do not rely solely on tests generated by the same AI assistant that generated the code.
    • Have a colleague review important or complex code where appropriate. In a team, use collaborative code review practices like pull requests and pair programming, see Code Reviewing Process explained in The Turing Way
    • Avoid relying on AI-generated “magic numbers” or parameter choices without understanding their justification and ask the tool to explain why, and verify against literature where relevant.
  3. Researchers need to be able to explain and defend the methods implemented in their research software. Understand the generated code before incorporating it:
    • In your prompt, ask the AI tool to explain the proposed approach before generating the code.
    • Use prompts that encourage understanding rather than blind code generation. For example, instead of asking: “Write the complete implementation.” consider asking: “Explain three possible approaches and their trade-offs.”
    • Simpler and established implementations are generally easier to understand and validate.
    • Prefer established algorithms and well-maintained libraries where appropriate.
    • Check that external libraries and dependencies are appropriate and record their versions in a file (e.g., requirements.txt, or environment.yml) so results remain reproducible.
  4. Automated checks provide an additional layer of protection:
  5. AI-generated code can resemble existing open-source code, while its provenance may not be obvious. This is not fundamentally different from a researcher finding and reusing code online, but AI can make the source less apparent:
    • Pay attention to copyright and software licenses. Read more about software licenses from Research Software Support at The Netherlands eScience Center.
    • Do not assume that modifying code removes the obligations associated with its original license.
    • Preserve required copyright notices, attribution, or other license requirements.
    • For substantial AI-generated code blocks, do a quick search for distinctive variable names, comments, or algorithm structure in public repositories (e.g., GitHub) or use code scanning tools if available. If a close match turns up and its license is unclear or incompatible with your project’s license, reimplement the functionality yourself rather than copying the matched code.
    • Generic boilerplate usually needs no special investigation. If AI-generated code contains unusually specific logic, comments, naming constants, or a polished implementation tailored to your problem, spend a few minutes checking whether a similar implementation already exists. When in doubt, treat the result as a starting point for your own implementation rather than incorporating it unchanged.
  1. AI assistance can become part of the research process. Documenting it makes the development process more transparent and potentially easier to reproduce.
  • Depending on your research process, you can also increase transparency by citing the specific AI chat that you used. Duke University Libraries guide to citing AI has a useful guide for citing AI tools and chats in a variety of styles.
  1. Prompting Strategy:
    • Continuously refine prompts and instructions to improve output quality.
    • Regenerate code with a different prompt, or a different AI tool entirely, to cross-check for consistent results, especially for non-trivial logic e.g., statistical computations, numerical algorithms.
    • In prompts, ask for ideas, trade-offs, or pseudocode rather than complete solutions, especially for the core scientific logic of your project.
  2. Use the right AI models and tools for your needs:
    • Large AI models offer advanced capabilities. However, they also consume significantly more computational resources, leading to higher CO2 emissions, land use, and water consumption. To minimize environmental impact, consider using smaller models for simpler tasks.
    • The use of tools (i.e conversational tools, IDE-integrated assistants, or autonomous agents) depends on specific development tasks, your preferences, and project constraints.
Callout

Good practices when using Agents

In addition to the above, when using agents, consider the following:

  1. Agents can, and will, make implicit assumptions about your project; careful planning can mitigate this:
    • Before implementing anything, spar with the agent about the task at hand, until you have a shared understanding of what is to be built. Capture this plan, in e.g. a Markdown file.
    • Break the task down into smaller, isolated steps. Use a strong model with good reasoning for this.
    • Use fresh context (or even better: a different model, e.g. one that is good for producing code, it does not need strong reasoning for implementing a plan), and have the agent read the Markdown file and implement the plan step-by-step.
    • After each step, or a chunk of steps, clear the context (or even better: use yet a different model, which is strong in analyzing code) and have the agent review the changes made.
  2. Telling an agent to not touch these files is not enough, set up programmatic guardrails, if necessary.
  3. Because AI agents have limited context windows, it is useful to break codes into small, well-contained, testable chunks. This also makes it easier to review, understand, and validate each step of the implementation.

References

Tips and guides for AI-assisted coding in science

Resources used in the text

Further reading