Software Engineering

The Rise of AI-Assisted Software Development

In twelve months, AI assistants have gone from a novelty to a daily tool for a large share of developers. Copilot is generally available, ChatGPT and GPT-4 have changed how people learn and debug, and new assistants launch monthly. We look at what these tools are good at, where they fall short, and how teams should adopt them.

By EthicalHat Research · · 3 min read
Infographic: AI coding tools go mainstream

A year ago this month, GitHub made Copilot generally available. At the time, AI code completion still felt experimental. Since then:

  • ChatGPT (November 2022) gave developers a conversational assistant that could explain, write and debug code.
  • GPT-4 (March 2023) made that assistant noticeably more capable.
  • GitHub announced Copilot X, with chat and pull request features, and Amazon made CodeWhisperer generally available in April.
  • Open models such as StarCoder showed that capable code models are not limited to a few companies.

Developer surveys this year consistently show a large and growing share of developers using AI tools in their work, or planning to. AI-assisted development is no longer a question of if.

Two modes: completion and conversation

It helps to separate two ways developers use these tools.

Inline completion (Copilot, CodeWhisperer, Tabnine) predicts the next lines as you type. It is excellent for:

  • boilerplate and repetitive code,
  • following patterns already present in the file,
  • writing tests that mirror existing tests, and
  • remembering API signatures you would otherwise look up.

Chat assistants (ChatGPT, Copilot Chat and others) are better for:

  • explaining unfamiliar code or error messages,
  • sketching an approach to a problem,
  • translating between languages or frameworks,
  • generating regular expressions, SQL queries and shell one-liners, and
  • rubber-duck debugging.

Many developers report that the biggest productivity gains come from the second mode, understanding faster, rather than typing faster.

What the evidence says

GitHub's own controlled experiment, published in 2022, found that developers using Copilot completed a defined task (writing an HTTP server in JavaScript) about 55% faster than those without it. Other studies and our own experience suggest the benefit varies widely by task:

  • Largest gains: well-defined tasks in popular languages and frameworks, test writing, and working in unfamiliar codebases.
  • Smaller gains: novel algorithms, complex business logic and large-scale design.
  • Negative value: when a developer accepts plausible but wrong code without understanding it.

Where the tools fall short

Confident errors. Models generate code that looks right, compiles, and is subtly wrong: off-by-one errors, missing edge cases, APIs that do not exist ("hallucinated" functions).

Outdated knowledge. Models are trained on data up to a cut-off date and may suggest deprecated APIs or old library versions.

Security. Research on earlier Copilot versions found a significant share of suggestions in security-relevant scenarios contained vulnerabilities such as injection and weak cryptography. Models reproduce the patterns in their training data, including insecure ones.

Context limits. Assistants see a limited window of your code. They do not understand your architecture, your conventions or why a piece of code exists.

Licensing and provenance. Occasionally a model reproduces training code closely. Most tools now offer filters for matching public code.

Adopting AI assistants as a team

1. Write a clear policy. Which tools are approved? What code and data may be shared with them? Enterprise plans that do not retain or train on your prompts matter for proprietary code.

2. Keep humans accountable. The author of a pull request owns its correctness and security, however the code was produced. Reviewers should apply the same standards.

3. Strengthen automated checks. Tests, linters, SAST and dependency scanning catch many of the mistakes assistants make. They matter more, not less, when code is produced faster.

4. Teach effective use. Good prompts include context: the language, framework, constraints and examples. Encourage developers to ask for explanations and tests, not just code.

text
Prompt that works better:
"In our Express 4 + TypeScript API, write a middleware that rate-limits
requests per API key to 100/minute using the existing Redis client in
src/lib/redis.ts. Return 429 with a Retry-After header. Include Jest tests
for the limit and for the reset window."

5. Measure outcomes, not usage. Track cycle time, defect rates and developer satisfaction, rather than "percentage of code written by AI".

What it means for developers

AI assistants are changing which skills matter most. Reading code critically, testing thoroughly, understanding systems and communicating intent clearly are becoming more valuable. Memorising syntax and APIs is becoming less so. Junior developers get a patient tutor, but also a tempting shortcut past the understanding they need to build.

The bottom line

AI-assisted development has arrived faster than almost any developer tool before it. Used well, with good judgement, strong reviews and automated checks, it removes drudgery and speeds up learning. Used carelessly, it produces more code with more hidden defects. The difference is in how teams adopt it.

Our secure SDLC team uses AI assistants within a reviewed, tested workflow, and our agentic AI security practice helps organisations roll them out safely.

EthicalHat Research

The EthicalHat research team: penetration testers, SOC analysts and engineers writing about security, software engineering and AI.

Want an expert opinion on your own systems?

Email info@ethicalhat.com or call +1-844-838-4422. Our security engineers are happy to talk it through.