The Future of AI Code Generation

• 

  Andrew ParkEditorial Lead
  Heavybit

AI Code Generation Is Still in Early Innings

AI code generation tools are still relatively new. The first tools like OpenAI Codex and GitHub Copilot arrived only a few years ago, and the most valuable affordances–the ways these tools enable you to use LLMs while coding–are redefined every few months as new models drop. As a result, we’re all still early into our journey of using, building, and making the most of AI-assisted coding tools.

At this point, it’s unclear how many developers have even adopted AI code assistant tools yet. Some studies suggest that 76% of developers “are using or planning to use AI coding tools,” but 31% don’t trust the output of their AI assistants and 45% suggest that AI tools “are bad or very bad at handling complex tasks.” At the 2024 AI Engineer World’s Fair, Quinn Slack (CEO & co-founder of SourceGraph) estimated that only ~5% of professional developers were using AI code assistants as of June 2024.

Is there a missing piece of the puzzle? What’s stopping teams from using them more frequently and gaining the benefits they’re supposed to offer? We discussed the issue in depth with Ty Dunn, co-founder at Continue.

Heavybit: In your conversations about AI code generation tools with developers, what seems to be working for them?

Affordances are how users can interact with a product--whether by design or by users finding stopgaps. Image courtesy Sketchplanations

Ty Dunn: We've talked to thousands of developers over the last year. So far, these seem like the only three affordances that early adopters are using regularly:

• Autocomplete, which gives you inline code suggestions as you type to either tab-complete or ‘reject’ by continuing to type out the rest manually
• Chat, which replaces researching quick coding questions, makes it easier to get familiar with new languages / frameworks / libraries, and allows you to iterate on a code section before you put it back into the text editor
• Edit, which lets you modify some code without leaving your current file by simply explaining the changes you want to make using natural language, making it easy to optimize your code for performance, quickly adding comments, making a code section more readable, refactoring it to use a different library, or just generally transforming it some way based on instructions.

Any other affordances beyond these three are not regularly used at this point by developers. They still seem to be experiments for early adopters and enthusiasts to try. Nevertheless, these alternative affordances are evolving quickly as labs release more capable models, developer tools companies try many different UI/UX approaches, and \developers learn to better leverage both of them.

Up to this point, most tools are being used in existing IDEs, primarily VS Code and JetBrains. However, there is ongoing debate about whether existing IDEs can support the affordances needed to most effectively leverage LLMs in the long run. In my view, most developers won’t switch to a new IDE in the near term.

However, I do think we will eventually move beyond existing IDEs as model capabilities improve. If you’re in the Bay Area next week, I’d love to discuss this further with you at the DevGuild: AI Summit III event, where we’ll be talking about codegen tools in a private, open-circle format.

HB: What’s preventing more teams from using such tools?

TD: One of the main reasons expectations have surpassed adoption is because the majority of AI code generation tools are black boxes. Black box solutions don't reveal how or why they come to specific code suggestions. When you don't have visibility into or control over the code generation system, you can't measure, understand, or improve it.

The lack of governance is also the biggest reason why enterprises have been cautious in using such tools; they're understandably concerned about code quality, security, and compliance issues. We believe that taking an open-source approach—that emphasizes transparency and customizability, making it easy for you to monitor and modify the systems and decisions tools make as needed—will be the difference-maker.

We believe in giving you the freedom to keep building software using whichever unique processes and tools you’re already using because they make sense for you and your organizational requirements. Breaking out of the black box will offer organizations the control and visibility needed to embrace AI code generation tools with confidence.

The Next Generation of Tools and Developers

HB: What will that better future–one where AI coding assistants offer what teams need–look like?

TD: Well-designed AI code generation tools should lower the activation energy required to get started on a task, make it easier to overcome roadblocks you run into along the way, and let you get more ambitious with every project. Using them offers the opportunity to bring up both the floor and the ceiling of the software each of us creates.

AI tools in software development are expanding who can participate in the creation and maintenance of software. People who never identified as developers before are increasingly able to engage in software development tasks. This shift will further democratize the creation of software, making it more inclusive and accessible to individuals with diverse backgrounds and skills.

Product managers, data analysts, designers, engineering managers, and other folks in the business will be able to more deeply contribute to the software development process. The boundaries of what has traditionally been considered "the engineering team" might even become less defined.

Companies are already witnessing scenarios where individuals with little coding experience manage complex website updates using AI code assistants—a task that previously necessitated bringing in a specialized front-end engineering consultant.

HB: How will changing roles affect the next generation of developers? Will there be fewer career opportunities?

TD: Many folks who might have previously viewed coding careers as out of reach will gain the confidence to become junior developers due to these tools. While Steve Yegge speculated about the demise of junior developers in his article "The Death of the Junior Developer", I remain optimistic that AI code generation tools will, ultimately, proliferate the very existence of junior developers in the workplace.

There's a very real problem of falling into the trap of overreliance on AI, which impedes learning. However, based on my conversations with developers, once you do this a few times, you notice and adjust your approach to using the tool. These instances where you are led astray can become valuable learning opportunities, which sharpen your ability to evaluate and understand how to use these tools.

From an organizational perspective, AI code assistants offer junior developers the chance to break the mold of slow learning cycles. Instead of waiting for senior engineering input to get unblocked, junior developers can instantly consult their AI code assistant and explore other possibilities they wouldn't have considered alone.

This not only accelerates learning but also allows junior developers to reserve their limited interaction time with senior engineers for crucial, in-depth guidance, thereby optimizing their growth and development within the company on top of the time saved for senior engineers.

HB: So, how do AI assistants change the world of senior engineers?

TD: Part of becoming a senior engineer will involve the mastery of knowing when to use an AI code assistant to accelerate your development versus when it would just slow you down. Senior developers will thus gain the benefits of AI code assistants, while largely avoiding the drawbacks. These general text manipulation tools will enable them to get more done, faster without compromising the quality of their work.

And as this happens, it will become clear that spending some of their newly unlocked time improving their AI code assistant (to unlock even more time) is particularly valuable. Developers will increasingly shift more and more of their time to working on their code generation tools and systems.

The work that engineers did previously–such as writing code–will become something that the system does, while engineers work on the implementation, operations, and improvements of the system that does those tasks. So if your job is to build and maintain the API services in an org, you'll increasingly move to building and maintaining the system that builds and maintains those API services.

You’ll effectively end up one abstraction level higher, similar to how we moved from assembly to higher-level programming languages. I believe moving towards this higher level of abstraction will amplify developers to become more powerful and more capable.

HB: What impacts will AI assistants have on overall code quality and how teams build things?

TD: I believe, in the long run, that having AI code assistants that give developers the control and visibility they need can absolutely pave the way for many more (and more capable) developers. In that future, there will be more software, which frequently will be much better.

Both the way people build software–and the way they learn to build it–is changing. Whether or not you have been coding for 50 years or just started today, if you have an open mind to learning when and where and how to use these tools, you will be helping to define where this change takes us.

Of course, there will be growing pains throughout the transition. There will also be a lot of bad practices propagated, much more insecure code shipped, many more bugs created, the increased use of software for nefarious purposes, and likely many more difficult to predict, unintended consequences.

But this is why we need to start talking deeply about those issues now–not just building the next cool thing but deeply thinking about and actively shaping the future. We need to not only build the next cool tool or affordance but also think deeply about the future we want and work towards it. The entire Continue team is excited about the potential for a better future.

But rather than just hoping things get better, we think there’s a lot of value in trying to better define the future we're working towards and getting more people involved in shaping it. We’ve compiled thoughts from the Continue community into amplified.dev, which explains why the best future is one where developers are amplified, not automated. If you believe in this future, join us by adding your name and opening a pull request here.