The Future of Coding in the Age of GenAI

• 

  Andrew ParkEditorial Lead
  Heavybit

What AI Assistants Mean for the Future of Coding

If you only read the headlines, AI has already amplified software engineers into superheroes who complete coding projects more than 50% faster. All that’s left is to sit back and bask in the golden age of AI permanently setting software dev on “easy mode.” Right?

If the above assessment isn’t quite the same as your own experience in development, you’re not alone. While there are many exciting news stories about the transformative power of AI for generating code, documentation, testing, and other areas, there isn’t much of a common framework for benchmarking or clearly understanding what makes AI assistants “good.”

To help the dev community better understand the opportunities that AI coding assistants offer–and what dev teams can reasonably expect from them, we tapped several experts in AI-related software development. For more insights into AI code generation and other AI development tools, join the DevGuild: AI Summit III event in San Francisco.

AI Assistants Will Mean More Software and Devs

Ty Dunn is the CEO and co-founder of Continue, the leading open-source AI code assistant. He has worked in and around machine learning and software development his entire career, serving a tour of duty as the Founding Product Manager at Rasa. Some of his top observations about AI code assistants include:

Early Innings and Growing Pains

AI code assistants are still relatively new, and not all developers have even incorporated them into their workflows. There are still questions about whether existing tools, including IDEs like VS Code, and JetBrains, will make sense for AI-driven development over time.

Valuable Use Cases for AI Code Generation

There are a few genuinely valuable use cases for AI coding assistants that developers are incorporating into their day-to-day work now:

• Autocomplete: Inline code suggestions have seen increased popularity, though some developers find them intrusive and their suggestions unhelpful
• Chat-Based Assistance: IDE sidebar interactions can provide on-demand answers to quick questions as well as guidance for unfamiliar technologies
• Editing and Refactoring: Inline transformations of code sections following natural-language instructions can speed up development processes

AI “Black Boxes” Are an Innovation Blocker, Especially for Enterprises

Many modern AI code assistants still suffer from an understanding and governance gap. Several of these tools are “black boxes” that don’t offer the visibility and control that developers and organizations, particularly enterprises need. There’s an ongoing lack of trust among many organizations for AI tooling that’s blocking greater adoption.

Additional Thoughts on AI Code Assistance with Ty Dunn

The Continue co-founder notes that there are still many X-factors shaping AI-assisted coding. “We're really early in this journey of learning to use code generation while building software.The vast majority of developers haven’t adopted them into their daily workflows yet. The initial autocomplete, chat, and edit affordances, while valuable today, are likely limiting us from integrating LLMs into our workflows more deeply.”

“People are exploring a lot of alternative affordances–you have AI labs constantly releasing more-capable models, like the recently launched o1. You have developer tools trying so many different UI/UX approaches. And then you have a third piece: Developers actually learning to leverage both of them.”

Future Effects on Developers

Dunn suggests that AI assistants can streamline away grunt work, and from a hiring perspective, they could absolutely make assessing a new hire’s skillset more challenging. However, it’s highly unlikely they’ll lead to an extinction event for junior developers.

“I don’t buy the idea that AI code assistants will lead to the end of junior engineers. I think there can be a trap that new engineers fall into–relying too much on these tools, which inhibits their learning–but to be honest, I think AI code assistants are going to lead to more people building software, ultimately.”

“And junior devs are going to be that much more capable and productive. If I had these tools when I was learning, I would’ve progressed a lot faster.” Dunn suggests that a professional upside of AI code assistants is they can help junior engineers unblock themselves more quickly, and save their consultations with senior engineering leaders for truly pressing questions.

“I also think that more people in organizations are going to participate in software development. Beyond what we’d traditionally call ‘an engineering team,’ we’re seeing people who might not consider themselves to be devs using AI coding tools for technical projects, such as making website improvements they might’ve needed a Web consultant for previously.”

How “Engineering Teams” May Look Different

“I think we're going to see more product managers, analysts, and business folks–just generally people across the organization participate in the development of software. Over time, I think software engineers will increasingly move from the work they do now to working on code generation systems and tools. What they used to work on will be what the AI software development system does–and they will work on the system.”

“If your job is to build the microservices in your org, you'll increasingly move to building the system that creates the microservices. You’ll end up being one abstraction level higher. Sort of like how we moved from programming in assembly to higher-level programming languages in the past.”

“What will the net effect be–whether we need more engineers or less? I’m not completely sure, but I'm pretty confident we're going to move toward a higher level of abstraction that is more empowering for people. And as a result, it’s going to continue to accelerate the trend of more software–for example, more custom internal software–being created and operated.”

“The future we believe in is one where developers are amplified, not automated. We created an initial sketch of what that means at amplified.dev, which we’d love for people to read and give us feedback on.”

“It's critical that we think about what we want this future to look like–and that we work together to create a movement towards that vision. Because that is what’s going to ultimately shape where this goes more than anything. Rather than just hoping (and not thinking) about it and just building one thing a day at a time, there's a lot of value in defining the future we're working towards and getting more people involved in creating it.”

Properly Learned, AI Assistants Will Be a Force Multiplier

Steven Schkolne is the CEO and founder of MightyMeld, the visualization and creation platform for complex React apps. He has been an engineer, artist, writer, and academic at such orgs as The Vain Company, The Huffington Post, and UCLA. Here are some of his observations:

AI Assistants Are Evolving Beyond Low-Level Work

After a formative period of automating away low-level tasks, AI assistant tools are starting to take on higher-level coding tasks, such as providing insights on APIs or the structure of existing code.

There’s Still Plenty of Important Dev Work for Humans to Do

While AI assistants may purport to take on repetitive, low-level work, there are still plenty of jobs requiring human intervention that also don’t quite justify investing hours from senior engineers.

Control Matters

Developers need to find a balance between the rapid automation that AI code generation promises and having control over the quality and relevance of code that such tools output.

Additional Thoughts on AI Assistants with Steven Schkolne

The MightyMeld founder is generally bullish on AI code generator tools’ ability to help developers crank out code quickly, as well as providing on-demand support via chatbots, and pushes back slightly against growing skepticism in AI for development.

“What we do is help developers do visual-level work with AI code. It’s a really exciting way to work visually. I tend to think that some of the criticism coming from cynics like Goldman Sachs is in reference to other areas of the AI ecosystem. But the advances in software development? They’re here to stay–I don't think we can go back at this point.”

How Software Devs Will Progress in Their Careers in the Future

Schkolne suggests that AI assistants are changing development as a career, but possibly not in completely straightforward ways. “A senior dev can get a lot more done, definitely–but I wouldn't use the metaphor that AI assistants are ‘a virtual junior dev.’ That's not really my experience working with AI tools.”

“It's more like having this killer autocomplete that can take a lot of the grunt work out of getting stuff done. Has it diminished the importance of junior developers? Maybe, but conversely, it's a wonderful tool for junior developers to skill up.”

“I think it's a lot faster to skill up to being more senior, but I think [AI assistants are] moving the level at which you're operating as a software engineer to that higher level: Where you're thinking about where the bigger building blocks go, how your system works, how things flow.”

“I think, maybe part of what we used to think about as ‘the junior-level role’ is disappearing. As a senior dev, I might, for instance, give you the function calls, and the inputs, and here's a comment...now go implement this API for me. But you don't need someone to do that anymore. So that particular part of the job may be disappearing.”

“I think engineers are getting a lot more productive at the senior level. And if you're new to the game, and you're trying to advance in your career, you have to just skill up to that higher level to be productive. That's what I see happening with junior developers.”

“On our team, we have a mid-level developer we work with, and there's still a lot of things that we want to hand off to this person that still make sense for someone who's not quite a senior dev to work on. They're not necessarily the deepest problems in the world.”

“But this person is still coming in more at that mid-level and doing some real problem-solving, taking some high-level work off of a senior engineer's plate. Mid-level devs can still do high-level problem-solving. They’re still working in areas of the codebase that might be a little more secluded from the main complexity.”

What Engineering, and Eng Teams, Will Look Like

Schkolne sees leverage as the biggest contribution of AI assistants to the future of development. “I think we’re already seeing things like no-code tools letting people build things for which they previously needed engineers. But at the same time, I think the complexity level of applications is going to rise tremendously because engineers can do so much more.”

“I’m in the camp of people who look historically at what other automation technologies have done–which is letting people do a lot more and build a lot more. You can make experiences for people that are so much more powerful. And for customers? Expectations are going to grow.”

“You're going to see more sophistication and complexity in certain products that couldn't have it before because in the past, three engineers couldn't make it as awesome. Now three engineers can make it that awesome. And so, I think the engineering team of the future will look similar to how it is today, but with 10 times as much stuff being done by that same engineering team.”

Schkolne suggests that the biggest immediate challenge is giving developers control. “While we’d kind of been riding the complexity of AI–starting out with small-scale styling edits, we’re now doing component-level, page-level edits, adding new things to your page. Some of these recent model releases are just making things even faster and more powerful.”

“The question we encounter is: How do you give an appropriate amount of control to the engineer to kind of keep everything tidy? If you do enough iterations with an LLM on a codebase, at some point, it isn’t going to stay tidy. MightyMeld is working on how we can give front-end engineers control while they're iterating really, really quickly and balancing those two things.”

Using AI Tooling to Focus on What Matters

Craig Alexander is the Director of Engineering at Tonic.ai, an industry leader in providing synthetic data for AI programs. He has been a technical lead and engineer at Gather, Kabbage, and IgnitionOne. He offers the following thoughts on AI development assistants:

AI Tools Are Useful for Non-Code Generation Use Cases

AI tools offer other useful additions to development, such as complex query generation and infrastructure-related tasks such as generating JSON reports or Docker configurations, which can save hours of writing syntax-heavy queries.

AI Tools May Change, But Fundamental Properties of Software Projects Might Not

Whether or not AI tools dramatically change the career trajectory of software developers, certain aspects of building software aren’t likely to change soon. At every stage of their careers, developers will always have to reckon with the size of, and relative importance of, their projects.

Avoid Getting Lost in AI Hype, Find Tools That Are Right for You

It’s easy to get lost in AI headlines of multi-billion-dollar funding rounds and multi-billion-parameter models. For teams looking to actually build and ship things, it’s a better idea to narrow your scope to finding models that are a strong fit for your use case and finding tooling and processes that increase your performance and lower your costs.

Additional Thoughts on AI Assistants with Craig Alexander

The engineering lead offers numerous suggestions on valuable AI use cases, particularly for data management, but tempers his outlook on AI across every single aspect of development. “AI tools have certainly saved me time in key areas–like query writing and query syntax, especially for non-SQL languages.”

“AI tools have also helped me with infrastructure–specifically, data tasks. As a developer, I'm very comfortable with code and running things locally, but sometimes I just need, let’s say, a Docker file to do something simple. Something like, ‘Hey, give me an engine X container that mounts this folder to serve up some HTML files.’”

“As for writing code itself, I don't really use [AI tools] that much, personally. I still find myself asking questions to other people. But occasionally, if I need to do something that's relatively simple, but I’m not 100% sure on the fastest way to write it, then I’ll consult [an AI code generator]. I find them to sometimes be a bit behind on new language features.”

“Now, there are lots of other areas where I do use [AI]. A big part of what we do at Tonic is writing complicated queries. We introspect database schemas and query a lot of system tables that are very different from technology to technology. I guess that's not directly writing code–it's very, very tangential and plugs right into it, though.”

Scoping Your Career as an AI-Assisted Developer

Alexander suggests that organizations will likely look for a “minimum literacy level” with AI tools and what to watch out for, though the tooling seems intuitive enough–and his own hiring will likely be focused on highly specialized needs.

“When I think about devs at different points in their career, there are two main factors to consider: There's the size of the projects that they work on and how heavily [those projects] influence the rest of [the engineering org] in terms of practices and patterns and things like that. I don't see those two things changing. I just see the existing levels getting a lot faster.”

“Let's say I'm a junior engineer and I just signed on for a small-to-medium-level project. I still have to work with the product team to assess the requirements. I still have to think about user experience and maybe work with the design team. I have this coding system that can shoot out all this code at me, but I still have to put it together and make sure it works correctly.”

How will team composition change as a result of AI assistants? Alexander offers that the makeup of engineering teams may not change much due to functional reasons. “It's been my experience that senior engineers don't have junior engineers to ‘assist’ them by writing small sections of each project.”

“It’s been more of a situation where we rely on everyone to integrate all of their stuff together. It's not like you have a senior engineer who takes these four pieces from these four junior engineers and stitches them together somehow. Engineering teams might end up being smaller just because everyone's gonna be faster.”

Rather Than Getting Lost in “The Future,” Focus on Your Future

The engineering leader suggests that despite all the hype, there may be tools out there that provide ROI for you–and those should be the ones to focus on. “If you have a tool that saves your engineers on average 30% of their day–that's 30% of their comp you can directly attribute to these tools and however much they cost.”

“Some of these AI tools–I know they’re not especially cheap, but if they can improve your productivity, it's definitely worth the money. And, in my opinion, there's also the opportunity cost of not [getting those productivity gains]–what are your engineers working on longer and not shipping? What competitor products are they not beating to market?

“If you’re trying to get any kind of ROI, keep in mind that with AI, as with many other systems, it’s ‘garbage in, garbage out.’ You’ll need to feed the right kind of data and avoid overfeeding it with bulky, unnecessary information. You’ll also need to make sure that sensitive information doesn't get in there to avoid leakage risks and privacy breaches.”

Alexander notes that putting all your faith–and your attention–into the endless march of technology might not be the most productive use of your time. “Something to consider–before we had LLMs, the kind of ML models we were using were pretty different. Then, all this new stuff came along, and seemingly overnight, the utility went through the roof.”

Reflecting on the relentless pace of new LLM releases, each more powerful and trained on more data than the last, the engineering lead suggests we may hit a ceiling faster than we think. “It's going to take some other huge paradigm shift or a new type of technology to get us to the next level. I'm not sure that throwing more data at existing techniques is going to be helpful. I think we're at the point of diminishing returns there.”

“None of us can practically say where this new phase of AI and LLMs will go, but at some point, the cost is not going to be enough to justify further development. At that point, more-specialized models might make more sense. For example, people are already starting to build specific models for healthcare data, financial data, government data, and so on.”

“I would suggest that if you’re looking at using models for internal use, take a look at evaluation tools to know which model will work for you today–understanding it might not be the best thing for you three-to-six months from now–and that you should be able to switch your model.”

“You should be able to measure which model works best for you, and what outputs are the most cost-efficient for you. Sometimes, models can generate responses with an incredibly high accuracy, but also at a very high cost. Maybe you won’t need 95% accuracy–maybe you’re good with 90% accuracy but it's going to be 80% cheaper to do it.”

“So if I were thinking of building or using existing models, I wouldn't worry about which model out there is ‘the best’ or has the most parameters. I’d rather think about what are the best models and tools for me and my use case today.”

Continue the conversation with 200+ techical leaders at DevGuild: AI Summit III event in San Francisco.