AI’s Hidden Opportunities: Shawn "swyx" Wang on New Use Cases and Careers

• 

  Andrew ParkEditorial Lead
  Heavybit

This article covers thoughts from AI engineering expert Shawn "swyx" Wang on new opportunities in AI for use cases and engineers looking to learn more about AI. Learn even more about AI engineering at the AI Engineer World’s Fair event.

GenAI’s Hidden Opportunities for Use Cases (and Engineers)

Generative AI is an exciting and rapidly-developing field which, from an early stage, came to prominence for capabilities like using large language models (LLMs) to spin up large amounts of content quickly. However, there may be other, less-visible upsides to this emerging technology that offer unique opportunities. We spoke with Shawn “swyx” Wang on unexplored upsides of GenAI and what they offer in terms of career opportunities for engineers looking to make a pivot. Here are the high points:

• Generating Summaries is a Hidden Strength of GenAI: Beyond spitting out thousands of words in response to prompts, GenAI can be an effective way to summarize large amounts of information–provided users curate it properly.
• Out-of-the-Box GenAI Isn’t Enough: We’re (hopefully) beyond the era of randomly typing things into ChatGPT and expecting a perfect, polished output. Like many use cases, generating effective summaries requires additional context, the ability to customize to user preferences, and a watchful human eye.
• AI Engineering is a Young Field With Many Opportunities, Particularly for Specialists: For career software developers, it’s not “too late” to become an AI expert–and one approach to accelerate your learning is to focus on AI knowledge specific to your current domain (such as your particular field of engineering), rather than trying to learn everything about AI in general.

Below are more in-depth insights from swyxand his perspective on GenAI’s hidden strengths, and the opportunities it offers for engineers looking to try something new.

GenAI as a Summary Tool: What’s Needed and What’s Missing

While GenAI is known for spinning up large amounts of content rapidly, it has other strengths by virtue of its core reasoning and “intelligence” features, such as the ability to summarize information quickly. However, like with many projects built on and around GenAI, creating summaries that are actually useful and contextually relevant to readers requires a more-thoughtful approach that adds functionality missing from out-of-the-box LLMs.

I'm happily listening to something that is generated by bots–but still has value because the curation happens somewhere else.” - Shawn “swyx” Wang, AI Engineer and Content Creator

“It’s interesting to me because I think that this is the one application of generative AI that doesn't add noise, but actually tries to remove noise,” swyx notes. His first attempt at using AI for summarization was the newsletter AI News, which he built in a week, but was quickly recommended by notable AI figures like Andrej Karpathy and Soumith Chintala, getting more than 15,000 subscribers in a week because it ingests 200K-300K words from the entire AI community–across all disciplines, as well as social media channels like Reddit and Twitter.

The original idea was inspired by the Hacker News Podcast, an auto-generated show that reads through the top posts and comments on the well-known engineering news website, and surprisingly, became his most-listened-to show. “I'm happily listening to something that is generated by bots–something that I know is generated by a bot that doesn't have any human input–but still has value because the curation happens somewhere else. And the summarization has value.”

Identifying and Building Core Product Elements Missing from LLMs

It should be noted that standard LLMs can spit out summaries of whatever text you input. But as swyx notes, a standard “please summarize the following copy-pasted text” prompt doesn’t take into account whether you’ve been reading the content for the past six months. He points out: “If you've been commenting on the same thing for the last six months, any human summarizer would definitely say something about that. So adding context to summarizations, and taking advantage of the ability of having a really strong opinion on what a certain issue looks like–that’s something I think is under-explored.”

Adding context to summarizations, and...having a really strong opinion...that’s something I think is under-explored.”

The core contextual elements missing from a basic summary aren’t new–academics look for four criteria: fluency, coherence, consistency, and relevance. Fluency covers whether sentences are well formed. Coherence covers whether a summary makes sense and is logically organized. Consistency covers whether a summary accurately reflects the source (or is hallucinating). Relevant covers whether a summary focuses on the most important aspects while sorting and filtering correctly. swyx mentions he also looks for a fifth criterion, “structure,” covering whether a summary presents information in a context that makes sense–and where relevant, whether it accounts for previous context as well.

Despite the need for extra context, swyx clarifies that LLMs themselves do have core functionality to support this use case–functionally that will hopefully continue to improve and scale in the future as LLMs continue to improve. “Though some people refer to this capability as ‘intelligence,’ the easiest way to think about it might be that the modern GenAI paradigm just adds a better ability to reason.”

swyx is bullish on the potential for LLMs to provide a strong backbone for summary-based use cases. “That potential to understand things like law, physics, economics–using that general intelligence to summarize things and rank the importance of things–cannot be understated. But although it's so much better than what we used to have before, it still needs the ability to rank subjectively, to match human preferences.”

Where Humans Fit Into the Loop

For this particular work, the most exciting future is one where humans continue to tell stories but can do so at greater scale and velocity, thanks to AI-augmented abilities to ingest and parse multiple information pipelines. “For me, the ‘secret sauce’ is in the pipelines that I set up to do things like multi-stage summarizations in response to events like when people drop links into a Discord. I actually click through the links and get a summary of those things as well. And that adds to the context of the discussion and it models real human usage much more in the sense that when someone sends you a link in your message, you read it and then you talk about it.”

Having people judge the signal does help, and I don't think that [human curation] is really going to go away.”

As an AI-augmented news editor, swyx utilizes the strength of LLMs to recursively break up and summarize incoming information and generate topics that might be interesting to readers. “So, a pipeline that runs over these few hundred thousand tokens every day that executes with debugability and reproducibility. And then finally, a thin layer of human curation, I think that is important. But I actually run four pipelines. And then I've just picked the best of the four. That's my role as an editor.

Interestingly, swyx has seen the highest demand for this use case in the world of biological sciences, presumably from busy scientists looking to keep up with medical research and drug approvals. But he asserts that the potential here goes beyond simply “too long, didn’t read.” By using human curators that take the time to onboard and vet information sources, it’s possible to keep the quality of incoming information consistently high.

“Having people judge the signal does help, and I don't think that [human curation] is really going to go away. LLMs are not smart enough to automate that part yet. But they can automate a lot of the routine reading and summarizing.” swyx suggests a new future for all information distribution–potentially a new paradigm for news in which humans work on vetting story quality, accuracy, and tone while LLMs handle the things they may be consistently better at anyway. “You can still keep the human touch. And I think that combination of human and machine is something that people really like.”

Bridging The Gap Between AI and Software Developers

As a software engineer by trade, swyx has his own ideas about how organizations can bridge the gap between how many AI/machine learning projects in production may be designed by data science PhDs, but expected to be maintained by software engineers with very different backgrounds (and potentially different incentives for engineering, data science, and management teams). He suggests that the AI engineering today is much more focused on the product side than the research side, but concedes that AI/ML can be intimidating for novices to try to get their arms around.

“AI engineering is a field that's still pretty young. There's a lot of tools, techniques, and papers to keep up on. But I don’t think it’s that hard to learn if you have the right curation.” He advises beginners to seek out informational sources that can curate the vast and constant stream of AI news down to something relevant to their own space.

swyx also recommends that novices in all fields, not just engineering, apply their domain knowledge to their learnings. Setting specific goals about specific concepts or areas to learn helps newbies focus on what’s most important for them to learn. “There's no way you can keep up on everything. The field is so hot, and there’s a lot of noise. So I recommend focusing on the things that work for you and are specifically relevant to what you do today. And once you learn that stuff, you can ship with it.” He adds, “you might also want to be a bit skeptical about anyone talking about working with AI and Web3 together–usually those things have no relationship, either you have a valuable standalone AI proposition or a valuable standalone Web3 proposition.”

Interested in learning more about AI engineering? Join the AI Engineer World’s Fair event.