Ep. #40, The Open Source Secret Agent with Dave McAllister of NGINX

Marc Campbell: On this episode of The Kubelist Podcast, Benjie and I were joined by Dave McAllister to talk about his long and storied career in open source, going from punchcards to modern computer graphics, to today where he's a technologist at NGINX.

Benjie De Groot: Yeah. This one was really cool, we went a little long but we covered a lot of stuff. His early work at NASA, early work at DEC, punchcards, beginning of open source. He was there from the ground up, and how it evolved over time.

Marc: It was really cool because a lot of that was before open source was open source, as a thing too, so you got to hear some of the stories.

Benjie: We also talked about his work at SGI, bringing open source in there. Jurassic Park, Adobe, bricking open source, and you just kind of go down the list of all these companies and you're like, "Oh wow, Dave was responsible for that."

Marc: You get to hear why he's known as the open source secret agent.

Benjie: Yeah. You just learn a whole lot and a lot of great anecdotes. Learned about NGINX, Unit, talked about wasm and wrapped it all up with the future of open source and obviously AI and where we're going with all that. I really love this episode. Again, it's a little long but I think it's an edge of your seat episode so really excited to share this one with you folks.

Marc: And welcome back to another episode of The Kubelist Podcast. Today's a treat, we have Dave McAllister from NGINX with us. Welcome, Dave.

Dave McAllister: Hi there. Thanks for having me on.

Marc: And Benjie is here too, of course. Benjie, hi.

Benjie: Hey, Marc. How's it going?

Marc: It's good. All right, well, let's just dive in, let's get started. Dave, really excited to talk to one of the, quote, "Open source secret agents." I wonder if you can get us started and just share a little bit of your background. I think we're going to spend a lot of this episode talking about your career and what you've worked on.

Dave: Sure. Again, thanks for letting me come on and have some fun just chatting about life in general. But I've been in technology my entire life. I've worked for NASA and built massive projects for NASA. I've done hardware design and software design. But in the early 90s, I was working for a company called Silicon Graphics. Some of your listeners and some of y'all may actually remember SGI and our graphics capabilities and our supercomputers.

In 1992 I met one of our engineers, a gentleman named Jeremy Allison and he was working with this really cool, free software project called Samba and it was amazing. All of a sudden, this light started coming on around this concept that says, "We can innovate in software ourselves, without having to be told what to do." From Samba, I jumped into Linux.

He got me into Linux about version 0.93 thanks to a connection with John 'Mad Dog'Hall. Since then, my entire career has been based around this concept that says software should be driven by the needs of the masses, and software creation should be driven by those needs. Everything since then was open source, even though it wasn't called open source until January, 1998.

It's always been this concept that says the source is available, and you can do with it whatever you damn well want to. So that's the short and dirty of where I've been there. Since then I've been involved in, if I'm recounting correctly, 13 startups, one that's still in existence, two failures and the rest of them we sold for little chunks of money here and there.

The biggest problem, by the way, is I'm usually about seven years too early with my really cool concepts and it's notorious.

I'm notorious at being too early to the marketplace and about the time mine craters, somebody will come out and say, "Hey, I've got this really brilliant idea," and they go off and make a fortune.

Marc: Yeah, so what is coming in 2030 then, for us, about seven years from now?

Dave: The internet as you know it will no longer exist. It will no longer exist as you know it. Let's put it this way, it will still exist but the problem is that as we get into Large Language Models for AI, AI Gen, heaven help us, the strong singularity event happened, all of a sudden we may actually see the internet becoming a true virtual assistant rather than a, "Lets go Google something."

Benjie: So Clippy is what you're predicting, you're predicting 2030 Clippy.

Dave: Right, 2030 Clippy which means, of course, the internet goes away completely.

Benjie: Interesting. We're going to get back to that. I'm going to back up the truck all the way to before SGI because I know a little bit about your background, because you were at DEC.

Dave: I was at Digital Equipment Corporation, yes.

Benjie: Which is like the godfather of-

Dave: Everything.

Benjie: Everything, partially. So I want to talk about this a little bit. I want to understand how did you get into computers in 1985 or whatever it was. What was it like? Where did this start? We've got a lot of people listening and they started their careers when the internet existed.

Marc: Or even Kubernetes, some of them started their careers in Kubernetes.

Benjie: Yeah. A lot of people, some of the smartest people I know, they started with Kubernetes 1.2 something. It's just like, "Okay, that's insane."

Dave: Oh, so they're the Lite guys. July 15th, 2015, Kubernetes 1.0 released.

Benjie: Exactly. But that's 20 years past where I want to start. Where did you grow up? I'm starting there.

Dave: Yeah, so let's go all the way back. I grew up in Virginia, so I'm a native Virginian in a little place called Chesapeake, Virginia which is right next to Norfolk, Virginia and you can tell I'm a local because I say Norfolk, not Norfolk, for this which is also on the border of North Carolina. Went to school in Moorhead, Minnesota and then ended up at Oldham Indian University and got a degree in statistics.

When I came out it was really interesting. There were no jobs for statistics, unless you wanted to be an actuary and actuaries are the only people in the world that can make accountants look interesting, so I decided I didn't want to do that. I got this offer to go work for a contractor for NASA doing statistical analysis of flight data.

Literally at the time they would take a big B-52, wire it up with sensors and fly it from Norfolk down to Florida and back, and my job was to build statistics that analyzed the vibration studies on the plane. They would do all sorts of things like run the plane slow, run the plane fast, what happens if you lose an engine, all sorts of interesting things. It was fascinating work.

Benjie: Well now, the B-52 has been around for like 80 years already so what year was this? Because that's not a good barometer.

Dave: Probably around 1980, probably about 1980 timeframe, I'd say.

Benjie: First off, that's cool. You worked at NASA, or you worked for NASA.

Dave: Worked for NASA, yeah.

Benjie: And you're doing this statistical analysis. Were you doing that by hand or was there-

Dave: We were doing it on one of the largest supercomputers in the world, a CDC Cyber 76 with a whopping 256 kilobytes of core memory, and my program was written in FORTRAN. It was 17 boxes of punchcards and had 23 overlays. For people who go back far enough, we didn't have enough memory so you would swap in the program in and out of the system as the program pieces became necessary.

Benjie: Okay. I knew what FORTRAN was, I knew what punchcards were, but I didn't even know about the overlay thing so that's already learning some stuff. Okay. So you are a computer programmer using punchcards and FORTRAN. How does the next step go? How do you get to DEC?

Dave: Right. So I moved from there down to Singer Lake in Houston, Texas. We built basically special simulators, and so they got me into the space program aspects. From there a couple of quick hops to this... I've forgotten where I was working at the time but Digital needed to have somebody who had significant technical skills, deep knowledge of NASA, that they wanted to go in as a sales engineer, a systems engineer.

So I applied for the job, I ended up working back at NASA but supporting Digital, and I was the only SE for the Digital facility for Johnson Space Center. I had eight sales reps to support, and every product that Digital offered, I was responsible for some level of knowledge on. By then computers had gotten a little better, okay? They weren't gigantic things, we had a little bit more memory to work with, and in fact during the time I was there, Microvaxs came out. Ooh, really cool. I can now have a Vax underneath my desk. That was seriously cool stuff.

Benjie: So you're in Houston, it's 1985-ish. You're at Johnson Space Center so you're embedded at NASA, and so you're really at the forefront of... Was Windows... DOS was around, I think.

Dave: DOS was around. Windows came out in, I think, the early 90s. Maybe the late 80s.

Benjie: Yeah. It was like '89 or something like that. Okay. So we're not even there yet, so you're on the ground, first real computers, and software at the time... I mean, this is part of this education for everybody, is software, when you say closed source software, literally the only operating system that you had... What operating system were these Microvaxs using at the time?

Dave: Microvaxs ran VMS, but they also could run a flavor of UNIX. Believe it or not, even the Berkeley distribution, the original BSD UNIX was designed and built on DEC equipment, way back in the dark ages. Just like UNIX itself was originally built by AT&T Labs on DEC equipment.

Benjie: Right. And UNIX, you pay for UNIX? You can't just get Berkeley... I know it's called Berkeley, that always confused me, that it was Berkeley-

Dave: Berkeley Standard Distribution, BSD. Yes, Berkeley Standard Distribution.

Benjie: Right, exactly. But that was created at Berkeley the college, right? That was where it came from?

Dave: Yes. It was.

Benjie: But yet you still had to pay for that, to use that operating system.

Dave: Not totally. You could get it on nine track tape by paying for the tapes, but the problem was is that the computers were all so distinctly specialized that you had to write your own drivers. BSD didn't have drivers for a lot of the things that you may want to use.

Marc: That's interesting, you have the two things that came together to make this possible. Like open source, but also the commoditization of hardware to make it possible to have that value.

Dave: Right. And in fact, open source, Microsoft is the, quote, "Enemy of open source." Microsoft is the reason that open source is successful.

Marc: The PC?

Dave: Yeah. Completely because of what you just said. Commoditize the hardware so that it would run anywhere on any hardware, no matter what you built it on.

Benjie: Okay. Yeah, that's crazy. Okay. So you're at DEC, you have a pretty successful career there for some time, and-

Dave: By the way, I do get bored easily so I do have a tendency to jump from job to job.

Benjie: Right. So at DEC you're in Houston, you're working at NASA, helping sell into NASA. So how'd you end up at SGI? Or is there anything else that we're missing from your amazing DEC stuff?

Dave: So the DEC, the Digital side, kind of hated it when anybody called it DEC. But so we went with Digital. DEC, by the way, is fine, you can say that and Ken won't come haunt you. But I transferred from Houston out to become the Western Regions Operating Systems Consultant and at that time we were heavily focused on VMS.

But at the time I think I was considered one of the 12 people in the world who actually understood UNIX internals and VMS internals, and so came out here, ran around, did lots of talks, created several internal programs for Digital, what was called the VMS Partners Program. I was a member of the Altrix Program, part of this program. Created the CASE Computer, Ada software engineering for those who started with Kubernetes, CASE Partners Program.

There I jumped quickly into Silicon Graphics because they needed somebody who both knew operating systems for VMS and operating systems for UNIX because they had this program called the Vaxcelerator Program. We could take most Vax codes and run them on NIPS processors, and they ran about 20 to 60 times faster, and so they needed somebody to do things like figure out how to deal with odd byte alignment in the operating system because Vaxs don't care, and turns out RISC chips really care.

Marc: And this was like late 80s, early 90s?

Dave: This would've been the early 90s. From there I reinvented myself at Digital multiple times, from OS engineering team to product management, product marketing and management for what I called Dead and Dying Products in Fringe Markets. So I had real time as a market, I was end-of-lifing a number of different hardware products, and then moved over to purely software.

We were really successful in promoting certain things around the software side. We were actually extremely well known in Realtime, we had what was called React Realtime Executive for accurate computer timing. That lead me into this structure that I kept looking at the operating system and going, "Okay. Why do we have to build all these things?" And at that point in time I got introduced into this concept of Linux. We brought in Dave Miller, who went to Red Hat afterwards as an intern one summer. He put Linux on to one of the NIPS servers, no graphics, just the server side functionality. It was mind boggling.

Benjie: What year are we talking about?

Dave: '94, '95. It was fascinating. All of a sudden, you had this operating system that from a server side was incredibly capable, incredibly powerful, maybe didn't do every single bell and whistle that a VMS or an IRIX or a Sonnet, Solaris, any of those things did. But from a server basis, this thing was seriously cool and people were giving it away. Every time you turned around, somebody would go, "Oh, I've got a problem. This doesn't work," and two days later it'd be, "Fixed it." The response capability, the fact that everybody was so involved in making this thing work was just mind boggling.

Benjie: So talk to me a little bit about the early, early days. We didn't have GitHub. Linus obviously wrote Git, I don't know what year that was.

Dave: 2004, 2006.

Benjie: Okay. So this is way before we have Git. How are folks communicating and iterating on the kernel, on the Linux kernel, at this time?

Dave: IRC and email.

Benjie: IRC and email.

Dave: That was our communications processes. I've forgotten what the repositories were, I think it was CBS, if I remember correctly. CBS, Brian Berliner wrote. He was one of my co founders for a company as well.

Marc: You're just like IRC discussing, emailing patch files around? Obviously there's some maintainers of Linux that still like to email patch files around.

Dave: Yeah. In fact, at NGINX, if you want your patch files to be considered, you email them to us on the email distribution list.

Benjie: Old school.

Dave: Yeah. We're very much old school. It's kind of interesting, it's hard to say I could go back and take a Slack or something and say, "What are the communication archives back to when NGINX started in 2002?" Slack didn't exist. But I can go back and search the email archives all the way back to the original message that kicked off NGINX. Likewise, if you think about Linus, his original message was out on one of the pre-software foundations email list saying, "Oh, it's never going to be as big as... But here's this kernel I've been playing with."

Benjie: That's good. Okay. So in this career of yours, FreeBSD is in there as well. Did you have any thoughts on that, or should the secret agent stick to Linux right now?

Dave: No, let's talk about it. The BSD, we were involved in a lot of BSD pieces, but I actually had a company called Threeware and this is one of those too early to the market. We built one of the worlds first and fastest iSCSI servers in the world, and it ended up being overly priced, it didn't need to be priced where it was but I didn't get to have a say on the pricing.

We were a little early to the marketplace, and we wrote that as the basis on FreeBSD. Because, just like everybody else, BSD still has the best TCP stack in existence and the biggest arguments we had were we going to use FreeBSD, NetBSD, OpenBSD? There were multiple BSD flavors. That's the biggest problem, because BSD was actually a significantly more mature system than Linux was at the time. But when you had competing variants inside of BSD, it splintered the marketplace.

Benjie: Right. And that was one of the early criticisms, right? Where it's like you just have... it's the tabs versus spaces thing at OS level if you're not unified. This is an interesting topic of Linux being very opinionated is a big reason why Linux is where it is, I think. I'm a big fan of opinionation.

Marc: Wait, Benjie, talk more about that. I want to understand. I don't know if that's a positive or a negative. The opinions make it where it is today, is that a good thing or a bad thing?

Benjie: I think it's a great thing, I really do. I think that one of the issues that we're dealing with in the Kubernetes ecosystem and the CNCF is there is too many ways to do the same thing, and so a lack of opinionation causes splintering, to what Dave just pointed out, and it causes confusion in the marketplace. Now, optionality is great, but having a consistent vision I think is what had made Linux become the foundation of, I'm going to go ahead and say it, society, if you will, right now. Maybe not society, but computers which is basically everything and so, yeah, that's my opinion. But what do we care about what I... Dave, what do you think?

Dave: Honestly, I tend to agree with you, as long as the opinions are reasonable, factually based, I don't care how you communicate them. Linus is actually a really nice guy, he's great fun to hang out with and so forth. But you don't want to necessarily challenge him when he says something looks wrong because, one, you're going to lose and, two, it's going to be painful to lose because when he comes out and speaks, ex-cathedra in a sense, generally speaking he's right. He's absolutely brilliant on being able to take very complex environments, break them down and say, "Oh, this is going to work, not only now but in the future," and that is a really rare skill.

Marc: Is that story, just observationally watching other people interact with him? Or do you have personal experience where you were like, "No, no"?

Dave: I've always been on Linus' good side for that, and partially because my submits to things in the Linux space were actually fairly small at the time so I wasn't a person that went off and rewrote the memory management system or changed the way the clock timing interactions hit.

Marc: You're kind of chop wood, carry water, make Linux a little better, not big objectionable PRs or patch files coming in?

Dave: Right, there was not a lot of those, "Oh yeah, here's my 10,000 lines for you to fit into your 9,000 line piece of code." Those things do not work well. But also, keep in mind at the time that everybody was doing micro-kernel approaches when Linux came out, and Linux was a monolithic kernel. Okay. So micro-kernels were really useful if you had a closed construction team where everybody in the team could get together and say, "I'm making this change."

And everybody would go, "Oh, that's going to have to be this way." Because of this monolithic approach, every person could build the entire kernel and try their changes out before they ever submitted them, knowing ahead of time who they had to tell had changed in those environments. So moving to monolithic is actually also one of the reasons that Linux became as popular as it did in this open source sense, because anybody could work with it and anybody could build it at any time.

Benjie: Right. So you could iterate on Linux from the get-go and know what you're going to break on every commit?

Dave: Absolutely.

Benjie: Well, once you build it, but yes. And so by micro-kernel, again it's kind of like the FreeBSD thing, kind of, where you have this implementation for the TCP stack and you have this other competing one and then there's some other timing thing, OTP or whatever it is. So that was another design decision at this time that made it so influential. By the way, I just want to point out as a responsible nerd, that SGI is the reason that we have Terminator 2, it's the reason that we have Jurassic Park, The Abyss, I believe, was also done on SGI machines. So you were working on the machines that invented the entire 3D graphics... I mean SGI is Silicon Graphics. So give me one cool story about something cool that you worked on at SGI that I can be like, "Hey, I talked to the guy that did this."

Dave: I'm going to tell you the first time I ever worked, and it was after I moved into product management. I was at a trade show booth, I can never remember where but sure it was. I was working on the servers, I was talking about supercomputing in realtime, but over the monitor the SGI logo was just spinning around and the SGI logo is a three dimensional cube and then we flattened it out to make a flat logo out of it.

But it was a three dimensional cube and people were coming up here and you could grab this thing with the mouse and spin it all the ways you want to and it never flickers, it never bows, it was ray traced, it was perfectly shaded, you could change the lights and so forth, and people were just going, "Wow, I can't believe this!" That was originally written and that time was running on a machine that they built in 1987. From there it was a case of, "Let's show you what we can do now."

We always knew that there was a big movie coming out because the studios would show up and they would take over one of our testing labs and they would put brown paper over the windows that could look into the testing lab and they would render the movies.

Because at the time, nobody had all the power for that and so when Dreamworks kicked off, the three founders of Dreamworks showed up and did a full blown discussion with the entire SGI engineering team in Mountain View, California. Out in an open field, on a stage, everybody shows up and we're out there listening to these guys talk about their vision of what movies could be and it was all because of the SGI side.

You mentioned Jurassic Park, we had a special screening of Jurassic Park with the head graphics designer showing up to give us the feedback of how they built it, why they built it the way they did and what was capable of being done that they had never been able to do before. So those are the things that we were really proud of.

Benjie: That's super cool. Okay, this is me growing up in LA shining through here a little bit, so this is not the computer nerd of me. But I remember there was a story, I think there's actually a documentary about this now, where they were going to do the T-Rex as animatronic and then some guys were like, "I'm going to use this SGI box and I'm just going to do the whole thing," and then Spielberg walks in and he's like, "Oh my god, do that."And this guy almost got fired and then he became a hero, or something like that, right?

Dave: It's something like that. The original was going to be the guy in the rubber suit and then there was going to be some animatronics in it and there was going to be some amount of computer graphics work. But little things like just being able to show the ripple in a cup of coffee, which you could do in realtime graphics at the time, was mind boggling. When you showed it to directors they now could say, "Oh, I am now no longer limited by this group or that group. I can literally build anything I can think of and communicate to the designer."

Benjie: Yeah. All right, that's not related to open source but it's still cool. Jurassic Park is awesome.

Dave: Well, you mentioned The Abyss as well. The Abyss is actually I think the creation of what was known as alpha channels. Alpha channels were how you have an invisible channel on top of your red, green, blue channels so that you can do things that affect the colors without affecting the color channels.

Benjie: Right. And that enables everything today, I'm pretty sure.

Dave: Pretty sure. In fact, including those ugly scrawls across the bottom of your screens or where they stick the logo and cover half your screen.

Benjie: Or the backgrounds we have on our Zoom meetings and everything else. Okay. So you were a part of that, that was really cool. Now, SGI was running IRIX, I believe. Is that the right operating system?

Dave: Yes. We were running IRIX. We also acquired Cray Computers so we had UNICOS, we also had a Microsoft product at the time, and in 1998 Curt Aikley, one of the co founders of SGI, and I went on a year long mission to get Linux as an accepted member of the SGI portfolio.

Benjie: Did that work?

Dave: It did, and in fact we became a major player in that space in shipping for CPU servers out to the marketplace and when SGI started declining, pretty much the things that were keeping them going was the Linux business. Actually, before they completely collapsed, that was the only thing they were doing, was selling with Linux.

Benjie: So you were one of the key players that got SGI, this behemoth of the computer industry, you got them onto Linux and now you're talking about... I remember one of the things about SGI boards, I have a friend that has a framed old SGI board and it's got the four big processors, so multiprocessor. In kernel, is that more SGI? Is that where that came out of?

Dave: The multiprocessor family had been around for a while. What SGI really made realistic was something called CCNUMA which is Cache Coherent Non-Uniform Memory Access which meant that any processor could feature to any memory wherever it was. It meant you no longer had to move data from one place to another place. Memory data movements are slow, whereas CPU performance is fast. And so CCNUMA let us build these massively large environments without having to worry about the impact on data locality.

Benjie: Right. So in my brain I'm thinking about the Erlang actor-model type thing, which I think that's sort of right. But the Linux support for multiprocessor, was that because of SGI?

Dave: It was also partially because of IBM for that. There were a number of players that came into there. The biggest single named contribution that SGI did that moved it directly into the Linux kernel was the filesystem XFS. XFS was one of the first 64 bit filesystems. I used to know the number of how many files you could open at a time, but we open sourced XFS. We put it under GPL2 so that it could become a native filesystem for Linux.

That would've been probably late '98, early '99. About the same point in time we released the specification and standards for OpenGL which is the language of 3D processing, even though we didn't release the code at that point in time. We built a consortium of industry people with a technical advisory board so that everybody had a voice in where 3D programming was going to go.

Benjie: Right. So OpenGL, I think a lot of people listening will recognize what that is. This is showing you guys figured this out because of the success of Linux and these open standards, open source. So again, you were just at the forefront.

Dave: We tended to push the curves, and what we found was once we got the engineering people to understand that Linux didn't mean their jobs were going away, that all of a sudden they were fully onboard. Linux was cool, Linux was easy, Linux could let them try out things they could never try out in a commercial operating system, and all of a sudden we started seeing both not only the external innovations coming from around the world around Linux, but we actually saw our engineering teams innovating and creating new concepts and new ideas around this because of the ability to access Linux.

Marc: So in the 90s, I think you mentioned earlier January, 1998, as being a date where open source became open source. But back then you had multiple challenges. We talked a lot about the commoditization of hardware and things like this and some of the technical challenges. But on the other side, you want to use this open source software and you want to spend company resources, time, contributing back and sharing it back, not making proprietary bits and code that you're running. That had to have taken quite a bit of buy-in. You mentioned the founder that you were working with, right?

Dave: Yes. Like I said, it took a year of doing this. It literally took a year of finding the right people in the company and teaching them. It was very much a grassroots effort. Literally one of the most brilliant people running open source today, a lady named Nythia Ruff, was at SGI as the third level escalation for customer support and she called me up once and said, "I need to understand what this open source stuff means and what it's going to mean."

She's now in charge of open source for Amazon, so she really, not only understood it, understood what its impact could be through Comcast, through all these different places. Nythia is a great example of people that when they start understanding it get to extend it themselves. Open source, the code is one innovation, but open source the concept is a whole different category of innovation.

Benjie: That is so accurate. I have a question, was there a moment where you were like... Obviously early on for you, you had that internal epiphany, obviously. We've covered that, I get it. But was there a moment where you saw some externalities? Obviously SGI seems like one of them, but was there another moment where outside of SGI, basically, but as a greater whole that you were like, "Holy wow, this is going to be a thing?"

Dave: Yeah. So keep in mind that a lot of my business was around supercomputing, realtime, those type of things. I did a lot of work in that space as well. But when you suddenly go down to visit Shell Research Center and they don't want to talk about your supercomputers, they want to know what the impact of Beowulf Clustering, which was a Linux clustering model for supercomputing, was going to be for their work.

That's an indication that the world has suddenly changed because, I hate to say it, companies are not exactly innovators. They tend to be very compute intensive, but they don't tend to necessarily try every new thing when it comes out, or when you suddenly roll into Midtown Manhattan and visit a lot of the trading firms and if you're not talking around things like JBOS or you're not talking around things like Linux as a server environment for compute processing, all of a sudden they don't want to talk to you anymore. That's an indication because they are early innovators for this.

But like the government, they're one of the few places that know how to make money magically appear. So all of a sudden you could start saying, "Okay, there's got to be something to this because I've got the slow movers, the people in the late majority talking about this, and I've got the people who are pushing the curve talking about this." It means not only has it moved into mainstream, it's moved into mainstream in such a way that nobody is willing to give it up, and that was the really A-Ha Moment that hit.

Benjie: What year? What year is that?

Dave: '99 through to about 2001, sometime in that timeframe.

Benjie: We all know that curve, the early adopters, the curve you're discussing. What I find fascinating is that you literally have been on the entire curve for the entire thing. I don't think there's that many people really, really on the ground that were as big a part of that. I know that you're named one of the top 10 pioneers of open source.

Marc: And the secret agent.

Dave: I'm very proud of that. I'm really proud of this and I refer to it all the time, because honestly, nobody knew me that much. I wasn't a name like Eric Raymond who wrote Cathedral and Bazaar, which if you haven't read it, it's still worth reading because it really does describe the software development process between big and little incredibly well.

Linus Torvalds could go to a conference, a Linux expo and have his groupies trail him. Until the first time he showed up without glasses, he had gotten his contacts and nobody recognized him for the first day, which was quite fun. Mad Dog Hall who still today... if you have Mad Dog, you can guarantee a few thousand people will show up just to hear what Mad Dog has to say, from the original days of creating UNIX all the way through to what the world looks like now.

I tended to really focus on sneaking into companies and all of a sudden they'd have an open source practice, all of a sudden they'd have an open source product or they'd be making use of open source. So I got this weird coin of being the open source secret agent, sneak into companies and all of a sudden open source would be part of their culture, whether they liked it or not.

Benjie: That might be a good transition to talk about your time at Adobe because that was a big part of your career as well.

Dave: I went into Adobe, interestingly enough, I went into Adobe, I got hired into Adobe basically for something else you mentioned, standard work. Adobe had decided that they needed to consider moving PDF, Portable Document Format, into an actual official standard. Now, the specification for PDF had been available since day one, anyone could go grab it and build anything they wanted to from it.

But there's a big difference between specification and standards because it meant that nobody saw it until Adobe was ready to release it. So I got hired in, we spent some time and we ended up with one of the fastest acceptances in ISO, so PDF is now ISO-32000 and is no longer an Adobe product. It is now part of the International Standards family.

But similar to what I just talked about, I was going to run standards but I wanted to get into open source space, and so we started looking and saying, "What projects do we have that are in the open source world and how can we better make them available for this?" There were two basic rules, just to let you know, these are Dave's two rules for company open sources, give credit where credit is due and return equivalent value.

It does not mean open source everything you own, which is the way everybody looks at this. But if you're doing something cool and you can open source it, give it back. We had a bunch of products and at the end of the second year, first year was mostly PDF work, second year we had about 30 active projects that had come out from the open source side which lead to a couple of notable ones that people today might even still recognize, both of which now live in Apache Foundation. One of them is called Flex, it was the Adobe Flex, it was the language for writing in Flash.

Benjie: I have to interrupt, my first startup was a language translation company and we used Flex extensively.

Marc: I've written production Flex code myself.

Benjie: Yeah. Flex was actually great. I mean, I'm embarrassed about this but Flex was great because it's Flash and you're not supposed to like Flash. But Flash, okay, we're going to get hate mail for this one, but Flash was... I mean, it was the only thing. JavaScript did not work universally remotely at the time, early in the browser days. Yeah, and Flex was the programmatic version that ran ActionScript 3 and Flex was great. I'm pretty sure it's not in use anymore, but I loved Flex back in the day.

Dave: Actually, Flex is one of the most active projects at Apache.

Benjie: Like I said, Flex is one of the most active projects.

Dave: The other one was a project that when we acquired a company called PhoneGap and we took PhoneGap and moved it into open source in Apache as Apache Cordoba. Those were the two projects that really committed Adobe to being an open source oriented company. Now, the activity levels have been settling. It's not at the same level. But I will tell you my Adobe R&D team, you can go out and find the R&D site on GitHub, Adobe's R&D site, is still active because those people want to share their work because their work is theoretical and there work is going to push the curve.

Benjie: Yeah. Adobe is one of the quiet cornerstones of Silicon Valley that I don't think gets enough credit. It reminds me of SGI where it's like Adobe just... I remember when I was a kid and there was Adobe Photoshop 2.3, that I'm pretty sure statute of limitations is up so I'll say it, I was able to get from a Warez site. Adobe 2.3 from my friend, and it was the most amazing thing of all time.

Marc: I think it's safe. I think not only the statute of limitations, most of us were downloading.

Benjie: Yeah. But Adobe has been really impactful to the industry, and you were the special agent or the secret agent that got in there and infiltrated and turned Adobe into an open source supportive company at the very least. Now, there must've been some real conflicts that you dealt with there because traditionally that was not their... Well, like you said, the PDF format was always open which, by the way, I had no idea. When you said that, I thought you were making a joke. I always think of PDF as just being like a nightmare.

Dave: Yeah. The biggest change was the original PDF specification included PostScript in the first two chapters, and when we moved it into a specification we had to go in and remove all of that and any references to those first two chapters in the specs because it had to be able to stand by itself for that. But the reason PDF is successful is the fact that early on the decision was made to give the reader away for free, bottom line.

Benjie: That's really interesting. I never thought about that part of it. There's a lot of value, there's a lot of value creation. Infinite value creation from a computer display perspective with the PDF stuff. So your career at Adobe, that was what you did there and now I don't want to fast forward too much but we are going to run out of time. So NGINX and F5 is where we're trying to get to. Is there anything you have to tell us before we get there?

Dave: This is one, NGINX is part of F5. They were acquired before me. But it's interesting that, again, an open source project started in 2002, basically runs the internet. So if you did away with NGINX, probably about half of the world is not going to have their websites running, somewhere in that neighborhood. It's dead, gone. So from an open source viewpoint, think about what that means.

That means you can't just blindly accept code, that means you've got to go through and spend a lot of time making sure that whatever the changes are don't break the internet. Talk about bad press, the heck with password theft. "Oh, look. They broke the internet." That would be a bad press day. But people always stop and think, "Oh, NGINX is a one trick pony,"and yet when I go out and look at the GitHub repositories that I have, I have like 84 repositories, of which 16 are really, really active repositories.

We have this thing written by the same guy who wrote NGINX, by the way, called Unit. Unit is an application, multilanguage application server. You can run multiple languages on it, and it's going to get me to one of the points I want to get to in a minute, and change things on the fly. You can update things on the fly without shutting down the application or the underlying structure. So all of a sudden, applications no longer have to worry about what they're feeding into, what the load balancer, cache coherency models all look like. It's all just done for them.

Benjie: Unit, I don't even know Unit. Tell me more.

Dave: Unit, basically think of it as an app server that doesn't care what language it is. So the language elements are sandboxed into a server that provides all those things like TLS encryption, end to end encryption capabilities, load balancing, reverse proxy. It's all just magically done for you and if you change your application, you don't shut down your engine, you change your application.

Benjie: So hotswapping?

Dave: Right, hotswap, change the configuration, change your load balancer constructs without shutting things down. And because it's multiple languages it fits into what we now call platform ops really, really well, and application developers don't have to worry about that pesky internet stuff. It's all magically handled for them. So Unit is one of them.

Most recently, near and dear to my heart, we added WebAssembly capabilities to Unit so Unit is now capable of delivering a WebAssembly application in that same format. WebAssembly, to your point early on about fragmentation, there are lots of WebAssembly runtimes and right now if you're going to do WebAssembly, your application has to worry about all those little, weird, internet thingies.

TLS, et cetera, you've got to worry about all that junk. If you run it inside of Unit, you don't. It's done for you. So application developers get to write applications and the networking stuff is just handled, and so Unit is one of our best kept secrets. We're really good at keeping it secret.

Marc: Yeah. I'm actually poking around the dogs in the GitHub repo. We'll include links here, of course, but it's not that hard to find. I was unfamiliar with it. I like it, it's like eight different runtimes. Is WebAssembly just one of those? As long as the language I'm writing can compile down into WebAssembly, Unit can run it?

Dave: Yeah. It doesn't even have to compile it, if you write it in Ruby, it runs it, if you write it in JavaScript, it runs it, if you... I think if you're writing in C you're going to have to go into WebAssembly to make it run. But the web languages are really, really useful because applications, our internet is no longer a static world, our internet is now application based and so we're really proud of the fact that we did WebAssembly.

We actually learned a whole hell of a lot about how memory interacts between multiple runtimes by having Unit. Linear memory feeds or how do you have a pointer, because WebAssembly wants to do one pointer but your application may want to have a different pointer, in your web environments you or I need a third level of memory pointers, and so how do all of those work together?

Why would any application guy, writing a calculator, want to know that stuff? So Unit, we're very proud of Unit, we are very proud of the fact that there with this thing called wasmTime, we're very thankful, again open source type routes, a company called Fermion jumped in and just helped us understand all these different things. This is where we can innovate together, no matter what we are as a company. We get to innovate together.

Benjie: Okay. I have a quick, pragmatic question. If you're running Python, for example, and you're using UWizzKey or G-Unicorn, can I just swap in Unit for that?

Dave: Yup, I believe so.

Benjie: Interesting. Okay. Well, that's blown my mind because I will say this, NGINX is absolutely a trusted open source vendor for me, and I would hope for everybody listening to this podcast.

Marc: And Unit is all just open source or is there a commercial product too?

Dave: Not even a commercial version, it's all open source. We use GitHub issues to track the issue capabilities, the team is incredibly responsive. You can actually go onto... we have a little NGINX Slack community, you can get answers there. But the thing about it is that, again, this is purely open source stuff and it solves a specific problem, and that's where we really get into this. Other things, by the way, for your audience. If you haven't spent time looking at WebAssembly, do your career a favor and learn about WebAssembly because it is changing the way we approach web applications.

Benjie: He's drinking the Kool-Aid. That's Benjie Kool-Aid. I'm a hype guy. I love the EBPFs, I love wasm. But actually we've had Matt Butcher on the show, we actually caught up with him on our KubeCon podcast and, yeah, it seems like there's a real lot of momentum about that.

Dave: Hopefully you got a chance to talk to Matt about things like Helm or the thing he's best known for, which is the Child Sky to Kubernetes, the book. I saw the first live reading of that when we were at Engine Yard. He actually created Helm as a weekend hackathon, also at Engine Yard.

Benjie: There you go. How did you end up at NGINX real quick?

Dave: Okay. So I was working for a company called Splunk, Splunk was a lot of fun. I am a monitoring guy. Remember way back in the first part of this I talked about statistics? Monitoring is all about numbers, and the fact is that our systems are so good that nobody actually understands what the numbers are telling them anymore.

I love doing a talk on what the stats are actually telling you and looking and saying, going, "Okay. What's the difference between a mean, a median and a mode?" The three principle statistics types, and watching people get their heads wrapped around what those three things mean, and then me looking at them and going, "It's a trick question, there's lots of different means."

So when you start looking at this, I moved here, I got a cold connect, they said, "Hey, can we just chat?" And sat down with this person, this is during the pandemic era so all on Zoom. What they were looking for was somebody who had built open source processes and capabilities, at the same point in time was perfectly willing to get on stage and go, "I can make you believe."

And it was really an interesting fit. NGINX is an incredible name, we have brand recognition beyond belief. But nobody knows about the rest of NGINX, and so looking at that challenge said, quote, "Heck, I'm in." We fast forwarded. When I came in the door I started screaming, "Open Telemetry. Why are you not part of Open Telemetry for this massive Kubernetes presence?"

Some of our most popular projects are things like the Kubernetes Gateway, the API Gateway Project for NGINX, or the Ingress Projects for Kubernetes. Why are you missing the OTel boat? We solved that problem, we're not getting to the OTel boat, NGINX now has a tracing module. That's one of my more proper projects. Not only do you see the trace, you can see the trace as it interacts with NGINX, as it flows from the user to the backend for that.

Marc: One of the things that I noticed on Unit when you were talking about that earlier is top level nav, when you talk about the functionality of it, there's usage statistics and it looks like that layer, that is something that it's built around from the beginning.

Dave: Yes. Again, you think about who uses the product, if you think about it from that viewpoint, NGINX is like the pipes in your wall. As long as they're not broken, you just want to know that water is flowing through them and that requires monitoring.

Marc: I mean, I can't remember the last startup, the last project that didn't have NGINX as a key part of it. You're right, it is critical to the internet. Lets go forward a little bit. You mentioned the Ingress controller, you mentioned Gateway API. Some of the interesting stuff, obviously this is the Kubelist Podcast, we talk a lot about Kubernetes and the CNCF ecosystem. There's clearly a place for NGINX to fit in and really advance that. What is outside of those two? Or we can talk more about those two, I'd like to talk a little bit about how you're interacting with the Kubernetes ecosystem, the Kubernetes community, the Kubernetes projects right now.

Dave: Okay. So we are, of course, members of Linux Foundation and CNCF, yada, yada, yada. We could play alphabet soup with the best of them. NGINX maintains a separate presence in this, just like Red Hat and IBM, even though they're the same company now. We maintain a separate presence because our role is different than what F5's role would be.

I have people who sit on the Kubernetes guidance committees, I've forgotten the specific one off the top of my head. But Dylan sits in there and he discusses with this and brings back both the insights from where the community wants to go to where our engineering team wants to go, as well as what our engineering team thinks they need to consider. We then make the contributions that cross there.

API Gateway has been something that's really near and dear to our heart. The issue with Kubernetes has been... I think Benjie you said this earlier, a little fragmented for this. This starts letting us have an approach that says, "Okay, everybody can go in and out this way, and innovate outside of that." And so that becomes a really important role for us, and so we were, I think, launched a couple of months ago. It was one of our big presences at KubeCon North America as well.

But we really believe that the world of the internet is going to be powered a lot by Kubernetes in the future, good or bad. It is not the easiest thing in the world to work with, by any stretch of the imagination. You can cut fingers off easier than you can make things work well.

Benjie: I would go as far as to say that's a generous analogy. It's a tough game out there.

Dave: Yeah. And so what we can do is starting bringing things together that, say, the commonality elements of things like Apache and NGINX, using a Kubernetes API gateway means you can now write to this and be pretty confident that you're not going to have to make changes to your environment as you move into the internet world. This is where things start stabilizing, and so kind of interesting that Kubernetes is taking this long to really get to a stabilization point that's not just code.

Benjie: Right. It's the community.

Dave: And it's a community that is speaking on behalf of the community, not a community speaking on behalf of individuals.

Benjie: Wait, say that one again?

Dave: Okay. So in other words, you've got a lot of companies, I've forgotten how many companies are in the Kubernetes side, it's like 1,000. Okay. They make decisions for the good of the industry, not for the good of Acme Anvil Corporation. That is a massive, massive difference. By the way, we're just really kind of getting there with things like WebAssembly. WebAssembly is fairly fragmented, but it's an accelerated coalesce that's happening. We're seeing this happen faster partially because we have W3C driving a standard approach, at the same time we have the open source world by Code Alliance driving a delivery code approach.

Benjie: Talk more about the CNCF because one of the things that I think is what's made Kubernetes and this whole ecosystem so vibrant is the community itself. I'd love to just hear the Dave McAllister comparison for the CNCF today versus, let's say, the email list of 1992 for the Linux kernel.

Dave: Wow. So back in 1992, et cetera, it was very much we didn't call them these, but it was very much a, "There is a person driving this project, and there are a very small number of contributors." So if you think about it in Linux terms for this, Linux came out but there weren't a lot of people contributing. The people that were contributing to it though were pretty brilliant and the code was really good. Fast forward, it's now 1998 and Linus can't necessarily look at every line of code and so he has built this set of trusted people to which the subsystems flow in for that.

In fact, we got to a point where there's Linus looking at the next generation but there's Werner looking at this and coming back and going, "Okay, I'm going to work on the stable maintenance model, what we're doing now. You look at what's coming next." So all of a sudden we have this going from a single point of view to this is truly a community of contributions and the community of contributions know how to funnel their communications in both directions.

That is the single biggest change. Lets fast forward the next step, and we now have foundations appearing. So Linux Foundation came out of the Open Source Development Laboratories and Linux Standards Base merging together, and that became LF. LF's principle mission was to make a Linux stable model and pay certain salaries, so Linus Torvalds is paid by the Linux Foundation to continue to work on Linux.

We now have foundations approach here. What foundations did was emphasize the meritocracy approach over benevolent dictator approach. Next step happens, all of a sudden all these companies came in and said, "Hey, wow. This open source thing is really cool. I'm just going to add open source to my marketing material, and, look, I'm set." It doesn't work that way, guys.

But we did see a lot of projects come out, we saw companies develop around open source initiatives. As long as they were willing to be open and not what I call a Walled Garden, "It's my code, I'll do with it what I want to, I'll let you have it but don't talk to me,"model, it became important. About that same time we had this thing called Public Cloud appear, Public Cloud changed the rules on monetization of code.

Now people could get paid for the usage of code, not for the delivery of code. That changed the rules of the game, and that's why we've had things that have showed up such as the Business Source License and the companies that have recently had to change their licenses because they've got to pay their developer staff at the same point in time that others are making use of their brilliant innovations.

Foundations solve that problem. Everybody has an even playing field, and that's why the foundation works. CNCF, do we really think that OpenTelemetry could exist without the backing of CNCF and the 800+ companies that contribute to CNCF? Probably not, because telemetry is hard, it's really, really freaking hard. And so a foundation let's everybody share a little bit of the pain for the benefit of everybody.

I think you've got to see foundations as a way that people are going to approach to avoid the Walled Garden approach. It's not going to be quick, this is going back to your 2030 timeframe when this happens, but foundations have shown that they can innovate as fast, if not faster, than an individual company.

Marc: That's a great way to look at it, and I think you definitely do look at what's happening in the CNCF and Linux Foundation... KubeCon was just a couple of weeks ago at the time we're recording this right now in Chicago, and it's not moving slowly. The pace of innovation and the pace of changes is so fast and it's mature, right? You can count on it, you can rely on it, so I think it's good but it's not a trade-off of velocity for maturity or stability either.

Dave: Right. You have to be careful though, because the enterprise rules is, "I don't want to be updating my software every two weeks." So in the NGINX space we have a mainline version, we have a stable version, a mainline version and whatever they call the bleeding edge version for here. But mainline comes out like every four months and stable comes out once a year, and they all get updated with fixed and so forth.

But if you want to be safe, solid and secure you stay on stable. If you want to be bleeding edge, you can do that as well. Flashing back, I think it was Linux 2.4.9 or 10, a long time ago, it came out with a new release that everybody went out with and then they found out that the memory management system fell apart under load for this and we had to come up with a 2.4.11 really fast. Everyone who was in a production environment under load, if they went to 2.4.10 broke. That's why you need to be safe and stable as well as innovate.

Marc: Yeah, and Kubernetes is good. I think even Microsoft has been pretty active, but again it's really the community and the governance model around even finding the right time and the right process to have an LTS version of Kubernetes for long term support because, early days in the 1.0, 1.2 Kubernetes, you needed the latest version because that's the only way to get anything to work. They were missing functionality. Now you get Kubernetes working, and you don't want to have to every four months go through a full upgrade on every cluster you have.

Dave: Right. And in fact this is one of those that if you're in the Public Cloud environment, you let somebody else worry about the upgrades. You just want your code to work.

Benjie: Well, there are some public clouds that force you to upgrade, but I don't want to talk about that. They do things without telling you. But look, I think the key thing that you're saying here, and this is a little mind blowing to me, is you just drew a direct line from the early days of Linux straight to why CNCF is successful. I'll speak to being on the floor at KubeCon, I talked to all these people and one of the themes was, "Are you incubating? Are you sandboxed?"

And they all are like, "Oh, I have to do this, this and this." And it was something I really picked up where the structure was tangible, they knew what they needed to do. Yes, there's a security audit, but then they're like, "Oh, but they pay for it and there's all these vendors and we just need to get it going and you do it." It's like there is a framework to move very quickly and you have some of these projects that are in sandbox or incubating or whatever it is, and they're being used by big companies.

I think it speaks to this entire ecosystem that's been built out. I'm going to go ahead and say thank you, Dave, one of the pioneers of open source, for helping us get there. But genuinely, I mean seriously, especially after this conversation. So we're way over time, but I don't care because I love this conversation. I have to touch on one last thing, ish, one last-ish thing. Okay. We got Dave, he's been here, let's talk a little bit about AI, LLM, open source.

Where do you see us going? I, as I've already alluded to, I'm full Terminator 2, the war against the humans and the machines, I think it already has begun. Kyle Reese needs to go back in time, we got a whole situation happening right now. But maybe someone with a little more rationality and experience can tell us where we're at and where you think we're going to be at. Then, whatever you say we'll add seven years, so we know that already.

Dave: Yeah, it's kind of interesting. I was actually in line to do a talk for an upcoming event and they called me up and said, "We're only going to let you do the talk if you include some mention of AI into this." And I said, "Oh, fine. Sure. Okay." So I went off to the principle conversational LLMs for this, and I basically said, "Here are 10 numbers. Please give me the average of those 10 numbers." Top six, all six were incorrect.

I spent 30 minutes just because I wanted to see what would happen, walking ChatGPT 4 through this exercise and I basically said, "Okay. Take these following two numbers, add them together. Take that result, add this number," and walked them all the way through this and I said, "I want you to count the number of numbers that I've given you," and it came back with the right number.

Then I said, "I want you to divide that number into the sum," and it came back and gave me the wrong answer. My next statement was, "Why are those two answers," pointing it to the one that it originally said, "different?" And it went off and it did its little generating, I think is the phrase it uses, generating. 20 minutes later it came back and said, "Whoops, I made a mistake."

So it was capable of learning from its mistake, and for the rest of that session it could do the correct math correctly. I started a new session, nope, gone again. So I'm not too worried about conversational LLMs taking over the world in the next three years. I don't think we're going to see a strong singularity breakthrough no matter what, pick-your-favorite-technologist, says is going to happen in the next three years.

I think we're going to get closer to this. The problem with a strong singularity is where the AI is capable of improving itself, is the model phrase. By 2030 we may be not the smartest thing on the face of the planet anymore. That is kind of scary in some ways, and it scares some people more than others.

However, given the current controls and intelligence shown around the world right now, if you have not watched any news, spend 30 minutes and watch your evening news and you'll understand what I'm talking about, I'm not sure we are the most intelligent thing on the face of the planet right now either.

Benjie: But that's my argument for, again, my Terminator 2 view of this where... First off, LLMs being AGI-like. Yeah, right. That's not remotely where we're at today. The thing that scares me about LLMs is that bigger is better, that's what we've learned so far. Where does that stop being the case? I don't know, but there is, this is a weird reference, but there's a movie called Lucy. Did anyone see that movie?

Dave: Yeah.

Benjie: Sorry, spoiler alert. At the end of this movie, this thing builds its own USB stick basically, which is everything that it needs. I don't want to ruin it too much. That, to me, I have a weird sneaking suspicion that maybe we're going to throw all this horsepower at this thing and it's actually going to be able to create the thing that is the scary thing.

Assuming we get to the scary thing, I.e, AGI, my concern is that human stupidity is the biggest threat to the AGI. The first thing that anything that was sentient would say is, "What happens if the entire climate changes and all traditional power generation goes away because the Gulf Stream stops working? Well, the only ones stupid enough to cause that problem are humans, so we got to get rid of them." Or, "What's the second biggest threat? Nuclear war."

Marc: But if power went away as we know it, that would probably be an existential threat to that AGI.

Benjie: That's what I'm saying. So what I'm saying is that's my nerd movie fear.

Dave: That's the risk on strong singularity, are they for you or against you? It could also be, "Okay. So your job is to protect humans." And it may look at this and say, "Okay. Well, if I'm really going to protect humans, nobody is allowed to drive a car." It can be as simple or as complex as that, you're not allowed to drive which means shipments can't happen so cities starve to death because the infrastructure is not set up to have the city have food for more than about two days.

So you've got all these different pieces that are inside of there. For my viewpoint, those are manageable. As you were pointing out, Marc, here right now if I was really, really nervous about something suddenly becoming strong singularity, I'm planting an EMP device under the main systems so I could push the button and wipe things. I'm not to that point yet for this. Where I'm going to be worried is when the first AGI comes up and says, "Oh, let me build Gen 2."

Marc: Yeah, because that only increases power exponentially and decreases the time in between those iterations. So right now you're not worried about that. What should we be doing with AI and LLMs right now? How do you use it on a day to day basis?

Dave: Okay. So interestingly enough, NGINX, y'all both have a background in NGINX. NGINX's config files are somewhat of a nightmare to deal with.

Marc: I've built some.

Dave: When I look through my community, I can guarantee that there's going to be the number one grouping of questions is, "Something is wrong with my config," even though they don't know what's wrong with their config. Number two question is, "Oh, I installed NGINX and I built a website. Why is it showing me the welcome page?" It's probably my most popular question and it's like, "Okay. Go change the config file."

The conversational models are actually decent at interpreting what's in your config file. They're not great, they're about 50% accurate right now for this, but that would be an easy fix. It would be easy to teach the conversational side what an NGINX config file needs and how to put it together. And so I can see it doing that extremely well. That's one of the ways I do, if I see something that I don't understand I will grab it, throw it into one of the AI tools, some of them are better than others for this, and basically say, "What is this telling me?"

And it's really good at coming back and going, "This is trying to do the following things." I also will honestly tell you, I'll grab a days worth of Reddit questions off of the NGINX tag and say, "I want you to sort this list for me and put common items together so I can see what the community is having problems with." And it's actually pretty decent at that too. It does a better job than I do.

Marc: Yeah. It's really good at summarization. What about taking what you were talking about with configs even one step further? Instead of, "I have a hand crafted config that I'm having a problem with"... Look, I think that the models that we have today were trained on data that's publicly available and NGINX has been around forever, so there is a lot that's publicly available, there's a lot of knowledge already built into these models for how to write an NGINX config. Why not just make it conversational? Instead of me even having to even start looking at and understanding that NGINX config? Is that something that'll come?

Dave: Yeah. I think that you'll see that sooner or later. I know that I'm not a part of what's going on, but I know that there are a number of AI research going on around here that's looking at things like how do we do this. How do we make docs AI intelligent? So those type of things are easy approachables and I think that that will change the nature of how people build for the future.

Benjie: Do you have any thoughts on open source's role in LLMs and in AI and tying that back a little bit?

Dave: Yeah. There are two aspects underneath of that. One is the engine aspect itself, and I think that the engines themselves should be as open as possible. One, because I want to be able to see when this thing is going to hit singularity model. The other one is the background data information, and this is where copyright and trademark law and all that junk starts creeping into it.

Honestly, the stupid of humans is going to creep into this as well. Okay. I am going to load my machine with nothing more than every history of every war in the world. Nothing else. Just the wars in the world. It's probably going to have a pretty bleak view of the human race for that, so open source is going to have to be part of this to be able to make sure that our learning data is as balanced as possible.

Benjie: We're completely out of time, we didn't even get into this. But the Wikipedia component of open source and open source knowledge, and so figuring out a responsible, moderated way for the training data... Because inherent bias is one of the biggest problems, if not the biggest problem.

Dave: Inherent bias and survivorship bias.

Benjie: Exactly. Yes, exactly. Survivorship bias. Not to get into this too much, but just for listeners, I don't know if you've ever thought about this. But if you're using artificial intelligence to give loans out to folks, well, up until very recently and arguably today, there was a lot of inherent human bias towards certain groups of people and not giving them loans. And so you can't just take arbitrary data, shove it into something and not equate for the human bias component of it. Oh, very important question. Kubectl or KubeControl?

Dave: I actually love Kubectl.

Benjie: Dave, correct answer. You won. That's my vote. I ask everybody that question, and Kubectl is the correct answer.

Marc: I think they have a cuttlefish for the logo now.

Dave: I think they do too. But yeah, it's cute.

Benjie: The campaign is working. The campaign is working, Marc. All right. Look, on that note, on the ctl note, we're going to finish this off. Oh, this episode is going to be coming out beginning of 2024, NGINX has a community day. Tell us about that real quick and then we're going to wrap up.

Dave: Yeah. Real quick. On February 5th at San Jose Convention Center, we've got an NGINX sprint which is our community grouping, community day coming up. We're going to spend some time, we're going to get the guy that runs the division in place, we're going to get one of the founders in too. We're going to talk about where NGINX is going, and all of those cool, little projects I mentioned earlier on that nobody was aware of. We're going to spend some time talking about those. We're going to talk about WebAssembly, we're going to talk about OpenTelemetry and monitoring capabilities. So we're going to let you not only know where we have been, but where we are taking you in the future.

Benjie: That's super exciting. Thank you so much for coming on, Dave. This was a great conversation, really appreciate all your time.

Dave: You made my day. Everything else is just going to be downhill from here.

Benjie: All right. Thank you, Dave.