Ep. #4, Backend Development with Paul Biggar of Dark

Joe Ruscio: All right, everybody. Welcome back. We're really excited to be joined by our guest today, Paul Biggar. He is a Heavybit alumnus, the founder of a Heavybit company and one we're super proud of, CircleCI, if you're hopefully doing testing then they're probably at the top of your list for continuous integration.

He's also the host of a popular podcast on the Heavybit network, more popular than this one, in fact. Although I'm gunning for it. To Be Continuous, which is great, Paul and Edith from LaunchDarkly have a great podcast.

Last but not least, he has a new thing going on. He's the CTO and co-founder of Dark, which is super descriptive. We'll get into that more, but it's at DarkLang.com, is that right?

Paul Biggar: Yep.

Joe: All right, Paul. Welcome.

Paul: Thank you. Thank you for having me here.

Joe: Pleasure's all ours. So, we have a couple of things I wanna talk about today. This podcast is about helping modern software teams get better, get more productive, get more done with fewer people, have fewer failures, etc.

And so one of the things, especially now that you've built multiple companies, and especially companies with challenging infrastructure. C ontinuous integration play is actually in the details, I mean--

Paul: We built a platform as a service, essentially, for CI.

Joe: Right. It sounds run/test, but there's-- This is at scale, a big thing. And so what kind of learning, going from "garage" to seed stage, to bigger?-- One of the things that I always found fascinating, and I'd like to get your take, having gone through that myself, is that the things that work really well at one stage, that are the best thing for one of those stages. They break in horrific ways at the next one and you have to redo it.

Paul:

There's the standard thing of: each inflection point you're going to need a different set of tooling to last you for the next 10x scaling. But I think people often don't scale that down and think about, "When I have zero customers or one customer or the first 10 customers, can I do something that's even cheaper than you often do?"

Joe: It's sort of like the inverse is true as well.

Paul: Yeah, exactly. You see all these companies where the first thing they do is they set up their Kubernetes-based microservice before they've written a single line of code, and it's like "What are you doing?"

Joe: It's not required to get things going.

Paul: Right. One of the things that we do with Dark is we look at how people write new pieces of software, and a lot of people look to hackathons. There's this idea of like, a hackathon is a thing where you go and you write code, you build this app in a weekend. We're actually getting to the stage now where people aren't writing anything in hackathons because it's just too much work to build an application.

Joe: Right. Particularly the Rails community. It's people who are there long term and bemoan what used to be-- When the beauty was you can get an app up and running in five minutes. Now it's like, "There's 30 JavaScript frameworks you have to pick from, and you got to get npm installed, and you've got to get this other thing--"

Paul: None of them work together.

Joe: Right.

Paul: You have to get your Webpack set up, and you've got to continuously deploy it, and once you've done all of that then you can finally write software.

Joe: Start writing some code.

Paul: There's so many people I've talked to that have a side project that they were so excited about, and then after 16 hours of doing this "infra-framework" they just didn't care anymore. They had no passion left for whatever it was they were building.

Joe: What's interesting, you hit on this at the very beginning is a lot of these things which are now being considered boilerplate were designed originally-- Let's pick on Kubernetes a bit. Obviously we love a great technology, but it was designed by Google to help them manage PlanetScale clusters.

Paul: Right, right.

Joe: If you can run your entire infrastructure on three servers, maybe it's not solving the same set of problems.

Paul: Or one. Or, less than one.

Joe: Right, yeah. Exactly.

Paul: I think one of the--

Joe: I'm dating myself with the three servers. It used to be three.

Paul: I'm sure that almost anyone starting to build a service today wouldn't even know how to get started with acquiring three servers.

Joe: Right.

Paul: Putting them in a colo and buying the parts off the internet and then putting them together, and putting them on a rack somewhere, then doing some sort of installing an OS, and--

Joe: Or even, I--

Paul: Imagine.

Joe: I was thinking of even three instances, because you want to be in three availability zones. Right now--

Paul: So quaint.

Joe: Yeah, right. That would be half of all availability zones, pretty much, at the time. You hear more and more, and I think the reaction on this side that I hear more and more now is serverless first.

Paul: Right.

Joe: Which is this emerging doctrine about, "Let's try to make it serverless first, and then if we can't make it work there then we'll drop back to a 24 by 7 running process that we have more control, and we can right-size resources for." What's your notion on serverless in general? Besides, yes, it does in fact run on servers. We don't need to go there.

Paul: Right. I think that serverless as a concept, and looking at broader than like Lambda or cloud functions or whatever, I think is a wonderful idea. It's disappointing though. The entire space around serverless is starting to have the Kubernetes-ification problem. Again, it started with "We just want to run a function in the cloud."

Joe: Literally, effectively like a cron job type function.

Paul: Right. An actual function in the programming language sense. Maybe it has to be JavaScript, or whatever, but there's an input and there's an output, and it's done. It's a lovely concept. There's just so much more that you need to do to actually run a thing. You need to monitor it, you need to scale it, and Lambda does some scaling but it has a cold start problem, and all this other thing. So now, I'm sure you saw at re:Invent, AWS brought out Lambda layers.

Joe: Layers, yeah.

Paul: I didn't even look at what it was called, I was like "I'm done with the whole thing. It's already too much." When we went to put some stuff in Lambda as a way of trying it out, it took like eight hours. It's not at all the simplicity that you are hoping for, it's huge complexity around a simple concept.

Joe: It's the kind of thing where I feel like they often focus on the initial abstraction, which to your point was a single function. Where it's like a web hook or cron job kind of thing, but it's not even just that it's a pure function with one input and some processing and something comes back. But it exists in isolation from literally, conceptually, "What does it do? What does this one thing?"

Now I think as people start and try and build more complicated applications out of it, what I find is a fascinating challenge. Especially with this infrastructure that's really-- Or products that aren't really built for that at all. It's bad enough when we've gone all in our Kubernetes microservices and we've crossed Dunbar's number of microservices, and nobody in the org actually knows what all of them are or what they do.

Paul: Right.

Joe: Saying "We're gonna take those microservices, each of which do five or six related things, probably more, most in practice. Then break them into individual functions."Now I have tens of thousands of things that are related somehow, and have flows that go through them.

Paul: And you've got a management problem, a version control problem, a continuous delivery problem, a testing problem. I think that the thing that Lambda and serverless is trying to do is good. I'm not a huge fan of how it ended up.

Joe: It seems like we're definitely clearly still, if you subscribe to the Gartner Hype Cycle model, we're still clearly rocketing up. We haven't hit the trough of disillusionment yet, which is some of the things you're hinting at.

Paul: I think people need to keep looking at this "Less" idea. The amount of accidental complexity that is in how we build software today is crazy. And so, doing--

Joe: What are some examples of that?

Paul: The very obvious one is all the infrastructure stuff. You have to learn Kubernetes and Docker, and you have to learn all that stuff. At the same time you have to build your continuous delivery pipeline, and your continuous integration, and testing. You have all this tooling challenge, and you see the amount of companies that come through Heavybit and all the other companies in the space. They're all providing one piece of the pie. So we have 50 things that you need to consider before you even start writing any software at all.

Joe: I've personally found the explosion of complexity in the development environment to be a fascinating problem that certainly most people aren't even talking about, unless it's commiseration, and certainly aren't solving it. Whereas historically, back again just to use an example, if you fired up a Rails app--

Paul: Right.

Joe: You had one application, one database and you could load up your local browser. That dev environment actually looked reasonably close to production.

Paul: Yeah.

Joe: Now, if you're using a microservices or even more granular environment, the number of moving pieces you need on your laptop to effectively replicate production is not tractable.

Paul: Right. It's not that these things started from a bad place. There are extremely valid reasons for microservices and places that monoliths break down, especially with scaling and deployment velocity. You don't necessarily want them all to be on the same thing.

Some things need to be extremely high reliability, some things need to be redeployed dozens of times a day. You do want to break those apart and you want to have some sort of boundary, but the end result, it just hasn't worked out particularly well. I think what you're saying about the trough of disillusionment is right. We're starting to see that steep decline.

Joe: It's something where we've made giant strides towards fixing problems in scaling production environments, and how quickly we can-- Like the velocity we can make changes at scale in large scale applications. But the trade off, because there's no free lunch, no silver bullets, is that we dramatically increase complexity of the development environment.

Paul: Right. This is where I should tell you about my new company DarkLang and how it solves all these problems, but actually, before we do-- I want to get to that later. Your original question was, someone is starting something today, they're doing something small. I talk to a lot of those companies and the advice I give them is "See if you can build it without software. Even if you're an engineer. See if you can throw it together with Zapier and Google Sheets, or Airtable or product management techniques like Wizard of Oz.

Have someone do it by email or buy a forum and there's an actual human behind it. See if you can get validation before you start to build the actual product, because if you build the product first you're gonna spend six months doing that. And once you've spent that six months doing that, you're going to find that it's wrong in some way that you don't know yet.

Whereas if you spent a week doing that in a completely slipshod haphazard way, you would only have spent a week to learn the lessons and to move in the direction of what you actually needed to do build."

Joe: It's great advice. It harkens back to some of the Lean Startup stuff that was absolutely all the rage. So, say you do that. You've got your-- Especially it is interesting, tools like Airtable. There are dramatically improved tools that you can plug together and actually get. They won't scale, and as you said there will have to be humans polishing edges and handling things, but you can actually get working systems with just plugging these together.

Paul:

The needs of an early customer, like how reliable or polished you need to be for that first customer, is often two orders of magnitude lower than people expect.

Joe: Right. Because often what you'll be doing is you'll be paying that hidden costs with humans. Whether it's your-- Not sales engineers, because you won't have any of them officially. But you as the founder acting with your sales engineer or your customer success hat on, you'll be talking directly to those people, finding out what's wrong and literally pulling them through those problems and manually fixing things on the back.

Paul: Absolutely.

Joe: How do you decide when it gets to the next stage? What's the trigger that says, "OK. I'm onto something here. I should go the next stage." What's that next stage look like? How do you formalize those things?

Paul: Personally, at that stage, one to two people or maybe one to five people, the process that I favor is "Fix whatever's on fire."If the thing that is on fire is actually just, "This whole thing is not working and we need to rewrite," and it's very likely that you're going to hit that if you literally build it with shoestrings--

Joe: If you're really committed to a minimal--

Paul: Right. But what you'll probably do is you'll probably-- The thing that works least will be rewritten on something that's a little more reliable, maybe you'll put it on serverless or something like that. Or maybe, honestly a Rails app is probably the right thing, or a monolith at least is the right thing as you start to rebuild. Then you can turn that relatively quickly into something that scales for your first 100,000 customers. If you're a B2B SaaS or something, different constraints if you're building a high scale mobile app or something like that, or a consumer app of some kind. But certainly, B2B, you can get away with that.

Joe: Especially a traditional B2B app, it's effectively a database with some business logic in front of it.

Paul: Exactly.

Joe: I always think those are-- It's kind of fascinating. The amount of raw compute you can get today in very small packages-- It's never easy to scale things, but certainly a lot less challenging than it used to be.

Paul:

I think one of the reasons that we end up with a lot of this complexity is because so much of the business value is now provided on the front end.

Joe: Yeah, which is-- Do you think that's part of why the rise of JAMstack, JavaScript APIs and markup, where we're moving towards this notion in a lot of places of fully-formed front end applications served down statically through CDNs that then connect back to GraphQL APIs.

Paul: Right. Exactly.

Joe: With headless back end services, do you think that's part of the reason that that's happened? That because the value is really focusing or pooling in the front end now?

Paul: I think that we solved a lot of what you can do on the back end, and made it at least a little bit easier conceptually. So now the great challenge is to architect these really well-performing-- I just wanna say good front end apps.

Joe: Yeah. So, your newest project gig, Dark. It's sort of an extreme example of this, right?

Paul: Yes.

Dark is an editor, a programming language, and an infrastructure compiler in one. Our goal is removing all of the stuff that we've been talking about and making it super simple to build back ends in the cloud.

Joe: So, effectively I mean you've got your own language, right?

Paul: It's our own language.

Joe: It's a language that's natively defined with a notion of production infrastructure inside the language itself, right? Things like--

Paul: There's a bunch of different ways that we describe it, but one of them is "A language designed for continuous delivery."

Joe: Right.

Paul: So it's built into the language is the concept of deployment of, "How does this get in the cloud? How does it scale?"It's holistic, basically.

Joe: Right. Including things so far as database, but maybe data at rest.

Paul: Yes. Persistent--

Joe: The persistent data.

Paul: Right. It's right in the language.

Joe: It's blurring all those lines, those hard lines that a novice user would have to, "OK. I know how to write some script so that I can run the command line. Now I have to figure out how to run a database."

Paul: Right. "And now I need to know how to write SQL, and now I need to know how to be able to do monitoring on my database." And that's-- We're only in the database side of it. Add that for the compute side of it, for the queues side of it. "If you want to have a service, do you need to be an expert in running Kafka? If you want to do analytics, you need to be an expert in running Cassandra?"

Joe: Right. You have these constructs, like message queues and other abstractions out in the platform and language itself, which I think is really interesting, especially the first time we talked about it. Fundamentally one outcome is to dramatically increase, potentially, the number of people who can write production code.

Paul: Yes. So the way that we targeted is by trying to remove all the bullshit that you have to do to write code and I mean, I'm aiming it at you and me and coders, but there's so many people who are on the periphery of coding who are--

Joe: Right on the precipice. They're almost there.

Paul: Right. They're learning to code, or they are in a coding-adjacent job like a designer or a PM or something like that. Where if we made it easier by removing all the bullshit, like this stuff that we're talking about with serverless earlier and with Lambda, if we just removed a lot of that complexity they would be able to do it. So it's not our first port of call, but it's definitely something that we're thinking about.

Joe: It's an interesting side effect. We definitely subscribe here, and I firmly believe that as much as is being done, and not enough. But as much as is being done to expand the access to people who can become developers, if they're interested in that, the appetite or the need for them I think will be almost impossible to fill as software just winds its way through every single part of every industry.

Paul: Absolutely.

Joe: Software is the kind of thing where you can always make it more complex.

Paul: Right.

Joe: So even if everybody can hire them, I think things that could dramatically increase the number of developers in the world stand to dramatically increase what gets done.

Paul: Absolutely. If you look at the number of people who could be developers or who could build software, and you think, "Are they going to go through the process of learning Kubernetes? Or of learning serverless? Or of writing Docker files?" Or any of those sort of things, when all they really want to do is take data from one API, do some compute on it and write data to another API?

Joe: Right. I think what's always an interesting indicator in these kind of things, there's a few others doing different things, but like semi related so there's-- Glitch is really popular, kind of in the same area. There's repple--

Paul: Repl.it.

Joe: Repl.it--

Paul: Yeah.

Joe: There may be others.

Paul: There's a ton, there's so many of them.

Joe: But it's always a good sign. It's a good sign. I think the fact that there is a signal that there's something happening here, that there's multiple companies coming together. What do you think we call this space? What do we call this solution? It's certainly server-less.

Paul: It's a thing that doesn't have a name yet. The only name I've heard anyone apply to it, I showed Jess Frazelle what we were doing and she called it "Deployless."

Joe: "Deployless?"

Paul: Yeah. I'm not sure if we love it yet, but it certainly applies to what we're doing.

Joe: I could see it's in the running.

Paul: It also applies to Glitch and Repl.it.

Joe: Right. Again, the notion that you're always working on production code. I mean, there's gates and version and everything, but there's not this distinct notion of like "OK, my code is finished in dev, and the pull requests are reviewed, and now we're going to deploy."

Paul: Right. The place that we started with Dark is we made a list of all the accidental complexity that we wanted to remove. One broad category was deployment. If you want deployment to be really fast, if you want to be able to get stuff out to your customers really fast, the fastest that you can do it is zero. Just completely removed. So it becomes, "What does it look like to build software if there is zero second deploy?"

Joe: Off the top of your head, what are the heavy hitters on that list of accidental complexity?

Paul:

The heavy hitters are infrastructure, deploys, using APIs, and then just the software development process itself. Everything from syntax errors to GitHub.

Joe: Got it. So things like actual code versioning and pull requests are native as well.

Paul: Yeah. All of those things for Dark are defined around this idea of deployless, or continuous delivery. As an example, every database table and every function and every type is versioned. You never go through the process of, "I make a change over here and it just takes out something over there."It's designed start to finish around this idea of, you're going to make changes and then you're going to carefully put them out route by route or group by group, use feature flags, et c.

Joe: It's amazing, especially building on now, going on 20 years of best practices, of what it means to deploy a web based application. Whether it's access from a mobile or a browser or whatever. Making those things native is potentially very interesting. Even like you said, just feature flagging, wiring that up. Certainly companies like Edith and LaunchDarkly have made this dramatically easier than when we had to literally get ZooKeeper and a bunch of other things and paste it together. So, are you seeing--? I know it's super early, but are you seeing production applications? That's always a question with new technologies. Like, "This is great for side projects," but people run their business on it.

Paul: At the moment we have two people running their business on it.

Joe: Excellent.

Paul: We're still nowhere near launch. There's so much to do still, but we needed to validate our early hypotheses of "Can people write software this way?" And the results that you get from Dark, it's not even the things that we've taken away that they care the most about, it's now that we have all this stuff in the same place the new things that we can do.

Like the introspection that we allow you to have into your application, those are the sort of things that I think are going to redefine how software development ends up working.

Joe: Just from an implementation perspective, which we get a relatively technical audience, and the thing that's always interesting-- First of all, you're learning a lot about programming language theory, and--

Paul: I kind of have a PhD in that.

Joe: Oh, so still learning some, but--

Paul: No, I'm learning a lot.

Joe: So you're actually going back to your roots.

Paul: Yeah. I mean, when I started Circle, it was very much like "What's the closest SaaS company that I could make that's like roughly 'compilers as a service?'"

Joe: It was continuous integration?

Paul: It was continuous integration.

Joe: I could see that.

Paul: Static analysis is not a massively good market, which was what I did my PhD in. But this one is so much closer, it's like actual programming language tools.

Joe: Then what are the kind of atoms that you're building this platform in? Are you able to leverage things like Lambda or Azure functions, or whatever?

Paul: Yes. We're building on Kubernetes.

Joe: OK.

Paul: It's running on GKE and PostgreS and Google Cloud, and there's this queues.

Joe: It's interesting then, I generally don't like to date these things, but re:Invent was just recent. Which is the annual Amazon rolls out 100 new things. Even now, especially it's like you see Google and Microsoft responding with their announcements based on what's come out. We mentioned Layers earlier, anything else jump out at you this year in particular?

Paul: Just yet again, they have launched many more things. That sounds good, right? People have launched new services, new products that you can use.

Joe: Shipping sounds good, right?

Paul: Yeah. But it's just more complexity. The suite of tools in AWS that you need to make cloud software has increased yet again, and it will increase at next year's re:Invent and the following year's re:Invent. Do they do a six month thing in between?

Joe: I think there's-- Especially these days, I think there's so much that things come-- They do the the small, I forget the name of them. Per city, they have a circuit of cities that go through and do the Cloud Days.

Paul: Every six months or so there's another 12 to 15 services that now you need to learn in order to be able to leverage AWS to build your startup, and that sounds like hell.

Joe: It's funny, as I was talking with a previous guest who focuses on Cloud and Amazon in particular, I asked him the same thing that I'll ask you. I've always found it fascinating that we're going on 10-15 years into AWS, which has changed the whole industry for the better. I think a lot of people would look at the way they're organized as if they're two- pizza teams, but as like very independent units that can prove a service.

Paul: Right.

Joe: The barrier to getting the service out into production is, I think, some people would find shockingly low for a company that large.

Paul: Yep.

Joe: But then the question I always come back to, I do wonder if at some point it starts to kind of collapse under its own weight. Because at some level there's no like directive for these things to play together in a sensible way.

Paul: Right.

Joe: Or for there to be any rhyme or reason other than the-- Now I think they're tiering. I think this is something new, where they're actually creating--They're trying to help you by saying, "Here's the fundamental tier of AWS services, here's your AR/VR tier, you probably care about these things if you're over there."

Paul: It's a good idea.

Joe: Some cataloging, right?

Paul: Right.

Joe: It does feel like it's dramatically more intimidating to sit down and create an AWS account and get things going.

Paul: Right.

We built on Google Cloud, and one of the reasons for it is that it really seems a lot more polished, and that the tools that they-- Everything works together and it works together pretty well.

Joe: It seems like when those things come out, there's one thing you can say for Google, is it seems always very intentional about how those things play together.

Paul: Yeah.

Joe: I was going to ask what you thought of the other players, or how important you think this is?

Paul: Google is either third or fourth in the cloud market, but I think it's by far the best product. The downside of Google is they have less services, but paradoxically, less services is actually better. I don't want to be-- Every new technology of any kind that I add to my app is more complexity that I need to reason about, more things that can go down, it's more risk basically.

Joe: There's a cognitive overhead.

Paul: Yes.

So, less tools that are better thought out and that cost less, is a much better situation to be building on.

Joe: Maybe there's some answer there, but the other question I like asking entrepreneurs, especially those building infrastructure of some kind, and especially those--because the one thing I always talk to entrepreneurs about in terms of the the million dollar question, "How do you compete with Amazon?" Which I know, especially now more in the venture industry, more and more people that's literally on their standard sheet. "How are you going to compete when Amazon announces this is one of their twenty five services next year?" And I always tell people, "Think very carefully if you're going to compete with Amazon on compute."

Paul: Yeah.

Joe: That's something you're charging headlong into. How do you think specifically about how an entrepreneur should think about competing with not just Amazon, but any of the cloud providers rolling out a version of what they do?

Paul: It's interesting. One of the-- Amazon has been working on for several years, on a thing that is like what we do. I don't know what it's going to look like, but I know that they're working on it.

The solution that I tend to find is that the things Amazon invents end up being very good. The things that Amazon copies ends up not being very good at all, so I would advise companies who are building a new thing to not worry about Amazon copying them. If you're building an open source situation that Amazon can just host, that's something entirely different.

Joe: Right.

Paul: But if you're building a new SaaS, yes Amazon will compete. It's "when," not "if," they compete, if that SaaS is in the dev tool space.

Joe: And the "when" is when you've become successful enough that they think the market is there.

Paul: Yeah, and they'll take a junior PM and they'll have them clone your thing and they'll probably make more money off that than you do, but it's not going to materially impact your service.

Joe: I usually talk to people, and there was a period of time I think when they first started these "Every year we're going to roll out our 10 favorite services' clone,"where I think for maybe about 18-24 months there was some real fear. But I think if you've been paying attention, I always call it the 80/20 rule. Though I say usually, they launched the 20% solution.

Paul: Right. Yes.

Joe: Which honestly is, if you've got five million customers or whatever, the attach rate you need to generate meaningful revenue is really low.

Paul: A lot of people who need a service they only need a small amount of it. They don't need all the bells and whistles, they just need the thing that puts data into a queue and takes it out the other end, as an example. And yet a 20% service solves that.

Joe: I think it's interesting from a competitive perspective, on the one hand you have to push further than you used to have to like 10 years ago. What you'd have to do, my product competed with Cloud Watch. I'm sure they have some terrible copy of what Circle does. Probably what we were doing in the early early days that we were able to get traction, which would be much harder now. So you have to build better product, but I think it's interesting that it also raises the barrier to entry for competitors.

Paul: Yep.

Joe: So if you're able to get past that, especially if you're the one driving the market, I think as long as you keep moving forward.

Paul: When I look at what companies are going to be successful in competing with Amazon, I very much look at it as, "Are these putting up a product that Amazon cannot do for some reason?"

Joe: Or, "Will not do."

Paul: Or "Will not do,"yeah. Replicated is a very good example of this. Replicated is building an on-prem installation of your SaaS thing. That's just not Amazon's world at all. It's doing some on-prem for people who really want to be in the cloud but want to have an Amazon looking thing on-prem, and then with Dark what we're doing is we're subsuming the entire thing. We're making it so that you don't have this massive complexity. And what Amazon thrives on is adding more services into the pile, more services into that complexity. So they're not building a holistic product that solves the need for AWS. It's not a thing that sits in their strategic vision.

Joe: I think even as fundamentally in their DNA, at least from what I've seen to date. What it would take to be required to get all these services to work together.

Paul: Yeah.

Joe: Which to their, in their defense, all these services have their own individual drivers and numbers that they are trying to hit.

Paul: Of course, yeah.

Joe: Whereas you have the "luxury" of being able to focus on this one problem.

Paul: Right. The other thing that goes with that is that it does drive up the cost of what it takes to be able to innovate.

Joe: Absolutely.

Paul: So we're building this product, but this is-- We've been at it a year and a bit now. We raised three and a half million to be able to do it, and being able to raise the money to do that pre-product isn't a thing that is common but is the thing that may have to be common to be able to compete in this space.

Joe: I do think in terms of downsides that's an interesting observation, because as impressed as people are with the large cloud provider's ability to innovate and roll out new things, I do think on a more somber note they'll never innovate and disrupt at the scale that millions of individual entrepreneurs will. The fact that the ability of those people to innovate and get a foothold, because it's never-- Again, if you look at every successful company, what they sold in their first year is a shadow of what they end up ultimately building. When that ability to survive on that initial product is made dramatically harder because some leviathan is standing in your way, I think that does at some level slow down overall progress.

Paul: The other way to deal with that is to build less and to build smaller things, smaller companies. There's a little bit of a movement towards bootstrap startups, and even investors investing in bootstrap startups. If you look at Indie.vc. If you build something that is only ever going to make $10 million a year, Amazon isn't interested in that. And if they can't-- If it's something super niche but there is an audience of people that you can get to that Amazon isn't going to, you can build a super protectable product there. The moat is your lack of potential.

Joe: Right. Yes. As long as you're going after an addressable market with a cost structure that that market can support.

Paul: Absolutely.

Joe: I think understanding your market and how big it is, and then how you should build your company to attack it, and what it means is such a critical thing that a lot of, especially first time entrepreneurs don't look at too hard.

Paul: I spend a lot of time thinking about business models. It's like a hobby. When I was looking at what to do next after Circle, one of the options was obviously Dark and one of the options was "Build a nice small SaaS." And the goal of the nice small SaaS was that it wasn't gonna be competed against by some VC backed company or a leviathan, or something like that. You can get very good very quickly at judging, "Is an idea something that is going to be VC investable? Or is going to be Amazon investable?" And if not, then that might be your nice one to 10 person business.

Joe: Yeah. So one thing, and we should probably finish up. But you did mention earlier, so I want to pull on that thread a little bit. In this discussion, the open source licenses, something that's becoming increasingly more contentious I think is this notion that these cloud providers will cherry pick the most popular open source-- And I think actually specifically, the most popular stateful open source projects.

Paul: Yeah, that's interesting.

Joe: And actually offer managed hosted blackbox versions of them.

Paul: Yeah.

Joe: Whereas historically, those companies-- If you look at some examples, there's Elasticsearch in Elastic, there's Redis and more recently Kafka. Just last week was announced within Confluent the company behind it offering a managed service, a Heavybit company it's a slightly different. We have Sidas who offers hosted PostgreS DB, which is actually a very interesting in between. Different than Aurora and RDS, so it stays where it is.

Paul: It has its own special sauce for charting and scaling.

Joe: Exactly, which exceeds the native capabilities. So that's actually one example of like, "How do you navigate that?" What are your thoughts on--? Because you're seeing things where people literally for the first time in 20 years are trying to invent new open source licenses to circumvent this.

Paul: I think there's a general thread in our times about the tragedy of the commons of open source, that so many people are interested in it that everyone else can exploit it. I'm actually not sure that there's a solution, and the reason being that so many people are making open source because it provides such value to them, and the value isn't necessarily monetary but it's in some cases it's a sense of pride. In some cases it makes it much easier to get jobs, in some cases it's just a thing you want to do with your life. But under that world we're just going to keep scaling up the amount of open source there is out there. And all of these challenges are not going to stop that from happening. So the question is, at the other end can it be protected or prevented? Because no one's gonna stop. Was it the common mark that they--?

Joe: Yeah, Redis I think, was--.

Paul: Redis Labs, was it? That came up with it?

Joe: Exactly.

Paul: And they were working with with FOSSA, which is now--

Joe: I think MongoDB was potentially looking into that as well.

Paul: Yeah. We're concerned about the same thing. Like, if we open source what we do, which we're probably not going to do for other business reasons, notably that no one wants to install Kubernetes as the first step of their--

Joe: Right.

Paul: If we're trying to make something that's really easy to setup, "Step one: Kubernetes."No one wants Amazon to run the service or the product that they spent a decade building. I think people are going to make licenses and half the open source world is gonna hate it, and it's just gonna continue to be.

Joe: It's interesting, there's already this play out, I guess about 10 years ago with AGPL which was focused at this initial threat of people taking open source and then using it to build SaaS platforms that they were monetizing.

Paul: And similarly, GPL 3.

Joe: Right. But then those all fell short of, or didn't address this notion like, "We're not going to build a service out of it, we're just literally going to run it for you and manage it for you."

Paul: At the moment people are saying that those are not free licenses. That the new ones are not free licenses, the things that prevent you from running it as a cloud service. But I wonder if essentially an anti -mooching provision--I mean essentially that's what the GPL was in the first place. It had an anti-mooching provision, and I wonder if an anti-mooching from cloud providers is going to be the next step that people accept as being OK.

Joe: I honestly, I've been thinking about this a lot recently. Because it's interesting in parallel to this-- Your point about the amount of open source exploding. I think increasingly the path to delivering software, at least outside of niches, not in real time embedded OSes or whatever. I think there are still niches where proprietary software where you purchase it and you literally get some binaries that you can't look at the source for, and you run them and trust them, is shrinking. I personally believe, at least, that we're moving towards a place where there's going to be really two kinds of software.

There's going to be managed software, whether it's like APIs like Stripe or things like Dark where you're hosting and managing all the bits. The tradeoff for complexity, I don't need to look at the bits, because you're signing up to own them. Anything-- And I remember building my company, we kind of quickly got to a point where like if it's been installed on servers that I am responsible for, it's open source from the CPU up. The OS, the libraries, the applications. It has to be open source if it's running in production, because I'm not going to be dependent on you in the middle of the night.

Paul: If you're running it yourself.

Joe: Yeah, and especially at scale. We saw this, things when we were running the Cassandra database, where we absolutely were-- I mean, very few cases but there were places where we would have to make changes to the core database itself.

Paul: Yep.

Joe: Just to support the thing we were doing, and oftentimes those would make it upstream but we weren't dependent on a vendor's roadmap to line up with our priorities.

Paul: Yep.

Joe: Which they almost certainly wouldn't because we were doing something very esoteric with it.

Paul: It's funny, because we do the exact same thing but Dark is sort of the opposite. I think that's going to be true of any providers. The providers are-- Their value is that you don't have to or get to look under the hood. If you're using Firebase or Stripe or whatever, things that you don't have to manage operationally, then those are the things that get an exemption in my view.

Joe: Exactly. There's a whole half which I think fit exactly that model.

Paul: I'm thinking AWS.

Joe: I think you're certainly right. I think it'll try to be solved with licenses, and I'm more hopeful that it can happen there because the alternative is just that there will be more bits held back in a proprietary fashion. You'll see more companies where there's a core useful open source thing to get over that hurdle, but then they will be holding more and more back because they'll need enough of a wedge against the large providers--

Paul: The stuff that the companies have traditionally held back have turned out not to be a big thing. So for example, the trademark stuff, open source companies have owned the trademark so someone else can't sell something which calls itself Firefox, which is the one that I'm thinking of.

Joe: Right. Not the pick, it's just interesting I just saw it, because Amazon has some small on-prem thing as of last week. So now you can literally "install"Amazon RDS already on-prem.

Paul: Right.

Joe: Which is MySQL or PostgreS, right? They have their proprietary bits, but literally it's come full circle.