Ep. #46, Kubefirst with John Dietz of Konstruct

Marc Campbell: Welcome back to another episode of The Kubelist Podcast. On this episode, Benjie and I had John Dietz from Kubefirst on.

We spent a lot of time talking about the Kubefirst project, their journey through a couple acquisitions, making a commercial product and then Colony.

It was a really good conversation talking about just really delivering best opinions into Kubernetes clusters.

Benjie De Groot: Yeah, I mean, you know me, I'm all about being opinionated and I think that the number of foot guns in our ecosystem is the thing holding us back officially at this point.

I've been kind of saying it for a while, but now it's just getting worse and worse. So I love projects like this that are like, hey, this is how you can get some best practices in place and build on it.

The other thing that's interesting, I think for those folks that are building companies from scratch right now, having these opinionated ways to set up your cluster helps you get into marketplaces with AWS, GCP, things like that is SOC 2 compliance.

I think that angle is also really good where you can, you can kind of generate docs and all kinds of different compliance side, but Kubefirst is a super interesting project.

I wouldn't say it was the most technical, like we didn't get into like, you know, string theory, but I think that the opinions matter and how to understand how to setup this stuff really matters. And I think what they're doing over there is really cool.

Marc: Yeah, enjoy the episode.

Benjie: All right, so Marc and I are here with John Dietz from Kubefirst. I'm excited to dive into that project and hear about the evolution of it.

But John, first off, just tell us a little about your background, where you grow up, how get into computers, first experience with open source.

John Dietz: Sure, yeah, thanks for having me on the show today. My name's John Dietz, CEO of Konstruct, a company that brings Colony and Kubefirst. Kubefirst is really the darling of our company.

It started as an open source project about six years ago. I was our first enterprise pilot and we have for the last six years taken it to market, gone through an incubator, went through that whole process and then just recently got acquired one more time and are sort of on our way.

But historically, I'm just a cloud native, cloud DevOpsy type of engineer. I started out a long, long time ago, came up through IT and the QA space. I was a automated tester back in the very early days of automated testing and then went into load testing and data and performance and then made my way into cloud and cloud native.

That's kind of the quick answer on a very long journey that kind of covered a whole bunch of verticals for a bunch of cool places.

I ran in a consulting business for 19 years, Integrity Test Solutions, and through that vehicle had just some really great experiences with a whole bunch of different worldwide organizations, USA Today, Northrop Grumman, Hewlett Packard, and got some really good platform building experience out of that and have been spending the last six years trying to codify that platform building experience into our product.

Benjie: Cool. I want to go back for one second. I do think that there's some grizzled veterans like myself that know a bit about the old days of automated testing, but I think that we have some younger folks that listen and would love to hear what it was like to be an automation test engineer.

You know, I assume in the like early 2000s, mid 2000s, is that right?

John: Yeah, you nailed it. We're talking 1999, 2000, 2001.

Benjie: So that's pre Selenium. Was Selenium around yet? Web driver?

John: I don't think it was yet. It was a lot of, not even .net, it was like basic and Sun Microsystems and we were just getting into the days of being able to build like three tier architectures and getting away from telco was like, you know, the big boom.

And being an automated tester in that day was incredible for me. Maybe not for everyone in the industry.

It was a weird time, like in 1999 or even further back in '97, colleges weren't teaching you test automation, test automation didn't exist.

Benjie: I don't know if colleges are teaching test automation right now, to be honest.

John: I believe that's true. They probably teach you some unit testing disciplines and stuff like that.

Benjie: Maybe functional.

John: Yeah. But back then it was like, you know, there were teams, QA teams that had to manually go through books and books and books of test cases and systems had to be specked out, literally like printed out on paper and test plans put together and manually executed and test automation sought to disrupt that.

And Mercury was a really big player. They eventually got bought by Hewlett Packard, but Mercury had Wind Runner and Load Runner as two of their main pieces of code and those were the two that I hooked into.

And you know, it wasn't anything multi-threaded or elegant. You literally had to have like a whole bunch of PCs and each PC could run tests for you. But nevertheless it was a huge evolution in being able to deliver quality software.

So I did that for, you know, maybe about like five years or so and all the while I was, I had my left eye, you know, sort of eyeballing the software development space, looking at the DevOpsy problems of performance and performance tuning of databases and stuff like that.

And that kind of just sent me off into a career where I covered a whole bunch of different types of verticals for a whole bunch of different companies through the consulting business I put together.

Benjie: Okay, cool. Yeah, no, I was just curious about how you did test automation in those days. I like to complain about Selenium, but that's more of aughts problem than a 2000s problem I would say.

And now we got Cyprus and Playwright all that stuff, but let's fast forward a bit. Okay, so obviously you started off as a test engineer a while ago.

You became a consultant, proper engineer doing all kinds of stuff and then you started Kubefirst and this is an open source project. So first off, what is Kubefirst?

John: Yeah. So Kubefirst is an instant platform for companies to adopt good patterns on free and open source tools.

So a lot of people when they go to adopt Kubernetes for the first time, they're net pretty quickly with a very tall CNCF landscape of products that they have to choose the right products and put them together in order to get good business value out of Kubernetes.

So when my co-founder and I started building our first Kubernetes platform way back, you know, talking 2017-ish, it was hard to do and it took us like about a year to build a platform for this startup company that had, you know, the infrastructure as code automated, that had application delivery to a bunch of environments, had the ability to manage Kubernetes clusters and all that kind of stuff.

And we both left that company going new companies thinking, well, you know, we spent all the time and research figuring out how to put good pieces together. It's going to be easy at the next place and it's just not.

And it frustrated both of us to the point that we began working nights and weekends on an open source project to try to build instant companies.

We know that there are certain disciplines that every project that's starting right now is going to want from single sign-on to infrastructure as code to a, you know, modern way to deliver applications and build applications and ship 'em to your pre-prod and prod environments.

And over the course of time, the Kubernetes landscape has also begun to centralize a little bit on some best of breed tools, if you will.

Not to say that any one of them can't be replaced, but there are certain tools that everyone tends to centralize upon. And it's with good reason. There are good free open source tools that can provide a ton of value to organizations if they could just put 'em together the right way.

So that's what our project is. Kubefirst is trying to be a very, very low barrier of entry to have a very, very comprehensive polished, production ready Kubernetes ecosystem that can build Kubernetes clusters, that can deliver GitOps applications, manage your infrastructures code, do governance and stuff like that.

Marc: But not like proprietary. You're saying you're like, so I remember like I'm going to go back, right? Like Replicated has been around since before Kubernetes. And when Kubernetes came out we were like, okay, this is really cool.

And now we have Kubernetes clusters and we have automation and we have GitOps and we have CNCF. We have like all of this pipeline and everything that we actually deploy code for. And sometimes you forget like how hard that was to set up.

Like even things like, oh we're going to set up Flux. That's what we picked back in the day and we're like, we're going to set this up. And I'm like, wait, how do I do Secrets? Wait, how should I structure this repo, all these decisions.

And so you're saying that's what Kubefirst is doing for you is basically kind of giving you that runbook, that playbook and saying like, just look, we're going to jumpstart you and give you the best practices here.

John: Yeah, that's exactly right. So we know that you're going to need a domain, we know you're going to need a cloud and we know you're going to need a Git provider and if you can just tell us the details for those three things, we can build a Kubernetes cluster in any cloud that you want and hydrate that Kubernetes cluster with Secrets management, user management, OIDC providers, so you have single sign-on into all your applications.

And then there's also all these chicken and egg scenarios of building up that management ecosystem. 'Cause you're right, some of these applications need Secrets, but one of your applications is a Secrets manager that doesn't exist yet.

So like there's all these chicken and eggs that you have to sort of sort out and we've gone through and established all of the orchestration to shake all of those chicken and egg scenarios out. So you basically just tell us what cloud you want, what DNS you want, and what Git provider you want to use, and hit a magic go button.

And then we build your Kubernetes cluster, we build your platform out, we build up your user ecosystem, hook it into all the applications on the platform, and then we give you the GitOps repository that has all your infrastructure's code and that has all your GitOps so that you can change anything that you don't like about the platform.

If you want to change ingress controllers or whatever it is someplace that you want to put your Secrets somewhere else or whatever it is you want to do, you're able to just start submitting pull requests to your new GitOps repo.

Marc: And like we're going to spend more time talking about it, but like, this is like the features. I want to definitely dig into that some more and how it works. But like you started this just as an open source project, right?

John: Well actually, so we didn't start it out open source, if you can believe it. So it's 2017 and Kubernetes is really hard, right? And you're reading blogs to try and figure out how Kubernetes is supposed to work, everybody's lying to you and it's just like a dark, difficult place to run software.

And so my co-founder and I, we were like, you know, look, if we can build this thing so that it's instant, I bet we can go company to company and get people to use this platform. But we were building it closed source at the time, it was just completely stealth.

We didn't want to open it until we could prove that it worked in an enterprise. So we went through a one year pilot with this company Virtru that does encryption for Salesforce and G Suite. It's a a FedRAMP medium environment.

It's, you know, FIPs 140-2 all over the place. It's just like a complex compliance environment. And we were like, you know, if we can survive this place with a hundred engineers, then this thing scales, it's ready to go, you can customize it to anything.

And that's when we wanted to try and raise capital. So we got through that pilot, we open sourced, had a brief relationship with Chris Aniszczyk, the CTO of CNCF and he was like, look, with that pilot we'll put you on the CNCF landscape right away.

And that was enough for us to feel like that was a signal that we were ready to go and try to raise capital. But neither me nor my co-founder were well connected in the space and raising capital-- That was probably the most difficult period of my life. That was just like hard.

Marc: That was like 2017, 2018?

John: Yeah, about 2017. Yeah.

Marc: Pre-COVID.

John: Pre-COVID. Exactly. Exactly.

Benjie: I want to talk about the company building, we're going to get back to that in a second, but I want to hear a little bit more about like, okay, so you said for example, it generates a GitOps repo for you.

Is that Terraform that's generating? What is the IAC, is Pulumi, what is it, what's going on?

John: Yeah, that's exactly right. So it's Terraform out of the box. You can migrate that to OpenTofu super easily if the license issues with HashiCorp products is out of bounds for you.

And the Terraform, we basically break up the infrastructure into four base modules that you get from the install, will create a Git module so that you can manage your Git repositories and the teams of your engineers and what repositories those teams should be assigned to in IAC.

We have IAC for the cloud, that's how we build out your network, your Kubernetes cluster, your S3 bucket for state store, stuff like that. And then we have a user's module, which is how we build a self-hosted single sign-on ecosystem so that you can start to, you know, just add your own engineers without getting some kind of a corporate SSO provider involved.

But you can obviously just shift it over to Okta or whatever it is you want to host for SSO. And then the last section is Secrets management.

We use Vault as our Secrets manager and a lot of the initial Secrets for the provisioning of the platform get created through Terraform and then persist through Vault, our external Secrets operator.

Marc: So once you spin all this up, right, and I'm not trying to minimize the amount of effort there because getting those best practices and those patterns is really, really key.

Equally hard, if not immensely harder is updating it and maintaining it for the lifecycle of this thing. So you do all this, right? And now new versions of Kubernetes have come out, new versions of like Vault are out.

How do you ensure that somebody using Kubefirst keeps like current?

John: Yeah, that's a really good question. So we have an upstream GitOps template repository, which is sort of the origin of the opinions of how you get your management cluster.

So when you do an install, we ask for your DNS, your cloud, whatever, and then we provision that in your cloud for you in a Kubernetes cluster that has all the tools in it that allows you to build and operate that platform.

But you're not typically going to put your apps in that cluster in most organizations, most organizations are going to have another cluster that represents their development, their staging, their production environments.

In more complicated environments, sometimes there's an east or west or a GCP and an AWS variation of those clusters. But that's where you want your workloads is in those workload clusters.

So the way that we've organized it, when you get that GitOps repository, it has the instance state of your management ecosystem, just like in clusters management.

But we also give you a templates directory, which has the definition of how workload clusters get created by the platform. And we have a user interface called Kubefirst Pro that lets you basically just click to create new clusters that are based on those templates.

So we'll start you off with some reasonable starting point templates for both virtual clusters and for physical clusters in your cloud that have your usual suspect apps like Cert Manager, Reloader, External DNS, Ingress, Nginx.

But we stop there because we know that the apps that you want in your workload clusters are your decision to make. So you basically just get to add applications to your templates and then anytime you create a workload cluster, those applications that you put in your templates will go in those downstream workload clusters.

And the workload clusters that you start from are in our upstream GitOps template repository. So you can pull those back down. And then from a management ecosystem, that's a harder upgrade to do frictionlessly.

And there's a handful of devils in the details, but at the end of the day, the detail to note is that as long as you're starting from our upstream on the latest version, you're adopting the latest version of Kubernetes, the latest version of all of our management tools, everything that we're upgraded, you get those upgrades for free.

That's basically the architecture of the fleet management system.

Benjie: Okay. If I'm listening to this and I'm like, okay, well I mean you've got helm charts, you've got Terraform stuff out there, or OpenTofu stuff, whatever, why would I want to use Kubefirst and not those?

I mean, I have a list in my head, but I want to hear why I would use Kubefirst from your perspective.

John: Yeah, sure. So there's a lot of different reasons that people choose us. Sometimes it's because of a time crunch.

There was one organization we were just talking to where, you know, they had an operational center responsible for hosting their infrastructure and for their application delivery and that had a bad relationship and all of a sudden they had no CICD anymore.

They had no way to manage their infrastructure and deliver their applications. It was a huge problem for them and they needed an answer fast.

And with the Kubefirst install, you have your entire company in that first hour where most companies, if they're going to build that themselves, even if you hire 10 of the best cloud native engineers you can find, that's still going to take you three, six, nine, 12 months to build that platform out. So a lot of companies don't have that luxury of waiting to get business value out of Kubernetes. They just need a operational platform right now.

And there's a different section of our user base that is unopinionated about Kubernetes and just wants good patterns. You know, you look at the CNCF landscape, there's, you know, got to be what, 400, 500 applications on there?

How do you pick the right one for your Ingress? Is it just whatever the biggest rectangle is? You have to trust somebody. And you can go to blogs and you can start to trust a sense of people with good opinions, but if you're not getting the full platform, you're only getting part of the story and you have to sort of cobble together those stories in order to figure out how to build a platform. And that process just takes a long time.

Lastly, there's a section of users of ours that really value the community that we built. So we have a Slack workspace that's free for our open source users, and there's somewhere in the ballpark of 500 engineers in there.

And these are all people that want to be centralizing on the same tools, using 'em the same way. So that when you lose that awesome SRE or that awesome platform engineer, you haven't lost all the knowledge, you still have a community, you still have a company supporting you and you still have a place to go to ask questions about how to run the platform that you're running.

So, you know, a lot of consultants will build bespoke implementations of how to run Kubernetes for your organization. And that's not wrong to do, but you're not left with a huge community of people using the same stuff the same way. So a lot of people find value in that.

Marc: So like if I'm using Kubefirst, right? And like let's say I'm like, hey, I have this, there's a couple of us, it's an early startup, and I say like, this is obviously I don't want to go spend the next three months or one month or whatever it is, like setting all this up and have my opinions, I'm going to click Kubefirst be done like in an afternoon.

How does that like grow with me? Like if I like then onboard an SRE team who maybe has different opinions and stuff like this, have you gone through that journey with like startups or do you end up just kind of selling to the larger customers and selling to startups and have a different motion for each of them?

John: We have a different motion for each of 'em. You're sharp to pick up on that detail. And the reality is that like they evolve however they think is best to evolve.

In fact, our first pilot with Virtru, they ended up taking our platform into clouds that we never even supported and go get 'em tiger. You know, you kind of get to take it in any direction you want.

With each decision that you make that deviates from the intention of the platform, you're kind of making a decision about like, well, am I just going to maintain my own updates for these applications?

Am I prepared when I start extending this into a new cloud, am I prepared to continually maintain the infrastructure's code for that new cloud? The answer's yes, then go right ahead. If the answer's no, then you know, maybe think twice.

But it's a very flexible contract free relationship to the point that like, you know, even the commercial user interface that we add to the platform at the end so that we kind of have some commercial viability behind the adventure.

If you were to delete that user interface completely and just for that matter, if Kubefirst, the project ceased to exist forever more, you're not impacted because you're not buying into Kubefirst. You're buying into these open source tools and GitOps.

So your GitOps repository is still going to manage your apps and Argo CD is still going to be responsible for delivering them and your infrastructure's code is still going to work the way that it works today and the speed with which you want those details updated.

If we're not going fast enough for you to adopt our updates, you get to just change it yourself. Or you can get our latest workload template updates, pull 'em into your repo and start using 'em to provision your clusters on our latest opinions.

Benjie: So, okay, first off, that's super helpful. Let's say that I'm an enterprise that's like, I need to go Kubernetes, right? Because I see that a lot at Shipyard. We see a lot of people transitioning, they don't even know where to begin.

Marc: So wait, it's 2025, you're literally seeing folks who are like, I'm ready to start my Kubernetes journey right now?

Benjie: Yes, I am.

John: I am too.

Marc: Really?

John: Yeah.

Benjie: So Marc, what I see with Shipyard a lot is folks in the enterprise space, there's one or two teams that have it and then everyone else is trying to catch up. But a lot of people don't necessarily, it's still a bit of a buzzword rather than a practical...

Marc: You're saying like you see these, like this many of the Fortune 500 or Fortune 100 have Kubernetes, but like there's a different graph which is, which are like actually running Kubernetes across the majority of their infrastructure versus like a pocket here and a pocket there?

Benjie: Right and I would guess it's like three of the Fortune 500 are actually doing across the org.

Marc: I go to some bigbankwhatever.com, like is it a Kubernetes cluster serving this or not? Like, probably not.

Benjie: Probably not. I mean I know JP Morgan's pretty good about stuff. But anyways, say that I have an initiative at an enterprise company to take more of my teams onto Kubernetes.

I download Kubefirst, CLI and then what do I do? I mean just very simple, just very simple version of this.

John: Yeah, but very simple. We have a few ways to install, we have marketplaces and whatever, but usually it's you download a binary CLI, you can brew install it, and then you just run Kubefirst AWS Create or Kubefirst CIVO Create or Kubefirst DigitalOcean Create.

And there's three flags that you have to give us some details on for your DNS provider, your Git provider, and your cloud. And that's it.

Benjie: I'll give you my IAM credentials, some type of Git.

Marc: Lots of secrets.

John: Yeah, it's like three sets of keys that we need for your DNS, your cloud and your Git provider. Yep.

Benjie: Okay and so then that literally it goes into my AWS account or my CIVO account, let's say AWS and it uses IAM role, it'll bootstrap like actual nodes. Is this going to be EKS or is it I can choose, it could be EKS or it could be my own distro of K3D or something.

John: Yeah, we do support K3D, but in all the clouds, by default it's whatever managed Kubernetes they have.

Benjie: Okay. So you're going to bootstrap my EKS, you're going to gimme some node pools and so at the end of running this and it's all local, that's super, I think I want to make sure that's clear. Like using the CLI like those Secrets are not giving to the cloud anywhere. This is all staying local.

Marc: When you create that EKS cluster versus, you know, an Azure cluster or an Oracle cluster or whatever, you're also using AWS's like VPC CNI, EBS CSI, all of the specifics for that cloud provider too.

John: That's correct, yeah. We try to adhere to the best patterns of whatever cloud we're in.

Like in the big three clouds, for example, we go so far as to like set up, you know, individual service accounts and find those service accounts through the OIDC provider so that you'd get, you know, at least privileged roles and stuff like that.

In the smaller clouds, it'll have a different fingerprint. It'll be, you know, what's your API key for your cloud?

And it's a lot less Terraform resources, but at the end of the day, yeah, we're just building up the management ecosystem in the safest and most production ready way that we know how to, and that's where you start.

Marc: So I'm on Kubefirst create and then I get this repo and I get like a bunch of Terraform in it. Then do I have to run Terraform plan, Terraform apply or does Kubefirst actually execute that for me too?

John: Yeah, we'll execute that as well and then at the end of it, we will even bind it to Atlantis so that from that point forward, when you submit pull requests to your GitOps repo, if it's a change to the Terraform directory, Atlantis will run a plan on your behalf, show you what the plan would look like if you wanted to apply it.

If you like it, then you can just type in your pull request Atlantis apply, it'll go through the motions of committing that, of locking your state so that your state's secure and stuff like that.

Benjie: Okay. So my example of being an enterprise architect or DP of engineering or whatever, this is really good for me because I can go back and be like, look, all of these accounts are locked down, this is best practice stuff.

So I assume that this plays really well with HIPAA and SOC 2 and even getting into the Amazon Marketplace, I'm guessing it helps a lot?

John: Man, you are saying all the right words this week. We are just starting our SOC 2, we're just starting our AWS Marketplace mission. So I think that you're exactly right.

What we would love to do is be able to provide not just a Kubefirst platform, but a result of a SOC 2 audit and a HIPAA audit so that you know exactly what the details are that you will have to go through from that default install in order to gain SOC 2 compliance in your organization.

Obviously there's this asterisk of, well, except for whatever else it is you might be doing in your organization, and that's just an asterisk that we have to carry.

But you know, we hope to be able to produce that deeply compliant ecosystem for everyone on first minute.

Marc: That's cool. So you're going to, it's not like traditional, like I'm a SaaS provider and I'm going to go through a SOC 2 audit.

Instead of that, you're going to like basically use Kubefirst to spin up the infrastructure in these different providers, go through a SOC 2 audit on each of those. Be able to say like, look, we don't feel like we're going to prevent you from getting SOC 2, let's start there.

Benjie: Yeah, exactly.

Marc: Yeah, that's cool. That's super cool.

Benjie: What's the pushback people say? Like, I don't want to use Kubefirst, I want to use Terraform, or I want to use--

Marc: I want to write it myself.

Benjie: Obviously everyone's going to be better at writing it themselves. Like there's no reason for anyone to not write everything themselves.

But putting that one aside, like what is, is there any actual, like if I'm trying, again, this is me as the enterprise architect at an enterprise trying to sell it internally.

You've given me a few good ones, but what else is there, what else is out there?

John: For the most part, the biggest challenge that we have is finding people at the right time in their journey.

You know, there's a lot of shops that we go to that feel very interested in the Kubefirst platform, but they've already built up opinions about what they want for their tech, for their cloud native tech.

You know, they just really feel passionately that, you know, they have to have not Argo CD, they need Flux CD and that that's just a deal breaker for 'em and we miss on those folks.

And optionality is very expensive for us. Just when you think about the number of clouds and Git providers and DNS providers we support, to add additional options on top of that is just variability that makes it prohibitive to test and to get exactly right for everyone.

So we have to be very stubbornly opinionated about the starting point. And that's a turnoff to some folks. Like, you know, if you just really, really needed your management cluster to always start a certain way.

In order to combat that, we've recently implemented workload cluster provisioning in our ecosystem and creating a workload cluster doesn't have to just be for like your dev stage prod workloads, a workload cluster could also be a centralized ecosystem for your CI tools or for your security tools.

And today, those tools live in management. So we are working through some stories to allow the end user a little bit more control over the components that get installed by default in that initial install and how to be able to reorganize things so that, well, it's fine that it starts out in the management cluster, but long term I would really like it to be in a CI cluster to get sort of those stories built into the installer process.

But for the most part, I would say our biggest challenge is just finding people at the right time. I hear over and over again, man, I wish I knew about you 10 months ago. You know, and that's frustrating to not have gained the popularity to prevent that from happening.

Like nobody goes into Kubernetes and doesn't know about Flux or Argo CD and we just have to get to that level of popularity so that we don't miss the users that are entering the scene, you know, and looking for that right starting point and not able to find one.

That's truly been our biggest challenge is just kind of getting the awareness around the project.

Benjie: Okay so the actual code, like, so it's a CLI, it runs some, you wrap Terraform or?

John: Yes.

Benjie: Like how do you apply Terraform?

John: How do you apply Terraform? So the CLI itself does it--

Benjie: Are you wrapping stuff? Are you bringing in, I'm just curious from a CLI perspective, there's always a, this is a little bit more technical obviously, but there's always a, like, do you install the binary for kubectl and Terraform or?

Marc: A vendor I'm in.

John: Yeah, it's soup to nuts no matter what, whether you're doing a binary install or a helm install or a CLI install, we're going to make our way into a Kubernetes cluster.

So if you don't have a Kubernetes cluster, the CLI will take care of that and create a K3D cluster literally on your laptop and install some installer components into that K3D cluster.

If it's a marketplace install, then you're already in a cluster. If it's a helm install, then obviously you're applying that directly to a cluster. So we're in some cluster somewhere with some install software.

Now what can you do with that install software? That requires downloading this GitOps template repo that has some Terraform in it. And then we have to hydrate that template with the details of your shop.

Like what's your DNS? What cloud are you going to, what's the account ID? Stuff like that. And then once we have that hydrated GitOps content, we can run proactively through the CLI execution, impact from that point, it's actually happening in a Kubefirst API that's running in that Kubernetes cluster.

And the API just has its own persistent volume. It's able to run Terraform applies. It's able to run really any sequence of operations until you get that cluster to a point where all of the tools are installed, they're installed in the right order, all of the certs have resolved, all of the DNS are propagated.

And then we give control back over to the end user and say, there's only one admin in this platform, this is the password for that admin, hop right on in.

And so at that point you leave this sort of ephemeral installer cluster environment and you move into your permanent management ecosystem that also shares that state, but it is being managed using Atlantis and GitOps.

And from that point you can always assume that you have your IAC square and you have your Secrets in place and you have all of the roles that you need in order to provision things and you can just start creating more clusters and shipping apps to 'em.

It comes with a sample application called Metaphor. Metaphor uses all of the tools on the platform.

So it's pre-dated with annotations and helm charts for Ingress NGINX, it's pre annotated with cert manager stuff and external DNS stuff so that you can really sort of feel a tangible way, like if you went to the metaphor repo that we give you during an install and just change anything in the main branch of the repository, a CI pipeline's going to kick off automatically.

And that CI pipeline, whether you're a GitHub shop or a GitLab shop, you'll have a self-hosted runner running in that cluster, in that management cluster that is pre-configured to be able to invoke Argo workflows.

And we have a whole bunch of Argo workflows, which are Kubernetes native CI automation pieces basically. And we deliver the platform with an opinionated and very decent starting point for how to build your first application container, how to build your first helm chart, and then how to do GitOps delivery in a promotional way to your development staging production environment.

Metaphor kind of tells that half of the story, that kind of day two on the platform story.

Marc: Yeah, there's like multiple things that you're talking about here that Kubefirst is doing that provide a lot of value.

One is just like saving me the days, the weeks of writing all of this stuff. That's one part of it. I think the bigger part is kind of the opinions that you're actually serving.

You know, you talked about like you can't handle the differentiation everywhere. Obviously if I'm like CloudFlare or Google DNS or AWS Route 53, like you have to handle that because I'm not going to switch my DNS provider, but like the other one that you were alluding to earlier was Flux versus Argo, right?

Like why should that matter which one I'm using? As long as I'm solving the purpose of GitOps, especially if I'm like new to Kubernetes. Like kind of going back though, there's a question in here.

The opinions are the interesting part and I actually want to dig into like how you're coming up with the opinions and what you do to ensure confidence in enterprises who want to buy the Kubefirst platform and be like, yeah, these are not just a set of opinions, but they're the right set of opinions and they're going to work for us.

And they're not just like saving me from making a decision, but there's going to be significant overlap with the decision that my own team was probably going to come up with. Not maybe every one of them, but like overlap,

John: Right. Yeah, no, it's a really good question and the answer's super nebulous. Like we have our community Slack and that's where we get almost all of our feedback from the ecosystem, direct feedback about like, hey, you guys should do this instead of this because I want my Secrets managed this way or that way, or hey, instead of a IAM credential, I want an IAM role.

That kind of feedback is just from people using the platform, finding something that is off with what they walked into with their expectations and then a conversation and a decision on whether or not we're going to provide that option or whether we're going to change to that opinion or whether we're going to let go of that type of a user.

With all the feedback that we get in the community, that helps us a lot. We're also incredibly studious and have a really good platform engineering team. They're just very, very talented at what they do.

They've been doing it for the largest organizations for the longest time, they're leaving these great companies just to work for my tiny company, you know, so that we can solve this problem for the world together.

And it's a blessing to have them, but all the credit in the world to what they bring to the organization. They bring just the best patterns and the best practices and they study hard and they pay attention to the industry and bring it back to the shop.

Marc: I mean it's such an interesting fun role I think for an SRE or somebody who has done this platform work before because you know, like you might be working for a large bank like an enterprise and you're like and I'm enabling all these developers.

But like you get like more leverage more or like your work is more people are using it when you're developing a platform like Kubefirst that like all of these other enterprises use. That's super cool. Probably helps you a little bit with hiring.

John: Oddly less pagers too. Yeah, like our production fingerprint is not very large. We have to make sure that you can brew install our stuff. We have some charts that we host.

We have some marketing stuff and a couple rather small services behind a dashboard in our SaaS ecosystem that we're just barely starting to build out. So yeah, it's a great place from an ops standpoint. Nobody wakes up in the middle of the night usually.

Marc: Yeah. The next series of questions I wanted to ask was like, I've been poking around on the website and like there's Kubefirst and Kubefirst Pro as two different products. Like explain that, like what are the differences?

John: That was really hard to do, you know, take a step back. It's me and my founder, my co-founder and we're just trying to open source a thing and see if we can make it popular and see if we can form some kind of business around the product that we're trying to build as an open source product.

Which like we took that a very long distance before it became a challenge of like, all right, well how do you commercialize a product that architecturally it was really important that we gave you the GitOps repo to me.

I didn't want an ecosystem where we keep the keys to your kingdom up in our cloud or anything like that. I wanted it to be very, you install and there is no contract with us. We don't need to live, we don't need to exist, you don't need to like our opinions, you can just leap the reservation and run with it. And because of that type of an architecture, it was very, very difficult to try to commercialize on.

And we decided that by investing in a user interface layer and making that our commercial angle, we could always have this open source platform that everyone could always benefit from and that would always benefit the very small founder that's trying to build something and get it off the ground, which is a really important part of our mission.

Like we're only where we are because companies like GitLab offered free software that we got to use and take advantage of and build stuff with and we want to do that very same thing for our users.

So introducing the commercial angle on the user interface was tricky and it was a very nebulous story for quite a long time. And that was while our company was also called Kubefirst.

So we had Kubefirst the company, we had Kubefirst the open source platform and we also had this user interface that we were calling, you'll never guess it, Kubefirst.

And after our last acquisition we got presented with an opportunity to build a new product called Colony. It's a bare metal provisioner that basically lets you repurpose your servers into Kubernetes clusters.

And when we took that opportunity we said, all right, there's a second product. We have a company and a platform and a product called Kubefirst and then one more product called Colony, that makes no sense.

So we rebranded our company last fall became Konstruct, and we said, okay, we're Konstruct and now we have two products called Kubefirst and Colony.

But then we said, ah, there's an opportunity here to make a stronger line of distinction between our open source platform and our closed source commercial aspirations.

So we decided to rebrand our user interface Kubefirst Pro, just to make it a little bit more obvious that that's when you're entering the commercial element of what we're trying to do.

Benjie: So Kubefirst Pro, is this on-prem interface or is it a SaaS hosted version?

John: Yeah, no, it's hosted in your management cluster. So we'll create your management cluster for you. And then the last thing we add to it is Kubefirst Pro and that's the UI that lets you press buttons and create clusters.

And we also have this like huge GitOps catalog of open source apps because like the science of building that opinion set of what should be in these clusters is daunting 'cause every company's different.

Benjie: I mean I think you just said it best when you just said the science of opinion. So yes, very much so. I think the foot gun of the CNCF landscape is something that we all navigate daily. But Marc and I especially.

So at the beginning we talked about the journey of now Konstruct. So just to review, Konstruct is the company, Kubefirst is the open source product.

Kubefirst Pro is the closed source commercial product and Colony is the bare metal bootstrapping Kubernetes. And that is open source as well. Colony is open source as well?

John: Colony has open and closed pieces. The good science is closed source. There is an open source CLI and we also have an open source vagrant data center that you can use to experiment with our Colony product.

'Cause like it's built for data centers and we don't want to expect that everyone's just got a data center in their basement. So we built an open source vagrant based data center with like spine leaf architecture and it's pretty wild, but you can spin up this Vagrant data center and then just start spinning up Talos clusters or K3S clusters.

We also support the creation of virtual machines like Ubuntu or whatever, and it's leaning on an open source product called Tinkerbell.

Tinkerbell does bare metal orchestration tasks and Colony uses Tinkerbell natively to be able to stitch those tasks together and produce more macro oriented automations like what you need to build Kubernetes clusters and stuff.

Benjie: And Tinkerbell is not Konstruct?

John: That's not ours, no.

Marc: But I think it is a CNCF project though.

John: It is CNCF, you have a great, great team too. The founders of that project are exceptional. They came from Equinix Metal back in the day, they didn't work there anymore, but a couple guys with a lot of just really great deep experience in data centers.

Marc: So colony like the vagrant VM, it's interesting. So it's not like I see, you know, again, poking around on the website here while we're talking, it talks about using CIVO Cloud for it, but it's not like a tight integration there.

Like if I just have some Ubuntu machines running in my basement or in AWS, I can actually run it there too as long as this like I can handle message virtualization on it.

John: Yeah, that's exactly right. That's exactly right.

Marc: Got it.

John: Yeah and we do have one deep integration with CIVO. CIVO offers a self-hosted cloud if you want to run a private cloud in your data center, kind of like I think EKS Anywhere kind of has a similar story.

Benjie: Are these machines or is it software?

John: Is what machines or software?

Benjie: Is that product that CIVO has, do I like drop in a rack at my data center?

John: Yeah, exactly, so they built the implementation in such a way that they expected to sell appliances and they can just kind of ship you some hardware, you plug it into your data center, it phones home, and then all of a sudden when you go to civo.com, your private regions show up as available regions, right in the cloud experience.

And if you wanted to host that self-hosted cloud on your private data center servers, you'd be able to use Colony to repurpose those machines into Kubernetes, specifically Talos for the CIVO enterprise stack.

And then we'll install the CIVO enterprise stack to that Kubernetes cluster and then phone back to CIVO on your behalf so that it's integrated with our cloud experience.

Benjie: Yeah, that's super cool. Okay, so you mentioned CIVO, you talked about this in the beginning, I just want to cover this because we're running a little short on time.

So Konstruct the company, Kubefirst at the time, you said you've been acquired twice. Just tell us, give us a 30-second because I think we skipped over that. I want to, that sounds interesting, I'd love to hear.

John: Yeah, no, it was a crazy adventure. We talked about sort of building it, running a pilot, open sourcing, trying to raise capital.

While we were doing a capital raise, we had a couple angels lined up and then we ran into Kubeshop. Kubeshop is an incubator that focuses on Kubernetes projects and gave us an opportunity to build a rather tiny team, but to go right now.

And so we said, let's do it. I got married, we were there for about a year and a half, I feel like, maybe almost two years. And we were able to build out a lot of the concepts for the platform and bring us right to our commercial go to market, at which point we got acquired again, which was the purpose of being at that incubator, frankly.

Benjie: So the incubator sort of acquired you, but really just funded you.

John: Yeah, that's what it was.

Benjie: Then CIVO came in a few years later and they actually bought the product. So Konstruct is a part of CIVO, which is a cloud. I'm familiar with CIVO, that's a cloud competitor to the big three, basically.

John: Yeah, they're a lot like DigitalOcean. They're based in UK and it's a little bit of a tricky setup. So we're operationally and from a corporate standpoint, we're distinct from CIVO. The main investor of CIVO shares ownership with us and there's a parent organization.

In fact, Kelsey Hightower just joined the board of that parent organization. There's also defense.com as part of that sort of network of companies under some shared ownership.

So we're not really acquired by CIVO. CIVO is the name that you're going to be a lot more familiar with, but we have a really good partnership and working relationship with them.

Benjie: Yeah, Marc and I actually were doing some on the ground KubeCon interviews and I believe we talked to the CTO of Kubeshop actually. So the name's been popping up a little bit these days. That's interesting.

Okay, cool. So now this an open source project, it sounds like you have some contributors outside of your direct company. If I wanted to contribute, where do I go? How do I find you guys?

John: The best place to go is probably just Konstruct.io and then up in the headers you can find your way to anything we have.

We do have a Kubefirst GitHub repository that's open source. It's at under the org Konstructio, all one word Konstructio.

Benjie: Yeah. And we'll give links in the podcast notes of course.

John: Yeah, thanks so much.

Benjie: You guys have community meetings weekly, monthly, anything like that?

John: We have a live stream that we do every two weeks. We try to talk about everything that we're doing, everything we're building, we bring guests on of relevant products that are adjacent to the platform environment that we're trying to provide for our end users.

We don't have community meetings, but we do have a vibrant community Slack. And if meetings were ever interesting to anyone, you just have to get our Slack and tell us.

Benjie: So Slack is the main place to contribute and get those opinions in there. Obviously using Replicated and Shipyard are part of the best practice for all CNCF projects.

Everyone knows that, so I don't even have to go on Slack for that one. No, but okay, so Slack is a good place to do that. The stack is Go, it's all Go, mostly Go, some generate Terraform.

John: Yeah, it's almost all Go. We got a little React stuff going on for the front end and yeah, the IAC is mostly Terraform. Sometimes it's a combination of Terraform wrapped in a cross plane provider.

Benjie: Okay.

John: And you have the ability to eject out a cross plane into Atlantis if you'd rather be in more of the human Atlantis side than the Git Ops cross plane side. We have stories to help accommodate all those needs.

Benjie: So even you guys aren't completely opinionated is the point.

John: Yeah, I mean how can you be fully opinionated these days? There's just too much optionality and there's so much nuance.

But yeah, you know, I think we are hitting the nail on the head as to a very good free open source ecosystem of great products that have been tried and tested.

The type of products that, you know, when you go to any new cloud native product and you start to look at how to install it, they always sort of give you an example with NGINX as the ingress controller and that it's details like that of us studying the industry for the last decade where we feel like we're setting you up very well for a great Kubernetes experience.

But yeah, it's impossible to be opinionated and right all the time.

Benjie: I mean, I would beg to differ. I'm always right, I'm always opinionated, so that's not true. All right, well, I think we're up on time here. Marc, do you have any last questions for John?

Marc: No, I mean, I think it's having been early in the ecosystem, I kind of wish something like this was around because like we went through the process of trying different tools, trying different paths.

You know, honestly, even right now I look at our GitOps repo and I'm like, ah, there's still some stuff in here that I feel like it's just way too, like set right now and having better practices early is really, really valuable.

Benjie: And the maintenance thing is really cool as well. Okay well John, thank you so much for coming on. It's Konstruct.io and again, it'll be in the show notes and really appreciate you coming on. Thanks for coming, John.