Ep. #112, WebOps with Josh Koenig and Steve Persch of Pantheon

Brian Douglas: Welcome to another installment of JAMStack Radio. On the line we've got Josh and Steve from Pantheon. Hey, Josh. How are you doing?

Josh Koenig: I'm doing great, Brian. Thanks for having us.

Brian: Cool. You want to give yourself a quick intro, and then Steve?

Josh: Yeah, yeah. Absolutely. So my name is Josh, I'm a co-founder of Pantheon, a platform for extraordinary websites. Pantheon got started about 10 years ago, and my background before that was as a professional web developer, built up a small, boutique agency in San Francisco, and we were working in the open source ecosystem around Droople, working on gradually bigger and bigger, more complex projects, and we were really learning how to apply DevOps to those projects so that we could do things like work with large development teams, leverage version control, continuous integration, continuous deployment.

Then deployment in a way that's ready to scale. Like the fifth or sixth time through that, we had done very different websites for very different clients, but the DevOps part was just getting more and more honed. We realized that we could turn that into a product.

We had bumped into a guy named James Lindenbaum in San Francisco and he took us out to dinner a couple of times and then basically said, "You guys are really nice and I like going to Delphina, but I'm never going out to dinner with you again if you don't start this company."

And so my co-founders and I decided to make the leap and we launched Pantheon to productize that experience, starting in 2010. It took us a couple of years to get on the market, but it's been onward and upward from there.

Brian: Excellent. Steve, what do you do at Pantheon?

Steve Persch: Yeah, Steve Persch here, director of technical marketing. This week I'm passing my seven year anniversary with the company and it has flown by. Yeah, mentally time traveling back to 2010, I was at that Droople Con in San Francisco where an early version of Pantheon kind of soft launched, and it really appealed to me because I saw the need for that.

Around that time I had tried out Jenkins, and I think that Droople Con was the one where I heard the term DevOps for the first time and I was just baffled by it. I was like, "Well, yeah. Of course we need servers and there's this stuff to do." I eventually realized that's a whole other job and that I didn't want to do it, so I quickly became a satisfied Pantheon customer.

Putting a major project on it, I was a Droople site architect at the time and was very happy to have someone else worried about Nginx configuration and MySQL tuning, and I could focus on building the features that my customers were requesting, and the billable hours had to be tracked against. So it's been a whirlwind seven years now at Pantheon.

Brian: Amazing, yeah. So 2010, that's before I knew how to write code properly so I wasn't actually deploying anything outside of maybe WordPress.com. Honestly, I didn't even know JavaScript at that point either. Anyway, we don't have to be nostalgic about this. But that's pretty early in my understanding of DevOps, so at that time... and I know about Jenkins as well, so Pantheon, can we talk more about the offering and what Pantheon provides to folks who want to do DevOps or basically want to abstract that away from not doing it?

Josh: Yeah, totally. It's interesting, because we were not on the forefront of DevOps, it was already fairly well established as a practice in general software development, but I think we were early in the wave of applying a lot of that to the web which is why we call ourselves now a WebOps platform because it's more targeted at this use case of people who are working with the web, people who are building websites.

There's a couple of things that are wonky about that because you're not just thinking about shipping code for your app, you also have to worry about content. That complicates some of the workflows a little bit, you have to think about that and be intentional. We also think about it as being a little bit more inclusive of some of these other personas that come in and are really crucial when you're doing the web or websites, so you're not just thinking about developers and operators.

You're also maybe thinking about designers and content creators, and the people who are trying to get business value from the web. That's in the whole DevOps culture, it's like, "Bring this cross functional team together, focus on winning together," and we just try to do that in the context of the web. What that means practically is we get everyone up and running on version control. We run a lot of WordPress sites still now, and a lot of those people are still climbing up that ladder.

It's a big jump for them to get onto version control for all their projects, and then we help them run a nice, continuous deployment, continuous integration workflow. We help them manage lots of individual site instances if they need to scale out, like you've got not one website but a company with 10 or 50 or 100 or 1,000 websites, or an agency, right? Like Steve used to work at and I used to work at, where we have 50 clients. So we help people with the operations at scale, and now as we'll talk about, we're getting into some exciting, new stuff that's particularly relevant to the world of JAMstack.

Brian: Cool. Okay. Well, I'll bite. What's the new stuff?

Steve: So the new stuff we're calling Frontend Sites. So for Pantheon's whole history we've run the LAMP stack style of doing websites, Pantheon started with Droople, quickly added WordPress. They run on a nearly identical stack of technologies and it's not a coincidence that Pantheon started about a decade after that stack started to emerge and solidify as a leader in the web development ecosystem.

Droople came out in 2001, WordPress in 2003, and it was pretty messy for the 2000s, of the different ways of doing Droople and WordPress. The 2010s were pretty messy for all of the ways that you can do the JAMstack, all of the ways you can do JavaScript centric sites. So now about a decade into that, nine years after the release or the open sourcing of React, we're coming out with frontend sites as the way to do frontend frameworks on Pantheon.

So under the hood, it's developers pushing their frontend framework code to GitHub, the Pantheon GitHub app sees that code push, build process runs and then deploys either to the static storage or a Node.js container running behind our global CDN. That's provided us a ton of value for many years for those CMS sites.

Josh: Yeah. And the idea here is there are already great platforms out there for frontend and frontend first, Netlify, Vercel, we have a ton of respect for the products that they have built and they're continuing to push the envelope on so many fronts. We see our role as trying to fill and provide an answer to the wider web of what's a really good way to do this when you want to use an open source CMS as your backend?

There's lots of cases where people want to do that because they want open source for freedom, or flexibility, or controlling their own roadmap on that side, or it's the right tool for the job. Headless WordPress is a neat thing to spin up quickly and throw in for a lot of solutions, and it gives your content editors a much better experience than giving them a GitHub repo.

Also, being able to bring our experience with really large scale web use cases, because we've grown a lot over the past 10 years. We already run hundreds of thousands of websites, we're reaching over a billion unique visitors a month and we're very experienced with handling high end enterprise requirements, high traffic sites, mission critical sites, and being able to speak to that part of the market and say, "Look, you want to build the best possible user experience and that means you need to use a modern frontend framework.

Then you need to think about where you're going to store your content, because the frontend doesn't do that anymore because you've decoupled these two things." A lot of people go down that path and are deterred, or at least made very uncomfortable by the complexity of having to wire too many different platforms together to get what they want.

Our strong hypothesis, and we're finding out as this starts to get into market this Fall, is that there's a real thirst for someone to simplify that a little bit more. We really made our bones in the past 10 years on being an opinionated platform for the right way to do these traditional open source CMS powered sites, and we think we can provide a similar value in the brave new world of modern frontends and decoupled architectures.

It's not that our way is the only way, it's just that our way is a really well paved path that's very guaranteed to work and be sustainable over time for people, because I think that's what a lot of folks are looking for.

Brian: Yeah, sustainability, I think with your experience in the LAMP stack, that was a tried and true path for getting stuff up and running. It was, like you mentioned, the opinionated way. But it's strong opinions to get people shipping Web2.0 sites. I'm curious, as the ecosystem has evolved, because now there's a lot of these frontends, you now have a frontend sites platform, the evolution of this stuff... I'm curious, from your purview, obviously invested in frontend sites, but we also have new runtimes, we have web components, we have all these new ecosystems. Is that something on the back of the mind or the roadmap for Pantheon?

Steve: It's something we're absolutely thinking about. One way I've thought about it the last few years is through the question what computer assembles the websites? That's something I've built into presentations the last few years, that in the early days of the web, it's literally the brain of the Webmaster is hand combining the templates or the markup and the content. CMS sites, the LAMP stack made the web server a PHP web server, be the computer that combines the templates and the content with iPhones and single page applications.

Those iPhones became the most powerful computer in the whole stack, like, "Let's make the iPhone be the place that assembles the website." That had a ton of downsides, performance implications, so, "Let's move it back behind a CDN, have a build process do it, or maybe a Node.js runtime or maybe a serverless runtime."

So that answer is always changing, and what we're seeing companies like CloudFlare and Fastly, as one of our partners, are making the edge be an increasingly viable answer to that question, "What computer assembles the website?" We see where that is going, however we also have this somewhat guiding phrase that came from a customer of ours who said, "We need you to be state of the art, not bleeding edge."

Another customer put it, said, "There's a lot of bleeding on the bleeding edge." And many web teams don't want to be covered in bandaids, some teams are willing to accept some number of bandaids in order to get the best possible user experience. A lot of teams are not willing to accept many or any bandaids, and so as we watch more teams become successful with these newer runtimes like Deno or WebAssembly become viable answers to the question, "What computer assembles the website?"

We need to make sure that there is a successful path before we encourage our slice of the market to go down that path because the slice of the market that we have... Well, one slice that we've been highly adopted in is higher education and they're thinking on very long timescales. Some of these universities are centuries old and for them getting onto WebAssembly in 2022, or 2023, or 2024, 2025, is not that big of a deal for them. They need to make sure that they are staying within successful guardrails, and whichever academic year they make the jump to the next new thing is not that big of a deal for them on a decades long timescale.

Josh: I think with WebAssembly and edge computing in general, everybody is still hunting around for the killer app because the types of work that are appropriate to do at that layer with that runtime are different than certainly the traditional CMS workloads. Even assembly, build step workloads at that level, you've got to think about it really carefully. People are edging around features that are about tweaking the experience at the last mile.

We did some work on that earlier this year with this idea of edge based personalization which shows some promise, but it's actually kind of complicated to get right and there's lots of ins and outs with that stuff. The ability to do it at the edge is one thing, but do you have a strategy for how you're going to vary what for whom based on what information? And you really have to connect all those dots to make it a really killer feature.

We're definitely keen on that. You also mentioned web components, and one of the things that we're also actively interested in, one of our principle engineers just did a talk on this at a conference around decoupled sites last month, is where do design systems fit into this equation? Again, everyone's got like, "Yeah, you can see it in your head, the vision of designers get to use Figma, and that seamlessly goes into some other system that then let's you deploy updated UX packages out to your mobile app and your mobile web and your other web thing and your digital signage. And all of that just works."

I know some people that have done really big consulting projects at companies like Nike and elsewhere, where they did a ton of work to make that dream come true, and it worked and it's there. That dream is real. But again, you need a Nike level investment of energy to get it up and running, and then even more to maintain it. So I think we're still in the early days of fitting all those pieces together, but it's certainly something we're actively, actively interested in.

Steve: Yeah. For a while I was trying to coin the term zombie style guide as I heard people throwing around the phrase, "We need a living style guide. We have a style guide where we decide on our styles, and then that gets synced with the CMS or the frontend framework." But in practice, many of the teams I saw had zombie style guides where they'd figure out the design in a style guide, it gets ported over to the production system and there's no living system actually connecting the two. What you end up with is a slowly decaying design system, and yes, it can often take a Nike sized level of effort to make it work. That's the kind of problem space that we want to bring within reach to a broad majority or a plurality of web teams.

Brian: Yeah. I love it and I love the approach. You know your customers and you know the persona, so serving them first as opposed to delivering bandaids for everyone who wants to jump on the bleeding edge, I think it's pretty valuable. I actually have not done any edge computing myself for any projects because I am limited in bandwidth, and also I prefer having established documentation out there as well. So as soon as everything's well documented, that's when I'll start using that stuff. But yeah, until that day I'm happy just to use stuff that I know will deploy and work, and make sure the green light is always good every time I merge from GitHub.

Josh: Yeah, absolutely. You want to stay in your power curve and that's a good place to be.

Brian: Yeah, yeah. I know there's so many different tools out there and it's interesting to see the evolution where your product is going, so I'm curious with the LAMP stack folks. Do you see a lot of folks moving over from the Drooples or going from WordPress API, to putting up fancy Next app or Astro app on top?

Steve: Yes. One way I think about it is every year that goes by, what's going to happen to the percentage of frontend developers who expect to be working in PHP based tooling and the percentage of developers that expect to be working in Node.js based tooling. I don't know how fast it's switching, but I know it's switching.

So timing that right has been a challenge because this wave has been coming for a decade, and because some teams have gotten cut on the bleeding edge, they have stepped back.

A leading agency in the Droople space did their own website as the very first React to Droople website I think in 2015 or something. A few years later, they brought it back to monolithic because they realized that in chopping off Droople's head, they lost a lot of what they were taking for granted and they lost a lot of the internal resources they had who understood fully monolithic systems.

So that's the kind of story that's happened a lot over the past few years, teams dipping their toes in the water, sometimes getting burned by it. But still they know the promise is there, and if they can get the green light and not feel like they've wasted a day or a week exploring broken tools, then they'll be much more confident taking a full leap rather than dipping toes in the water.

Josh: Yeah. And I think that the thing that I see that's a shift that's happened, maybe in the past couple of years, is the first wave, at least from our perspective and, again, we have our corner of the ecosystem, right? So it's people who are already up and running on these mature, open source, CMSes. Not people who were just born anew into the JAMstack world and that's just where they started. What we saw was a first wave of people who were really chasing technical possibilities.

It's like new frameworks, new ways of working, new capabilities and it was a very developer led rush to a new set of tools that showed that there's absolutely a there That's your classic innovator wave. But to Steve's point, some people like coming back with bandaids and saying, "Yeah, but maybe not for me yet." And now what we're seeing is another wave which is driven more by a clearer business case, if you will, because the truth is you can inarguably deliver a higher quality end user site visitor experience with this modern tooling, and web properties that have better user experiences and companies that provide better customer experience. They get better results and they win.

The people who have poorer web experiences will eventually improve or they'll die. And so you're seeing a lot of folks saying, "Okay. This is important, we need to get to this because it's what our customers expect and that level of expectation around quality of experience is always going up." Then I think you're getting lot more rational conversation of people thinking a little bit longer term, planning ahead and saying, "Well, what's the best way to go about doing this?"

Also because some people went ahead and blazed the trail, there's more known pitfalls and so forth, there's less surprises as it goes along. I think in our corner of the market where the preponderance of the work of the teams that we serve is still in the monolithic realm-- Maybe they're saying 10, 20% of their projects now starting with the idea of a decoupled frontend.

We're also starting to see it's really interesting having conversations with people that are coming from the frontend first world, and realizing that open source as an answer on the backend is interesting if it were easy. Right now you can get that Contentful thing up and running real quick and that's really attractive because it gets you up and running, but then is Contentful the right partner for you long term? Is that the right solution for this project?

For all the same reasons that your frontend is really open source so that you have full control over it and you can chart your own destiny, a lot of people are realizing the same degree of avoiding vendor lock in and other things on the backend is also powerful. That's something that I'm very curious to see about, when we're able to show people a really well lit path from either direction to using this architecture and being open source on both sides to see what kind of attraction that generates.

Steve: I think there's growing philosophical alignment between the backend ecosystem and the frontend ecosystem. WordPress has had in its documented philosophy for a number of years the idea of decisions, not options. WordPress as a community prefers to just implement the best practice, rather than Droople in comparison will sometimes expose a million check boxes and say, "Good luck picking the best configuration."

WordPress is better at saying, "No, these check boxes are prechecked or no check boxes at all."That appears to be a shift that's happening in the frontend ecosystem as well, a few years ago you might have spun up the latest frontend framework and ended up with a huge node modules directory and a giant JSON file that you are now responsible for maintaining in perpetuity.

For the more recent frontend frameworks that have come out, they seem to be taking inside the framework, more responsibility for more decisions because there's some fatigue, I think, out there among teams that don't want to be responsible for figuring out every single detail of a frontend framework. Those two things seem to be coming into sharp alignment where there's just a preference for companies like Pantheon or open source frameworks, both on the back and front and side to make decisions that work for a broad majority of websites.

Brian: Cool, yeah. I'm intrigued and I'm sure listeners are intrigued, so I'm sure someone is probably listening and they're working with a Droople or some other... all these other CMSes that were tried and true. What's the getting started path for Pantheon for folks who are listening?

Steve: Sure. The getting started path for us right now involves a feature flag, so you go to Pantheon.io/decoupledCMS, fill out the form, tell us about what project you might be interested in doing. We add the feature flag to your organizational workspace and then you can, in the dashboard, spin up. Right now we have NextJS plus Droople, NextJS plus WordPress, and Gatsby plus WordPress baked into the dashboard.

We're exploring expanding those sets of recipes. We have preset opinions on both sides, there's the start state for the CMS that's preconfigured with things like that GraphQL plugin or the appropriate modules in Droople turned on, caching turned on in appropriate ways. Then on the frontend framework side, our engineers are working on a suite of npm packages that, again, preset the appropriate ways of reading that data out of the open source CMS.

Josh: I think the old school is not something to be ashamed of. I use the term mature, which-

Steve: That's better.

Josh: Again, in a business context, yeah, it carries... There are some things where you want it to be mature, honestly. Like in your stack, thinking about where you're working with real innovator tooling versus where you want to have really established tooling. Everybody's mix may be different, but very few people are all innovation all the time because that gets pretty wild.

Steve: Yeah. All these systems still are telling time by counting the number of seconds since 1970, so referencing a few decades back doesn't bother me.

Brian: Amazing. That is so true. Well, I appreciate y'all coming on the podcast, talking about Pantheon and your new ships and the new focus. So folks, definitely check it out. I did want to check out some picks, these are JAM picks, things that we're jamming on, could be music, food, entertainment, tech related, nothing is off limits. And if you don't mind I'll go first, and then whoever wants to jump in next, feel free to jump in.

I just had a guest on this podcast who's the author of Learning TypeScript, it's an O'Reilly book. Really enjoyed that conversation because I have been using JavaScript for a while and had been using TypeScript for a while too, but not really using TypeScript. I'm one of those folks who get to come into inherited code, there's already TypeScript in the project and I just learn on the job, but never really took the time to learn, "Okay, what is the reason for TypeScript? What are the tools?"

So I've been learning that pretty quickly as building a new TypeScript app from zero to production at this point, and this book was super helpful. I just read the book cover to cover, which I don't do a ton with O'Reilly books, but I just felt like I needed to have a good understanding of what the benefit and what the sort of features are in TypeScript that I should be aware of.

It was a weird situation where TypeScript now, the 4.9 I think is the latest version, the latest version is actually pretty useful. I say that because it wasn't useful two versions ago for me, and I had bad practices from my old TypeScript days to current. So I highly recommend checking out that podcast, but also pick up the book, Learning TypeScript as well. Either of you, do you write JavaScript TypeScript? Is that in your wheelhouse?

Steve: TypeScript is not in my wheelhouse yet. At Decouple Days, I interviewed someone who had presented on ReScript, which I hadn't even heard of. I thought, "Oh no, I need to keep track of more than one way of typing my frontend code?" But I see the appeal of TypeScript, but I have not gotten past Hello World myself with it.

Brian: Fair enough.

Josh: Yeah. I've only used it in the limited context of some proof of concept code around edge compute, because if you can get it to TypeScript than you can compile to Wasm, and if you can get to Wasm then you can get it to the edge. But I would say only one and a half steps beyond Hello World, so maybe that's a good one for us to pick up and have on our shelf.

Brian: Yeah. It improves the language of JavaScript in so many different ways. My historical background for TypeScript has always been library authoring. So having the understanding in the VS Code, Merlin type autocomplete and stuff like that, TypeScript, that was the gimmick to get people to use TypeScript in a lot of projects, and I think it's gotten so much further past that that it's definitely worth checking out.

Things like TRPC which I'm going to do a talk at KubeCon next month on TRPC, where you can build APIs using types in TypeScript and it's a beautiful way to approach the problem. It is new, so obviously it is bleeding edge, so take that into consideration. But if you're into contributing back upstream, definitely the best time to get involved now.

Josh: Cool.

Brian: Yeah. And I had one more pick which is a show I've been watching every night with the wife. We caught up on the first season a couple of weeks ago. Literally I think we binged it, and now we're on the second season, which is Only Murders In The Building.

I've been hearing folks talk a lot about this. I think if you like true crime and drama podcasts, that is the premise, is that three tenants in a building... I won't give away the plot, but anyway, they create a podcast while they're trying to discover what happened to the building, which is why it's called Only Murders In The Building, and there's a... What's the serial podcast type host that's in there?

Played by Tina Fey, Steve Martin and Martin Short. I did not realize how funny they are together, but I also did watch all those shows like The Three Amigos and stuff like that, or the movies, rather. They do a great job even for their age, which they're up there. I don't even know how old they are. But worth checking out.

Steve: Definitely 70s. Yeah, I really enjoyed that show. I was late to season one, so I think we binged season one and then we watched season two as it came out each week. Really enjoyed it.

Brian: Yeah, I'm surprised on how much I enjoyed it. So definitely, I highly recommend folks check it out. Steve, do you have picks?

Steve: I am soon to be jamming on Astro. I've been reading the docs for Astro on my phone during downtime, and I respect their opinionated stance, the way they lead with the statements like, "We are preferencing content heavy sites, and we have the view that these other frameworks have preferenced web application-like experiences."

While that's not a hard black and white divide between content focused sites and web applications, Astro is leaning hard towards content focused sites. That's probably the bulk of the websites on Pantheon, so that's one that I need to get myself into. Their focus on server side rendering aligns well with what we're doing at Pantheon, and the way they're soon to be relatively agnostic about which other frontend libraries you might layer on seems to be appealing to me.

Brian: Yeah, I'm a fan. I did have Ben Holmes come on to not talk about Astro, but he now works at Astro and I did have an opportunity to build an Astro site. Which is no longer an Astro site for reasons, the whole website and web application.

It's the paradigm that I want to lean into more websites stuff, but I was building it as an application and that's how I learned Astro and what its use case was. That site is Hot.OpenSauce.Pizza if anyone wants to check it out. Some of the features we added were very web application heavy. But I will be shipping a new Astro app soon, so stay tuned.

Steve: Well, I'll have to check out Hot.OpenSauce.Pizza.

Brian: Correct, so Hot.OpenSauce.Pizza.

Steve: Nice.

Brian: It's a discovery tool for finding trending open source projects. There's a whole meme and a branding going on over there.

Josh: Yeah. I checked out your OpenSauce.Pizza before when I was doing my due diligence, and that's a really cool project and now I have to see what the Hot.OpenSauce is. I got to have it.

Brian: We're serving up another pizza site pretty soon too as well, so also stay tuned.

Steve: Sweet.

Brian: Yeah. So Josh, do you have any picks for us?

Josh: Yeah, I've got a couple, and I'll do one tech and one non tech. The tech one is a project called the HTTP Archive, not to be confused with web.archive.org that screenshots every website over history. That's cool too. But the HTTP Archive is a really interesting project where they're running a crawler across close to 10 million sites twice a month and pumping all the data into a publicly accessible Google BigQuery project.

It's got down to the complete web page test, all hard data, all the details. So I'm plugging it because while we were getting... This is now going back maybe a year, year and a half when we were just starting to get the team together to really go all in on this frontend sites project for Pantheon, I was using it to do all this research. Like, "Okay. How many websites are actually running this stuff?"

There's cool tools out there like BuiltWith that'll give you, for a business user, a high level overview of things and a chart which is nice. But if you want to answer a question like, "Well, how many people are using what version of NextJS and also have this other thing? And what's their average time to first paint?" That's data you can pull out of the HTTP Archive, and the fact that they have it going back several years, it's just super awesome. I guess over the past year and a half, I don't get to write as much code as I used to.

But the place where I've been able to stay close to the mettle is messing around with data and being able to get a real, true big data dataset in BigQuery and just thrash around with it was super fun. I really enjoyed that experience, and I found it to be useful in a lot of ways. So that's my tech pick. My non tech pick is a book that I read this summer called Cloud Cuckoo Land that blew my mind.

It's a multilayered story that some of it's happening in the future and some of it's happening in the past, and pushes all the buttons from a, "What's it like to live in a world where it feels like the apocalypse is always just around the corner and technology is changing everything?" I don't know, I haven't read a book like that that left me feeling... It actually gave me goosebumps at a couple points, and so if you like speculative fiction and can hang with a longer book, it's like 700 pages, it's a real treat. So those are my picks.

Brian: Love it. Yeah, I dropped links in the shownotes as well. Yeah, looking forward. I don't know if I can commit to that book yet, but when I do have some downtime, it's definitely going on my Amazon wishlist for sure.

Josh: Yeah, you're going to want to be able to... It'll prevent you from sleeping, you'll be like, "Okay, one more chapter," and you got to time that right for your life.

Brian: Perfect. Well, speaking of time, we're out of time. I appreciate y'all coming through and talking about Pantheon. Folks, definitely check it out, reach out and keep spreading the jam. See you next time.