Ep. #43, SpinKube with Kate Goldenring of Fermyon

Marc Campbell: Welcome back to another episode of The Kubelist Podcast. Today, Benjie and I are joined by Kate Goldenring, a senior software engineer at Fermyon.

We're going to talk about SpinKube and probably some IoT Edge, Wasm, lots of different stuff. Welcome Kate.

Kate Goldenring: Hi, Marc and Benjie. I'm excited to be on the show.

Marc: So to get started, we'd just love to hear a little bit about your background, what you did before Fermyon and leading up to Fermyon, what you're doing now.

Kate: Yeah, so I am unique, in that I kind of started my career in tech in the open source space. It was pretty lucky place to start and it was actually right in the Kubernetes space.

I was at Microsoft and we were looking at what does Kubernetes on the edge look like and how do we handle all these IoT devices around our clusters on the edge.

And so from there, I got started maintaining a project that's now a CNCF project, called Akri. And I am co-chair of the CNCF IoT Edge Working Group.

And I started to get involved with WebAssembly stuff while I was at Microsoft a little. And then when folks went to go found Fermyon, I thought, okay, I want to go join this, this nascent technology that has some really powerful concepts and ways that it can improve sustainability, which we might talk about a little bit.

So that's when I joined Fermyon to work on serverless WebAssembly. And now I'm in that space, working on open source technology there, such as Spin and SpinKube. And I work also within the Bytecode Alliance, which is the organization working on the WebAssembly on the server specifications, such as the component model.

And I am co-chair of the documentation special interest group there too. So I'm always happy to hear questions people have about the component model, because that's probably gaps that we have and ways we need to fill in our documentation there.

Benjie De Groot: Next time you can just say the things that you aren't part of, because I think that would be easier for us. No, I'm just kidding.

Kate, that's super impressive. We're going to give links to all of the things that you just said, but I want to talk about a few of them. Before we do, tell me about...

So you went to school and then you went to computer science school and then you went to start at with Microsoft. And then I just think it's interesting how you were able to navigate inside Microsoft, especially at that time, going straight to OSS.

So I'd love to just hear about what it was like early days at Microsoft there.

Kate: Yeah, like I mentioned, it was kind of a great seamless order of events there, but when I joined Microsoft, I was on the CoreOS team. So we were actually focused on a minified version of Windows, but we were transitioning to other ways that people were using operating systems and Kubernetes specifically.

We basically, as a new hire, they kind of put the kind of R&D projects in your hand. So I was lucky and Akri became one of those projects. So one of our senior engineers, he designed the project and I kind of got the opportunity of implementing it.

And what that project was focused on was resource usage within Kubernetes. So I got to go pretty deep within Kubernetes and look at all of that, and that kind of threw me into the Kubernetes world through the device plugin interface, which is kind of the specification within Kubernetes that I worked on.

But yeah, from there, then it was just connecting with other open source teams. So Matt Butcher, who's CEO of Fermyon, worked on the team that was under this umbrella group within Microsoft, called Deis Labs, which you might be familiar with, but that's where a lot of the cloud native projects within Microsoft are.

But that was a separate team than mine. And so when my team decided to kind of venture into this world of CNCF and Open Source, their team, specifically Karen Chu on that team, kind of mentored us, told us what open source community management looks like, what is the CNCF and kind of brought us under their wing, and that's also how I met them and some of the projects they were doing there, such as Krustlet, which was their first attempt at getting WebAssembly and Kubernetes to work together.

Benjie: Okay, so Deis, I always say that one wrong, but they were like a really early acquisition in the space by Microsoft. I remember when that happened. Were you there when they were acquired, or you came in after they were already acquired?

Kate: I came in after, so a few years after that.

Benjie: Okay. So Microsoft had already kind of started like, "Hey we're going to think about Kubernetes a lot." So what is this like 2018, 2019 maybe?

Kate: Yeah, 2019.

Benjie: Quick side note, my favorite socks that I had for years, I had three pairs of Deis socks that I got at the Tectonic Conference 2015, they were one of the early people giving away socks at these conferences, so shout out to Deis.

Kate: I say Deis.

Benjie: Deis, there you go. I've heard "Dice" before, which I kind of like. But anyways, but the important thing is, is that Matt was over there and the whole team and they were kind of helping you guys navigate that. Internally in Microsoft in 2018, 2019, it seems like that's when that transition to open source really started to take off and you were there from the beginning.

Was it just smooth sailing and you're like, "I want to do this open source thing." They're like, "Let's go." Was it really that cool already over there?

Kate: I wouldn't say exactly smooth sailing. So I think the one thing that made it feel like it was supported by leadership was the fact that my team at the same time was working on Mariner, which is Microsoft's Linux distro.

And they knew that when they were going to be working on that, they were also going to be doing that open source. Akri wasn't the only project that was venturing into those open source land within my org, so that was helpful.

The part that was hard was just building community, as with any open source project, and getting more engineers within Microsoft to be able to work on it was also a struggle, given that Akri was always our enhancement to Kubernetes, it wasn't the project.

So Akri is about how you discover devices around your cluster, so it's kind of an add-on to what you have with your cluster.

And so in that sense, prioritizing it sometimes was hard to motivate, but we really found that there wasn't any project like that in the CNCF, and for that reason, when we submitted it, we were pleasantly surprised to get accepted on our first try, we weren't really expecting that either.

And so that was kind of why everything flowed a little easier than we expected, is that we were filling a gap.

Benjie: And that's when you just kind of... You really got into the CNCF and all this stuff. So Akri, is that still a sandbox project?

Kate: It is and I think we might be looking at incubation soon.

Benjie: Oh cool.

Kate: There's already several companies working on it, so I'm still involved with it from Fermyon and then Microsoft still works on it and CISO's gotten fairly involved with it with K3S and their edge offerings, so we could move it towards incubating in a 1.0, and so we're making some changes there now.

And there's also a new specification within Kubernetes, called Dynamic Resource Allocation, that takes Akri further that we're looking at porting it onto. So we're still evolving it and definitely trying to grow it.

Benjie: That's really cool. I love these projects that just kind of just keep growing, 'cause they're needed.

Okay, so you were at Microsoft, you kind of got into that stuff, then you got exposure to all this stuff and you kind of fell in love with it sounds like.

So how long were you at Microsoft and when did you end up leaving? Was it straight to Fermyon, or was it something in between?

Kate: Yeah, I was straight to Fermyon. So I was at Microsoft for three and a half years and then I ended up starting to work with Matt Butcher's team, the Deis Labs team to help collaborate on Krustlet.

So we were trying to get Akri to work with Krustlet essentially and be able to run WebAssembly. And with that I got to work with that team and collaborate with that team and got to learn about WebAssembly and got really excited about it as a technology and its instant startups, its isolation and its scalability.

And so essentially when that team departed to go found Fermyon to basically take this big push and effort towards serverless WebAssembly, that was a pretty exciting venture and something that I really wanted to be a part of.

And so I also joined about eight months later and have been at Fermyon since.

Benjie: So wait, how did you find out about Wasm, because that's super early? I guess Matt obviously, but in general, tell us what it was like to see Wasm back then. What was your introduction to Wasm and how did you get into it?

Kate: So this goes back to the original storyline of the fact that my team at Microsoft tended to take the new hires and interns and give them really cool R&D projects.

So we had an intern that summer, Diego, who we were trying to come up with an intern project for him and we had this Edge IoT project, called Akri. And then we had this really cool Wasm on Kubernetes project, called Krustlet that another org was doing.

And so we're like, we'll make Diego bring these two really niche projects together and make it work. And so I was his mentor for the summer and got to be involved with that.

And then we ended up both presenting the work at Wasm Day, I think the first Wasm Day North America in LA, presenting that work.

But that's basically how I got to work on Wasm is that, yeah, we drew another awesome crazy project at a new hire and it led to some really cool collaboration.

Benjie: Man, that's kind of the whole point of open source, right? That's a really cool way that it came into it.

Marc: So you then you left to go join Fermyon, you're describing Fermyon as serverless WebAssembly, can you explain a little bit more there, what the product was, what the team was that was leaving Microsoft to go create this?

Kate: Yeah.

So I think there's a lot of ways to use WebAssembly, and so I think some people use it for plugins. And for Fermyon specifically, we were interested in making it easy to run event driven applications where those applications are actually WebAssembly.

And so WebAssembly itself can be thought of as two things. So WebAssembly is a portable compilation target for your code. So it's this one target that you can compile any of your favorite languages too.

Not any, but a lot in different languages to, over 20. So the Rust, Go Python, JavaScript, take your favorite language and now you compile it down to a .wasm file.

And the really exciting part about then with WebAssembly is that you can run it anywhere you can put a WebAssembly runtime, and you don't have to cross compile it.

So whether that is on Linux, windows, MacOS, whether it's on x64 arm, you don't have to cross compile it. And that WebAssembly runtime executes it in a secure sandbox. And so in that way it's also thought of as a really good isolatable unit.

And then the third part of WebAssembly is that it starts up in sometimes sub millisecond times. So when you take together its cross language polyglotonous, its sandbox environment and its instant startups, that sounds like a perfect serverless solution.

And so that's why Fermyon, when people have been trying to do serverless in the past with other units, such as Lambdas, or specifically micro VMs or containers, and it just wasn't really catching on, when they saw these characteristics in WebAssembly, they thought, "Okay, I think we have a better way of doing serverless, let's give this a go."

Marc: Got it. And so jumping ahead a little bit, we set this up and we said we're here to talk about SpinKube.

So you've been there for a while, working on serverless WebAssembly at Fermyon. And SpinKube, what is SpinKube and how does it help deliver on that?

Kate: Yeah.

So first of all, Spin is a developer tool, an open source developer tool that Fermyon's put out that helps you quickly create these serverless WebAssembly application.

So it's a simple CLI experience where with a SpinNew, you choose your language, SpinBuild, you now have that .wasm file, and a SpinUp, you're now running it locally. And people really enjoyed that experience.

We have over 80 contributors and 5,000 stars on GitHub. And people were putting it places and running it and distributing it with various methods, like NgenX unit, you can run Spin apps and you can run it on Fermyon Cloud, which is our hosted platform where we use Nomad as the orchestrator.

But people still love Kubernetes, and so people kept asking us, "How do I put this on Kubernetes?"

And so SpinKube is the conglomeration of four different open source projects from several different companies, namely Microsoft, Liquid Reply, SUSE, and us at Fermyon, to make it easy to run these Spin apps on Kubernetes. So have that seamless experience, where instead of deploying containers, you're deploying WebAssembly applications to your cluster.

Benjie: So I want to go back to a few things, but before we do, tell me-- What do I type to do this with SpinKube?

Kate: So we talked about the SpinNew, SpinBuild, and now it's SpinKube, we need to put that WebAssembly application somewhere, namely an OCI registry.

So your next command is a Spin registry push, and now it's hosted in a registry, that Wasm artifact, this isn't a container, it's actually a Wasm artifact type for that OCI.

And then you do a SpinKube scaffold and now you have a custom resource that is defining this Spin app for your cluster.

And you apply that and now there's a deployment in your cluster that is actually a deployment of WebAssembly applications instead of containers.

Marc: When you're pushing that Wasm artifact to the OCI registry, is it just the Wasm artifact, or is there more information that you include in there?

Kate: Yeah, there's a few layers there. So you have your Wasm and components, so those .wasm files, and then you also have the Spin manifest, which is a TOML file that basically defines the capabilities and endpoints of your applications.

So it says, "Hey what are the functions of my serverless applications? So what HTTP endpoints do I hit?" Or you can have other triggers for Spin apps.

So it could be like what MQTT topics is it subscribed to, or what Redis do. There's also a command triggers if you just want to directly invoke it. So we have several different ways you trigger these serverless functions, and you can actually have several in one Spin application.

There's actually no limit to the number of endpoints you have. So that manifest defines all those endpoints and it also defines what resources each of those components have access to.

So say one part of your Spin application wants to cash things in a key value store, that's where you specify what is the source of that and hook it in, or that's actually specifically, that's where you say that that component can access that.

And then we also have other files that map what is the exact store to access that, so that way you don't have to change your development life cycle to change what resource you're connected to.

And then the only other thing that is also a part of the OCI artifact is any data. So if you have a personal blog, you might have images there, or any files or any type of static content that you want to be packaged with your application.

Marc: So that artifact is everything that you really need to run the application. How much of that, those layers in that packaging format, is proprietary to Spin, versus either a standard, or shared by other platforms that do the same thing?

Kate: Yeah, so the layers themselves for the Spin app is not proprietary to Spin, but it was our guess that a good way to lay out a Spin application OCI. The WebAssembly working group within the CNCF actually has been working on standardizing that and they are almost done with that.

So once they've finished, I believe that's either been recently approved, or it's approved, we're going to port our OCI format onto that and there should be a standardization for when people are distributing their WebAssembly applications and OCI formats, whether that's a Spin application type of OCI, or another type.

Marc: That all makes sense for delivering a Wasm application. Taking it one step higher, rarely is an application probably written as a single Wasm application, it's a lot of different microservices.

Is the use case and the common pattern you see, people will deploy an entire platform full of microservices as a bunch of Spin applications, or if I have an existing legacy, like Kubernetes application, is it a good way for me to onboard, like throw SpinKube and be able to start using Spin as a subset of my entire platform?

Kate: Yeah, so I think there's definitely several different approaches there.

So to first talk about how do I go from my traditional Kubernetes approach to my microservices to serverless WebAssembly, moving to serverless WebAssembly is definitely not just a quick port. There are several steps, because serverless applications are built differently, they don't have servers, they're purely event driven, so you do have to first make sure your application's event driven.

If you're already using something like Dapper, it makes sense to make that transition. So that's one part of it.

The way whether people use multiple Spin applications or just one, is actually somewhat interchangeable, there is a maximum size for Spin application, so that would be one of your limitations of why you wouldn't use one, but you really could have one Spin application with many different endpoints.

So for example, as kind of a demo application we have, just to kind of show how fast WebAssembly is, we have finickywhiskers.com, which is this one kind of fun game where you try to feed the cat exactly what the cat wants as fast as possible and they keep changing their mind between tuna and veggies and meat and all this over and over again.

But that one application has about eight different WebAssembly components in it, to keep a high score, to tally, to change the UI, all of that. And then another use case that we've seen is sometimes people have like data processing pipelines.

So there might be one part of your application that is triggered by a change in some sort of data bucket and another part of your application that processes that further down the line.

So say you have two different teams that are managing your applications, you might choose to have those be two different Spin applications, so that the development lifecycle one doesn't have to also be a part of the development lifecycle of the other, so that might be another reason why you would keep them separate.

Benjie: Okay, so at Shipyard we have a lot of folks that do leverage Lambda and serverless.

So if I understand this correctly, and please tell me if I'm wrong, I could hypothetically take a Lambda, which is a Python Lambda or whatever, and I don't actually understand how those work that well to be honest with you, but I think I do, and that's an event driven blob of code, and I could throw it into a Spin application and then I could do Spin push and then actually run it inside my application.

And the only thing I really have to do is change the wiring from my main application or whatever it is that's calling out to Lambda, And now I could actually point internally to that CRD, or something to that effect, is that sort of right?

Kate: Yeah, the only thing would be that last part and where the Spin application itself doesn't need to know anything about Kubernetes. It doesn't need to do anything-

Benjie: No, no, I'm saying my... Sorry, the architecture in my head that I didn't communicate very well, is I have a monolith that used Lambda for processing images, and so I could support that from Lambda over to internally and make it a SpinKube function for lack of a better word.

So is that something I can suggest to people, is that a way to get everything out of proprietary infrastructure and into Kubernetes?

Kate: Yeah.

Benjie: Okay.

Kate: And I would love if you made that suggestion, because we want to see a case study of someone porting from Lambda, that's one that we've talked about being a viable one, but we haven't partnered with anyone on that.

So if someone wants to partner on that, I would love to see what that journey looks like.

Benjie: We'll talk offline on this, I might have someone to introduce you guys to.

Kate: Okay.

Benjie: Okay, so that's where my brain goes. Tell us some real world instances of people using SpinKube, if you can share any of those things.

Kate: Yeah, and right before that, I wanted to just tag something on to what you said about being kind of locked into AWS and how you could maybe move to non-proprietary architecture with Kubernetes.

I want to add something beyond that. And that is, one of the really cool things about the component model is that you are defining components that have access to the host through standard interfaces.

That's what the WebAssembly, WASI stands for WebAssembly Systems Interface, but Oscar Spencer, who's one of the big contributors in WASI, likes to think of it as WebAssembly Standard Interfaces.

And so another great part about this is that even if you choose Spin for your way of designing your WebAssembly components, you could actually later change your mind, because a lot of the different serverless WebAssembly providers out there are conforming to WASI, which means that you're also not locked in to what you package your components in, you can change your mind there as well.

And so that's why WASI is particularly exciting for that part of your application development as well. So I just wanted to point that out too.

Benjie: Yeah. I am now Googling "component model," so rather than me synthesize this website, will you actually tell me about... I mean, I think I now understand, 'cause I've spent all three seconds and I listen to you. But this component model thing's really cool. Dive into that a little bit more please.

Kate: So WebAssembly, as it sounds, is made for the browser. And around 2019, people started thinking about what if I could run this outside of the browser. But if you're running things on servers, there's access to certain host level resources that are different from that of the browser.

So how do I access files and sockets and IO, et cetera. And so people started defining in an IDL, or Interface Definition Language, called WIT, all these different interfaces for accessing host level resources.

So defining what it means to send an HTTP request to a WebAssembly function.

And what the component model does is it basically takes a WebAssembly module, which is what browser WebAssembly is, and wraps it in this packaging that basically tells the world, these are the type of functions I accept, so these are the calls you can invoke on me and these are the calls that I want to be able to make. And the host must provide all that it wants and it must provide all that it says it provides and it can't do anything more or less than that.

And so all of these WASI interfaces are defined, so wasi.dev defines the ones that have been voted by the Bytecode Alliance, which is the open source organization that governs the WASI specifications.

And yeah, essentially these are defined basically blueprints for WebAssembly components. And so every component has a blueprint essentially that says what it can and can't do.

Benjie: Every time that we start talking about Wasm, WASI and all these things, it just gets cooler and cooler and more and more like you guys have already done all this.

That's my general take on Wasm, is wait, this all works? Which is maybe the consensus, or maybe I'm just ridiculous. But okay, so the component model is a blueprint. You're kind of blowing my mind here.

So it's a guarantee that hey this is going to work on any of those given architectures that you were talking about before. By the way, I always think about this, running this stuff in a browser.

I mean, I actually think I know the answer to this one, but I'd still love you to articulate it. Why did it start in the browser and what's so cool about the browser that made it a good place to start?

Kate: Yeah, so WebAssembly was created for the browser in order to enable people other than JavaScript developers to write browser language.

And so people think of WebAssembly as the second language of the browser. And because it was made for the browser, it has all these other great characteristics, because browsers have tons of untrusted code running next to each other, so it needed to have a secure sandbox.

And browser, people are impatient, there's that 100 millisecond latency that we're aware of, so WebAssembly needs to execute faster than that.

And so all of these characteristics are what made it so appealing outside of the browser too, but it really was made to make the browser more polyglot.

Benjie: And the sandbox thing is the big deal to me. Obviously the browser is actually, ironically, one of the more trusted things out there at this point.

And so when did WebAssembly actually... When was the first commit on it?

Kate: Okay, I believe 2017.

Benjie: Okay.

Kate: But we can double check that. I think the timeline is 2017 WebAssembly, 2019 is when people started thinking about WASI and the system side of it.

Marc: Pretty new.

Kate: Yeah.

Benjie: And right around the time that you were doing Akri. Super interesting. Well, we have a whole host of other things to talk about.

I do want to talk about a few of the other projects that you work on, 'cause I think they're kind of cool.

So why don't you go down the list and tell us a little bit about each one of the things that you do, Kate?

Kate: Sure. So we can go in I guess chronological order. I mentioned Akri at the beginning, but I guess I'll just give a little bit more of an overview of it.

This is Kubelist, so people understand Kubernetes. And one of the things that Kubernetes has is resources, so when you define your pod spec, you can say, "Hey, this container gets this much CPU and memory," but what you couldn't say, is "I want this USB device or this IP camera."

And so that's what Akri does, is it creates Kubernetes resources for both local and network-based devices and declares them on a node level, so that when you then go to deploy your applications, you can actually request IoT devices.

And then when you request those devices, what it does is when your pod is deployed, it injects connection information as environment variables.

And it also will, if you use the controller or the scheduler part of Akri, it will also like automatically tear down your workloads when those devices go offline and it'll bring them back up when the devices come back online.

And it also manages how many different pods can use this device at once, so what is the capacity of the device? So it does all of that, and it's very pluggable, so say you have proprietary devices, you can create your own discovery handlers for that and deploy them as their own pods, and it's supposed to be extensible in that way.

And it's similar to Spin, Akri is also developed in Rust, so it's got that low level secure programming language and is somewhat unique in being a Kubernetes operator in Rust, but more and more are these days.

Marc: And that operates completely outside of the device itself, right? The device doesn't have to know what Akri is, or that it's going to run in that type of an environment.

Kate: Right, exactly. The idea of Akri came from the fact that these devices are too small to fit a Kubelit on them, so you can't make them a part of the cluster, so how do we bring them into the cluster?

Benjie: And then also I guess there's really cool mocking stuff you can do also with that for testing if you don't have those physical resources.

Kate: Yes, we have end-to-end tests where we have to do a lot of that. You can mock video cameras pretty easily on a Linux, and so we have some fun demos and stuff where we do that as well.

Benjie: Yeah.

Kate: I guess you were prompting me by saying, what are some of the other projects I've worked on? So I'll just mention involvement in the CNCF as co-chair of the IoT Edge Working Group. The way I got involved with that is that we actually presented Akri there.

So we were trying to talk more about Akri and share with people, and then I started joining the meetings more. And one of the co-chairs left, and so then I took the opportunity to run to be one of the co-chairs.

But the work of that group is really involved in how do we find a more terminology and understanding in this space that is just so heterogeneous. And so sometimes all we're doing is just chatting about the latest ingredients, where are the conference is going on, other times we're writing white papers to design best practices for applications on the edge and operators on the edge.

So it's really a catchall and we're always kind of looking for a new way to pivot and sign new spaces. So if people are interested in that kind of discussion, please come join. It's within the runtime tag of Kubernetes.

Yeah, and then from there, within WebAssembly really I work on SpinKube and Spin, but within SpinKube, I haven't yet talked about what are those support projects made of. And one of them is the one that I focus the most on.

But we kind of talked about the user interface of what SpinKube looks like, you create that Spin app custom resource that defines the Spin app deployment you want to add to your cluster. And it also has things like you can use the horizontal autoscaler, volumes, secrets, all the things you're used to with Kubernetes, you can use for SpinKube as well.

So it's bringing that Kubernetes native experience to WebAssembly on Kubernetes. So in order to use that custom resource, just like any other Kubernetes operator, we need a controller. So we have the Spin Operator, which is our controller that sees that Spin app and creates your WebAssembly deployment.

And then if we go a layer down, we have what executes these WebAssembly applications. So you can't just put WebAssembly, an OCI reference, in a pod running on Kubernetes and it works.

You actually have to provision those nodes. And what you're provisioning them with, is a Containerd shim. So it is a shim that Containerd calls out to that executes WebAssembly instead of a container.

And so that's the part of SpinKube that I maintain and that's what the Microsoft folks developed. And they also developed a project called Run WASI, which supports many different WebAssembly Containerd shims, not just Spins but also WasmEdge and Wasmtime as well.

So that's the project I work on and is kind of the lowest level of what SpinKube is. And we have a runtime class manager, which is the only other part of it which makes it easy to get that shim on your nodes, basically make sure that binary is present.

Benjie: So how do you divide up your days? What do you do? Do you sleep? What do you do? I don't understand. How does this work for you daily?

Kate: Fortunately, all these projects and groups are definitely groups and teams, so I am not doing all this whatsoever.

So I don't find it to be too crazy. To answer your question, I also have a lot of hobbies and don't want my work to be my life, so fortunately that has not become the case.

Benjie: Wonderful. And I know you're really passionate about sustainability. I met you when you were giving a talk at DevOps Days Seattle about sustainability. Let's talk about your thoughts on that and also how the work that you're doing is making a difference there.

Kate: Yeah, so I originally wasn't intending on being a software engineer. I thought that I would do something within the environmental sciences.

And found that as a degree, that was very sprawled, there wasn't much expertise in the program that my university offered. And so that's how I ended up spontaneously pivoting into computer science, I wanted some tool to express change in that field.

And so I've kind of always been like spider fingers, looking for a way to see that in tech. And when I saw WebAssembly, I kind of latched onto that for that reason.

And when we talk about instant startup with WebAssembly, that also means instant tear down in the case of WebAssembly. And so what this really means is you can actually run code only when you need to.

And so the talk that I was giving at DevOps Days, I talk about how with WebAssembly and serverless, we can actually do more with less hardware.

So we can put dozens, 100s, even, we've done some performance testing on Fermyon Cloud where we had like 5,000 applications on a single node, and it was like an eight core node, and you can scale them up and down in milliseconds, such that as long as these workloads are different tenants and they're uncorrelated, you can actually have each of their-- when they're execution times and they're not execution times that they balance out-- you have a high level of multi-tenancy. And that's where I get excited about WebAssembly.

So when Fermyon started, we joked that our cloud spend was cheaper than our T-shirt spend, because we didn't need too many nodes to run our website, because our website runs on Spin. And that's kind of where I get excited.

I really think serverless is important, but I think it only works if you have the right unit underneath it.

Benjie: And not to go back, I want to talk to sustainability, but how is it so resource efficient? I still haven't wrapped my brain around this, why can you get 5,000 services on one node? How are you doing it? And safely, it's safe. It's safe?

Kate: Yeah. So to start, these applications are very small, so their memory footprint is tiny. So what might be a 300 megabyte express JS container is like a three megabyte JS Spin app. So already like the memory footprint and the data footprint is smaller for having all these applications on one device.

The other part of it is the way the WebAssembly runtime works, that is what is the virtual machine aspect of WebAssembly, is that all it does is it basically creates a four gigabyte memory sandbox and loads that WebAssembly component into it, and it's isolated there in linear memory.

There's not an extra OS there, there's no extra dependencies, it's really lightweight, and so it's that load and go. The only extra thing the WebAssembly runtime has to do is it does have to take that portable Bytecode format, that .wasm file, and compile it to machine code.

So that's a a prerequisite step that usually happens upon the first provisioning step when you first deploy an application. So with the Containerd shim, we do that right when you schedule your application the first time, but that's really the only prerequisite step and then it's just kind of a load and go, which makes it very efficient.

Benjie: So is that Bytecode stored in memory, or where does that go?

Kate: Yeah, so the or the Bytecode itself, like the .lawsn file is just on disk, but when you then create a WebAssembly instance, which is what is ready to invoke, that is in memory, that's that sandboxed instance of WebAssembly that's ready to be invoked.

Marc: Got it. Going back, taking a 300 meg JavaScript application and turn it into a three meg deployable WebAssembly application in Spin, how, how, where... Is JavaScript just that inefficient that you're doing it, or where are you saving that space?

Kate: I don't think it's the JavaScript in that case, it's the container.

Marc: Okay.

Kate: Because I'm not talking about a native application. If you were to just compile that natively, it would probably be very comparable sizes.

Marc: Got it.

Kate: I haven't done that test, I would love to, I'm going to verify that after this, but it's the containerized form.

It's the idea that I don't want to run it natively, 'cause I still want isolation, but my container choice is bulky, but WebAssembly isn't.

Marc: Okay, so most people are running JavaScript applications in unfortunately a Debian container, or something like this, which has lot of weight.

They get rid of that, it gets small, but now they don't have the isolation. So you can bring the isolation in through WebAssembly without having to bring the container overhead in?

Kate: Exactly.

Marc: Got it.

Benjie: All right, going back to sustainability stuff. So there's some initiatives over at Fermyon, I believe, to kind of proselytize that stuff. Tell us a little bit about that.

Kate:: Yeah, so in the sense of what are the ways within Fermyon, we're trying to spread the word about sustainable WebAssembly?

Benjie: Yeah.

Kate: Well, I think one, we're just trying to get the word out about that.

I personally am trying to talk more about why serverless could be a good way of achieving a greener cloud.

We talk about software carbon intensity and how to measure it, which the Green Software Foundation has come up with a formula to measure carbon intensity, and so we try and put it in a measurable format for people to see how they can actually see if serverless WebAssembly can cause a benefit there.

The other thing to keep in mind is that Kubernetes, another number that we'd like to share, is Kubernetes, the sysdig cloud usage report showed that 67% of cores are unused in Kubernetes. So there are so many resources there to be used that we just aren't taking advantage of. And so that's another thing that I like to share, is there's a lot of potential and we have a lot of wasted resources because we have to over provision a lot of times with containerized deployments.

And then the other thing that we've done that I think is interesting is that right now everyone's talking about AI, but inferencing can be expensive. And so there's a really interesting company, called Deep Green.

They basically capture the heat externality of doing inferencing and use it. So we're using SIBO Cloud's Deep Green integration, and they tell you where yours is running too.

So our servers, which have GPUs, are heating a public swimming pool in London, and so that's another way to try and have carbon neutral inferencing, is by capturing the heat and actually using that for a purpose, namely heating a pool.

And so all the inferencing requests we run on Fermyon Cloud are as close to carbon neutral as possible for that reason.

Marc: Cool, it sounds like sustainability is coming in smaller footprints so you need less compute, quicker scale up time and scale downtime, so that you don't have to sit there with as much idle compute ready, and then this where you're able to... You don't have as much waste, because you're able to reuse that.

Did you mention at the beginning, Kate, that you were working with a SIG, or part of the CNCF around sustainability also?

Kate: There is the sustainability tag within the CNCF, which I'm a member of, I don't do leadership there, but I am, within the Bytecode Alliance, I am a co-chair of the documentation SIG.

So what we do there is basically when people ask, what is the component model, what is WASI, we try to make sure that there's information out there. So we have two books, we have wasi.dev, which is explaining what WASI is, what are the current interfaces there.

And then we have the component model book, which is component-model.bytecodealliance.org, and that has an explainer of what is the component model, how do you get started in your language of choice, what run times can I use, et cetera.

And we're always kind of trying to look to improve the way that we have tooling and documentation there as well. We don't currently have a sustainability SIG within the Bytecode Alliance.

The documentation SIG we just formed four months ago. And that might be something that as the organization grows, we find room for.

Benjie: Super cool. So let's go back to... I mean Wasm, Spin, all the things, but anything cool on the roadmap? I'd love to hear about the roadmap, what that looks like.

Kate: Yeah, so for the SpinKube side of things, within the roadmap, it's adding more triggers. So right now you can trigger your applications based on SQS messages, command, so for sidecars, for Kubernetes, MQTT and Redis triggers.

One thing we want to add is that MQTT trigger. I submitted a CFP for KubeCon to talk about using Akri with SpinKube based on discovering MQTT based devices and triggering those applications, WebAssembly applications.

So I kind of want it in selfishly for... If that gets accepted, I want it for the demo. So that should hopefully be coming soon. And then with Spin, what we're really trying to do, is make sure people are getting all the benefits of the component model.

So WASI 2.0 was announced in April this year, and what that did was it announced the component model, which was basically what I was saying, providing that wrapper and blueprint around WebAssembly.

But one thing I didn't mention about the component model, is that it makes it easy to compose components, which is where that component comes from, that word. And so I basically can take my favorite URL library from Python and compose it with my Rust application into one binary, and that's what the component model does.

And with Spin, we want to make it easier for you to define that within your Spin application manifest. So we've been doing work within Spin on that and I believe that would be enough of a manifest change that that's what's going to trigger a Spin 3.0.

And there'll be other changes as a part of that too, but that's a big one.

Marc: Got it. A lot of this is open source. SpinKube and a lot of the other stuff you've been mentioning is open source.

How would somebody get involved with the community, with Fermyon, to work with you on these projects?

Kate: Yeah, so for people who are familiar with the CNCF, CNCF Slack, you can find us there. So SpinKube has a channel there and so does Akri, Akri has a channel there, and so does... The IoT Edge Working group has a channel on the Kubernetes Slack.

For Spin, we have a Discord, so fermyon.com/discord will get you there, and that has like over 1,000 users and we're pretty active there. And then there's like a fun show and tell channel that people share a lot of their latest and greatest apps that they have developed there.

I personally am a Survivor fan, so I'm honing my Survivor draft app as we speak and may share that. And beyond that, the other one I would mention, is if you want to get involved in the work developing WebAssembly, you want to provide new specification.

For example, some people from the Embedded space are working on creating new WASI embedded specifications, so how do we make NCU class devices, like be able to work well with WebAssembly and the component model, you can get involved talking directly with people who are working on that through Zulip, there's a Bytecode Alliance Zulip.

So if you go to bytecodealliance.org, you'll get there and that's where you can really dive in deep to WebAssembly.

Benjie: What's Zulip?

Kate: Zulip is another chat interface, if you can believe it. It's actually fully open source, you can contribute code to the Zulip source code, which is kind of why we use it with the Bytecode Alliance.

And it's very... We had someone within Fermyon who gave a whole presentation about how amazing Zulip is and how to capture its advantages, so it has some huge fans who are advocating for it. So I'm just going to spread the word on Zulip on their behalf.

Benjie: Hey, no problem, it's open source, we love open source here. That's the whole point. So, okay, but now I have to have another chat client is what you're saying.

Marc: I haven't moved to Discord yet, so I'm still on Slack.

Benjie: Yeah, I'm XMPP. I'm using Gchat from 20 years ago.

Okay, cool, so this is the most important question, Kube Cuddle or Kube CTL or Kube Control, how do you say it?

Kate: I say Kube CTL, and I don't think people like that.

Benjie: That's not right, it's Kube Cuddle, it's Kube Cuddle is the right answer.

Kate: I think Kube Control probably is what I would like to transition to, but I do say Kube CTL.

Benjie: No, it's Kube Cuddle. My mission is to get people to say Kube Cuddle.

Okay, so we can go to the Slacks, we can go to the Zulips, we can go to the Discords, is there any other events around Wasm, or anything else going on where we can contribute, learn more, be a part of the whole Wasm movement?

Kate: Yeah, I personally love going to conferences, and there are some really good WebAssembly focused ones.

And these are more focused on WebAssembly on the server, but Wasm.io, which is in Barcelona, that was in March, obviously we're past that, but I would highly recommend looking at the recordings for that, especially if you want to understand what is the component model, where is it being used, et cetera.

Sometimes I just watch those over and over again while I'm drinking coffee, 'cause they're just so well done. And coming up, if you want to be in person for one KubeCon North America for one of those pre-event days, there is WasmCon.

So that'll be a great opportunity to kind of hear even more talks. And it won't be focused on Kubernetes, but it will be... A lot of it will be cloud focused and developer tool focused and a really great opportunity to dive in as well.

Benjie: I think it's technically the first day of KubeCon is that day.

Kate: Yeah, it's that one day before now that they don't do two.

Benjie: Right, so in November. Well, I will be there, splitting my time amongst a bunch of stuff, 'cause there's a lot of cool stuff going on there.

I'm really looking forward to it. Really appreciate you coming on, Kate. Learned a whole lot. I have a lot of research to do. Looking forward to finding all the links for this episode, just to like-

Marc: The show notes are going to be long.

Benjie: The show notes are going to be long and wonderful and I look forward to spending a weekend actually reading up on all this stuff. But really appreciate you coming and keep up the good work and keep up all the good stuff.

Kate: Great, thank you both for having me. And I'm around, you can catch me anywhere and pass me those questions as people listen and as you all listen again. So I really appreciated the opportunity to share all the things that I'm passionate about.