Ep. #9, Falco with Dan “Pop” Papandrea of Sysdig

Marc Campbell: Hi, I'm here today with Pop from Sysdig.

Pop is the Director of Open Source Community and Ecosystem at Sysdig.

And we're here today to talk about Falco.

It's an incubation project that Sysdig contributed to the CNCF, welcome Pop.

Dan "Pop" Papandrea: Hey Marc, thanks for having me.

Marc: Thanks for coming. So, before we start, I'd just love to talk about your role at Sysdig.

What you do, what the day to day is, and anything else you'd like to share?

Pop: Absolutely, so I am the director of Open Source Ecosystem and Community for Sysdig.

Prior to that, I was a field CTO, so I was interacting with a lot of end-users and how they're utilizing, not only the commercial product but also our open source.

We at Sysdig believe that we have capabilities we built off of our open source core.

So if you think about like Loris Degioanni, who's our founder, he was one of the people that put together, tcpdump and Wireshark.

He's like a legend. I was on the interview with him, I'm like, man, you saved my life in so many occasions.

I'm like, I feel like I owe you money or something, right?

He's just a fantastic dude.

And so, moving off of that kind of field facing role.

And this year I started a podcast called The Popcast, and I was much more immersed in the Kubernetes and cloud native community, getting to interact with like the community.

I love it so much, and I'm like, I've been on that revenue and field side for a long time, almost 15 years.

And I just, in terms of my career progression I'm like, I just want to integrate more from that perspective.

Because I love open source, and I was just made a CNCF ambassador and I think what the CNCF is doing is phenomenal.

And so I wanted to be able to do that and also like again, integrate my work day job.

Which is Sysdig and then, again, this incubator project as you said with Falco, being able to take that and grow it out and see if we can graduate with just this amazing set of team, this global team that we have with Falco.

So that's a little long-winded way of answering somewhat of your question there, but.

Marc: That's great, yeah, we're definitely going to dive in to some of that a little bit more.

And congratulations on being named a CNCF ambassador, that's a great responsibility.

Pop: It's an honor. And again, it's like I am in such respect, God rest his soul, Dan Kohn was a great person.

He inspired that community, he was a New Yorker as I am.

And seeing what Priyanka is doing as well, on the team with Chris, Anna, Zach and all of those folks.

I feel it's an absolute privilege and I cannot wait this year just to do more with the community.

I'm just going to double down with what I did with the community in 2020.

Marc: That's awesome. Yeah, we're both running podcasts and we both have day jobs that are just totally immersed in the CNCF ecosystem, so that's awesome.

Let's get started, let's dive right into Falco.

Can you kind of give us a high level and explain what the project is?

Pop: Sure, so Falco is like a runtime behavioral engine that basically helps you from a security perspective, that gives you prevention, allows you auditing capability for behavioral and monitoring and intrusion.

So things like syscalls are being used as the underlying fundamental interface between an application and the kernel.

And so syscalls allow you to like basically have the program, like, if a program is doing a certain task, processes or a network or a File.io or any of those things, those are things that are touching the kernel.

So it's the most truest way of getting, details. And so we have a kernel module that taps into eBPF natively.

Again, this is a native infrastructure.

I just definitely want to enforce that, where an IP helper that's jumping into the kernel that's able to do things like take suspicious events and send out like things through standard out or files or even if there's a side project, a fantastic project.

Which I talk about in a blog, it's CNCF Blog that we could link to here.

But basically it's called Falco Sidekick, where you can output to so many different mechanisms.

We're talking like Slack or PubSub, Kafka, AWS, SNS, those types of things.

And so you have a fault set of things that you can notify.

And also you can create these rule sets, macros, conditions, event types to be able to have it do anything.

And like a standard use case that I've seen is folks looking like there's a CVE about men-in-the-middle attacks.

I think it's called CVE-2020-8554.

And that was one where our security researchers are able to find the way to look at this and be able to see a Falco rule if a service was created with an external IP and give you that audit capability.

That's been so valuable for customers that, again, I think that security is only a software supply chain. Which is great, having a vulnerability management system that's blocking those things. It's something we do in the commercial product as well, but doing it at runtime to me is something that you need to have. You can't just say, "OK. I'm going to rest on my laurels, a nd I'm going to be protected from a vulnerability perspective from my pipeline."

If I'm running something, and I have this kernel capability to be able to see into those things that are wacky, that are out of sorts and then be able to get notified and then to an action of some sort, that is the most valuable thing you can do, and that's what I think the value of Falco is.

Marc: Yeah, so it's not to really replace the software supply chain security, it's really security in depth, put security at all the layers and protect that software supply chain.

We've definitely all familiar with recent stories around the SolarWinds Orion hack and like that problem with the supply chain.

But don't just trust that, is what you're saying is let Falco actually watch the runtime environment in case there's some other attack that's happening.

Pop: So, I did a talk with Steven Terrana from Booz Allen at KubeCon this year, and we talk about the complete pipeline.

We're talking all the way from like binary off to post-mortem and incident response and then forensics capability.

So to me, if you rest on just that you don't have a complete security solution.

And if somebody is telling you that then they're completely wrong.

So if you look at this and you do things like benchmarking and you do things like of the nodes that are running Kubernetes or vulnerability management or even compliance, like rule sets for like--

We just introduced that at KubeCon as well, native Kubernetes network policy creation and visibility.

Those are things you need a lot of solutions in one.

If you want to-- Again, there's open source solutions if you want to cobble together all of those things.

That's fine, you can do that or you can have a complete solution in Sysdig or, again, from a runtime perspective you can utilize Falco from an open source perspective to be able to go forth and do whatever you need to do from a security perspective.

But I totally agree with you, it's not a single point solution for security.

You need a host of other things or you're going to run into exactly what happened in that incident.

Marc: And so you mentioned that it's native in eBPF but it's a CNCF project and we talked a lot about Kubernetes already.

Help me understand a little bit, like, does it run as a Kubernetes daemon, is that a pod?

Is it something have to run on every node? Like what's the architecture of Falco?

Pop: So basically we can run as a system D, as a daemon running on the host itself, right?

So you can run it as a-- running of a Kubernetes or you can deploy us as a daemon set and you can deploy via Helm.

So for instance, I spin up through cube admin I set up a cluster.

I can very simply run a manifest to be able to deploy or I can install on those underlying nodes if I need to.

So if you go to Falco.org, there's an install capability that shows you how you're able to do that.

The other capability, like I said, if you're spending it up, if you let's say use EKS or AKS or GKE to managed solution, you can also use Helm to deploy these.

And we give you like a default set of rules, which again is very cool.

You already get a default set of rules that you would protect against, what we see as a local set of rules.

And then you can also create your own rule sets through this Falco rule syntax.

So it's very cool, you can even install it as a Red Hat package or a DBM package.

We have an installation script, so, or even to run it as a Docker container.

Marc: That's cool that you can actually just Kubectl, apply it, and get Falco all up and running inside a cluster.

Pop: Oh, it's very cool. Not only in Kubectl, but you can use like Helm or something like that or like, to be able to deploy.

I've seen customers and users in the space and deploy it very quickly from that mechanism.

Marc: You mentioned in the introduction here that Loris is the CEO of Sysdig, was one of the founders of Wireshark.

And so this is like something packet analysis for anybody who's like new into the infrastructure ecosystem in Kubernetes.

Wireshark is definitely like it's a great tool that most of us used back in the day and continue to use it in certain cases.

Can you help me understand a little bit, like how has the Sysdig DNA been baked into like forming Falco?

Pop: That's a great question. And how it is, is basically, again, if you look at the syscall capability that we have with Linux P and Litscap.

We know, those are basically where we're tapping into the ring buffer capabilities with capture, and so the underlying story there is, there was an end customer that was using Sysdig, and they're like, wow, you're capturing a lot of this information.

Like we said, File.io process, network details.

You ever thought about taking this and turning this into some type of behavioral rules set engine?

And so utilizing syscalls like that are trapped into ring buffer.

Again, if you look at EDP functions, folks are creating IP, like helper applications in the user space.

We were doing that before anybody. We even contribute back to eBPF those capabilities.

So like Lorenzo Fontana, one of our amazing developers on the Falco project, he wrote a book on eBPF.

So in terms of our kernel module tapping into user space, that's the capability that we have for being able to like go very deep in the kernel.

But we've been doing that in an eBPF VM, like the virtual space for a lot longer than other tools that are out there.

So you mentioned Wireshark, this capability is not just network details.

It's literally every single system call that you can then create a rule set that says, I want you to look for this specific system call or utilize the rule sets that we have out of the box.

I think that's one of the most powerful things out there and that's where we're seeing adoption of Falco grow and grow and grow.

And I'm just loving it. It came from Loris, took a summer off and created this and came back and said, look, this tool is how we should go forward.

And it's such a blessing working with that man.

Marc: Yeah, that's actually interesting. So, Falco didn't exist when Sysdig started?

Sysdig created this platform, they've been around early in Kubernetes, and then you said Loris took a summer off and actually created this.

That's an interesting story and pretty different than a lot of CNCF projects that created the company out of the project.

Pop: Again, we've always had open source roots, right?

So if you look at our-- this capability for being able to like parse Linson P like events, right?

And then donning that back to the CNCF, because there wasn't a project out there that was doing runtime capabilities.

And I believe that Falco is the default runtime behavioral engine for Kubernetes, period.

And not because I worked for the company, and also I worked with the project, and I'm responsible for helping to get growth and adoption of it from a community perspective, but it is an amazing tool.

You can see every single system call that happens, not only on the single nodes, but every single node that you installed there daemon set on.

I don't know of any other tool that can do that in the native capability of the eBPF. I will go on record.

I'd love to challenge anybody to bring their tool against Falco. We will win every single time.

Marc: Yeah, I don't know of one, so that's great. Let's actually dive into those rules for a minute though.

You said that Falco comes in, it analyzes, it inspects all the activity, the syscalls that are happening and then Falco ships with a set of default rules, but you can write your own.

What types of rules does it ship with? What can it protect against?

Pop: There's a multitude of rules that you have. And also, we have the SecurityHub.dev, where you can grab rules for things like Nginx or FTB or those types of things.

So, yes. Those already come out of the box in terms of the rule sets, but you can also augment it with your own.

There's even like folks building things for like Fluentd or Elasticsearch and those types of things.

Or even like for specific CVEs, for, like I said, I mentioned earlier.

There was a RunC container breakout a year ago, right?

Look at that, that's basically like Dockerd and if it's opening right to prox self those are things that you can have a rule.

So you can create a condition and then send that output saying, ”OK. Further scripted, that name is open to right by process.”

And it shows you then that's doing that.

Couple that with something like Falco Sidekick, which basically I can then send outputs to gnats or LowKey or Prometheus or SMTP or OpsGenie or Datadog if I need to.

At the end of the day, you can do a ton of that stuff.

So the rule sets are plenty that come out of the box that are doing things for file intrusions or network intrusions or proc inclusions.

And then again, like we even created ones that like map to like the MITRE ATT&CK framework.

So you don't have to do the guesswork there. And then if you decide, ”I don't want to have to build all these integrations.”

That's where you look at our, Sysdig's commercial tool.

Sysdig's commercial tool basically has vulnerability management, native Kubernetes network policy capability, the runtime capability, but also an audit capability of everything that's going on.

So both tools, if you look at it the commercial office are all based on an open source core.

I know of no other project out there or company that's based specifically on that capability.

Marc: How much security expertise do I need to have in order to run Falco?

So like I'm wondering, if I'm a small startup and a couple of people, but I don't have any dedicated security personnel and I'm running managed Kubernetes in the cloud somewhere, what you just described in the rule set is great, but if I don't understand what some of those rules are actually protecting against, what's the learning curve to get started?

Pop: So again, it's always a culture thing as well, because look, developers or operations folks are not going to know about all of the security capabilities out there.

If you look at, I think there was a panel that the discussion with Ian Coldwater and Duffie and Marc Eisman and all those.

There's a lot of best practices out there. The Falco rule set has a default set of rule sets that we feel are kind of the best practices that are out there.

So you can adopt those and pick those up very quickly, and the other part of that too, is you can basically like utilize that security hub like I talked about and grab new rules as you go.

So that's the capability.

Now, the other part of this too, is if you decide, "Look. I don't want to have to create all of these integrations as a tool and all of those things," then that's where you would probably flex to our commercial tool to be able to do that. However, again, the open source out of the box has a ton of these kind of security paradigms that you're going to need anyway. So like from a runtime perspective, like I said, with file or network or specific applications or even if you're running on a cloud provider, we can audit Kubernetes audit log capabilities.

So think about this, let's just say, I'm a startup and I spin up a replicated cluster of Kubernetes, I'm going to have the best practice based on runtime right there.

However, if I want like benchmarks and vulnerability management and runtime capabilities, kind of that full workflow, then I would probably look into Sysdig's commercial tool.

Marc: So we're all running Kubernetes right now and Falco makes total sense.

Like what tools did this replace?

What existed before we were on Kubernetes that people would use to solve the same problem?

Pop: So it was probably a lot of different tools.

If you look at it you would use, AuditD capabilities, some of those capabilities to be able to do that.

But again, because containers are black boxes, you don't have the context, that's the beauty of this.

So basically an output, I can say this thing happened in pod A that happened in cluster B.

You know what I mean?

I had the context that they have because, basically the Falco daemon set is interfacing with the Cube APIs, to be able to tell you, ”OK. This is replicated pod one that's running in cluster B,” those types of things, right?

That's the thing. So, you could have utilized all types of older tools to be all that.

And there's a ton out there, I'm not going to name all of them. LSOF, those types of things.

Like OSquery, that are specific to just querying but they're not able to do like behavioral capability.

So again, there's a ton of legacy tools.

But also, you can use Falco on a standard host as well.

So you can utilize that. So, it runs the gamut.

Marc: That's cool, so I can actually just run it without Kubernetes, if I wasn't running Kubernetes yet?

Pop: Correct, It runs as a system D on the host itself, so yeah.

And we have, again, if you go to Falco.org the installation tools are right there, so.

Marc: You talk about the custom rule sets that I can write.

Once I have Falco up and running I can create my own custom rule sets to protect against things that I know about or that are important to me.

Do you have any examples or stories that you might be able to share around like unexpected use cases where you saw somebody using it in a creative way and you're like, wow, that's actually, we didn't think about that when we built Falco.

Pop: I've told this story on a couple other podcasts, but it's so phenomenal.

I love it so much and I'll say it again.

So basically there was this very large investment bank that we worked with.

And this is where we came up when I mentioned earlier the whole MITRE ATT&CK framework, capabilities mapping Falco rules to it.

They had 36 rules that they wanted to create. They gave it to us on a Monday.

Like 24 of them were available to them by Wednesday and the 26 were done by the Thursday of that week.

Phenomenal, and again, there were the rule sets it checks.

We can set check for SIM links, ownership changes, executing shell binaries, as the Sage binaries, core Util executable, log and binaries.

They were like, all of these things check the box.

So they were able to create those rules that all using the Falco rules syntax very quickly.

That's one of my favorite stories, because the end-user will utilize Falco and then actually uplifted to the commercial tool, because they needed other capabilities.

All based on the power of Falco's rule sets.

And the syntax was so simple for them to create them.

Marc: That's cool, I assume the rule sets are just YAML at the end of the day that you're writing, right?

Pop: Correct, and again we have default local rule sets there that you can discern.

And what I'm trying to do, and that's something I think we'll talk about a little later, but having folks contribute more.

So we'll talk about a roadmap as well, but we're trying to make it more easier for folks to sandbox and create rule sets and then contribute those back.

I would love it if Replicated had a default set of like-- and I'm sorry, I'm going to put you on the spot, Marc.

But a default set of rule sets that we can include in Falco, and then say anybody who's running Falco, "These are the default rules that you would get that are omitting things that are processes that need to happen from a Replicated perspective.”

I would love that. So be giving that to Amazon, for Amazon to do that for Fargate or something like that, because we can also support Fargate clusters and server lists and those types of things.

So, those are the things I'd like to see more folks contributing from that rule set capability.

Marc: That makes sense.

I think there's a huge, getting everything baked into the default is super important and making it, so when I spin up a Kubernetes cluster, I should just have Falco and I should just have default rule sets running it. It shouldn't be, I just created a Kubernetes cluster and now I have to think about which security framework I'm going to add in.

Pop: Well, that's the beauty of it. We already ship you with a ton of them.

And then if you go to SecurityHub.dev, you'll have a ton of best practices there and folks contributing back into that as much as possible.

So yeah, you have that. Because again, all rules, trading a rule, you need these keys, you need rules, conditions, descriptions and prioritization.

And so basically there's conditions that you can do, you can version them as well.

So I see folks using things like, Terraform or something like that, to be able to create these rules.

And then say, “OK. This is 1.2 and this CVE came out. let's call this 1.3 and this will address this.”

So almost using the Falco rules as code, we have that capability and we want to extend that out as well.

Marc: Then how do I, like if I'm running Falco and I get alerted that like there's activity in my cluster, that's the CVE?

Does the SecurityHub.dev or does Falco, or is this more of a commercial offering assisting?

Help me understand what that means to me, like what the risk is to me, what I should do to remediate against that capabilities or anything like that?

Pop: Great question.

So again, if you're looking at Falco as itself, this is the detection and audit capability there, right?

So it's going to tell you, but then you can use like something like Falco Sidekick, which again is an open source project to send it out to something like gnats or have it run, like let's say a Lambda process that's going to do some type of action.

So there's that. The commercial tool, you have the capability of using those Falco rule sets then have it trigger an action, like stopping the container, pausing the container or running like a web hook.

Which you can, by the way, run a web hook even on the Falco side to have it do an action of some sort, right?

So that's what I see as the capabilities that the terms of remediation.

So understanding what's going on.

If you talk about that CVE that happened, if I didn't have that capability to even detect that which is something like, signals that are different than the application that should normally do, then I would want to have that.

But again, if you look at this in terms of detection and auditing and those types of things, prevention is not enough. You need complimentary and not mutually exclusive approaches as well.

Marc: Does the team assistant responsible for creating those default rules in the SecurityHub.dev or like you mentioned, you take contributions, where are those coming from?

Pop: Security researchers.

Look, I mentioned Brad Guzman earlier, he contributed a rule set that fixed an issue with registries.

Specific container registries that was addressed by him.

What I would like to see this year , we're actually working on making the rule sets easier to digest and have like, sandbox capability so you can go and play around and create your rules. I want to see more people outside contributing. I want to see risk security researchers create new rule sets that are out there.

And companies like Amazons, Googles of the world, the Replicators of the world, that are just like, "I want to kind of contribute this rule because I saw this in my cluster. I want to be able to do that.”

So that's what I see. And that's, again, part of the thing, if we talked about my CNCF blog, I wrote earlier.

To me, I want to see more contribution, I want to see more adoption, because at the end of the day I want to see Falco, we all want to see Falco as a graduated project, just like Kubernetes and Prometheus, because we feel strongly that it is the best one-time behavioral engine on the planet.

Marc: Yeah, and I think there's something interesting too.

These CVEs that come out in Kubernetes, they get announced, but they've been under embargo for a while and cloud providers and upstream distributions have an opportunity to patch them, which is great before like the details are disclosed.

Does the rule set in Falco? Do you have a path where you're going to be able to start like having the rules to detect a CVE on day zero of the announcement?

Pop: So in terms of CVEs, like we have a security research team that's creating these blogs and on the capabilities of the Falco rule sets.

And those are merged as the beauty of open source, right.

Is they're merged into those local Falco rules sets.

Again, I mentioned Kaizhe Huang, he kind of created that one based on that CVE I mentioned earlier.

Or like, if you look at our security hub there's ones like, 2019-5736, which is a minor mitigation for a runtime capability.

So to answer to your question, in terms of the embargo process, we work with hyperscalers, we work with the companies that are out there to be able to help address those things and be part of the overall kind of CVE process whenever possible.

Marc: So, that makes sense. Let's transition for a minute and chat about just the roadmap of the project.

Today, Falco is an incubation project. You mentioned you want to see a path to graduation.

Can you talk a little bit about like the timeline and the requirements and what Sysdig is doing to help push that agenda?

Pop: So, we're just as prone to any other, the TOC kind of capability of trying to get us fitted into that.

We've done a ton of work in the year, like we've migrated for instance to a prowl process, which is phenomenal.

We wrote as a blog on it to be able to do handle our build process and all of that. That's been great.

And so like we're adhering to the CNCF process for getting graduated, but in terms of roadmap capability, like we're going to donate Lin SP, capabilities to the CNCF.

So other people can utilize the capability to do like linscap and Linux P capability.

That's kind of the first thing. The other things we're trying to do again is get more contribution and adoption.

We had over 20 million downloads last year.

And so it's just been phenomenal in terms of like the utilization, but I want to see, I only have like 12 adopters right now.

And I'm like, wait, I know there's a lot more people utilizing Falco.

I'd love to see more in terms of the adopters.

So having people make it easy so people understand, that's my gospel and mission this year, is to make sure folks are contributing and they feel part of this Falco familiar.

We're doing such an amazing thing. And so having people involved in it, just if you think about like Kubernetes.

I'm part, of the Kubernetes community, I'm part of the contributors.

I love that community, everybody in that community is phenomenal.

They want to help people get through. And we're trying to get Falco to be a place for that as well.

Last year they did a ton in terms of getting it in a place for us to be graduated.

And I see this year, us creating more capabilities to make it easier to adopt and contribute as well as adhere to whatever the CNCF requirements are for being graduated.

Marc: And outside of the security researchers that you have, when you talk about just engineering effort that goes into building Falco, do you have a dedicated team?

Are there folks working full time on the open-source Falco project at Sysdig, or is it really just kind of split among the engineering team as part of the Sysdig platform?

Pop: No, we have dedicated engineers.

We talk about, again, Leonardo Di Donato, we have Leonardo Grasso, we have Lorenzo Fontana.

We have engineers and there's also, you know folks on the main commercial team that will contribute.

Because we feel strongly that open source is always the core of what we do.

And we've all dedicated folks but also there's an expectation that the engineers will work on it.

But also beyond that, we have contributions from IBM, AWS, VMware, Mattermost.

If you look at our CNCF stats, there's companies all over doing it.

I would love to see Replicated on that list, to be honest with you, Marc, Do you know what I mean?

So like at the end of the day, it's like, ”I want more people to contribute, I want to understand where they're utilizing it.”

But again, any project, if you think any successful project, if you think of like Prometheus or the Helms of the world, those type of things, they didn't just have like a certain setup.

They had a high level of maintainers but they had folks contributing all over the world.

And that's what I would love to see.

Marc: We're going to get Replicated on that list too, because like we've run Kubernetes, and obviously enterprise, Kubernetes distribution security is super important and like Falco is a part of what we're doing right now.

So we should do that.

Pop: And I'm a big fan of you all.

Originally the deployments are backend used to utilize Replicated. It's a great tool and anybody out there, again, I'll give you a shameless plug for you, but it's just so quick to be able to like just run process to get, your applications.

Then it might've been running on a SaaS from an on-prem capability.

I haven't seen you all capabilities recently but I was so floored with that.

I got to be kind of a Replicated expert, because then when I originally started with Sysdig I was wearing many hats.

So I was deploying and doing customer success, marketing, all that stuff.

And so I really had fun with Replicated and I appreciate working with your support teams, your phenomenal team over there.

Marc: That's awesome, yeah, and it's fun as being a part of a startup like Sysdig, where early on you wear all these hats and you actually find the one thing.

You said you were a field CTO and then you were able to spend enough time on the revenue side and go find this like open source community project that you just wanted to expend all your time doing and being able to like eventually stop wearing all these hats and being able to like put some focus in.

Pop: Yeah, but you know what I love though, I love speaking and being with end users, it's my favorite thing in the world.

I love just sitting down with a customer and saying, I solved this problem this way and I use your solution this way.

Tell me what's out there. My favorite part of my day is doing that.

It's not being in meetings, it's not coding, it's sitting there with somebody from Bloomberg or something like that, where they're doing such amazing things and you're just floored by what they're doing.

That to me is worth the price of admission.

And I love that my new role, I get to do more of that and interact with the financials, the farmers and any end user that's out there in the world. I'm not just limited to a specific geo now, I'm literally now all over the world that I'm so excited for that.

Marc: Yeah, I imagine being so deep in the security ecosystem and in Kubernetes puts you in a unique place where you have developers for these large, regulated industries that want to adopt Kubernetes but they're hitting these obstacles inside their org and you can actually unblock those and like create a path to like more Kubernetes adoption inside these large, regulated industries and organizations.

Pop: Oh, without a doubt. There was just a stat, a usage stat that looked at-- It was like almost $1.6 trillion lost or something on like either misconfiguration or cybersecurity.

So, think about that. People are looking like they, even legacy hosts, there's cybersecurity issues and all those things.

And then you couple it with adding like an abstraction layer on top with Kubernetes, that's I think the power of things like Falco and the other tools that are out there, is that capability to give that peace of mind kind of give that confidence in running your cloud native pursuits.

Marc: Yeah, even the SolarWinds Orion hack that was in the news recently.

Nobody, like no organization today controls the supply chain of all of the software that's running in.

We’re all relying on either open source or commercial software running, and those have their own supply chains.

And so having something like Falco running in the cluster, which is able to do more than just query those vendors and projects to figure out if they've been impacted, but be able to like have some confidence in it is really empowering.

Pop: There's no truer place than the kernel to be able to look at that stuff.

If a binary is doing something out of the ordinary Falco will tell you.

If it's writing to a director it shouldn't be, Falco will tell you.

So at the end of the day, like you can't just rest on your laurels on having a vulnerability, like container scanning thing from those things.

It's great to have that, because again, you need protection from all layers.

You're not going to like build a wall.

If you have a castle, you're going to build a moat around your castle, not just on one side of the castle.

So at the end of the day, that's the thing.

Wow, did I just go "Game of Thrones"? Anyway, yeah, so.

Marc: And so, going back to the CNCF landscape and being able to move to a graduated project, you mentioned a couple of things.

You mentioned adoption and contributors.

You talked a little bit about various contributors and we definitely have to like, make some contributions that we'd like to actually see in there too.

But you said that there were only so many listed, like, I think you mentioned 12 listed adopters right now.

Pop: Yeah, those are adopters.

Yeah, And I don't think we had a focus on it, because people are using it, but I want to see more of that.

Even the adopters that we have are-- Shopify is one, right.

Those are ones that are part of our adopters list and it's growing.

So it's like, I want to try to get more folks talking their user stories they're using, utilizing Falco, how are they using it?

And so that's part of my job, is I want to be in there interact with them.

And then, so exactly what I mentioned earlier is talking to end users of Falco and seeing how they're using it, it's the highlight of my day.

So I really want to do more of that.

And so helping folks like understand what Falco does.

And hey, I'm having issues with deployments, I want to understand how to fix it.

I want to be able to channel those and help to adjust those things.

Marc: Yeah, that's one of the challenges of open source in general is there are people out there using it and that that's great and that's what you set off to solve.

But then you often just don't get a lot of information about who's using it and how, like there's some stories you're definitely missing and not knowing the full extent of what it's being used for, is this just in general, a challenge of open source.

Pop: Yeah, and I'm going to plug, shameless plug podcast, everyone just follow @PopcastPop on Twitter, you'll see links there.

But on my podcast I had the HashiCorp folks on and they were talking about like, they don't build crippleware, right?

So like there's capabilities on Falco that's going to do ton of capability.

And if there's things that you need to do, like from adoption perspective, they know that they're folks that will never buy commercial tool from an open source perspective, right.

It's just, they're not going to buy it, they're going to utilize that.

But there are so larger implementations, larger enterprises that need additional capabilities.

That's where they may would go with a commercial tool.

But if they can then extend the basics, the underlying engine here through an open source platform like Falco does, I don't know of any other vendor that does that or project that does that.

If, again, going back to the adopters list.

If you think of Booz Allen, Coveo, Frame.io, Gitlab, Logs.io, Shopify, all of those folks that are utilizing as an adopter of Falco, they're using it as a core capability of doing runtime protection for their Kubernetes or cloud native hosts.

Marc: Do you ever have to think about enhancements or features in Falco and have a internal conversation with the product team wondering, like, is this actually going to take away sales from the commercial offering if we make this open source?

Pop: No, because we do everything out in the open, and also like we have key contributors that are part of their community meetings that are on the commercial side.

Look, you're going to balance the things that are necessary, but you're also going to balance the thing that the community is asking for.

I think we've done a really good job of it. At the end of the day, Falco is what it is.

It's an underlying rules engine with a default set of rules and a default set of outputs that you can stand with Falco Sidekick.

That's what we do and we do it really well.

And we're going to extend those capabilities, make it easier to adopt, make it easy to contribute.

But at the end of the day there's commercial capabilities that we use with assisted secure that extend that capability even further based on that open source core.

Marc: So, I think one of the messages here is if you are an adopter, if you are using Falco open source, let's get you listed so that we can actually help push the Falco project through to the graduation phase.

Pop: And we're in Kubernetes Slack right now, there's a Falco channel.

Join the channel, talk to us about what you're doing. That's the best way.

And if you're running into, ” I'm having issues with getting this deployed,” let us know.

Or, “ I want to create a rule set, how do I do it? How do I go back and contribute back?"

That's what we'd like to see.

And again, shameless plug for that CNCF blog we wrote, it really shows a lot of the things that we're looking to do.

I'm trying to get more folks to utilize it and also tell their stories, “How are they using it? What are they doing with it?”

So, I'm excited to interface with you all, I'm in there constantly, I always check it.

My wife tells me I have to put my phone down, but I don't care, I love what I do, so I'll always be looking at my phone and wanting to help people.

Marc: That's great. Yeah, and we'll definitely make sure we link to the CNCF blog post in the show notes here.

So outside of more contributors and more named adopters, what's on the roadmap?

What do we look forward to in 2021 for the Falco project and feature sets?

Pop: First, I would implore you to join our community meetings because you can help shape what we're doing from a roadmap perspective.

At the end of the day, we're doing a ton.

Like I said, the biggest majority of the things is doing that donation for Linux P, the libraries to the CNCF, that's first and foremost.

But also again, make it easier to adopt, talking about the rule sets, make it easier for folks to do rules, trading the sandbox capability.

We also have a VS code integration now for creating rules, that's being merged in.

That was from a contributor, from actually the assistant commercial team, which is awesome.

So there's so much extent, then the capabilities also, making it easier so there's no dropped syscalls.

We're doing a lot of that to fine tune the backend.

We're also doing a ton to enhance like the creation through something called driver kit of like kernels that are being created from any eBPF perspective.

Because that app that we have, uses the eBPF but it's constantly looking at specific kernels, versions of kernels to be able to do that.

So the boys had done, and the community has done a ton of amazing work, but the boys are going to look to doing more enhancements of that as well.

So it's just, it's phenomenal. And then also I love, I'm going to do a shameless plug for Thomas.

Thomas is one of the guys that works on Sidekick.

And so Sidekick, again, is this integration between various outputs of it.

He created almost 23 in the community.

To create almost 23 outputs in a matter of less than six to eight months.

It's phenomenal the growth of that. So I'm really proud of that.

And lastly, we're revamping our website to make it easier for folks to interact with Falco, to do things like have, like a specific developer level, questions answered from API perspective.

So again, doing a lot of things to help with adoption and also fine tuning the engine and rule sets to make it easier for folks to utilize Falco.

Marc: That's cool, yeah. The Sidekick project sounds like really cool.

I think having alerting when these events happen is really powerful obviously, like that's where everything starts.

But like what really matters the end of the day is making sure the right people get notified and not have to like carry another application or open up something else.

So integrating into that existing tool set is really cool.

Actually, probably one of the big drivers for Falco adoption is the Sidekick project.

Pop: I would say that as well as like, the integrations that we have with--

If you think about AWS capabilities of Fargate, we're seeing a lot of traction from that perspective as well.

And again, kudos to Jonah Jones, he's one of the contributors who works at AWS.

That dude's been phenomenal. Another person is Carlos Panato.

I got to give shout outs to him as well, Carlos Penato is at Mattermost, and now he's at VMware Tanzu and he is phenomenal.

So he's created, I think he did over 30 PRS in a matter of a month.

He's like an Helm expert. And so, yeah, it's just been great.

It's like, I love seeing the community grow.

I'd love to see, again, anybody who's listening to this Pop@Sysdig.com or @PopcastPop or @DanPopNYC on Twitter.

Reach out to me, I want to make you part of this project.

One thing, the shameless plug today, somebody actually contributed a running Falco in WSL 2.

If anybody's familiar, that's windows subsystem for Linux running on windows 10, I was floored by this.

They compiled their own kernel to be able to do this.

And they provided a step-by-step log on how to do it.

So you geeks out there that want to have like me that a lot of have this in a sandbox and want to kick the tires, this is your best bet.

Marc: That's pretty intense. Like running Windows, running WSL with Linux and a custom kernel to have it all available locally on your laptop, that's pretty cool.

Pop: Did you ever see the movie "Inception"?

You got to spin that top, you're spinning the top right now.

Marc: Yeah, that's amazing.

The last thing I'd love to chat about is just like. if I don't have Falco right now, maybe it's new to me or I've been exploring it, what's the best way for me to get started.

Like, should I just dive right in and install it, custom rules, get everything out of the box, hookups some integrations with Sidekick, or is there, is there a path that you've seen work really well for new adoption of Falco?

Pop: That's exactly it. Again, we have support on Kind or Minikube if you want to kick the tires there.

Or if you want to spin up like a GKE cluster run, or manage cluster or even running it on a single host.

Like, “You can run this locally and have this run.” So that's what I see folks doing.

And then being able to say, "OK. I created this rules, I know what this thing does because now I'm seeing the stream of events happening from the kernel. And now I can create these rules sets because I understand what's going on in terms of the conditions of these rule sets." So to me, that's the easiest way for somebody to get started is installing it, understanding what the outputs are, creating rule sets.

Then using something like a Falco Sidekick or something like that to output that to whatever mechanism they want.

Like let's say they want a Slack alert or something like that.

And that's what I see as the easiest way for somebody adopt it.

We got everybody, just go to Falco.org. It has all the steps that are there.

And if you need anything, we are in the Kubernetes Falco channel on Kubernetes Slack.

So we'd love to help you all. I want to see all of you in that channel that's listening to this and yeah, that's it.

Marc: Yeah, that's cool, it's definitely pretty straightforward and easy.

We have it installed and I can actually just speak to the fact that like, it is watching and alerting for things that like we have no other way to like monitor for today.

So like it's a critical part, I definitely recommend it.

Pop: Loris says the kernel doesn't lie. And I agree with that, it doesn't lie.

You're going to see things that now we're going to see but then the whole thing is the power of it.

And we want to make it easier for you to say, okay, we see all these things, now what can I do with that information?

This year is the year that we make it way more easier for you all to do.

Just like again, I would say at the beginning of last year it was a little more difficult for you to install Falco.

We made it a lot more easier.

And our plug right now is how can we make it easier for people to output or create rule sets, or deploy the core engine, that's the big thing we're trying to do here.

And also just grow out some of the capabilities based on what the community requirements are.

Marc: So, do you think we might see Falco apply for graduation sometime this year?

Pop: That's our hope and it's what we're shooting for.

But again, I want to see more adoption out there but absolutely I would love to see this.

And again, being a CNCF ambassador as well, I have to be agnostic to all the projects that are out there, but I just, I love this project.

I would not have moved from. I love the job that I did at Sysdig and I love this job here because I feel so strongly about this project and I feel so strongly about the community's adoption of it.

Marc: That's great. Look, we love everything that Sysdig's been doing been around early in the Kubernetes ecosystem.

The Falco project is great in contributing that as a CNCF project, it is awesome.

The Podcast is a great podcast, I would encourage anybody who's listening to like go, follow on Twitter and subscribe to the podcast.

You have some good stuff planned for this year, I think.

Pop: Yeah, so season two is here.

So if anybody who hasn't listened to the podcast, the podcast is basically where I looked at this as an opportunity to like connect people, the humans behind the code, but yeah, it's been a godsend and I'm really happy about the podcast success and just being out there in the community.

Marc: Yeah, that's great, it's good to work on it too.

It's definitely gone from brand new to one of the ones that I look forward to and subscribed to all the time to.

Pop: Appreciate it, your checks in the mail.

Marc: All right, is there anything else that you want to talk about Falco or Sysdig, anything?

Pop: Nah, just go out there and use Sysdig, and I appreciate the opportunity to talk to you today.