Ep. #8, VideoLAN with Jean-Baptiste Kempf

Matt McClure: Everybody, welcome to Demuxed. Today we have JB Kempf from the VideoLAN team. Really excited we were able to get him in person, following FOMS and Demuxed. So thanks for being here today, JB.

Jean-Baptiste (JB) Kempf: Thanks for having me.

Matt: Real quick, today Phil and Steve couldn't join. So again, we have Nick joining us, Nick Chadwick.

Nick Chadwick: I'm very excited, this is the second time. I think I'm going to become a regular, I can tell.

Matt: Yes, yes. It's the accent factor again. JB, why don't you give us a little background about yourself.

JB: I'm a French engineer, I'm 34 years old, I think, if I'm not mistaken. I'm the president of the VideoLAN non-profit organization that was created in 2008, and I've been working on video-related technology since 2005, maybe even before. I'm a developer, obviously, a geek. I have now a startup around video technologies.

Matt: I had no idea you had a startup, outside of VideoLAN?

JB: Yes.

Matt: Cool, what is it? Or that you can talk about, at least, I guess.

JB: VideoLAN is a non-profit, and Videolabs is a startup. Because one of the problems we have with VideoLAN is the lack of funding and the lack of involvement of people. But doing a startup is a cool thing to do, right?

Nick: It is pretty cool.

JB: But we can talk about that bit later, I guess.

Matt: Great, cool. I guess it probably makes sense to give people a little background about VideoLAN itself. I think anybody that's been in the industry for more than 10 minutes recognizes the cone and the hats.

Nick: Or anyone who was a student.

Matt: Or anybody that's ever watched a video on their computer when it didn't work and needed to download something else. Yeah, I guess let's just start off with what is VideoLAN, and how you got involved.

JB: VideoLAN is a project, a software project, like Mozilla is, and there's many software developed by the VideoLAN non-profit and the VideoLAN team. One of them is VLC, but that's not the only one. x264 is a VideoLAN project and we have a lot of other, mostly libraries, for the developers related to video, codecs, demuxers and so on, that are made by the non-profit.

"VideoLAN Client" became VLC, is the most known software, but it is not the only one. So in fact, VideoLAN was a student project. At the beginning of 1994, it was called "Project 2000." Project Network 2000 was a way for the student of the Ecole Centrale Paris campus, which is a university south of Paris, to get a new network.

Because they wanted to play video games, and they had a very old token ring network, which was great in 1994, except video games because the latency was crap. So they wanted to go and get a new network, and no one wanted them to get a new network because it was a lot of money. So they went to see a few companies, and one of the companies was a French podcaster, and says, "You know, TV by satellite is the future." Well, it's easy to laugh now, 20 years after.

Nick: Wow, so TV by satellite wasn't around when VideoLAN was created.

JB: It was just starting in 1994, right then.

Matt: That's incredible.

JB: And that was just the idea, right? And the problem was there was 1,500 students on the campus. That meant you have to put 1,500 boxes of decoders, of MPEG-2 decoders, that costed 2,000 box. And also 2,000 satellite dishes.

So one of the guys says, "Okay, if you're able to have only one satellite dish and one decoder, and streaming on the network, then decode on the PCs, then I pay you a new network." So of course, that was 10 years before YouTube, right?

Now it's obvious, but at the time when they started the project we are talking about, processers were the 486 and the Pentium 60 and 90. So that was like science fiction, to do MPEG-2 as the decoding on a machine. But the students said, "Okay, deal." So it started the project which was Project Network 2000 and took more or less two years to have a proof of concept that managed to stream two pieces of video.

Forty-five seconds, the machine was packed with 64 megabytes of RAM or something, and of course, everything would crash after 50 seconds. But the demo was 45, so that was fine. They got the network, and that's how the project Network 2000 started.

The only idea was to play video games with low latency, so it had an awesome network. Very expensive at that time.

After that, 1997, a few guys were also students on the same place: "Well, you know, maybe other people care about this VideoLAN streaming ptoject, and maybe we should do that as open source." So they restarted the project and called it VideoLAN because it's video on a local network.

There were three important PCs: the VideoLAN server, the VideoLAN client, and of course, some stuff doing some switching on the network level that made no sense outside of this student university network. And that's how it started.

It took them two years to make it open source because the university wanted to keep that, you know, to resell MPEG-2 decoders or something like that. It completely failed. And so in 2001, it became open source. Both, all the software, there was like, VLC, VLS, VLCS, VLM, and all others.

VLC, when they started to make it open source in 2001, it started to really get traction.

Especially because it was ported on the Mac by some guy from, I guess, Netherlands. And that was the only way on Mac OS to play DVDs.

Nick: That was pre-QuickTime?

JB: QuickTime couldn't play DVDs. For a long time, you needed iDVD.

Nick: Okay, so, one of the first features that came out of this VLC project, after it was open sourced, was DVD playback.

JB: Yes

Nick: Because DVD's MPEG-TS under the hood?

JB: Well, exactly. So it's MPEG-PS, but as soon as you have your MPEG-2 audio decoder and your MPEG-2 video decorder, then TS and PS are not that difficult and not that dissimilar. So that was easy to add. And as soon as you could hack and destroy the DVD CSS protection, which was not really protection since it's so weak, they added that and after being ported, being open sourced,

in three months someone ported it on the Mac. Then in six months, someone ported it on Windows. That's how VLC started getting actually popular.

In fact, VideoLAN is the project that did a lot of those stuff related to network and video streaming and servers and clients and so on. Normal people just use VLC as a client which was, at the beginning, only playing MPEG-2. Then someone added another decoder and then someone added libavcodec at some point.

We were in 2003 and that eDonkey, Kazaa, eMule or whatever those stuff that none of us have ever used, of course, went around. That was one of the best ways to play DivX, because DivX, MPEG-2, MPEG-4 ASP was so great. You could put your DVD on a CD for the same price. So that's the difference between VideoLAN and VLC.

The second very important project was started in 2004, I believe. Which was H.264, the H.264 encoder that was started by Laurent Aimar for his project and that became probably the most used encoder in the world, I guess?

Nick: Probably. I'm not sure there's any doubt at this stage.

Matt: Yeah.

Nick: I'm fascinated just by one little snippet you had there, is the core innovation that VLC had that got it off the ground was this software, MPEG-2 decoder. But the university couldn't find a market for it? Like, there was no commercial market for an MPEG-2 decoder?

JB: Well, there was. The problem is that it was a student project which would be a major project. The allocated time was 150 hours per student and students put 150 hours per month. So they said, "Well, you know, we want to be open-source." It was like pure GPL. They didn't care at all about making money.

There was so many questions about patents and licensing on MPEG-2, that didn't really care. So they were more focused on having a good technical background and having fun. That was one of the reasons VLC became popular. Is that because there was no marketing, you know, business around, was just like because we care,

it's going to be good engineering and we want people to be happy. That's what drove VLC for a long time.

What had the VLC really start on Mac was the MPEG-2 decoder for DVDs, which is a bit different from what made it popular on Windows.

Matt: I don't think you talked about why, how you got involved. Is it while you're in school? Is it afterwards? And then from there I'd love to hear your take on why you think VideoLAN is important in today's world.

JB: Sure.

Matt: Besides from being the most-used encoder.

JB: I went to this university called Ecole Centrale Paris, in 2003. I was a student there and I was a vice president and manager of the network, of the student network. Because that was managed by a non-profit that was only managed by students. So the whole campus network was done by students and not by the university at all.

Matt: It's incredible.

JB: It's incredible because, in fact, it's a private campus on a public school which never exists in Europe, at least not in France. But for historical reasons, everything on the campus was managed by students. Which is great because it helps you to learn so much more than listening to some old professor dude that, you know, is just going to take give you theory.

In real life, we care about projects and doing projects. I was a student there and I started to be the vice-president and the manager of the whole network. I think that in the end, we were only two to actually work seriously on the network.

The second one is Rémi Denis-Courmont, who's still a VLC developer. Probably he's the biggest VLC developer from the same year. When I was inside the university I was mostly doing networking and streaming and multicast and so on. I spent a lot of time also deploying 70 wi-fi access points on the campus. That was my second-year project. But I helped already on the VideoLAN project, mostly on the streaming part in 2004. But I was not coding, right?

Nick: Was that still working? Was the central goal, you said, was to take a single satellite feed, decode it once, and stream to everyone who wanted to watch TV on the campus? Was that still up and running?

JB: Yeah, yeah, yeah, exactly. That was till we had around 120 TV channels. And some of them were encrypted, so we had developed something that was just decrypting and removing the key, and re-streaming that. So, that's a VideoLAN project called libdvbcsa which is just removing CSA, or opening, or whatever you call that.

Matt: Liberating, I think is the word.

JB: Liberating. Okay, I need to use that word. I'm not breaking the DRM. I'm liberating the media. And that was still working and quite well, right? And also, we were doing some H.264 tests very early. I don't think it was 2004.

VideoLAN was one, the campus one was where a lot of testing was done by broadcasters from Europe and so on. Because there was an awesome network and good technology. I didn't really code on VLC at that time. I started mostly coding when I was doing an internship in 2005 and I was bored as hell because there was not enough to do during the internship.

I started helping, mostly because I didn't have really a programming background. In my university I did generic engineering. I didn't go that much to lessons also, so didn't do a lot of development. 2005, I started to help on the form. Helping people and giving a few patches, like normal. Then, in early 2006, I believe, was when I was in another internship that was also extremely boring. That's how I started to actually code a lot.

I had to work a lot on porting on Windows because no one maintained the Windows port. That's how I started coding. Because I had no idea how to code at that time.

Nick: Wow. So we wouldn't have the Windows port of VLC unless your internships had been boring?

JB: Well, you probably wouldn't have a VLC those days.

Nick: I'm glad you were so bad at picking internships.

JB: Then I moved to San Francisco for one year working for the French consulate. That was on Bush St. at that time. And also that was extremely boring, but I stayed one year, it was amazing. I went to meet so many startups and so many cool tech people. But when I came back, I said, "Okay, we need to do something about that."

During that year, in fact, I started recruiting new people. Because in 2007, early 2007, we are the only two left on the VLC project. So I spent a lot of time during 2007 and 2008 to get new students, to get new people, to ay, "Hey, you know, it's open to all. It's cool." And so on. Also help from people to actually get involved, because it was quite difficult at that time.

When I came back from the U.S. in the end of 2008, I created the first VideoLAN Dev Days meetings and I created the non-profit at that time. I spent all my time to get all the assets, IP, whatever, from the school and move them to the VideoLAN non-profit that was started in 2008. And I've been the president since.

Matt: That's incredible. Like, start getting involved?

JB: Well also, I spent a lot of time doing what no one wants to do: answering forms, answering e-mails, support, compiling for Windows, working on Windows XP and all of those horrible operating systems that no one wants to use.

Matt: So things that are only marginally more fun than the boring internships. That's kind of what I'm hearing here?

JB: Yeah, I spent a lot of time on VideoLAN doing the stuff that no one wants to do, right? And that's how you make yourself useable, useful. I now know how to code correctly in numerous languages.

But compared to many others, I'm just like, well, I know how to write good C and decent C++ and write some assembly. I can read it without a problem and a lot of scripting, but there are some people who are way better, right?

One of my big roles is to understand most of the technologies and be able to review patches. And also understanding the bigger picture and spending time so that those people who are better than me can actually be very efficient. That is one of the main goal I've set.

I think it kind of worked, right? VLC moved from 2003 where we're like a total of one million downloads, mostly on the Mac, to probably one billion in 2011, and now, I don't know, two, three?

Nick: No big deal.

JB: Well, it doesn't matter that the number of downloads for us. Because, first, download doesn't mean active user. It could be, like, an update. And also, a lot of people download outside of our websites. Because they go to Download.com or VLC.de or other website, right? And a lot of our users on Linux, right? And of course, they go through that distribution.

It doesn't really matter, the number of downloads.

It's really good for the press people who love to have big numbers. But in fact it's not what is important.

The importance is that are people happy and do we serve a good purpose? That is the most important part.

Nick: I'd love to hear, the story you gave us, you came back from San Francisco in 2008 and set up the non-profit and started to just, trying to build the community. That was almost 10 years ago now. So what have you been up to in the 10 years? Did you come back out to San Francisco and work again?

JB: No. When I was in San Francisco, I worked on a Franco-American startup which was doing peer-to-peer streaming video in 2008, that failed very badly. I remember meeting some guy, I think it was Justin from Justin.tv at that time, who said, "You know, bandwidth is not worth the complexity." So that failed.

Then from 2009 to 2012, I worked in a CTO of a small startup that was doing some great stuff, and it was kind of video Spotify, basically. The problem is that it takes a lot of time to actually decide what you're going to watch. It looked like a TV, but it was, like, recommended VOD, but we started that with video and with music videos. And you know, you played one and it moved to the next one and moved to the next one.

But if you showed that to my mom, she would say, "Well yeah, it's a normal TV but the only thing is that you can select or next." And that's a bit like what you see at the end of YouTube videos, right? Where they move to the next one.

Nick: Or things like Pluto.tv which are doing quite well now.

JB: Yes, but that was 2009 and a lot of the problem on that was, basically, working with content providers who are obtuse, that they don't want to move to anything. You know, "Yeah, but I don't have the rights, and blah blah blah."

The scaling and the timeframe for the development of those stuff is like 18 months, and so on. That's not a timescale that works for a startup. So, I think we died in 2012. At that time, I was also doing more and more VLC work and it was taking, I mean, I had two jobs, right?

In 2011 I went to the non-profit and said, "We need to do, like Mozilla, which has a non-profit and a corp, we need to do something like that." Well, most of the non-profit is French, so I was just like, "Ah. You know, what will happen in five years if the company dies and who will develop VLC?" And, "What, blah blah blah?" I have no clue. I can't tell you, right? I had no idea.

I would like to say, "Well, we've been doing great." I've no idea. Which was the honest answer. In 2011 we didn't do that, but the problem was, if we wanted to go to smart TVs, iPhone and so on, you can't just work on your laptop, right? Developing formal value is extremely painful and extremely costly. It's even worse now.

So in 2012, I've decided to do my own consulting company, separate from the non-profit. The idea would be to do the commercial arm of the non-profit, which was in end of 2012. And I've been working on this company and leading this company ever since.

Most of the development today for mobile is developed by the company, but a large part also on VLC. However, the owner of the copyright is still the non-profit, the editor is still the non-profit, but the rest is on the company. So, if the company dies, VLC doesn't, right?

Nick: Right. Got it.

JB: It shouldn't die, though.

Matt: Before we move on or beyond background, I'm very curious where the cone came from, in all of this.

JB: That's a good question, and usually I refuse to answer that one. Especially when it's recorded, but I answered to that on "Ask Me Anything" on Reddit last week or the week before. I won't get to all the details, because some of the details are too private. But it's really linked to today's non-profit that was managing the network in the Ecole Centrale Paris.

When I arrived in 2003, in Ecole Centrale Paris, the geek building, all the geeks were in the same building on the campus. They had a collection of cones that was insane. And when I say insane, like, on the floor for 24 people they had maybe 200 cones.

There was this absolute insane cone culture.

When I mean cone culture, it's like they were doing competitions. There was acrobatic cones where they were throwing themselves from the first floor on a big mattress with cones to do acrobatic jumping and all that. We're judging that.

There was, of course, a cone ball where they were playing kind of like, I guess a kind of baseball with a cone. It was also just, like, a competition of cone, every team. So they did lots of shots of cones around the campus, fighting. And there was this huge...

Nick: There's living that cone life.

JB: Worshiping in, of course a joking sense, not anything weird, but just like worshiping on the cone. And it was extremely funny.

That kept the question most, that's always what I said, what people ask me is, "How did it start?" Well, it started when they had this very fast, new network. And this very fast, new network got connected to 100 megabytes per second to the internet.

Nick: Wow.

JB: Sure. Like, yeah. We're talking 1999. And that starts to be quite fast, right? Of course, students did stuff. They did do downloading stuff, right? The people, basically, someone said, "You know, that's not okay." Especially since it went to the network from a French education network. So, that's absolutely not okay.

The university went to see the students, who say, "We know who he is. We'll manage that." But they didn't know what to do, right? Because they never forbid, actually, to do that. And also didn't want to give the name and so on.

They had this dinners outside of the campus every month in order to discuss about the problems of the non-profit outside of the campus. One of the times, they actually discussed about this problem. And so on their way back, they might have had a few beers, I guess, there was some constriction just next to it.

Some guy actually took a cone and went on the campus, and while discussing that, they starting screaming the name of the person with the cone. So the first thing, the first use of the cone, well actually to use that as a porte-voix. I don't know.

Matt: Megaphone?

JB: Ah, the megaphone, right? And when they went back to bed they kept the cone. Almost at the same time, and this is before my time so I'm not sure, but almost at the same time, one guy rewrote the X11 video output for Linux on VLC and he finished at four and a half. And there was a lot of fighting.

Not fighting, but competition in all the VLC developer days where people wanted to put features that the other didn't. And in order to just show that he actually finished the X11, because that was a big, before another guy did it, he actually checked-in the code and put a small cone because it was just after the megaphone event that was around.

He put this cone and said, "Well, I pushed it at five a.m., but I pushed it first and here is a cone." And so the next morning, people can actually go on from what he started but he was the one who started it and the cone's stayed for a bit and the cone stayed.

So those two events, and then it started the cone culture or cone worshiping, and that kept, the thing is,

it's completely insane and stupid to use a cone for video software. But it's so great. No one in their sane mind would do that. But that's so distinctive.

You do a web browser. Okay, I'm going to take earth and add some stuff. Yes, of course. So, I'm going to take video player on YouTube. Oh, I'm going to use a play button and maybe a square, you know, that would remind you to the TV.

One of the reasons VLC got popular is because a lot of people don't use VLC. They use a cone player.

That is one of the reasons why, even though we receive one e-mail per week, saying that, "Oh, you should change your logo." We will not change the logo. But in the beginning, it was just kind of a funny mistake and no one cared. It was, like, 20 people who are using the software at that time. It was not even open sourced. It's like a funny mistake, right?

But it is a bit like the Christmas holiday season hats that VLC has every last 10 days of the year. It was not meant to be live. But it was during the development of the of the CUTE interface in VLC. It was just like, we knew it was going to be very long to actually release, but for the developers to have fun, we added that. Well, Rafael added that.

But then, when we pushed it to projection, we kind of forgot to remove that thing that was changing the cone to a Santa Claus hat. That's how it started.

Then people starting insulting us for having this holiday season hat. People were so insulting that we said, well, you know, we are going to keep it.

Nick: That's so French, I love it.

JB: There is no way you insult us on something we do on our free time for no money. You are not our clients. We do something. We're happy if you're happy. But never insult us or attack us because of some silly mistake. So that's why also decided, well, you know, we keep it.

Matt: I love the icon and the logo, but that story was so much better than I expected it to be. "Cone culture" is a phrase I never expected to hear.

Nick: I love that you guys knew that there was one person on your network who was pissing everyone off. And some guy, at one a.m., was screaming his name through a cone.

JB: Well, I don't know how the US campuses are but that was, like, a very, very... It's a small campus and it's a very friendly environment and it's and engineering school. So that wasn't out of line, right?

Nick: Yeah.

Matt: Totally. I love it. I think it'd be awesome to talk a little bit about what the current world of VLC looks like. So, at FOMS you told us a little bit about VLC3. Do you mind giving the audience a little feedback there?

JB: We started being very serious for the Android port of VLC and the iOS one in 2012 and '13. The whole infrastructure of VLC wasn't actually perfect for that. Mostly because on mobile, you need hardware decoding, there is absolutely no discussion, right?

The code base from VLC on Android, on iOS and the one from the desktop was a bit different. That gave us those absolutely insane namings. Where the iOS version is at 2.7. The android version is at 1.5 and the WinRT version is at 0.5 and the desktop is 2.2. I mean, it makes no sense.

So a lot of work we spend on 3.0 where structure we have one version that we're going to release at the same time on all the platforms and have hardware decoding everywhere by default. Because now we're talking about UHD, right? And 4K and your laptop is using now 12 watts and you can't keep up, especially since we're starting 4K, 60fps, 13 megabytes per second.

That's not insane, but just in software your machine is almost dying. 3.0 has hardware decoding, 4K, HDR by default on all the platforms. HDR also depends on the platform because there is lots of discussion on that. We have everything related to 360 videos.

You can play your 360 videos and, of course, we also have third-order ambisonic, so, 3D audio. I'm not sure there's many players that actually do third order ambisonics and not first order. But the result is so much better. That's the first really important part, hardware decoding and 360.

The second important part is everything related to browsing your local network. VLC was always able to do UPnP browsing, but very badly. And now that your laptop is, like... And a lot of people have NAS. It makes sense to have a better improvement and support of that.

Now we can actually list files on Samba shares, DLNA, UPnP, NFS, for the five people who actually use that, SFTP, and so on. That was kind of like a large work we did, and finally, supporting a lot of new copy formats. Because what I try to get people to remember is that VLC plays everything and runs everywhere.

But the thing is, it's not always everything. There are a lot of weird cases that we didn't support, so we spend a lot of time, especially, two guys in the team has been, in the last two years, finding weird samples of the old QuickTime days, on the 1990s, finding some people who put some 608 inside MP4 in a different way than you should do.

Some non-defined MP4 boxes, and some people who put VP8 and VP9 inside OG because they can, or work on stuff like web VDT or TTML, and so on. The least of the number of formats that we support is huge. That's mostly for 3.0, then there is so much other stuff.

We can now store dialogs, we can store passwords for your Samba shares and so on. But it's encrypted in your hard drive and not just plain text and a lot of stuff around that. It's good that they always the first version that is the same everywhere. So I'm quite happy about that. But we need to accelerate the big releases, I think.

Nick: Is the feature set compatible across all, can I play a file from a Samba share on my Android?

JB: Yes. Well, you can also play an AVI and an SRT inside ZIP, inside a folder, inside a ZIP on your SMB file, and on your SMB share. And it's going to get open the ZIP, open the RAR, open the folders and get, and find there is a subtitle directly next to it and automatically detect it because it's the same name.

So, yeah. A lot of stuff like that would be closer to a lite media center but, like, we are very far from a lite media center but that make a bit more cool.

Nick: Makes sense.

Matt: Very cool.

JB: So, after that on probably VR. We have patches for the VR headsets. I have no idea exactly what's going to happen with all those VR headsets and if people will care, but we have everything ready for that. And probably start working on the UI and media library.

Nick: So, getting Oculus support into VLC, are we able to download VLC from the Oculus store and play a video?

JB: It actually works now. I got to try patches for that. The question is whether we would send that from the Oculus store or just, you run VLC, the normal VLC, and if you have an Oculus it automatically detects that. So not a new 250 megabyte download, you need the app that is extremely slow.

Just like, you know, you got VLC. Well, I do take the recent HND so... Oh, also on the 3.0 we added support for Chromecast, which was asked for so much time.

Nick: Right, yeah.

JB: When we did that, we didn't do it just for Chromecast. We did it in a way so that we can extend for the future, so we can get airplay, UPnP, renderers, HiCast or whatever the new format that will be.

Nick: Sounds like a little work.

JB: It is. But it's cool.

Matt: How many commits, would you say, were in this last release?

JB: Fifteen thousand for the core and maybe 4-5,000 for Android and maybe a bit less for iOS.

Matt: That's incredible. That's incredible amount of work.

JB: It is. It's a lot of work and it's also very rare to have an open-sourced software that is managed like that. Because now, almost all of the open-sourced software are developed by big companies. Basically, Google and Facebook are basically, paying people to do that. It's their day job, right? They just, in the end, open source it.

Doing the way the hippie, communist, whatever you call, old way where we don't have features we develop, it is quite rare. But also, most of the old way of doing open source and people who actually care about calling that "free software" have gone, right?

The reason why I started the company was because startups are cool and if you want to have new people investing time, you need to do a startup.

So I'm doing an open-sourced startup but because else you don't get people to actually develop on that because they could work for and get a quarter million to half a million per year at Facebook.

Matt: There's something else you mentioned at FOMS, which kind of blew my mind a little bit. But VLC.js. Can you tell us a little bit about it?

What's the phrase that Steve Robertson used? This is a common phrase but, "If it can be written in JavaScript it will be written in JavaScript." So VLC now in JavaScript?

JB: Well, the goal is not to have it in JavaScript. Let's be very clear on that. One of the question I always ask is, What is the future of VLC?

Everyone is going to watch movies, stream directly online, whether it's legal or illegal. And so what is your value?

Of course there is, there will always be value. But the value will be decreasing to the users. So there was always a discussion about how do we get VLC to run inside a web browser? Whether we develop some kind of Video DS clone or, when we discussed that Video DS didn't exist, but that was a long time ago. But stuff like that.

Well, we are not really the JavaScript guys. And in 2013, I guess, the Firefox OS people asked us whether we could compile VLC for Firefox OS. And say, "Okay, sure. Give me threads." And they say, "Well, no threads." And say, "Well, no VLC."

You cannot have a media player without threads, because, at least, you need an audio and video thread and an input thread, right? And the loop for the maximize. So, I said, "Well, no way." But then started last year a discussion about WebAssembly.

One of the goals of WebAssembly was to have pThreads and modules. Modules is exactly how VLC is, so that completely fits how we are, and threads say, "well, you know, we could do it." and there is a lot of value. When I was talking about that last year, people would just laugh and say, "Well, JB, you're insane."

I actually did a demo. Why is it important? Because today people will use HLSGS which is basically demuxing TS and HLS manifests in JavaScript in order to remux it in MP4 and then in order to basically feed that to the MSC which just reads fragmented MP4. Really? Is that a sane way of CPU cycles? I don't think so.

Let's be honest. I'm very very respectful to the people who are doing that because, like, wow, that's like an insane amount of work. Extremely difficult work. And after writing a TS demuxer, I know that writing a full TS demuxer is extremely difficult and also the web is getting a bit ridiculous.

Like, oh, the reason you've had, so let's add a web thing so we have web VR, and then we have web joysticks and at some point, the APIs are way too big, right? So, in my opinion, it should just like have low-level APIs, and then you write your own stuff above it.

Some people from the internet archives say, "Well, I want to play DVDs." And I'm just like, "Well, you know, VLC was done to play DVDs." And they say, "Well, what would it take to actually have DVD playback?" And I say, "Well, you know, we send script and pThread. It shouldn't be difficult to get that."

That's how we started just to do a proof of concept. Which actually worked. And that is the surprising part, is that I spent some time this summer with another student and actually, it kind of works.

The question is we have a proof of concept, right? The question is whether this will move to alpha, beta or actually something you can use.

Well, this mostly depends on the browser people. But imagine, in one year we got WebAssembly. We got threads in WebAssembly. We got assign.wasm and we managed to do something which cost compile, transpile yasm assembly file to assign.wasm. the Chrome people actually explored the API the way exported to PPRPI to do the hardware decoders and we have modules.

Then, well, downloading the core VLC just a few hundred k's of byte stream or whatever. And then depending on the format is going to download the right module for your right format. And if you have had the hardware decoder for your format then it's going to use the hardware. If you don't, it won't. And then you can manage to do stuff like that.

So, AV1 decoder, write it in C in assembly. But thinking it will compiled to web assembly and then assign to .wasm. As soon as you get hardware decoder then it will be offloaded but if not then you can still use all your machines and not say, "Well, I'm going to serve three time the content because I need, one, it needs to be in C for Mac OS and iOS and AV1 because YouTube does that for 4K.

"But for the rest, I need H.264 and, you know, why is using some HLS but the other one is using CMAF and the last one is using dash. Oh now, wait. It's not C, it's not dash. Three difference of dash because these version to understand this one and this one is this one." And I'm not even talking about DRMs, right?" Like CBC, CTR and so on.

As soon as you got modules, maybe you want to write your own DRM. Write it as a VLC module. It will work in VLC.js, it will work on VLC, on Android, and libVLCs. So you can use it and write your plugin for your custom format and your custom codec or your custom DRM. Because maybe you can do stuff that makes sense.

Maybe MP4 or MKV are not the best muxing formats, and we can do better with less overhead. Maybe you have a very specific use case or you're seeing that, well, HCVC is great but I can do something slightly different. Then you write your own. VLC would be a good framework there, where you actually plug your stuff.

That's also something people don't know, that VLC is not a player. It's actually a multi-media framework with around 500 modules that are loaded and are in time.

Most of the people just click and use it at libVLC, but a lot of professionals can have their own format. It's very possible that this actually works but it will mostly depend on whether the web browser people are going to do the right stuff.

Nick: I think that's a fascinating thing you've touched on there. Which is, in a world where we have WebAssembly downloading, essentially by code, we have all of these different APIs essentially replacing the OS APIs we knew and loved, with something that's kind of cross-platform. It really does such bring into focus, "Okay, well, why do we have things like MSE and not hardware decoders exposed?"

Because, when you're in this context, we already have people deploying out to production HLS.js. We're already deploying and running in the web browser full TS demuxing followed by full remuxing into fMP4.

Like, why? We already demux the TS, if that was an AAC decoder, if that was an H.264 decoder. If there were primitives available to us on these platforms, we could use them, right? We're good enough as developers to do that. Whereas, right now, our hands are being tied.

JB: Yeah. And you're tied by what happens if, in two years Chrome decides that they don't ship an H.264 decoder anymore for, I don't know, patent reasons or because they want to push their own format, what do you do? You're just, you spend so much time on that.

Or why they decide that, because of how they're doing CMAF. they're not going to support these fMP4 in the same way as they used to. Then you need to adapt. And also the schedule of releases of Chrome and Firefox is very difficult to follow.

Sure, it's okay for a startup. It's okay for Justin.tv and Twitch TV, and all the people like that. It is not okay for broadcasting people or people who have a very slower ecosystem. And there are people who have slow ecosystem that do make sense.

You can't just, well, everything is going to be replaced by startups because, you know? There are cases where you don't care and you need some strong innovation. And the problem is that now you have too much power, in my opinion, on a few actors.

So far, they're great. Chrome is doing an amazing job and Firefox is too. But what happens in two years? What happens in five? You can't play your files any more? What about 10? Right? Now we're talking 10 years.

Also, today we have DVDs.

On the internet I have hundreds of thousands of DVDs. They can't play them because there is no way.

And the only solution we have now is, oh, a demo of a MPEG-2 video decoder in JavaScript, which is great. But the difficulty of the player is not just the decoder, all right?

It's having good audio/video synchronization, understanding bit-rates, having a lot of features, detecting other files and so on. And so I think that's one of the main points for interoperability and also, not only in space and number of formats, but also in the future and in the past.

Nick: Sounds like a very interesting project.

JB: Well, I'm not sure it's the most interesting project I have. We have a project of running a kind of Chrome, sandbox-like inside VLC and I think that, in the research, is more interesting than the VLC.js.

Nick: Okay, well actually I think up next on our schedule, Matt was talking about some of the future and what are the hard problems that you are working on now?

JB: Well, the hard problems are, there are the short-terms ones and the longer term one. On short-term, now that the playback engineer is way better and the same everywhere, we need a better UI and a very Lite media library so you can, because a lot of audio software went away, iTunes is pure crap.

You have a lot of files, and how do you manage that? A lot of people are using VLC for that and that pains me. Well, maybe we should do something for people to play audio.

But that's the short-term, right? We have also the media libraries that are already done for Android and iOS. It's in C++ we're going to use that on desktop. That's for VLC4. VLC4 is also polyVR and some UI related to the media library.

But that's not really difficult, right? It's already planned. We're also already working on that. These will happen. The more important stuff I'm working on are VLC.js and the sandbox stuff. I have no idea whether that's going to land in six months. Probably not. One year, two years, three years. I don't know. But we'll find out.

Matt: So, what is the sandbox stuff? Would you mind giving a little bit more information there?

JB: Yeah. So, the current approach of sandboxing is very bad. Well, it's great but it doesn't fit the VLC module. What happens for VLC is that, you're going to put VLC in the sandbox and you're to give it all the rights.

Well, I need to have access to your file and without a file picker, because I'm going to open playlists or MKVs that are linked or MXF that are linked, or MP4 that are linked, I need to open the subtitle that is the same name next to the file, etc., etc. You're going to give me access to all your files on Windows on your documents.

Already, VLC has access to way too much. On Linux, we had the issue where some security researcher sent us an exploit and the exploit was actually not existing in the calculator like they do. But actually reading .ssh and .gpg keys, private keys, and send that to a server, right? And that's where we realized that, holy shit, VLC has access to stuff it shouldn't have.

Well, we need hardware decoding and video display so we are on Windows and DWM mode which is half the kernel. On Linux, X11 and DRM so you're root, so you can do anything on the machine. I need to have access to your audio output which is, I got access to your /dev/dsp and I can do so much with that.

I need access to networking to play files. To play networks and streams, I need to open also sockets so you can control VLC. But I also need access to your /dev read-only on Linux to get V4L, DVD, Blu-ray, ASI, and so on. In the end, we realized that VLC has a good use case like the browsers for all the features that a sandbox could have. And that's bad.

So what I want is not to do that, but to do exactly what Chrome is doing. Which is, basically, you would split VLC into multiple processes. Basically, the demuxer would have no rights because it's just data in, data out. The video decoder and video output would just get access to the graphic card which has access to no files system, no network, no audio. The audio has audio, but no...

The input is just depending if it's a network input or file input, then it just just has access to a network or file, but only your home and not the whole system files, etc. And that's very difficult because we need to be able to pass huge buffers between processes in a cross-platform way, and that for every input we recycle this by all this process and we do that.

The closest to that is a Chrome sandbox, but in performance, it's not yet there. They have different cases and in port that's not exactly what we need. And also, the code base for the sandbox is too big for us to maintain. So I want to do that.

We have a proof of concept based on thread, not processes, that actually can offer on Linux. But now we need to move that to multiple-process VLC and also, it's really important, the OS is applying the rules.

That's one of the points where VLC is very good, is that if you would do that with your streamer, every module would actually say, "I need this right."

You do a raw module which needs all the rights and boom. You do that in VLC because it's a bit more static framework. We launched a thread and this thread is a video decoding thread and this one will load modules.

It's easier to restrict the thread than having the modules, which meant that the co-VLC can apply different security modules on the different threads' slash processes and not the module themselves. It's way more secure. Also, I can ask the kernel to actually apply the security and force it.

I think it's a very big project. But it's quite interesting. I don't think anyone has done that correctly for now, so we'll see.

Matt: It sounds like a huge undertaking. I mean, how much of that is re-writing the world and how much of that is just kind of breaking things, you know what I mean?

JB: Yeah. It's not that much. Because when you do a VLC module, it's usually 1,000 lines of code. You basically have an API that is basically, like for demux, read data and output data. Output that high can get a stream.

Whether this function is calling you on a pointer that is just your normal pointer or it's your memory, isn't a big deal. What's important is the sandbox will not change a lot on the modules. VLC code is 100 thousand lines of code, right? The whole VLC module is around one million, and all the libraries, maybe, 50 million. But the call is quite small. And most of the strangers will only be downloading the call.

It's a lot of work, mostly because we need to find a good way to do IPC in a cross-platform way, but it's not insane. We're talking all about a lot, probably four people full-time for two years, let's be clear on that. We're talking about one, one and a half million euros. But I think interesting enough that we should think about that.

Also, if it'll give us the VLC multi-process, which is most interesting for people who care about streaming. Because, basically, your VLC could play whatever it plays and then pass that to the stream output which would be actually in the different process or maybe a different docker and you would have a duplicate that, duplicate that.

Then, in a different docker, you have this stuff that is taking the data and re-stream that to RTMP and other one to RTSP and the other one to HLS and the other one to Dash, because VLC can do all those stuff. And then, if one crashes, well, it doesn't crash the rest because those are different processes. If we manage to move libVLC framework to multi-process they'd give us other ideas that can be quite cool.

Matt: That sounds amazing.

JB: Yeah.

Matt: What's the biggest problem you guys can solve in the next five years? I guess let's take that a little bit more meta, not just technically in the code base but at a higher level, like in the video industry or consumer-video world.

Nick: Who's going to buy your startup?

Matt: Those can be two different questions.

Nick: Okay, fair enough.

JB: So, I'm going to answer in different way but I'm going to get to the point. I'm going to answer I think two or three ways. First thing is that, on a very close time I think we're going to invest quite a bit of time on AV1 and AOM.

VideoLAN, in my opinion, one of the good entities to actually have decoders, new encoders, new tools around that. Because VideoLAN can bridge between Facebook, Google, Microsoft and so on, and the open-source community.

We need some good open-source tools if you want to get AV1 good. Not like the VP9 mistake.

The second stuff is that, where do we go for VLC? Well, you mostly know, right? And I'm going to spend a bit more time on other stuff where related to VLC. But the VLC needs to go in the JavaScript direction the same time the sandboxing. The more faster and better filters and so on, and also improve the libVLC framework which is now, since VLC is without zero, actually useable and actually use in lots of applications.

But the biggest question is not that one. The biggest question is, does a project make any sense? Maybe yes for people who are in the video industry. But outside, why does my mom care about VLC? Should she?

I think that we are the problem that the question is not how do I place something because, basically, this problem is solved. Everything, from the open-source technologies, from FFMPEG-2, H.264, and VLC got taken, re-shared and so on, finishes inside the web browsers and everyone watches stuff online.

I think that the technical problem, how to play it, is fixed. I think that we played a large part on that part, but I think it's fixed. The biggest question today, when you talk to people, is not that one.

The problem is, what do I watch? And how? Because people are cutting cable, of course.

But everywhere they're just watching a lot of TV shows and movies and when you realize that the average Americans watching five hours of TV per day and the range around European is three and a half or something. If you convert that to just TV shows, it means that you watch a whole season in two days, well, two nights. So, there is a lot of question.

Matt: I've never done that. No, no.

Nick: I doubt it.

JB: I mean, how do people get that, right? So you got stuff from Netflix which is great. But in the end, Netflix's catalog is shrinking. And it's not that big. And so what happens is that you're going to ask your friend, "Hey, I finished Westworld. You should see it's so great." And then people are going to say, "Oh, yeah, Westworld." They're going to go, discuss on Facebook, IMDB reviews and so on.

Then what happens next? "Oh well, where should I watch it?" So they go, they start watching it. It's on five different platforms. The one is on HBO. The other one is on Netflix. And you don't care. You could even pay. It's just like, "Holy shit. I need to manage all that, and I don't know where to find it."

Then after you finish, you're just like, "Ah, what episode I was...? I don't remember. Oh, yeah." But you know, this is Thanksgiving. Outside of the US no one has any idea that during Thanksgiving there is no TV shows. Why do they not write during Thanksgiving? I mean they wrote it, like, six months before. And stuff like that.

It's a huge pain to know what to watch. Even when you know because you've discussed with your friend in real life, on social networks and so on, then you don't know "where."

Sometimes you watch so much stuff you don't remember if you've seen it.

People who say, "Well, it never happened. I start watching a movie and then realize that I've already seen it." Everyone has that, oh, you start an episode then you realize that, "Shit, I already saw that one but I was like half sleeping." So, there is a big pain point here.

I'm working for a project around that which would be, kind of, Last.fm for video/TrackT/IMDB/kind of a social network and aggregator around videos. Everything you watch, whether it's legal or illegal gets added to your profile, profile being public or private, I don't care.

You can follow your friends. See the timeline what they're watching, add that to your watch list, and then what you'll watch, you click on it and it gives you the link to the right platform. Whether it's legal or illegal. Whether your input is legal or illegal. I don't care. What's important is that people know what to watch and know where they are in the whole, all the videos that they've been watching.

In my opinion, one of the biggest issues for piracy is not people who want free. It's just convenience.

Try to watch, Game of Thrones is out. Try to see it in France in the correct version with French subtitles. It's impossible. It's so much pain.

I think that the problem for piracy is a question of demand. I don't know anyone left now who's pirating music. Or even video games. Steam and Spotify solved that. I hoped that Netflix would solve that and it's not going to solve that. They're focusing on amazing shows I love. If some people on Netflix, I love your shows.

Nick: Second.

JB: That's one of the projects I'm working on and that could be something that could get bought at some point. The second project that is related to that is, how do you do that when you are on your Apple TV and you have no plugins? It must be seamless.

It's like you install a small plug-in on your VLC, on your Kodi, on your web browser and everything you watch is sent. Legal, illegal, don't care, so that I know what I've seen and my friends see what I've been watching. How do you do that when you're in a closed platform?

Then you need to recognize what's on your screen.

We developed a video Shazam where you basically take your smartphone, more or less, shoot your TVs and it tells you what you're watching.

It's like video fingerprinting, but not in the usual way because most of the usual fingerprinting is done when you have access to the full screen, which is easier. We have that but also we have developed a part where you actually jump your whole frame and, basically, it finds a way even while not having the full correct...

Matt: That's so cool.

Nick: That's really cool.

Matt: Is that open-source?

JB: No, that's not open-source.

Nick: Smart.

JB: Well, the two last projects are not open-source because of what you said. Maybe some people would acquire that, however, if someone give me enough money, and I'm not speaking of insane amounts of money, but just enough, then yes, I could open-source that. If you've got 10 million, send that.

Why not, right? I don't care. I just need a way to keep putting good developers on important subjects around the open-source community and around VideoLAN. And if this is what it needs, then why not?

Matt: I really love your philosophy around open source. It's pretty awesome.

JB: Those are basically the projects I am personally working on. Mostly, with a startup. If any of those projects will actually fly, what will happen is that I'll split the company and split the people.

Nick: And spin them out into their own entities.

JB: Yes, because, for example, VCs would hate what I'm doing, right?

Nick: No doubt.

JB: "JB, it's insane, right?" You're working, you're supervising, in fact, five to seven projects. One is VLC underway in Java, one is in Go, one is in, basically, JavaScript. The two others are in C. Some are writing assembly codes for decoders and some people, we are deploying consulting for streamers. People deploying fMPEG and so on.

I'm also working on stuff will be recommendation engine and video shadowing. And there's just like, you know, you need to focus on one thing. I fully agree with them.

The thing is, all these go inside the same direction and that is the big picture that you usually don't see.

I understand that because it's more complex, then there is a higher risk, so the people are less interested in investing.

The recommendation part is basically a service VLC right? I could play anything, now, what should I play? And it's going to be integrated in VLC which is your own VLC account and the Shazam of videos because I need to be able to say, "What I've been watching, what do you think?" Etc., etc.

The stuff I'm doing on the VLC GS is basically, I need to get VLC to actually go on and that will go through the web whether we like it or not. And send for the sandboxing, right? There is probably 300 million people using regularly, VLC on the desktop, and their security is shit because we are just like a bunch, a small team and also because we are already and we're going to be more and more security target.

Because people now install on the machine, an office suite, a web browser, an image viewer, and a video player, right? And the first two are very secure. The last ones are not. VLC with all the libraries we demux and decode and pass so many formats, there are hundreds of thousands of security issues.

All that goes into the same direction. Most of it is going to be open-sourced. Some of it might not because, well, you know, if that's the way to get money and get good engineers to work on the right and fun project and not Uber for tombstones or wheat delivery or like, you know, I mean great, but like,

I want to do actual technical innovation and not business model innovation.

Nick: One of those sounded dumb.

JB: You know, Uber for tombstone would actually work.

Nick: Might might work a little better. That sounds like a big vision. I like it. Yeah, it sounds like you've got your work cut out for you.

Matt: And actually on that note, you've talked about a ton of, as you said, really cool things to work on, get your hands dirty. So, how do people get involved?

Nick: Yeah, what if I've got a spare afternoon one Sunday?

JB: Well, the first thing is, actually, report bugs.

This is the first thing I ask for, people to actually report bugs

because that's just like, "Oh, you know VLC doesn't play this file for six months." I'm just like, "Did you think about reporting this bug?" And they're just like, "No but I thought you knew." You know, I support millions of different types of files. If you have an issue, report to us.

The second thing is, actually, compile VLC and work on the small features that you actually care about. I did the Reddit Ask Me Anything last, like, 15, 20 weeks ago. And people who were there, "Oh, I need that." Those are like tiny features, and they're just like, "Oh, that would solve my life."

I'm just like, you know, coding that would take half a day. Then, if you care more then you can start actually working on VLC, on the security part, the JavaScript part or actually go down to codecs, encoders. We are going to have a lot of need for a neat, correct open-source AV1 encoder in the next year. How do we get to that, right?

There are also projects around VideoLAN because, in my opinion, VideoLAN is just the larger open-source multimedia community.

And VideoLAN today is not only about VLC. We are merging the Mac OS profit into the VideoLAN profit and people ask me, "Well, would you host MPlayer?" Yeah, sure.

My competition is not MPlayer. My competition is RealPlayer or BS.Player or all those people who still... It's fine, it's okay to run non-open-source codec packs on their machines. There are also other projects in VLC and stuff that are very interesting on the lower level.

There is lots to do on HDR. Today, HDR is very bad. You have no clue about quality. Quality of video encoding is a very big topic until last year. We are always the same PSNR and SSIM until Netflix came with vmaf which has great points but also weak points.

Why do we care about video quality only now, right? So, there are many projects around video that should be done as open-source projects and VideoLAN is probably one of the right places to actually join and do other projects around.

Matt: This was an amazing conversation.

JB: Thank you, thank you so much.

Nick: Yeah, thanks for coming in, JB.

Matt: Thanks everyone. See you next time.