Ep. #18, An Introduction to Service Workers

Brian Douglas: Welcome to another installment of JAMstack Radio, on the line we've got Eli Williamson once again.

Eli Williamson: Hey guys, it's good to be back!

Brian: Cool, and then in the studio, we've got Zack Argyle. Interesting last name.

Zack Argyle: That's the only reason my wife married me, she liked the last name.

Brian: Unfortunately my last name's a first name. Anyway, so Zack, why you here?

Zack: So I'm here to talk about service workers. I work at Pinterest, and I've been there for about 2 1/2 years and worked at PlayStation before that. And I am super excited about service workers and Progressive Web Apps in general. And it's kind of the future of the mobile web. I am a firm believer in it as a platform, and we need to dedicate resources there, so we've got to talk about it.

Brian: Cool. So, service workers, what are they?

Zack: What are service workers? So that's a great question.

Service workers are essentially a background process that runs in the browser that corresponds to your website that can be running when the user is not on your website.

This allows for some kind of unique opportunities for things like desktop notifications, and also, for things like intercepting the request when people want to go to your site, you can actually respond from the service worker, instead of actually going all the way to your server. So that's how we can enable things like an offline experience, where you don't even have to go to your server, and we can respond with HTML immediately.

Brian: Okay, cool, and then before we get further into service workers, I made the same problem when one of the Airbnb guys were here. Can you explain what Pinterest is real quick? Because I'm sure there's a lot of people who are really Pin-interested in knowing. You see what I did there?

Zack: Yeah. That's actually funny, because at Pinterest, we've got like a million different puns that are just everywhere. I mean, if you look at like our opensource projects, 1/2 of them have 'pin' in there somewhere. So Pinterest is essentially a visual bookmarking tool. We call ourselves the catalog of ideas. So people can come to the site, and it's 1/2 search engine, 1/2 browsing, it's a whole ton of images, each of the images has a website corresponding to it as well as some rich information. Things like recipes, or movie information, or information about locations.

Brian: And like salads and stuff?

Zack: Salads, yeah, lots of food. Most of my dinners these days come from Pinterest.

Brian: Oh, yeah? Cool, awesome. I joined Pinterest back when I was getting married years ago, when Pinterest was just up and running. And I used it for my wedding, and adding stuff, and all my Pinterests wer recipes from around that time, or wedding ideas.

Zack: Yeah, absolutely, that's one of our bread and butters, I think.

Eli: I think I started out on the food train as well with that, nowadays, I use it a lot more for UI inspiration, there's a lot of great designers on there. It's really coming up and killing it.

Zack: Yeah, I think people are surprised with the applications of it, because in general, people, at least in the US, still see it as like a female-centric site. But externally, it's about 50/50 male-female, internationally. Inside the US, it's probably like 70/30ish. But yeah, there's so many cool applications for it, so for example, the show Stranger Things, they used all kinds of private boards on Pinterest to plan inspiration for creating Stranger Things.

Brian: Interesting.

Zack: And so like, yeah, there's all these really cool, that's why we call it the catalog of ideas, it's this place where you can come and just kind of discover interesting things.

Brian: Cool! Obviously, if you guys are global 50/50, there's a lot of users, so what are some things that you guys implemented internally with service workers?

Zack: So, I guess in general,

Service workers are great internationally, where internet is not as good, and the idea behind that is that service workers allow you to cache request information permanently.

So I can leave your website and come back, and that cache is still filled with information, so I can do things like create an app shell for my website, and app shell being some base set of HTML that I can use to be the shell for my site, and then load from there. So I can return that app shell from the service worker cache, and people don't even need to go to the server to get any kind of HTML, so that means it's absolutely an instant page load, which is really cool.

Brian: So when you say instant page load, is that initial? Like, is there any performance hit by caching initially when you visit the site? So for example, if you're going to be caching whatever you see, like your pin board, is that going to take a hit on performance while you're downloading a couple gigabytes, or maybe a megabyte of data?

Zack: It can, and yeah, that's actually something that we do.

Brian: So currently, what does Pinterest use it for?

Zack: We use service workers for two things in particular right now, the first is for notifications and browser, and then also, for caching of our JavaScript files.

Brian: Okay.

Zack: And there's only small benefits to caching your JavaScript files, because we've already got them on a CDN as well. But it actually parses a little bit faster when it's cached inside a service worker, but the benefit being that

In the future, we can enable this entire offline experience where you don't need to come to our server or even be online. We can give you all the JavaScript, and then once we have an app shell, we can just return that to you, and all the JavaScript. And you can run the entire site without even being online.

Brian: Cool, that sounds ideal, especially if you need some pin action on the airplane.

Zack: Yeah, right?

Eli: Does that work for offline content? So let's say I have a board for a project I'm working on, and I want to go check it out, for example, on the plane. Can I go see that board that I had set up when I had internet, see all that content? Or does it just cache the JavaScript?

Zack: If you had been there before, yeah, the browser will cache all the images automatically for you, and we will have cached an app shell. So we're not quite doing this yet, it'll be in the next couple months that the app shell will be implemented, and merged in. But the idea being that the browser will have the images cached, we will have, in the service worker, the app shell and JavaScript and CSS, so that when you go to the site on your airplane, you automatically get the HTML, CSS, and JavaScript, and the images are cached in the browser. So you will be able to see everything, but you just won't be able to make any actions.

Brian: So when you do, like, a service worker, I was actually checking out the Pinterest site before we actually started this podcast. Actually, before I even got to the studio, and I was like, "oh, I probably need to check out the site to kind of prep myself to see what Pinterest was doing with service workers." And I know Pinterest's been doing this for a while. When you see Pinterest on the mobile browser, it says, "Our experience is better in a mobile app." Is that a service worker?

Zack: No. That's sad, that's what that is.

Brian: Yeah, so I guess it's just like some sort of mobile view, and they just made sure you put that on top?

Zack: Yeah, the idea behind that was, it was probably two years ago or so, and this was when noone cared about our mobile web. So they didn't invest any energy in it, or any resources. So the iOS and Android apps were just much better, right? And so the idea was, "if we just kick people out to the app, then they're going to have a better experience." And it turned out that it actually was good for them, people were better users in general, if they use the apps and we just force people out. And so we're actually experimenting with not doing that anymore, now that our mobile web is getting better.

Brian: So at Netlify, we have the opposite experience, where our web experience on the computer is actually much better. And we actually have data that shows all our users are coming from computers, desktops, and laptops. So I think we came to the agreement that we don't need a mobile app, but then, I personally think that we probably need a mobile app, and the reason why we have 99.9% of our users on the desktop is because we don't have a mobile app.

So anyway, long story short to say I'm actually working on a mobile app that hopefully will be done in a weekend, because I'm so close to it being done. It's in React Native. So that way I can see different stats, to actually see if some people actually care about using the Netlify.

Zack: Yeah, hard to compare when there's nothing to compare it.

Brian: Yeah, that's the whole thing, too! So going back to your Pinterest thing. iI you guys had a bad experience, I guess that was really smart on your part, at least you're not losing users because you didn't have time to make sure everything was responsive, and actually was working.

Zack: Yeah, part of the problem's our iOS app is so good.

Brian: Yeah. The iPad app is amazing!

Zack: It's, in my non-biased opinion, one of the best-architected apps in the store. It's silky smooth everywhere, animations are amazing. We're probably not going to be able to beat that in mobile Safari. And also, mobile Safari doesn't even support service workers.

Brian: Yeah, that actually makes a lot of sense, because Safari's kind of...

Zack: The dragger.

Brian: Yeah, it is the dragger.

It's painful to have actual feature flags in your code, in your JavaScript that says, "If this is Safari, don't even bother."

Zack: "Bail, bail, bail!"

Brian: Which, I've had to do a couple times, on certain things in our application, because it's just like, "wow, they just don't care, I guess."

Zack: It's one of those things where they make all of their money from the app store, so if they make mobile web better, then they lose out on money. So I get it, but at the same time, I'm like, "Ugh! All those people, internationally, that don't even download apps!" You're hosing 'em all!

Brian: Conspiracy theories.

Eli: It does bring up a good point too about service workers, in terms of supportability. Besides Safari, what support does it have across different browsers right now?

Zack:

Firefox, Chrome, and Chromium browsers all support service workers.

So it's pretty much, at this point, just Safari that doesn't from the main browsers. Edge, I believe, either does or is pushing it out soon. I don't remember which one it is.

Brian: Yeah, hopefully by the time this podcast comes out.

Eli: Yeah. So, I don't know if you know this, but I'm the creative director at Netlify, and have been marking on the marking site quite a bit, and we've actually been playing around with service workers on our site. We're still exploring them quite a bit, but one of the things, I think I've run into repeatedly, in my experience, is trying to debug them. They're a bit difficult, I'm wondering if you have any tips for that.

Zack: Yeah, so I created some tools that I think make it a little bit easier, and they're opensourced under Pinterest's GitHub account, it's GitHub.com/Pinterest/service-workers, and there's three things in there. One for generating service worker files from a config, a second one is generating a bunch of them via webpack, so that way you can have access to the hashing and stuff for your JavaScript files.

And the third one is for testing service workers. The testing service workers one is the one I was kind of most excited about. Because there are already things that generate service workers, but the testing one, I would highly recommend. You can use it with whatever your service worker files are, and it makes it really, really clear and easy to write tests against the service workers that you've generated. So it's not really perfect for debugging, per se.

Brian: Or preventing bugs.

Zack: Preventing bugs in the future, yeah.

Brian: So you just write like in, like, your Mocha, or whatever, JustCode, you just write, I'm looking at it right now at service worker mock, you just use that mock, mock that out, and it's confirmed that service workers are working how you would expect.

Zack: Yeah, it essentially mocks the service worker environment within Node, so then you can just start making assertions against what is happening in the environment.

Brian: Okay, cool! So then if I wanted to make sure, when the page loads, and I'm sitting in the notification, I could confirm my notification was sent as long as the service worker exist.

Zack: Totally.

Brian: Cool! So what's next with Pinterest and service workers? You guys are doing notifications? Actually, can you explain what sort of notifications are you sending in your service worker?

Zack: Yeah, so it's the same notifications that we have that we're using for our iOS and Android. We allow people, if they want to, opt in to get notifications on desktop as well.

Brian: Yeah, I've seen that a lot lately, I wonder if everybody's getting on that train. If you go to news sites, it's probably not a service worker, but you go to news site that are like, "Hey, you're in Buzzfeed, you should probably want "notifications locally." Is that sorta how it works.

Zack: Yeah, it's all service workers.

Brian: Okay, interesting!

Zack: Yeah, it's weird, because it doesn't feel like the same service worker's doing the same thing, right? Because, when I think service workers, I think caching, but then it's also doing this notification thing, and then it can also do things that allow you to add your site to the home screen.

Brian: Yeah, that's another interesting one too as well. Obviously, since you guys like your iOS experience. Would that be something that Pinterest be interested in doing down the road? Creating a different experience for people who just don't want to go to the app store, or have like a bandwidth cap? I know with a lot of apps, if it's over a certain amount, you can't actually download it unless you're on wifi, so that'd be interesting.

Zack: Yeah, we're definitely growing a lot internationally, and talking about mobile web is happening a lot over the last couple months. Yeah, I'm actually like a dedicated resource on mobile web right now. So it's kind of up to me to decide what's the right path to take for this? And I think that creating some kind of experience that can be like a native app, but it's in mobile web, so people don't have to download an app, internationally, that's exactly what people want, so.

Brian: Yeah, which makes sense, so I don't know if you talk to anybody at the Twitter team, but they recently made their entire Twitter mobile experience a Progressive Web App, which is a term we haven't even mentioned yet. But they have that whole experience, and it was an interesting experience, they have a blog post about it as well.

Zack: I talked to them a couple months ago about it actually, so I was kind of excited to see that they finally launched it officially. I use it exclusively on my Pixel.

Brian: Oh really?

Zack: And it's fantastic!

Brian: Okay. Get one of those.

Zack: Yeah, because we're growing internationally at Pinterest, we're pushing our employees to use Android more, to test out the Android app, so anybody who wanted to, they would buy them a Pixel.

Brian: Oh nice, I just was going to ask if they subsidize you. Okay, very cool.

Zack: Yeah, so I had been iOS before that, and it's been mostly seamless, but there was some hiccups switching from iOS to Android.

Brian: Well yeah, I kind of want to see how far our company will give us Pixels too as well, so we can be Android people.

Eli: I am curious about that transition, because my workflow is so integrated with iOS, I'm curious to hear how Pixel, and how switching to the Google ecosystem has worked out.

Zack: Yeah, so I guess in general, I'm not like a super hardcore phone user, so, what I used it for the most of is like taking pictures of my kid. Or taking pictures in general, and the Pixel's camera is absolutely phenomenal. I could not give it better reviews, I absolutely love it. So, that's 90% of my use, and then texting. So the funny thing is, texting was probably, by far the biggest hurdle that I had in the switching from iOS to Android, because of iMessage.

So any conversation I was in with like a group or something, where it was considered an iMessage chat, because everyone was in iMessage, I just stopped getting texts for the group. So I had to call everybody and be like, "hey, can we start a new chat? Sorry guys, can you delete that chat, start a new one for me? I'm not getting anything."

Brian: Yeah, I'm pretty sure you lost a lot friends doing that as well. "God, this guy."

Zack: No one likes me anymore, because I use Android.

Brian: "Pixel Zack is back."

Zack: The funny thing is, my wife always wants me to take the pictures now. She claims she likes iOS better, but she always wants me taking the pictures, so who knows?

Eli: That could be a pro or a con, I think. If you're the one that's like, " oh, here, hold the camera, take the photo."

Zack: That's true.

Eli: But I am actually quite intrigued. I'm a lot like you, I use my phone primarily for taking photos, and kind of just web apps, so, the messaging hurdle might be a tricky one, but I think it might merit a transition. I appreciate you shedding some light on that.

Zack: Yeah, no problem. Totally related to everything else.

Brian: Yeah, and if anybody wants to become Zack's friend and you have a Pixel.

Zack: Hit me up!

Brian: Yeah, definitely hit him. So you mentioned that you are the resource, as far as mobile web. So, I know you mentioned before, you have a team of engineers in the platform team. So how big's your team?

Zack: There's about 15 people on it.

Brian: About 15? Yeah, so, and then only the Pixel Zack is focused on mobile web?

Zack: Yeah, yeah, so we don't have a separate app for mobile web like Twitter does. Twitter has a separate team working on Twitter Lite, which is really, really cool, and I'm incredibly jealous of that. So ours is a responsive website, right? And so most of the team is building out things for, one, our core UI library, it's like our design language that we use to build everything. The second is all the build tools, we manage those, and then third is all the Node processes, we manage, basically, JavaScript on the front and back, our team.

Brian: Okay, cool, and I know previously, you guys were using Denzel?

Zack: Yes, you remember the name, the code name!

Brian: The code name, well, I'm actually seeing two talks about it. And I think you had mentioned to me, about a year ago, about it, and I was actually really intrigued, you guys actually had your own internal framework built on Python, was it?

Zack: Yeah, it was Django, Ginger, Backbone, and then on the client side, it was Nunchucks, which is like a JavaScript version of Python's Ginger templating language.

Brian: Yeah, and it seems to be a trend that teams in other companies that are making a transition to either the React or the Vue, are moving away from their own homegrown solutions. So as part of this whole service worker effort, and I assume you guys are Progressive Web App-ready. Is this whole transition from Denzel to React because you guys saw another opportunity to then get the good lighthouse core?

Zack: Yeah, so we were actually, we've been using desktop notifications for about a year, so this was kind of prior to our transition to React. And we've just now that service workers are starting to be a little bit more mature, we're experimenting with making them a little bit more integrated into kind of the daily life on Pinterest.

Brian: Cool, so have you guys been following service workers, like the whole ecosystem? Because I know it's been around for a while, and I know at some point, people started talking about it more.

Zack: Yeah, we've talked a lot with the people at Google, in particular. I've chatted with Owen, who's kind of the PM over most Progressive Web App stuff. Alex, who's kind of the Twitter loud voice about all things mobile web and performance for service workers and that kind of stuff. And they've helped us out with things, and we've kind of made requests to them. Same thing with Firefox, we met with the Firefox group and talked with them about service workers, and similar requests.

One of the things that came out of the conversation with Firefox was a talk about caching a bytecode for the JavaScript that we cached in service workers. Instead of just caching the file itself, we cache the bytecode that has already been parsed, so that it's even faster, when you come from a service worker.

I mean, it's probably only going to be like maybe 15 milliseconds, but still kind of cool.

Brian: Nice, so how do you guys get all these meetings? Know people that know people?

Zack: Pretty much, yeah. One of my coworkers, Kevin, who incidentally helped co-found the Brave browser, he's on my team, and he worked at Firefox for a long time, with Brendan Eich, which is how he kind of did the Brave thing. So he knew people at Firefox. Google just really likes Pinterest, and we like Google, so we've got great conversations with them about all the kinds of things, so.

Brian: And you guys use Pixels, too.

Zack: Yeah, right, so they like us even more now.

Brian: Scratch our back, you scratch ours.

Zack: Exactly.

Brian: And so Progressive Web Apps, we didn't really touch on it, what's your stance on Progressive Web Apps, its popularity, where it's going in the next six months to a year? I know it's a trend of people moving towards it, but you think it's worth it, now that you've put all this work into your service workers?

Zack: Yeah, so

In general, my thought is, it's bad to bet against the web.

Brian: Okay.

Zack: As a platform, because it's so accessible, it's going to be really hard for anything to ever replace it, and so I think things like Progressive Web Apps that are making the web even more powerful, and closer to native in general, are fantastic. So I'm like, all in on it.

Brian: Always bet on web.

Zack: Always bet on web, yeah.

Brian: So, don't add screens here, your app, that says, "Go use our iOS app."

Zack: Yes! I know. I was unhappy about when we did it two years ago, but, it was the right call at the time.

Brian: Cool, yeah, I appreciate you coming in and shedding some light on Pinterest a bit, and then also service workers. Eli, do you have anything else to add to the conversation?

Eli: I think he answered most my questions, related and unrelated, but I'm certainly probably going to go get a Pixel now. Google, you can send me one.

Zack: You can ping me on Twitter, we'll chat about it, we'll send pictures of each other.

Brian: Yeah, we'll just ask our nonexistent HR person to send everybody Pixels.

Eli: Yes!

Brian: And then when they get hired in two years, then we'll finally get Pixels. Netlify is a very small company, so. Well, relatively small.

Eli: It's growing a lot recently.

Brian: How big is Pinterest as a whole?

Zack: So I started 2 1/2 years ago, and it was just over 300, and now we're like over 1,400. So it's grown 4x in 2 1/2 years.

Brian: Nice, hopefully, 2 1/2 years, Netlify will be at least 300.

Eli: Yes!

Brian: Very cool. Awesome, well, with that being said, let's go ahead and roll right into picks, these are jam picks, things that you're jamming on. Sometimes they're tech-related, sometimes they're not tech-related. I am going to ask Eli to go first, because I am unprepared.

Eli: Oh man, putting me on the spot. Well, I have been really into some Spotify jam, because I've been doing quite a bit of traveling. So I guess my pick this week will probably be a bit of a throwback. But I've been really digging this song, Aqueous Transmission, by the '90s grunge band, Incubus.

They have been on repeat quite a bit lately. They're kind of up there with, I guess they kind of breathe that essence that is something in the vein of like Modest Mouse, that Dramamine song. It's very similar, it's got this kind of underwater feel, so yeah, that's been my jam. It's been kind of carrying me through these last couple weeks.

Brian: That sounds so depressing, but knowing Incubus, I guess that makes sense.

Eli: Yeah, yeah. It's kind of one of those beautiful depressing songs, but yeah, all happy over here.

Brian: Yeah, well thanks for going first, because that just sparked what I've been listening to, which is Kendrick Lamar. Actually, so I missed out when his first, his latest album came out. I was just so busy with work that I didn't actually listen to it, so I ended up listening to the entire thing last week, front-to-back and then back-to-front. It's pretty exceptional. I'm actually a big fan.

I like what he does, and I'm not upset that he stole my Twitter, my Twitter handle used to be @BryanLamar, because my middle name's Lamar, and Kendrick's middle name is Lamar, so he basically stole that name from me, so I had to change my name. So I'm @bdougieYO on Twitter, which also has been stolen by someone else. But anyway, that's a whole another story. So, I'm going to pick Kendrick Lamar as my jam, and Zack, did you have a jam?

Zack: Yeah, a song that I've been listening to recently is Ed Sheeran's "Castle On A Hill." For whatever reason, that song just like, I just feel good when I listen to it.

Brian: I can't say I've actually heard it.

Zack: Oh man, you need to check it out, it feels like it's a song you listen to when you're just like, going for a long drive somewhere.

Brian: Okay, I'm riding my bike to the bar, is that going to work?

Zack: Dude, yeah, it's perfect for that, absolutely.

Brian: It's only six blocks, but can I get that song in?

Zack: Oh, I think, if you go really slow, maybe.

Brian: Okay, well, I mean, whatever the reason, so when you're downtown, riding your bike, you stop at every single light, it's very annoying.

Zack: But not the stop signs.

Brian: Yeah, not the stop signs.

Zack: You're like the bike riders I see.

Brian: No, same thing in Oakland too, you don't stop at stop signs. Not even at street lights, just go right through.

Zack: So rebellious.

Brian: Cool, I'll have to check it out. So you mentioned you have a kid. I have a kid too. So Ed Sheeran, my only experience is when he was on Sesame Street, that's how I know who Ed Sheeran is. Cool, well, on that very sad note of my Sesame Street watching, I'm going to end this here. Zack, again, thanks for coming on, talking about service workers. Eli, thanks again for stepping up and asking some really good questions, and just being Eli.

Zack: We love you, man.

Eli: Thanks guys, it's a pleasure.