Data-Driven Progressive Web Apps (GDD India ’17)

Data-Driven Progressive Web Apps (GDD India ’17)

to look at data-driven PWAs, which is to say a lot of what
we show in starter classes for the PWA is how to cache
all of your static assets, how to cache an entire web
page or maybe even to cache a dynamic asset– dynamic page– not data,
but the page itself. But what if you’re building
something like Twitter, where you need user icons
and tweets, or you’re building a photo viewer, where
you need to pull the photos in and cache them, and
you want to make this data available offline? How do you do that if you’re
building the next Flipkart? Flipkart doesn’t cache every
single product in there. They cache things
dynamically as you search. So how do you do it? And in the classroom, we
do this pretty quickly. So we’re going to spend
some time taking this apart, going in at greater length. First off, if you don’t
have Node.js installed, find out who on your row
has one of the USB drives, and there’s images of Node.js
you can install from there, because some of the
tooling we’re going to use runs on top of Node. The labs for this entire class
are also on those thumb drives, so you’ll be passing them around
just to quickly copy things to your machine. There were two on each
row in the fishbowls, and people who came to previous
sessions may already have one. So until we’re done, we’ll
talk for a little bit, and then you’ll do some code. Then we’ll talk some more and
you’ll write some more code. And at the end, we’ll
have some questions. So it’s pretty straightforward. This is a mixture of
lecture and code lab, but sliced together in
about 10-minute slices. OK, so what is a PWA? A basic definition– so
very fast initial load. You cache all of
your static assets. They’re coming locally
from the device. They don’t need
the network at all. It may have an app
shell, though– the fact that it fast
loads throughout the app, that you have things
all loading up locally. App-like immersive experience–
things like full screen, add to home screen, potentially
even push notifications– make it look and feel
like a native application, except that anybody
can install it, whether or not you
have a Play Store available on your machine. And really, they use
very, very little memory. So the manifest is what
enables that add to home screen and full screen mode. And the last one
here, offline content, is where we’re going
to spend our time. How do we do offline data? So things that you download
using fetch or using Ajax– how do you store
it for offline use? So let me do a quick rerun
through service workers and cache APIs. Who in here is a
service worker expert? Who in here is kind
of sleepy after lunch? Ah, a few hands. OK. So almost nobody is a
service worker expert, except maybe Jake Archibald. Let’s do a quick recap. So the basic idea is that the
service worker is a proxy. It’s a network proxy that
sits between your code– your JavaScript– and the
outside of the browser. And so when your
browser, your web app makes a request like
a GET for hero.png, the service worker
intercepts that request. And it can do one of two things. Depending on the
code that you’re running inside the
service worker, it can either go to the network
and get the file and return it, or it can go to the local cache
and get the file and return it. So the service worker basically
lets you intercept any requests to the network and reroute it. Now, if you’re using an exotic
protocol like WebSocket, then this doesn’t apply. This is actually for
HTTP and HTTPS calls. The cache API basically says
it’s– the service worker is sitting here between
your page and the cache. And so the service worker
intercepts a request, goes to the cache
API, and says, hey, does this file
exist in the cache? And if it does, it just
serves it up from there. So here’s an example
of what would be in the cache for
the AirHorner demo app, which you’ve probably
seen a million times by now. So we’ll have index.html,, index HTML, the home screen, one variation. And so all the
variations of the ways you can get to the root
screen are all in the cache. And then the JS scripts,
the sounds, and the styles– and so this is all in cache. So AirHorner runs
completely offline. Everything that it needs
is stored in this cache. Now, this cache is not the same
as the regular browser cache. The regular browser cache– if you’re offline,
it will not generally serve new files up to you. Instead, it’s going to serve
up an error saying, hey, wait a minute. You’re offline. This file might be old. This is a secondary cache in
addition to the browser cache. That’s one that you
control directly. It’s going to– you use
JavaScript to say add a file. Remove a file. You have total control
over this cache, and it will always
be available offline. It can get removed
if memory fills up, and it’s basically a
least recently used kind of algorithm. So any apps that
are really active– this cache will stay active. Apps that are older may
get pushed out of memory. If that happens, they
may stop working offline. The next time
somebody goes online, service worker will
see, oh, wait a minute. The cache is gone. Let me build a new one. The app shell
pattern is basically the way you build really almost
any good-sized application. You put all the common
materials into a shell– so all of the common navigation,
the sidebars, everything. And then you load views
into it as you need to. In this case, those views are
HTML pages or other content that you’re rendering. And by the way, how many
people here use React? Right? A lot of people– more
than half the room. React works just fine in here. A lot of people say, I can’t
use React with service worker. No, you can use React just fine. Build your React application
normally, and then all you have to do is write the
service worker code to cache the pieces that
need to be cached– the HTML, the
JavaScript, and the CSS. So an example of building
that cache by hand is you make a list of all the
URLs in your application– all of your files. Notice that we have
both slash and index HTML, because there’s
two different ways to get to the index page. You wait. Now, this code is inside
the service worker. The service worker is
given an install event when the time is ready
to build the cache. So on the install
event, you call wait until, because otherwise
there’s a return immediately and your cache may not get
built. So you’re going to say, wait until I finish this job. Open the cache by name. So in this case,
it’ll be cache-1. And then add everything
to the cache. How many people don’t
know what this arrow is? Please be honest. Everyone’s got it? OK, a couple of people don’t. Let me tell you
really fast, then. That arrow is ES6
notation for a function. So you can use ES5, which
you’ve been writing. So I could write a
function in here, and then the function would
take a single parameter– cache. Or I can simply write the
parameter name, the fat arrow, and then the body
of the function. So this is a shortcut
for writing a function. Now, I could show you how
to write that thing by hand. There’s a problem with that. Let’s say you’re working
and you add a file or you change a file. You sure you’re going to keep
that whole file constantly up to date? Probably not. So Workbox is a tool, and
we’ll have a whole session this afternoon on it, that
automates writing most of the service worker for you. It takes a set of references to
where your static files live, including things
like wild cards, which service worker
doesn’t normally do. And it goes and builds
the exact list of files, and it calculates a
hash for each file and actually stores
those hashes so it can detect when a new version
of the file’s been uploaded. So it writes all
the code for you. You put it in your build step. Whatever build tool you happen
to love, it fits in there. So using Workbox, we have to
import the Workbox script, and import scripts is actually
a feature of the service worker that says, I’m running a script. Somebody used a
script to create me. Now the service worker needs to
pull some scripts into itself, so it calls import script. You create a new Workbox
software object– new Workbox service
worker, I’m sorry. And then you say
to that, pre-cache. And you give it an empty list. OK, well, this isn’t
going to do any good, but this is a template, just
like you use page templates. You hand this to Workbox
as part of your build. Notice, this is in
the source directory. And actually, let
me back up one here. You hand it to Workbox
as part of the build, and what is going
to happen is Workbox is going to fill in all of
your files into that empty list and generate a new copy
of the service worker that will actually be
your production copy. Right? Pretty typical
situation– you’ve got a source folder
and a build folder, and you have to
compile some things. This is just part of
that compilation step. So you can do it from the
command line or NPM scripts using Workbox CLI, or it
has built-in integrations for Gulp, Webpack, and Grunt. Or if you like some other tool,
you can roll it into there. It’s really very
straightforward. So for Workbox CLI,
you install Workbox in your package JSON for NPM. You basically say, Workbox,
generate service worker. And that’s all it takes. And then run that command. Don’t worry about
memorizing any of this. Everything is in the
lab that’s coming up. Now, how does it know
what files to pull in? You build a file called
workboxcliconfig– it’s a JavaScript file– and you say, everything’s
coming from the source directory at the top. My global patterns–
I want index– well, you see the files. I don’t have to
read them to you. You can include globals
there, and it will actually search and pull all
the files off the disk. This is a huge time-saver. You tell it where the
source service worker is, that little template. You say where you want it to
be written out to at the end, and you should ignore– if anybody’s using globals,
ignore the following files. So for example, don’t
cache this config file because it’s not
actually a runtime file. No. OK, sorry. This is the output code here. So Workbox software
pre-cache had a empty list. Now this is the built
one with the output. And it’s got URL. And it has a hash
for the file so that if a new
version of the file gets seen by the service
worker, gets picked up, this will automatically
replace it. You’ll keep your cache current. Now that’s static asset caching. For runtime caching,
for things that vary like the pictures
and the text in here, that’s runtime
caching with routes. And so what you say is if
it comes from this URL, this route, here’s the caching
strategy I want you to use. Normally, the cache strategy
you use is called cache first. That’s for all
your static assets. That means always get
it from the cache. And if you can’t get
it from the cache, fallback and hit the network. In this case, you’ll pick a
caching strategy depending on what type of data it is. So we’re going to say,
inside of your source service worker.js we’re going
to register a route. So any image file, ping, GIF,
or jpeg, we’re going to say, cache first, which
is try the cache. If it’s not there, get
it from the network, put it in the cache. We’ll say we’ll call
it the images cache and we’ll say it can
take 50 entries total. That’s all you have to do to
cache those dynamic images. How much time did
you just save writing this code as opposed to writing
a couple of pages of code and debugging it? So we’re going to do a
little quick lab break here. You’re going to follow
these instructions. You’re going to– if you
don’t have node.js installed, pull it from the USB. Hopefully most of
you already do. We told you before. Grab the lab code from
this folder on the USB. Open up the lab
instructions page. Now section 2 tells you to
basically download everything from the network. You can skip that
section, because you’re using a local USB. If we all tried to hit
the network here once, we’re kind of a
rolling DDoS attack. And then stop before step 5. So go up through step 4. And this will cache all
of your static assets. And this will cache dynamic
file assets, but no data yet. OK? Let’s go. [MUSIC PLAYING] What I’m trying to do is talk
through the next section. And then do whatever parts
of the lab you need to do. We’ve built in enough
time that most people can get through all the parts. And then there’s some
optional stuff at the end. And we actually timed
it for the optionals. So you should be
able to make it even with the extra time
for the installs. But let’s go ahead
and move forward. So we sent you through the lab. Now what do we do with
PWAs that rely on data? What if it’s really
a very simple shell? And everything’s coming
in from Ajax or Fetch? So for example, here’s a
clothing shop or Twitter. All of the information in this
page, the jackets, the prices, the pictures, the tweets,
the user icons, that’s all coming from the network. Those are not files. So you wouldn’t store
them in the cache API. So where are you
going to put them? Well, some people might
say local storage, but local storage
is super limited. Instead, usually
what we recommend is putting them in a database. Now, this is kind
of interesting. Because I’m going to walk you
through IndexedDB, because it’s been around and it’s
available pretty much anywhere the service worker is available. If I was starting today, and if
I was setting the workshop up today, I would probably
do the first workshop using Cloud Firestore that
we talked about this morning. It’s essentially a
document database built on top of Firebase. Because it will automatically
handle everything we’re talking about in here–
saving your data when you’re offline, transmitting it to
the server when you’re online, synchronizing the data. It does all of that. And under the hood,
it uses IndexedDB. So I’m going to teach you
how basically it was built. And then you could
choose to use it or you can choose to build your
own, whatever works for you. So IndexedDB, it’s
a NoSQL database. Technically, it’s
an object store. It’s not even a real database. It has a table with
vocal records in it. The records do not
have to be identical, which is really
confusing if you’re used to a kind of
relational database where you define what
the table looks like and the columns look like,
and they’re all the same. And not strictly true here. So you have a row number. You define some key if you want. In this case, it’s going to be
the value ID from the object, so it’s copied in. And then the actual
value is just an object. Now we’re showing the object
here in a sort of JSON format. But there are actually
regular JavaScript objects. You could even
drop a blob in here so you can get an
image or a video or literally anything that
could be stored in a JS object can be stuffed into a IndexedDB. And an IndexedDB appears
in the Application tab. You can come in and open
it up and look at it. And in this case,
I have two objects. I have an object
store called Witters. And I have an index
on it called by date. And so this is Witter. This is Jake Archibald’s
version of Twitter. So it’s really ideal
for storing data. You can do the usual
CRUD operations, create, remove, update, delete. You can do transactions,
which is actually really nice. You can string a
bunch of operations together and make sure
they’re either all going to succeed together or fail. Because one of the problems in
the database if you don’t have transactions is, what happens
if your set of operations gets interrupted in the middle? Maybe something else tries
to write to the database when you do. Without transactions, you could
have a corrupted database. With transactions
you can be sure that the data stays consistent. It has indexing, so you
can say, sort by this value and get them out in that order. And it’s a
cursor-based database, which means it doesn’t
give you a table, it gives you a pointer
into the database. You get your result and then
you ask for the next field. And the result and
the cursor moves. So in this case, we’ll
show two object stores. We’re going to show an object
store with some data in it. Both of them have data. This one also has two
different indices. So each index is a different
way of sorting the data. Actually, it’s a different
way of traversing the data in a sorted order. So we’ll open a date. We’ll create a function
called open the database. And we’ll say We’ll give it a name. We’ll give it a
version number 1. That version number,
every time you add a new object store
inside the database or change an index,
that’s essentially changing the schema. And so you would bump
the version number up. And then the last thing we
take optionally is a callback function. That callback gets
called– actually it’s not even that optional
once the database is created. And it says OK, database has
been built. Now do your work. So we’ll create an
open database function. Open it. Function upgrade DB. We’ll see, OK, does
the object store name contain objects store me? Do I have something called
object store in here? If not, I’ll go
ahead and create it. Notice that Upgrade
DB is an object that passed into this callback. All of the calls
for manipulating the structure of the
database are in this object. So this is the only
place you can get to it. And it’s the only way
you can get to it. This code may look a little
odd with lots of callbacks and event handlers. And you’ll see it gets worse. That’s because it was actually
designed about 10 years ago, back when event handlers
and callbacks were super common before we had promises. Now we’re going to
use an example here of something called
IndexedDB Promised, that Jake Archibald wrote, that
wraps IndexedDB in Promises and makes it vastly
simpler to write. But to get started
to do the creation, you have to write it
the old fashioned way. In this case, we’ll
also add an index on the object store called
price, so we can sort by price. And this is what the
database– we’ll get an example in the index of here’s one key. The key path is called time. So that’s one index. And that’s simply the time
[? caught ?] in the object. And then a second index,
key path ID, which is the ID field of the object. Now to get the data, this
is using IndexedDB Promised. Now Call Open Database,
that returns a promise. Promise then
resolves the promise. Pass it a function. Remember, we’re using
the fat arrow here. So this is a function taking
to single parameter call DB, the database. Start a transaction. So on the object store– and
it’s going to be read-only. Because you can have
multiple concurrent reads against the
same object store. If that had been read-write,
then everything else would have been blocked
from accessing it until the transaction was done. Because it’s read-only, you
could have multiple threads accessing it, all
reading at the same time, because nobody’s going
to modify the database. We’ll find the object store,
and then we’ll say store.getall. And what store.getall
does is returns back– in IndexedDB Promised,
it returns a promise. In regular IndexedDB,
it triggers an event that says, here’s some data. And then it has a cursor. And then you say, give
me the next record. So this will return the
first record of the set. So we can do add
get, to get one. We can do get all, to
do essentially a query, put, or delete. Kind of obvious. So the story is to
use them together. Anything that generally has
a URL goes in the cache. Now some people cheat. Let’s say you have
images, like, there’s a app out there that’s a
Pokemon index, called Pokedex. They actually use
data URLs and stuff– actually, they say sometimes
stuff the images in the cache– or IndexedDB. It depends. But you could stuff
images in here. You could turn something into a
data URL and stuff it in here. But if it’s got a URL
normally, it goes in the cache. Otherwise it goes in IndexedDB. And the code to
tell the difference, let Workbox start
things in by route and store them in the cache. And for all your data, just
go through IndexedDB code. So this is the overall
flow of the app. So go to the server and say,
fetch me some data or use XML http request. It’s actually better to use
the Fetch API, by the way. Things have been greatly
modernized over XHR and it’s more reliable. So get some server data. We’ll get back a promise
because we’re using Fetch. Then we get the data
from the network. In a function, we put it
on the UI immediately. This is actually one of
the first things to do. As you get the data,
put it in the UI because you want your
UI to be responsive. Now, save the data
locally in IndexedDB. And that’s all going to
happen asynchronously. That’s all Promise-based. So you update the screen,
stuff the data in the database. If an error happens, grab your
local copy of the event data. So try to pick it up
from the Index database and display it if you can. So this says, try the network
first and then fall back on the database. You could actually
write the reverse of this code to
go to the database first and then
try to work later. So this is the local save. It’s basically what you’ve seen. Open the database,
create a transaction, open the object store, put
each object into the store. If anything fails, call abort. Now here’s something
interesting. How many people in here have
programmed databases before? OK, a small number. Anybody who’s programmed
a database here, does something
seem to be missing? I started a transaction,
but what’s missing here? I’m not committing
the transaction, am I? Because IndexedDB is weird. It’s very strange. Here’s the way to
think about it. So this happens in
one chunk of code. And then somewhere
down here at the end, when we’re done
running JavaScript, the browser goes into its
idle loop, where it delivers events and all of that. Basically, that’s also
where all open transactions are automatically committed. So if you don’t call
abort, then the next time you get to the idle loop, that
transaction will be committed. Now in some browsers
like Chrome, the commit might
happen even earlier. But the model you
need to think about is whenever you’re
in the idle loop, all the open transactions
will be closed and committed for you. But it’s a little
strange if you’re used to writing databases
and you look at that and go, wait a minute. I didn’t commit this. The browser did it for you. It’s a little too clever. Getting a local event
data is really simple. Open the database,
create a transaction, call the object store. Call get all. No transaction required. Create a transaction read-only. Get everything, call get all. You don’t need to the
database or anything else here, because it
will auto close for you on the next time
for the idle loop. OK. Lab break. Steps 5, 6, and 7. Just keep working for a while. [MUSIC PLAYING] –everybody has network
connectivity everywhere. So it might be you get into
a cab, and you get Wi-Fi. Or you might walk into a
train station and get Wi-Fi. But you want to do some work
before that have it saved. And have it sent
to the server when you have a network connection. So here’s how you do it. It’s actually very simple. You use Workbox. You have to create what’s
called a queue plugin. So Workbox will
automatically run a series of plugins every
time the network comes up to synchronize
with the database. It’ll even track
the synchronization of your objects. So all you have to
do is say to Workbox, here’s where my data is stored
and here’s where it has to go. So in this case, we’re
going to write a callback. Replay did succeed is an event. There’s a callback that’s
called when we’ve actually sent an object to the
server successfully. So it gets a hashcode
and the response. And in this case,
we’re just going to go ahead and
post a notification. We don’t go through all the
details of background sync here on the screen. We do have them in the lab
notes and in the Workbox documentation, but this has
made life so much easier. This used to take an hour
to explain how to do it. Now we say write
the queue plugin. So in this case, what
we’re going to do is we’re going to say API add. So all of our data is going
to go to this end point. So it’s going to go to whatever
our URL is, slash API slash app on the server. It’s network-only. We say, here’s the
plugin you should use. And you should use
post to send the data. And then you just tell it which
database it’s coming from. So those are optional
steps 8, 9, 10, and 11. If you get through
step 7, you’re great. If you want to experiment
with the background sync, you can do the
rest of the steps. And I’m willing to go
ahead and just comment what I said earlier,
which is an easier way to do it now is Cloud Firestore,
because they built all of this into Firestore for you. So again, I’m Sarah Clark. If you need to email me,
[email protected] Good luck. Thank you. [MUSIC PLAYING] [APPLAUSE]

Be the first to comment

Leave a Reply

Your email address will not be published.