My name is Les Hazlewood. I'm the CTO, co-founder of Stormpath.
Stormpath is basically a user managementand authentication service for developers. We provide as a SaaS servicea REST+JSON APIthat automates security for your apps,the apps that you're building.We provide user security workflowssuch as password reset, account email verification. We implement security best practices.How many people heard of the LinkedIn breechesand the Twitter attacksand things like that due to password problems? The Sony Playstation network issueswith password reset?
We solve those problemsso you as developers don't have to worryabout that issue anymore. Of course, we offer developer tools,open source SDKs that wrap our REST+JSON API.That's us.Give us a shot. Just like you might use Stripeto offload payment processing, you can use Stormpathto offload user management security.
Tonight we are going to covera decent amount of stuff. I'm going to go through this stuff fairly quick and although it might look like a lot, there's a lot of stuff that pertains to problemsthat we solve everyday as REST developerstrying to implement best practices.You'll see a lot of things herethat you've probably seen before. A lot of the things we're going to talk about tonightare best practicesor de facto conventionsin solving these particular problem sets.
APIs. We're here to talk about APIs first and foremostso that means our focus is on building apps, supporting developers.
Developers are your customers. Whether they're internal or external, we want to make things easy for them.The easier it gets for themthe more likely they are to adopt your serviceand be happy and spread word of mouthand all that good stuff.
This presentation will focusmostly on pragmatism over ideology. How many people herehave heard of the term RESTafarian? You guys heard about that?A RESTafarian is an ideologue who won't compromise. We try to implement the best practicesand the proper techniquesbut every now and thenwe might diverge from some of themif they're overly complex.We want to focus on pragmatismso you can get your APIs out quickly and please your customers.
Then of course we're going to focuson adoption and scale. By scale in this presentation,I really mean distributed systems and web scale.In order to get large adoptionand to propagate or proliferate your service,things need to operate at web scale. And this is really, how many heterogeneous systemscan connect to yoursand how many can you connect to?That's really what I mean by web scale.Not so much scale based on performance.
Why are we talking about REST?There's 6 major points in Dr. Roy Fielding's thesison REST architectures.He lists them as scalability, generality,independence, latency with respect to caching,security and encapsulation.By these he means,again we just mentioned scale, when he talks about scalabilityhe talks about ubiquity.Internet, web level scale,not necessarily machine kind of performance scale.
Generality. REST is a general architecture.It piggybacks on the common HTTP methodsthat are already part of HTTP specification.It's very general in its approachin that it can be leveraged by pretty much anybodywho understands HTTP.
There is independence.Your implementation of a REST APIcan be completely independentfrom things that consume it.You might have Ruby on Rails back endor Python, Django back end, or Java back end.It doesn't matter.
You can be independent and you can interact with third party services and vice versabecause of the independent nature of HTTP.
Latency is also a very important pointthat he brings up in his thesisspecifically with regards to caching.It's something that people kind of step overa lot when they talk about REST APIs.Latency and caching are very importantin the REST paradigmand we'll cover a little bit aboutwhat that means later on.
Security is important.There are secure headersor things that support security in HTTP headersand you can leverage thosein REST based authentication schemes.There's security already built into the HTTP protocoland you can leverage those in REST.
Encapsulation. You can encapsulate details or complexityor migration paths inside of your applicationwithout exposing those details to your end users.
REST affords these 6 general propertiesas a general architecture.
This is a REST+JSON talk.Why JSON?I don't need to spend a lot of time on this.It's pretty much ubiquitous nowadays.I think there was a study published recentlywhere I think somewhere north of 55%of all application developmentis being done in JSON now, and of course that covers web-based applicationswhich I'm sure fill upa large percentage of that number.
JSON's very important.It's becoming useable and usedby almost all developers nowadays.It's very simple.The JSON grammar is incredibly simple.It's very easy to read by people,it's very easy to parse by computers,and it's scalable in thatit can handle many different use casesand it's flexible.
You can represent very many different data typesand data formats, data modelswith JSON's relatively primitive structure.
HATEOAS.How many people in herehave heard this term before? A decent number of people.HATEOAS is an acronym for, if you want to call it that, "Hypermedia As The Engine of Application State." That's really a big mouthful that means everything that a computer needs to knowabout your REST service is discoverablefrom a single initial URL endpoint.
If you can imagine yourself as a web browser,if you go to a homepage of aparticularly complex site via the links,you can reference every other pagethat's publicly accessible. You can get sitemaps,you can traverse linksthat reference resources that reference other resourcesand you're able to traverse that entiredocument graft, if you will, because you've got that initial resource.
That's really what HATEOAS meansin the REST scope is that,if I interact with the REST API,I should only need to interact with oneand only one initial resourceand then from thatI can figure everything out on my own.This is not a mandate for REST architecture.It's not something that Fielding coverstoo well in his thesis. A lot of people now who think about these thingsreally view that this is sort of a further restrictionon REST architecture.It is RESTful.The purists like to think this is pure REST,but it's really a further restriction on REST. You can be RESTful without itstrictly adhering to this principle. Keep that in mindif you see this term come up.
REST is easy, right?It's just JSON, it's just HTTP.HTTP has been around for forever so has JSON.It must be really simple to create a REST+JSON API, right? Not really.REST is really, really hard to get rightfor providers.If you're providing,if you're implementing a REST APIand you're exposing this to potential customersit's very difficult to get this right. And the reason why is,REST is not a specification.There's no RFC or W3C standard for REST architectures.
It's a design approach,it's an architectural philosophy,and because it's a philosophy or designthat doesn't have a concrete specification,it can be interpreted by people differently.
If you go on Stack Overflowand you do general searches on the internet,you'll find various opinions about how to do linksand how to reference other resourcesand all sorts of other techniques,and it's because this is not a standard.Everything that we're going to be talking about tonightis really an accumulation of probablya year and a half worth of researchon many different REST APIs,both JSON and XML based, many third party providers,public REST SAS services. And a lot of the things are going to bewhat we at Stormpath consider best practicesand de facto conventions,and we hope it's useful for you as well.
The punch line here is thatREST can be easy if you followsome of these guidelines and conventions.
In the course of this presentationwe're going to cover a bunch of examplesso you can see some of these principles in action. I'll use, just for the sake of simplicity for myself, Stormpath's domain.In Stormpath as a security and identity service,we model things like applications and directorieswhich hold accounts in groups.Of course, almost everyone hereis familiar with accounts in groups.
There's also associationsbetween a lot of these things.There's One-to-Many, Many-to-Many associations. How do you effectively represent thosein a REST or a JSON based representation?Also developer workflow, so password reset and account email verification, behavior essentially.How do I represent that in a REST+JSON world?I will reference these continuallythroughout the presentation.I think it's concept that most people arepretty comfortable with.Most apps need these stuff.
Let's talk about fundamentals.The first concept I want to introduceis that of a resource. REST as a paradigm says nothing about JSON,it says nothing about XML.It's basically a mechanism for data exchangeand Hypermedia traversal.What that means is you can represent datain any format you want.That means that we need to restrictthe vocabulary of how we represent resources.
Every time I talk about a resourceand as a best practice,you should always think of them as nouns, not verbs.
An account is a resource, a directory is a resource,a group is a resource, a user is a resource. These are things that can be interacted with.They have properties. But you're not going to hear me talk about resources as verbs.I'm not going to have a resource that says "get application"or "delete a group." I'll cover some use casesas to why in just a second.
Also resources are very coarse grained,they're not fine grained.If you have an account, you want to represent the accountwith all of its properties.You have no ideathat your consumers of your API are going to useor how they're going to use that information.You want to be very coarse grained. Give them as much information as makes sensein a single resource representationand then they will use that informationin ways that you probably never could have envisioned.
They might find value in the data or the attributesor how it's referenced to other links,to other resourcesthat you could have never really thought of. By putting the power in their handsto decide how they want to use their service,they can do a lot morethan you probably could have supportedout of the box. Keep your resources coarse grained. Represent data in larger chunksnot finer, smaller kind of resource representations.
REST is an architectural stylefor use case scalabilityand when you keep things coarse grainedagain, you help your user solve problemswithout having to interact with youfor very specific use cases.
As an example of why this is important: What if I am creating a REST APIand I make my URLs reflect operationsand resources?Maybe I would getAccount URLand a createDirectory or updateGroupor maybe I want to verify an account's email address.This looks fairly innocuous. It doesn't look very difficult or challengingor even that dangerous.The problem here is thatwhat if you need to introduce a whole lot more behaviorto support customers?Maybe some person wants to be able to search accountsor if they want to find a group based on the directoryor they want to verify account email addressesby a certain token.
As you can see here this quickly explodes.This is not maintainable, it's not scalable,it's not supportable as a development team.The more and more behaviorsyou add to the URLs itself,the more difficult it is for you to knowwhat's happening in your system. If you guys have dealt withlike RPC back in SOP days,this smells like bad RPC.Don't do this.This is not a good idea.There's definitely a better way.
We really want to keep it simple.We want to simplify resourcesand then how they're referenced via URLs.
How do we do that?There's really only two types of resourcesyou have to worry about to implement a REST API.There is what we call a Collection Resourcesand Instance Resources.A Collection Resource is itselfa 1st class citizen resource,it has its own propertiesbut it also contains other instances, other RESTful objects.It is a container. The other resource type is really an Instance Resource.It represents one data type in particularand not multiple things.It's just a single instance of a type.
Here are some examples:
We can have a Collection Resource, in this case it's labeled as /applications.The takeaway here is thatthis is a plural name. It's not /application, it's /applications.It's plural, it's self-documenting, it's readable.The name being plural is importantfor intuitiveness and readability.When you interact with this,you know that this thing exists for the purpose of one or more things. It's recommended that you keep yourcollection resources or endpointsin the plural. It's self-documenting and it's helpful.
Instance Resources, however,almost always tend to be a childor referenced or owned by a collection.In this case I'm showing an instance resource.It's owned by the applications collection,or it's a child of that collection,and it has its own unique identifier.Applications/a1b2c3 is a specific instanceof an application. That's it.There's really collections and instances.That represents resources,it represents state.
What about behavior?How do I not blow up my URLsto have all these different verbs inside of them?You can basically handle that through behavior.HTTP as a specification has 5 primary methodsthat are well-definedthat indicate behavior of how you interact with resources or documents on a web server.There is of course, GET, PUT, POST, DELETEand then there's HEAD for metadata.
How many people here think there is a1:1 correlation betweenCreate, Read, Update and Deleteand GET, PUT, POST, DELETE?Okay, we got a couple people. It can be modeled as such.
It's very important to understandthat it's not strictly a 1:1 correlation.GET, PUT, POST and DELETEcan be used for Create, Read, Update, Delete,but it is not a 1:1 correlation.This confuses a lot of people.
If you ever go to Stack Overflowyou'll see tons of questions about "Should I use POST or PUTin this particular scenario,and I don't know what to do,can you please help me?" Hopefully I'm going to make that very clear hereand hopefully pretty helpful.We'll cover some of the difficult ones in a second,but these really do have a 1:1 correlation.
GET is actually a Read: "I really do want to read something backfrom the server." DELETE does mean: "Please remove it from the server." And HEAD is a metadata operation: "I want you to give me back some information. You don't have to return the actual body,the actual resource." These really do exactly what you think. But PUT and POST are not really obvious.They can be both used for Create and Update,and I'll explain exactly the scenarioswhere it makes sense to do these things.
PUT, most people associatewith a general update command: "I've already got somethingand I just want to modify it or update it." But, it definitely can be used for Createif you allow your clientsto specify the unique identifiersor the identifier of that particular resource.In this case, maybe I allow the clientto specify the ID of an application.Maybe it's a global unique nameor something like that.You can use PUT in this scenariobecause you're giving the server everything that it needsand you're telling it where you want it stored.This is a legal operation.PUT can be used for Create in this scenario.
It can also be used for Update. In this example, I'm putting some datato an existing locationand it's got an ID already associated with it.But the key here is thatthe name and the descriptionthey have to be, in this particular example,the only two propertiesfor a particular application resource.There shouldn't be any other propertiesassociated with this requestand the reason why is:
PUTmust be a full replacement operation.It can't have only some of the propertiesand then some of them excluded.Every single property for that resourcemust be specified as PUT.The reason why, and this is really important,is that PUT is mandated by the HTTP spec. This is not REST, this is HTTP. The PUT operations must be idempotent.
An idempotent operationis any operation that when executedone or multiple timesresults in the exact same state on the server.
As an example,if I have a name and descriptionand I send via PUT and I send a name in only on request Aand then maybe a little bit laterI send in just the description attribute,the server state between those 2 operationscould be different.Maybe the name is updated by a different requestor the description has changedby some background process. You can't guarantee identical state on the serverwhen you only submit partial updatesor partial representations.This actually breaks a fundamental propertyof the HTTP specs.Just keep in mind PUTs must be full replacements.
Another important thing,idempotency as a property, again, if you do some research aroundon Google and whatnot, idempotency as a property only pertains to the server state.It does not have anything to do with the client state.As an example, let's say I use a PUT to create that application,and then it's created, everything works fine,there's no constraint violations.But let's say I have a unique constraintof application names in my system.It means any app that you createhas to be named the same.Or excuse me, they have to have unique namesacross all of the applications.
I can request this the first timeor I can send in a request the first timewith this payload, and then as a client,I can send that exact same payload in,and the server can respond to mewith an error condition stating, "Hey, this is a unique resource.There's already something else with that name.You're not allowed to save thisto the server again."While that is different behavior for the clientbetween the two requests,the server state is the samebetween the two requests.Therefore, idempotency is maintained.It's important to know that idempotency only mattersper the HTTP spec.For the server state not for the client. We talked about PUT.
This is POST being used as a Create.This is a lot more common.You'll see this referenced a lot more. POST can be usedto create things typically instance resources when you interact with a parent resource.In this case, I'm interacting with the application'scollection resourceand I am indicating, "I want you to create a new child of that collectionand here is the nameof the application to be created."
In this case 201 is returnedand it's important when a request is successfuland a new resource is created, you always return a 201, not a 200.200 OK just meansthat the request was successful. 201 means: "Not only was the request successful,but I created something for youand by the way, here's the location headerthat tells you the canonical locationof where that's going to exist."
You always want to return 201s when you create thingsand you always want to set a location headerto tell the client the canonical locationof where that resource now resides,and then it can reference thatin future requests to go interact with that resource.
You might notice herethat this only a partial representation,so if the application has a nameand a description property,I'm only specifying one as POST. The reason why that's okay isPOST is the only one of the HTTP methodsthat does not need to be idempotent. And actually they're pretty vague on this in the HTTP spec,but POST basically meansa server processing directiveand not a whole lot more.It means you can do whatever you wantwith a POST operation.That's why this is legalto create partial updates via POSTbecause this does not have to be idempotentper the HTTP spec.
This is Create.Same for an Update.In this case as you see,I'm only giving a partial representationwhen I want to update the data. The idea here is thatonly the name property of the particular applicationis being updated.I'm specifying its full resource URI/applications/ID. I'm only specifying the name.In this case 200 is okay â€” pun intended.
200 is an acceptable response codebecause you already know the identifier,you already know its location of where it resides.You don't need any additional databack to the client, 200 OK is perfectly acceptable. Again, that's because POST is not idempotent.You're allowed to do partial updatesand partial data creates.
Media types.This is something that people kind of come upand they have a lot of questions about.Should I use media type,should I not use media types?What are best practices?The reason why I bring this up as part of the fundamental section is thatthis is really, really coreto how REST as an architecture should function.Fielding specifically states two really important concepts that exist in RESTful architecturesand that's the notion of distributed Hypermedia, basically the ability of resourcesor documents to link toor reference other resources and documents.
The other half of that is media types, you should have theoreticallya rich media type library, if you will,or repertoirethat allows you as a developerto describe your resources in detailusing the media type specification.
A media type really is just a format specification, how data is structuredand paired with a set of parsing rules so that machines know how to interactwith that data structureand interpret it accordingly.
The client can tell the serverwhat media types it wants back from the servervia the Accept header.When the client sets the Accept header,it's telling the server, "These are the things that I accept,these are the media types that I understand,so please give me back the datain one of those formatsso I can parse the information."
The response from the server to the clientcontains a content type headerthat says, "I get it, I know you're askingfor these various different types of media types,but here's the one I'm actually sending back to you." Most of the time those two content typesshould be identical.Sometimes they don't alignand then you can do content type negotiationin various ways. We'll talk about that in a little bit.
Here's some example of some media types:
Application/json is something everyone is familiar with.Application/foo+json saysthis is not just a JSON document,but it's a JSON documentaccording to the Foo specification or Foo semantics.Then you can do other things.You can create key value pairswith semicolon attribute delimiters.You can provide more informationto the media typeso that clients can read thatand figure out how to do thingsin a more specific manner.We'll touch on media types a little bit pretty soon.
These are really important. The RESTafarians say this is mandatory,you should have all of your data exchanged and identified via media type.We might veer from thator diverge from that kind of recommendation.If you can do this,it's very important that you do it. This is a really good practice.Highly, highly recommended,but there are some other waysto solve this problemand we'll talk about that in a bit.
We talked about fundamentals,let's talk about designing.We're going to create a REST API from the ground up.What are the things we have to worry about?I almost hesitate to introduce this firstbecause URLs, I think it's just naturally somethingthat developers think about.What are my URLs going to look like?Are they going to be intuitive to understandand can I put resources there?
URLs actually don't really mean a whole lotin the REST paradigm.What does matter is links and HREFs.As long as you can have links from documentsor from resources to resources,that's really what matters.
But nevertheless, people like to think about these thingsbecause I guess it's easy to grab ontoand it's one of the first things you think about.
Talking about URLs,if you were a potential customer,what would you want to interact with?What would you want to seeif you're trying to hack out a script really quickto see if you can maybe put cURL togetherto test API interaction or integration? That's pretty obvious, right?Everybody likes the top one.It's easy to remember.You don't have to type so much.It's less error prone.
The bottom one, while totally validand there's nothing technically wrong with this, it's not pleasant to look at,it's confusing. Do I have to go /serviceor do I forget API or REST?You don't know what those intermediate paths meanand it's just distraction, it's noise.
You want to providea comfortable and pleasant experienceto your developers or to your customers. Give them an easy to use host namethat they could just interact with from day one.
People like the top one.
Sometimes I get questions about, "If I interact with that API via a REST clientlike cURL or command-line client,versus when I visit this in a browser,what am I supposed to see?" Because I had mentioned with content negotiation,the browser can tell the server, "Hey, I support XML and JSON and HTML and plain text.I support all these things."
The server can come back and say, "Oh,you support all those things.Well, I'm going to give you HTMLbecause that's listed as the firstin that list of Accept media typesand so clearly, that's the one you prefer. Here's an HTML pagethat renders that information,"which is totally valid. That's the whole reason why content header and negotiation existsand why it's functional and you can leverage that.
There's no right or wrong here.You can do whatever makes sense for your customers.We, however, at Stormpath chose,and I think a lot of other companies choose the approachwhere no matter what the Accept header says,as long as at least it says application/jsonsomewhere in there,we always return JSON back to the caller.The reason why we do that is that, again, we want to help developers.
If they're experiencing problems: maybe their REST command clientdoesn't work that wellor they've configured cURL incorrectly, or who knowsmaybe the command line toolor IDE tool that they're usingis just blowing up. It's really convenient to just open up a web browser, go to a URI,hit enter and see the data that comes back.
For debugging purposes and tracingand developer level information,we find it much easier to represent JSON to the browserand to command-line clients equally.We just feel it's more convenientand it's been helpful for people.
Again, this is just a recommendation.There's no hard-set fast rule.
Versioning.How do I handle versions?This particular topicbrings up a lot of debate in the REST community.The RESTafarians sayyou should only encodeor represent versioning informationin the media type.As you see here,the media type example showsthat it's a JSON document, according to the Foo specification,that represents an application resource,and the version is at version 1.The data that comes backis applicable to version 1 parsing rulesor data format rules.The top one statesthat you could just put a version identifierat the end of the URLand everything from that point downcomes back at version 1.
The bottom, the media type approachis the purists approach. It is the cleaner approach and it is the recommended approachby most people who advocate REST architectures.Roy Fielding recommends this as wellif it's possible for you.The reason why is that you're URLsdon't have to change over timeas you upgrade your service.They can remain the same.You don't have to redirect clients to new URLsif they're interacting with an older version.The server can look at that v=1 tagand knows that, "Oh, they're requesting a version 1representation of my resource.Here, I'll render version 1 for themeven though we're on version 4."
This is a much cleaner approach. The URLs stay the same.There's less changein the most important part of the requestwhich is the URI.It's backwards compatible.There's a lot of really great benefits for this.The big deal with media types though in this approachis that not many people understand this stuff.You got to understand, especially if you're a SaaS provider,you're going to have customersof all levels of education, kids right out of high school, PhDs.
You're going to have so many peoplefrom different walks of life.The media type recommendation or approachis a very technical answerto something that could be simplified.
A lot of people don't like using this stuffbecause it imposes a significanttechnical challenge on the end userwho's interacting with your API.It's correct, but if you're really going after adoptionand you want to make your customer's lives easy,the URL at the topis the one that most public service providerskind of adopt.Stormpath does this, I think Yahoo and Google,and a whole lot of people, Twilio. I think most of the public SaaS providers do this approachbecause it's just so easy,and you know that any of your customerscan go to that URLand everything from that point onis a particular version.
That being said,if they want to start using version 2,they have to change the URL and their source code.They have to go through maybe some configuration updatesto be able to interact withthese versioned resources properly.It imposes more change and more risk,but a lot of people don't thinkit's that big of a deal. The thing I would like to point outabout the top approach, though, is thatyou want to keep your version informationor version identifiers very succinct. Preferably atomic integers.
You might have internally an APIthat's at version 1.5.42or 1.5 or some date. Your customers don't care about that.As long as whatever is being served from that endpointis backwards compatible from day one when you produced your first resources from that version identifier,it can stay on that identifier. You don't want to introduce change.Every time you release a new version,you don't want to require your customersto go reconfigure the REST clientsor their scripts or whatever they've built.It's really annoying, really disruptive.
Think long and hard aboutwhen it's appropriateto increment your version numbersand keep them atomic integers for simplicity.
It's also kind of a good gut check if you want to increase your version numbers a lot, maybe it says there's a little bittoo much churn in your API,and you might want to rethink that as an engineering team as to whether or not it's a good idea to do that.Just be careful with version numbers.At Stormpath we do the top approach. We found it to be easier for our customers. The good news here is thatif a media type is specified with a version number,it can, if you want it to, overridewhatever is specified in the URL. That may or may not be a good approach.I'll explain a little bit later.
It's good idea to support thisif you can get around to it.It is the clean and appropriate wayand the customers who are savvy enough to use thatwill benefit from all of those additional benefits.
We talked about resource format already,we talked about content types.In this presentation we're going to be coveringapplication/json a lot. And, as I just said,start out with application/jsonand then allow or specify these additionalmedia type specs or media type representationswhen you have time to do it.It will be helpful.
Date, times and timestamps.We've gotten some questions: "How do I represent time? And there's like 20 different ways to do this."I've seen on various posts: "Just the time or just the datesor do I introduce a time zone?"I find it amusing because there's a standard that's been around for forever.It's called ISO 8601and it codifies how dates and timesshould be represented in a text format.Any time that you have to represent timeto an end user,stick with the standard.There's a lot of parsing tools and libraries that already know this stuff,that can parse this stuffwithout any work on the developer's endpoint.
In this case you'll see it's a year with the letter Tthen a time formatand then finally a time zone indicator.Here you see we're using Z for Zuluwhich is the same thing as UTC, Universal Coordinated Time.We recommend thatanytime that you represent timestampsto your customers, your developers,always represent it as UTC.We've seen from an implementation side,at least, there's a lot of problemswith how different data storers represent timestamps.MySQL, for example, handles this terribly.It's really hardto represent accurate timestampswith time zone metadata and MySQL.It never has really handled that scenario that well.Other databases don't do so well with it either.
But on top of that, your customers are likely to bein many different time zones â€”East Coast, West Coast United States,Pac Asia, Europe.Those are all different time zones as well.If you represent everything in your systemas UTC, ISO 8601,you can let the customer formator render that data in their time zoneas they see fit,and it doesn't cause any confusions or problems.There's no time collisionswith different representations of time. If you stick with Zulu or UTC timeyou're going to save yourselfa tremendous amount of problems and trouble.
We highly, highly recommendthat you use that format and you use UTCfrom the operating systemall the way up through your API. It's going to save a lot of people a lot of heartache,both yourself and your customers.I strongly recommend you stick with 8601.
HREFs.How do we represent linksand references to applications?Again, I mentioned previouslyHREFs and Hypermedia specificallyin Fielding's thesis is paramount. He listed as probably the most important part of HTTPand the REST architecture is that documents and resourcesshould be able to referenceother documents and resources.Every resource, therefore, should havea publicly accessible unique URL.
In Stormpath we encode this informationas a first level property of a resource.We happen to call it HREF.I've heard other people call it linkor URL or whatever.We call it HREF just to be a little bitsemantically close to the Xlink specification for XML. But one of the interesting thingsabout this particular representation where it's a first class propertywith all the other propertiesis you're going to see some really cool tricksfor entity or resource expansionbeing able to expand the linkinto its full object resourceand it's going to make the JSON representationpretty elegant and very easy to consumefor customers and people that are parsing the JSON.There's some really nice attributes or qualitiesto this particular approachthat I'll cover in a little bit.
Response bodies.Do you have to return themas part of the response?GET is obvious. If I'm issuing a GET request,I absolutely want that data back.With POST it's not so obvious.If I'm going to send some information into the server,maybe it's the entire resource representation,do I want to get that same exact data backthat I just sent to you as a response?There's no mandate for it. We actually do recommend that you do thatand the reason why is thatwhen a client gets the data backmaybe there's some data that's been updated that they don't have control over, then they want to see the freshest value.
For example, maybe there's an update timestamp.They have no control over that. Maybe it's only updated in the server,but when you return the data back to the customer they can see those fieldsthat might have been updated automaticallythat they don't have control overand also, they know that they're lookingat the freshest most recent version of that data. And it's important because if they instituteclient side caching,they can invalidate or update their cache locallyto use the freshest, most recent versionof that information from the server.
We do actually recommendthat you send the data back.It will greatly simplify client code as well.
If they know there is always data to come backthey can always relay itto whatever their storage mechanism isor their parsing mechanism.There's no special cases.It's easier to code for that particular use case. That being said,you only want to do this when it's feasible.If I'm uploading a 10 gigabyte video file,I clearly do not want that back.That's going to eat up bandwidth. Use some prudence with this.
You can also if you wantallow customers to control this themselves. If you specify a control query parameterlike _body=false,maybe you have a data limited API, maybe there's a quotathat they can only exchange so much data per monthon your public API. This control parameter allows themto reduce the amount of datathat's exchanged over the network.You can give them the optionto reduce the amount of data that comes and goes.Convenient if you want to use it.
Content negotiation.I'll just kind of jump over this quickly, we've covered this.The client specifies the Accept header.The important part here is thatthe preference for what you want to get backshould be comma delimitedin the order that you wish to receive informationor that the client wishes to receive something. In this example, application/json comes before text/plainand that tells a server, "Hey, I prefer JSON over plain text.Please give me back JSON if you can,if not I'll deal with plain textbut I prefer JSON so please send it."That's how clients tell the servers what they prefer.
Resource extensions.In addition to specifyingthe media types in the Accept header, you can specify an extensionthat goes on the resource URI.Instead of just the resource URI, I can have the URI .json or .csv. And this technique, if you want to support it, conventionally overrides the Accept headerif it might be set.The reason why is thatthe URI is the most important part of the request, it's closest to the identity of the request. If I specify JSON on the URLthat means that no matter what my browseris sending in the Accept header,I want to get back to JSONinstead of plain text or whatever.
You can support many different formats,.csv, .xml if you want to,but this is a nice techniqueespecially for developersthat might want to test different formatsand it's not so easy to set headersif you're just using a regular browser.
Resource References and Linking.Again, we said this is extremely important.The difficulty here is thatthis is really kind of tricky in JSON. Again, JSON as a format has no official specificationfor linking or referencing other data.XML has it. There's an actual W3C standard called XLinkthat is a specification by the W3Cthat says this is how we reference Hypermediaacross documents or across resources.Since XML doesn't have that,how are we going to do this?There's a lot of contention right nowon public discussion forumsabout what's the best way to do this?Should I have X, Y and Z attributes?How do I model this in JSON?
We went over and over and over again in our headsas to the best approach and we ended up coming up with, the KISS Principle won out for us.We really wanted to keep it super simpleand reduce complexity.The way we solved thisis we actually referenced, as you can see here in this example: I'm showing an Account Instanceand it's got a couple properties. It's got its HREF which is where its canonical location of where it lives, it's got a given name, a surname.The tricky thing hereis that there's this Directory attributeand the Directory is itself a complex objectand the Directory owns the Account. The Account here is referencing its parentbut it's a complex objectso how do I reference that in JSON?
We found that the simplest way to do thiswas just to use a complex JSON objectthat itself has an HREF property. One of the interesting things about this approachis that in anchors, in HTML anchorsas part of the XLink specification,they have this REL, R-E-L attributelike A HREF=blah, REL=blah. And REL means relation.In that REL attribute, you're kind of giving metadataabout what kind of relationship is this?What does this mean for that particular link?
The interesting approach here is thatin JSON, the attribute name itselfis kind of the REL in this case,the relation.It's a directory attributebut it's also impliedthat that's the relationshipthat's being represented here.The account is now referencing its directory and it does it via a simple HREF.Again, we're going to talk aboutresource expansion in a little bitand you'll see how this is really, really elegant.It requires no change of code for your clientsto represent more complex versionsof this resource.
If it's just a single property,you know: "I have an attribute,it's complex object, it's got one HREF property,it's clearly a link."That's how you can reference multiple resources.
This is particularly interesting.It's pretty useful and it's an often requested featurein REST APIs.It's also known as Entity Expansionor Link Expansion.Basically, the concept works like this.What if in a single request,I wanted to get back the Account objectand its parent Directory in a single payload?I want all of that information in one request.I don't want to have to get the account objectand then look at the HREFand then go execute another HTTP requestto get to the Directory.Please just give it back to me in one lump sum.
The way we've solved thisand the way we've seen other people solve it as wellis that we supportthis notion of an expand query parameter.We've seen people do this also as headersthat the server interprets to expand the data.We think it's kind of nice to have as part of a query parameterbecause it's sort of a directiveabout that specific resourcebut it's not the core identifierfor that particular resource.You can get additional metadata as query parametersand it's all represented fairly easily.You don't have to go through the effortof setting headers,but this is a personal reference. There's nothing wrong with using headers.We just feel that this is a little more easy to understand and self-documenting for peoplethat want to interact with URLs.
In this case we've specifiedthe expand=directory parameterand that's a directive to the serverthat says, "Okay, I'm going to give backthe Account resource, but in the directoryin addition to the HREF,I'm also going to populate all the other attributesfor that particular directory.If you look at the structure of this JSONit's identical to the previous examplewithout a directory.It's totally identical.There's nothing that has changedin the structure of the documentexcept for we've just added a few more propertiesto that complex object.
The beauty in this particular approachis that you're clientwhen it's consuming information,doesn't have to know, "Oh, I have to lookat this particular data format for links, and then I have to look at a totally different formatfor expanded references."This technique basically allows the client to say,"Oh, I've got one HREF and nothing else. Clearly a link. It's got more than an A HREF, it's been materialized.I don't have to worry about anything else.I can process them both identically."
Really, really nice technique, somewhat self-documentingand the fact that it's just a simple HREF propertyto represent a location meansthat you're providing a low complexityresource representation to your customers.It's easy for people to understand this approach.
What about partial representations?Instead of getting an objectand a bunch of others, what if I only want certain propertiesof a particular object?Again, maybe you have a quota limit on your APIfor data exchange.In this case, we've seen people offera fields query parameterwhere they can specify the attributesthat are inside of the resourceand then your resource only has to returnthat particular data.An interesting point that I should note hereis that when you see this query parameter,expand=directory and then the fields mechanism,you're interacting with the same resource both times.The same resource within your server.
The interesting thing to note hereis that the query parametersare taken into accountby caching in proxy serversas determining whether or nota particular resource is unique. Caching servers will see this URI and then they'll see the other one,and then even though in your serverit's the same resource,the caching server is treating them as two separate resources. Query parameters are taken into accountfor unique identities for caching.Just be aware of that. It may or may not have an impact on youdepending on if you cache or how you cache data. I just wanted to bring that up.
Pagination.What if I have a directory withhalf a million accounts in it?I'm not going to give you half a million accountswhen you request that directory.That would be absurd and maybe it would slow things downand cause frustration.Clearly you'd probably wantto chunk up a response like thatinto many different pages. We've seen a bunch of waysthat different people support this.There seems to be some consensuson the approaches to support pagination. Most of the time,the query parameters are actually named Offset and Limit. Offset is the starting position from 0. In the overall collection, what index do I start my page at?Limit is once I started that index,how many results are you going to give me back?
This particular query says, "I want to start on the third page of dataand I want 25 results back for that third page." You can use these two parameters and I've also seen people use a cursor query parameterthat might retain like a serialized data cursorthat they can usewith their underlying database technology. Maybe it's Oracle cursor into the actual tableor there's some kind of indexing mechanismin a distributed cacheor NoSQL data store. Using a cursor query parameter is alsopretty common as well.This is one approach you can use cursors.
One of the other things that's interesting aboutcollections and collection resources is thatin the spirit of HATEOAS,instead of the client having to worry abouthow to calculate pages â€” Do I have to worry about offset and limit? â€” you can provide additional attributesin your response representationthat are themselves links to other pages.If they want to get the first pageor the previous page or the last pageof the overall collection,they can just interact with that particular HREF. They don't have to worry about "what's my starting index"and "how many results can I get back in a page?"
You can provide this as a really nice convenience mechanismto simplify user interfacesthat might consume this directlyor people that just don't want to messwith the offset and limit query parameters.This is a nice techniqueif you want to simplify usage for collections.
In this case you'll see and this is a good representation againwhy collections we call them collection resources, they're first class citizens.You see they have their own properties â€” there's an offset property,a limit property, a first property.These are first class resource propertiesand then you have an items propertywhich is an array of the thingsthat it contains for that particular page.
It's a resource, it's got its own propertiesbut it does happen to havean array of things that it encapsulates.
Many-to-Many.How do I handle Many-to-Many relationships?In this example,accounts can be in many groupsor can be assigned to many groups,and of course a group can have many accounts.Via the URL that you see here,it's a unidirectional association.I have to go to an account firstin order to get its groups,but perhaps I have to go to a group firstin order to get its accounts.How do I represent data relationshipif it can be referenced in two different ways? You can do what I just said.You can have two different URLs that represent the datain that particular way.
We have chosen in our API,and we found this to be pretty convenient, I don't know if I'd call it a best practice, I think it is, it's been very useful for us, but we represent all of our relationshipsas a resource in it of themselves.In this case it's called GroupMembership.You can call it whatever you want.You can call it whatever mapping or whatever.We call this a GroupMembership.That means that you can interact with itas a first class citizen resource as well.Just like all the other resources,it's got its own HREF,it's got links to the account and the groupthat are linked togethervia this particular membership resource.
One of the nice things about thatis you can interact with the membershipand you can find out what the account or the group is.Maybe if you support querying,you can query for the account or the groupand get back the other.One of the nice things about this approach too,is if you choose to offer metadataabout the relationship at a future date, you already have the resource in place to do that.
Customers can already interact with it.It can already be part of your API. If I just added a property like: created byor made by John Smith at time, December 14th.You can provide additional metadatathat might be useful to your clientsif they want to know more informationabout these relationships.
By representing it as a resource in it of itself,you get a lot of extra power and capability.
Another also nice intrinsic benefit of this is if I issue a delete request to this URI,I immediately delete the associationbut I don't delete respective group or the account. I can add an account to a group; I can delete an account from a groupjust by deleting this particular resource.It's kind of a nice side benefit.You could attack this from many different anglesdepending on what makes sense to the customer. You'll see, how do I reference memberships? In this particular exampleI have my normal group's URIand if I go to that particular collection resource,I'm going to get immediately back a collection of groups,not the membership just the actual groups. But I can also additionally represent the membershipsas another collection property.
Forgive me, this is not a relative path, these are ellipses.I'm truncating the beginning of this for brevity.
But the idea here is thatyou can reference a fully qualified URLa collection if you only care about the groupsor I can also reference a collection resourceif I care about the memberships themselves,the actual metadata associatedor related to those associations.You can let the client choosewhich of the two that he prefersfor his particular or her particular use case.
Errors.How do I represent errors in a RESTful way?This is a really important pointthat a lot of people overlook.I got to give props to Twilio.I don't know if there's any Twilio people in the housebut they were one of the first peopleto kind of do this the right way, I think.We've adopted much of their particular approachand kind of expanded on it a little bit.This is how they representand we represent RESTful errors.
In this case you see, maybe I'm posting to the directories endpointand I'm trying to create a directorythat has the same exact nameof something that's already there.I will get back a 409 conflict from the serverin this particular case. The error code could be any number of error codes,but in this particular example409 indicates conflictwith a resource state on the serverand so I'm not allowed to createsomething that already exists. But I'm also in the response body returning a lot of information.I'm giving them the statuswhich is the same exact HTTP status codeas what's set in the response header.
The reason why I do that is thatyou'll only have one place to go to.You have one place and only one place to lookin the response payloadfor all the information that's important to you. You can of course, or your clients can of course use the HTTP headerand maybe that's more beneficial,but by repeating it hereyou give them another optionthat could be convenient for them.
There's also a code propertywhich is a code unique and specificto your specific REST service.The reason why is I think there's something like,I want to say like 23 or 24 HTTP error codesfor 4xx and 5xx.If you think about it, that's not a lot.That's not a lot of error codesto help you fully describewhat went wrong to your customers.
By having a company or service specific error codefor your API,you're able to convey particular conditions and errorsin a much more specific manner.
We also like to provide a code propor a code attribute in the payloadthat they know, kind of like you've seen Oracleand MySQL error numbersthat indicate very, very specific conditions.That's very helpful for people debuggingand trying to figure out why something failed.You can do that. In this case we've set a property called a property, and the value's name indicates "this request failed specificallybecause the name property of what you gave meis not valid."
Of course, they can choose to ignore thisand come up with something else,but at least you're giving them a head startand you're explaining things in a non-technical waythat's really helpful for people. The developer message then would bethe thing that actually you show to your customersin a very technical waythat helps them understand why it failedand maybe it gives them some resolution advice like, how do you fix this?You want to be helpful to themso that they know what's going on.
If all of these still isn't good enough,the beauty of the MoreInfo property is thatthey can go to a webpagethat contains a whole bunch of information.Maybe you have some sample appsthat you reference from therethat showed them how to solve this particular problem.You can fully expand on what happenedand why it's happening.You want to be as helpful as possibleto your customers so that they'll be happy to use you and they'll continue to use you over time.
Security.This is obviously near and dear to my heartat Stormpath.Unfortunately, I don't have a whole lot of timeto go through this stuff.But just in brief summary.Avoid sessions if possible.From a performance perspectiveit does help scale significantlythe less state you have to maintain â€” stateless architectures and asynchronousevent-driven architectures. They're all good for scalabilityfrom a performance perspective.
We recommend and we advocateauthenticating every request if necessaryor if possible.
For Stormpath for example,every single requestthat comes in our service is authenticatedand we do that for security reasons. We don't want any interceptionor any manipulation of our dataor our customer's data. One thing that's importantis, authorize based on resource content not the URL. You don't want to say, "Is the user allowedto visit the URL that matches this?"That's a fairly brutalauthorization policy approachbecause if you change your URLs for any reason,now your policies have broken.
What is a better approach isthat you actually authorizebased on the content of the resource.Maybe you say, "The current useror the current requester is allowed to seethe account with ID 1234." It's a much more concrete,specific authorization checkthat is orthogonal or separateor not tightly coupled with the actual URL format. What you really care aboutwhat resource is being interacted withand what they're doing with it.Not so much the URL.
If you can use this finer grained authorization approach,it will scale better.It will be easier to maintainfor your application over time.
How do you authenticate end users to your API?We really recommend using the existing protocolunless you're a security team.Oauth 1.0a, there's Oauth2.If you have to use HTTP basic authenticationonly use it over SSL. Even in intranet environments.It can be particularly risky.For example, certain infrastructure environmentsmight log requests as they come inor the request data, the request headers,and if that logged data or that logged messagecontains a base64 username and password,that's a potential attack factorthat people can use to find out passwords. If you have to use basic that's okay.Make sure it's only over SSL in all scenariosand don't log data.Don't log the authorization header in that environment.
Oauth 1.0a is pretty really secure.It's based on a digest-basedauthentication algorithm that does not require the passwordto be sent over the wire.That's a really secure techniqueand we recommend it.Oauth2 is a little weird.It's been going through a lot of problems.Recently the lead of the spec committee stepped downbecause they couldn't get any consensus. They were trying to solve all problems to all peopleinstead of being narrowly focusedon a particular solution.It's kind of gone its own wayward ways.
From an actual security perspective thoughif you look at the crypto behind it,Oauth 1.0a is actually still more secure than Oauth2because of the crypto algorithms they use.Oauth2 is basically SSLwith what they call a shared secret or a shared tokenor provider token. We recommend Oauth 1.0a as a security companyif your data is secure.
If your data is not super secure â€”maybe you're a social network,maybe like Google+ or Facebook or whateverand a lot of that data isn't crazy secure.It's not like bank information.Oauth2 could probably be a decent use case. But if you have security sensitive workflowsuse Oauth 1.0a.
And only use a custom authentication schemeif you really, really, really know what you're doing. There are a lot of different vectors for attacking authentication protocolsthat people are unaware of.
At Stormpath we like to thinkwe know this stuff pretty well.It's our bread and butter.We do have a custom authentication schemethat's very secure.It's very similar to Oauth 1.0aexcept it does a lot morein the areas of cryptographyspecifically around the request body.The request body is also encryptedor at least part of the encryption algorithm.
The other thing about custom authentication schemesis that if you create oneand you're sure it's rock solid,nobody is going to know how to use it.Even then, you would only want to use itwhen you provide client support for it.For example, when Stormpath uses our custom scheme,all of our publicly provided open source SDKsimplement that schemeand we'll use it when people communicatewith the REST API.The point is that we're distributing those SDKs.We went through the effortto implement that algorithmacross the various languagesso our customers didn't have to worryabout that complexity.
If you have to do this custom scheme,make sure you implement it for your customersotherwise they're never going to adopt it. The next best step if you don't do thatis probably Oauth 1.0a.
Finally, I don't have a whole lot of timeto talk about this,but use API keys instead of usernames and passwords.There's a lot of reasonsdealing with enter P and variability and other things,but a really simple kind of takeaway is thatif you don't use API keys and your customers use usernames and passwordsto authenticate your REST API,the second they change their passwordany of their integration with your service softwareis going to fail. Don't use username password pairsfor authenticating REST APIs.You don't want their software to breakwhen they change their password in your system.It's a bad idea.
To round out that part,we'll talk really briefly about 401s versus 403s.401 Unauthorized is kind of unfortunatebecause of the name,but it really means unauthenticated.It means the server says, "In order for you to access this resource,you need to provide valid credentials.I need you to authenticate with me.I need you to prove your identity."That's really what a 401 Unauthorized means.
403 is the actual real unauthorized kind of codeand that's really a Forbidden.The server is saying, "I get it.I know who you are, you've proven to methat you are who you say you are,but sorry, you're still not allowed to access this resource, it's protectedand you don't have thesufficient access rights to see it."Just make sure you understand 401 versus 403and when it's appropriate to use those things.
HTTP Authentication Schemes.We probably won't cover this really now,but it's a simple handshake.The server issues a challengevia the WWW-Authenticateand the client if it understands itcan submit what the server wants via the authorization header. Pretty simple stuff.If you have questions about thisfeel free to ask at the end of the presentation.
We talked about API keys.I'm kind of going to skip through thatin the interest of time. We have some pretty good blog articleson the Stormpath websiteabout why API keys are good,why they're beneficial.There's probably a good solid 6 reasonsas to why this approach is betterfrom a security perspective over other techniques.Feel free to check that outor ask me after the presentation.I'll be happy to fill in the gaps.
IDs.We've talked about this just barely.IDs should be opaque to your end users.They should be embedded within the URL,and the URL effectivelyis the new ID for modern REST JSON web services.
The client shouldn't ever have to knowhow to parse a URL or how to append an IDbecause if you give them a fully qualified URLevery time as your identifier,all they have to do is execute an HHTP requestin order to get access to that data.
They don't need to know where it goesas a token within the string or any of that stuff. They should be opaque.Clearly they have to exist for your benefitas an engineering team on the server side. But clients should never have to know about them, they shouldn't have to parse themand they shouldn't have to worry abouthow to specify them.
Of course, they should be globally uniqueespecially if you are offering a larger scale service. Sequence generatorstypically have to execute within a single processor you can segment it upand try to deal with segmenting IDs across a cluster. It's a lot easier if you use things like a UUID algorithmso you don't have to worry about contention on machinesif you get a sufficient load.
We actually have something that we use at Stormpath, it's not URL64, we call it URL62.It's similar to Base64 encoding,but it uses 62 characters that are URL safe. You could take an ID that we generateand drop it in a URLand it will be safe at all times.It basically does the same thing as Base64, it just uses 62 characters in the codec. That's nice because you want to keepURL friendly characters.
The less customers have to worry abouthow to decode URLs just to access a resource, the better.If it's just a drop in kind of ID or URLit makes their lives easier.
Method overrides are kind of interesting.I'll cover through this briefly.Some HTTP clientsdon't support DELETE and HEAD and PUT.Most of them, or especially the older ones,support only POST and GET.If you want you can provide an _method=DELETEor _method=PUT control query parameterand then the server can look at thatand says, "Okay, I know you're sending in a POST,but I'm really going to treat this as a DELETEfrom that point down in the request chain."
I should note that this has to be over POST.It's not okay to do this over GET. If you only support GET or POSTit's not okay to do this over GETbecause this might not be an idempotent operationdepending on what you're trying to achieve.
Caching and concurrency control.There's ways to support things likecaching and optimistic locking.For example, maybe when you create a resource,you can specify an ETag header to the clientand that client says, "This is the unique identifierfor this particular version of a resource." The client can then later send in a request to the server saying, "Give me some dataif what you currently have doesn't matchthe thing that I'm specifying." And then the server can say, "No.The version I have is the same one that you haveand therefore you don't need anything.I'm going to give you back a 304 Not Modified."
The response payload is much, much simpler.It's reduced, it's a minimal amount of bytesas opposed to a full HTTP responsemight have been. This is important for performance and efficiencyof caching servers,especially proxy caching serversthat you might have in your infrastructure.
Maintenance.If you have your REST API up and running,you've got all these best practices in place.What are some of the concerns that you might havefor maintaining the API? Use HTTP Redirects.It's expected that every client should knowhow to handle a 302 Redirect.If you need to change the URLsof where things reside,it should be expected to be okay to move them to different locationsas long as you automatically redirectthe old request URLs to the new locationsof wherever they're stored.
If you find yourself refactoringor you're cleaning up your serviceand you feel that things should berepresented in different locations,it's okay, use HTTP Redirects.Your clients can still get to their data.
Also, use an abstraction layer for your endpointsif you have to migrate functionality. Maybe the clients always interactwith a small thin interactionor a small thin layer of codethat knows "They're accessing legacy functionality.I'm going to direct it over to this kind of controller over here,or I need to have this handler over heretake care of this newer more recent request." Create an abstraction layer in your REST APIand then you can make those redirections in your codewithout ever affecting your clients.It's a really nice benefit.
Finally, again I know this might not be digestibleby a lot of people who aren't developers or they don't have a computer background but if you can, use custom media types.This is really the heart and soul of RESTand of course Hypermedia.These are best practices. Use them. They exist. That's why they were defined as a protocol. And you'll find, hopefully,as people started adopting RESTand they understand these principles more and morethey'll be more useable by many other peopleand then the benefits will start to increaseas everyone understands these things.
That's all I have for you.Thanks for the time.Again, we're Stormpath.It's free for any app.You guys can try it out.We'd love your feedback.We are, like probably many people here,a start up in the Valleyand we love hearing customer use casesand feedback and advice.If you have anything for uswe're definitely open to suggestions. And give it a shot.Thanks for your time.
Q: What are your thoughts on URL templates for token substitution?
A: You mean URL templates for token substitutionfor your customers of your REST API?I think it can be usefuldepending on the web framework that they're using.
The idea though is that unfortunatelyI think the design of those systems,specifically some of the REST architectures,like Ember has got a pretty good design,but I think it supports token substitutionfor IDs and whatnot.I think they do thatbecause it has not been clearor well-designed by so many other systemsthat they force this knowledge of IDs on youthat these frameworks have to support that notionbecause so many other servicesexpect you to know about IDsand where they reside in the service.
As you saw in my examples,if you're HREF and every resource that comes backis a fully qualified URL resource or resource URL, they never need to know about IDs because they will always have a handleto go execute a request with.
Again, that best practice is not implementedin a large majority of servicesso you have a lot of frameworksthat have to support template based mechanisms.
Q: How do you respond to requests when there is a large amount of shared or global states? For example: You've created an application to manage employee shifts, and would like to properly implement a REST API.
A: It might sound contrived, but it is possiblemost of the time I've seento represent those things as nouns.For example, there could be a shift objector there could be a shift plan objectthat contains metadata or attributesabout that particular planthat you could submit to the serverand the server can inspect thatand then asynchronously go offand trigger various things.But the point is that most of the timeyou can represent that as a nounand do a POST and then represent that informationin the context of a single request.
Also equally important in that kind of paradigmis to make sure that it's coarse grained. You want to be able to provide a lot of informationbecause maybe what's it's in a shift plandiffers over a couple monthsas you're developing things.
If you make it coarse grainedand you create a noun that represents that information,you can start to represent behavior as nouns.
I'll give you a really good example of that. In Stormpath's API,whenever a client needs to log in,whenever one of our customershas their own end userthat needs to log into their apps,clearly that's a behavior. I am authenticating with a service.You might not immediately think of a nounthat represents that use case,but it was actually easy for us to do.
We called it a login attempt.It's a login attempt, it's got properties, there's a date, there's information that has been submitted. And so for us,that behavior is encapsulatedas a login attempt resourcethat is posted to the application resource.That creates a collection or childrenof the application,of all the login attempts that happenfor that particular app. Again, it might feel contrivedbut it's amazing thatwhen you go through that process,you start to clarify and clean things upinto a coarse grained naturethat actually simplifies things over time.
Q: How do you respond to GET requests that might require large amounts of processing or have long response times?
A: Just to be clear, the question is: Sometimes GETs,HTTP GET methods can take, depending on the service,a really long time to respond.Maybe some return in a couple milliseconds,maybe another GET, if it was synchronous would return in 10 hours.
The way I would actually model that in a REST APIis that I would have a request objector a submission object like a report requestor report submissionthat is then posted to some collection endpointthat can receive that data. That will return a 200 OK indicating, "Yes,I have received the request, I'm going to go start working on it.Here is the location backvia the location headerof where you should checkwhenever that thing's going to be finished."
And you might in the response body sayas part of that response, "Hey, this information is going to take a really long time.The expected duration is 10 hours.Please go to this URL when it's finished." And then the client can pullvia whatever mechanisms it hasto figure out when that thing's doneand then try back to pull that information.
Q: How do you migrate APIs across HTTP methods? For example, when you need to change an API call from a GET to a POST?
A: Yes.Well, you can actually.You can do a redirect. This is actually where media types might be beneficialif you had technical clientsthat could understand this stuff.
As part of the media type in the response to the GET,you could actually provide additional informationthat tells them, "Hey, this resource is over hereand I want you to execute a POST."Unfortunately because you're kind of doing abackwards compatible migration mechanism,something like that's probably not in placeor you can't just tell them immediatelyto go check the new URL.
That being said,for me, I'm a big stickler in backwards compatibilityand not breaking things that already work,so I would keep the GET workingin that particular scenarioand then maybe support a query directiveor HTTP header that can be inspectedthat overrides the default behaviorto return or** redirect** to a new resourcethat tells them, "Hey, I need you to go pull thisin 10 hours or something like that."
I would be hesitant to break existing clients,but I would provide a mechanismthat allows them via header or a query parameterthat new functionality can be implementedand then you can go redirect based on that.
Q: Are there any downsides to using UNIX timestamps?
A: Any downsides to using UNIX timestamps?The biggest downside that I can think of is, well there's 2 really, your customer processoror whoever is consuming the information â€” might be a Windows platform.Not a big deal. Windows has the ability or the librariesthat it can call in order to figure that out.The other thing too, is thatmilliseconds since epoch I think can only go upto like 2024, or something like 2025,or 38.
If you have to represent timestamps,which kind of sounds weird. That's only 20 years away, 25 or 26 years away,it's not that far away.If you have data, maybe it's a construction planning site mechanismor something for like the Olympic committee, there are dates that aren't too far awaythat surpass that time. So 8601 never has that problem.You'll never have to worry. What did we have in 99? There was the crunch for the rollover.You might see the same thing in 2038.
Q: What value does adding human readable messages to software that is already in production?
A: For the scenarios where the app is already builtand ready to go,it might not have that much benefit.But that being said,there are always cases where bugs might come upor maybe something occurs.
Maybe there's an internal server errorwith a new error codethat represents informationthat you as a developer haven't seen yetwhile you were developing your app,and that extra information can help a debuggeror a person involved in debugging. "My app was perfectly fine for a year and a half.Now all of a sudden it started failing, why?"Granted it might be the server's faultbut at least you'll have that additional informationto help figure it outand maybe come up with an immediate fixor just get a fix out the door.
For us it's more of a best practiceto give as much information as humanly possible.But you're right, most of the time if an app you built is totally stable,it's never going to need that additional information. But your hurdle at least,I don't know about your particular use case, but if you're a public SaaS company the biggest hurdle to adoption is understanding APIsand being able to consume them easily.
It's much better to front load the workin providing human readable information to solve problemsbecause that is way more importantto your business modelthan not having that information.
It's really a holistic view of that stuff,but you're right.It doesn't really help in technical scenariosafter the service is well established.
Q: If you have a resource with a very large child collection do you paginate the child collection or provide a URL to the child collection?
A: That's a fantastic questionand we actually do both of those things.For example, in our API you have an account that,or actually it may be a better example as a group that has a collection of accountsthat's a collection resourcethat's referenced by the group. There could be thousands of accountsin that particular group.We actually support even in the expand query parameterthe ability to specify additional offset and limit valuesthat pertain just to that particular propertythat's being expanded.
We do support pagination for nested collectionsor referenced collections within a resource.That has been a real benefit to our servicebecause customers can still page datathat might be more numerousthan just the initial resource.
We do that in interesting ways.If you're interacting with an instance resource, ?offset=whateverlimit=whatever does not apply to that particular resource.What we would actually do isexpand=groups(limit:25,offset=whatever)and that means that any of those attributesin the parenthesis set applyto that particular attribute of the resource.
That's how we solve it.There's no best practice that I've seenabout how to address that.We thought that was particularly elegantbecause it was easy to readand that's just we've solved that problem. And it works well,and we haven't seen any issues with it.
Q: How would you model a password reset with a parameter for your users?
A: That's a fantastic questionand I like it because you could actually goto Stormpath's API and do that today.If you actually tried it outyou would see how we do it. The way we do it is the customer sends us â€” off the application resourcethere is a collection resource calledPassword Reset Tokens â€” they submit a POST to that collection endpoint. Again, when we submit a POST to a collection endpoint it typically means you want to createa child of that resource. We're actually creating a new token resource.Part of the request bodyneeds to specify a username or an e-mail addressas part of the payload.
When we get a submission to that token's endpoint,we inspect thatand then we actually create a resource with the tokenthat goes back to the customer. Then that can go in the emailthat they send out to their customer,or we actually can send out the email on their behalfso they don't have to deal with that particular use case. It is a behavior, it's a verb,but we have "nounified" it.
Again, it might sound contrivedbut oddly enough, the end goal is to actually simplify thingsand it's almost deceivingat how much something that simplecan actually have a good impacton a stateful, or at least a nounresource based REST API. We nounified it and turned used POST to return back a tokenand then they can do whatever they wantwith that token.
Q: Is it common to see RESTful resources that only accept POSTs?
A: Yes, actually for usbecause of security reasons,you can't do a GET on that particular resource.When you create a token,we'll give you back the response code,a 200 OK, and the actual HREFof where that resource resides. But if you executed a GET on that resourceit would give you back an errorstating that you are not permitted to see thator whatever because there's security information associated with that. And that's totally legit.
REST doesn't say you have to support all the methodsfor each resource.It just says that under certain scenariosyou have to adhere to certain behavior. You don't have to support CRUD for everything.
We end up not supporting it for that,and the reason why is thatyou don't need to look at that resource.In the password reset scenario specifically,when the end user gets their emailand they click a linkand it's got the token inside of it,**all **our customer has to dois just relay that token to us via a POSTand then we process itand then delete it on our system.All of that is automated.They never actually need to look at the data.It's just a passing directive to usand it works great.
Thanks guys!My pleasure.