John Sheehan on API design for humans

John Sheehan is co-founder and CEO of Runscope, a brand new startup based in San Francisco focused on API tools. Runscope is building the future of developer tools for API-driven applications. John is a pioneer in API platforms having worked at both Twilio and IFTTT. He’s a frequent speaker and thought leader on the business and technlology of APIs. He also runs the API Jobs site, curates API Digest -- a weekly list of the best API links and co-hosts the Traffic & Weather cloud and API podcast.

API Economist: Your article, APIs are Dead, Long Live APIs, created a little bit of a stir. What point were you trying to drive?

John Sheehan: I think it created a stir because people don't want to think that something so new could be 'dead.' I was hoping to make the point that they're far from it; that the amount of attention that the consumer APIs out there get is disproportionate to the amount of total API traffic they handle.

I spend a lot of time talking with people, finding out where they actually use APIs, where they apply it most. The two things I hear most common are, 'Yeah, we might use a social API here and there,' or 'Yeah, we might use an infrastructure API like Stripe, SendGrid, Twilio, or Parse' but the vast majority of our traffic is powering our mobile apps or internal to our company and never exposed externally. That's where most of the uptake has really happened in the last year.

These private APIs have taken the model that these big, public, high‑profile APIs have made publicly, and started applying that internally, and getting just as much benefit from it. We did a survey with ProgrammableWeb a couple of months ago, talking about where all the API adoption was and it showed that the vast majority of the traffic lives on internal APIs.

I call them "Dark APIs" in the article. I would estimate, and this is my own guess at what the number is, that 90% or more of the traffic out there is on APIs that never see the light of day. Again, mobile apps or internal API stuff.

API Economist: What are the benefits of internal APIs?

You get benefits that you maybe don't anticipate. For example, at the API Strategy & Practice Conference, a group from Target gave a presentation about how their internal API usage has started to change how their business units interacted with each other. They built an API traffic dashboard that gave them insight into the nature of their infrastructure that they couldn't see before. They're gaining new insight into where data is flowing within the company without having to do anything extra because they can watch that API traffic. Once you can see easily see where all your data is flowing you can extract business intelligence from it as well. The Target team’s presentation demonstrated just how important it’s become to them.

This is a great example of the sort of internal API uptake that is really driving the explosive growth of API traffic. While the public APIs get all the attention, they can be a distraction from where the real activity is happening.

API Economist: Speaking of the API Strategy & Practice Conference, you gave a presentation called, APIs for Humans. Where is API design headed?

John Sheehan: As people become more familiar with the things that are frequently called REST APIs, we'll avoid that whole debate on what is and what isn't REST. People are becoming familiar with simple HTTP APIs where they’re using simple URLs, readable URLs, and HTTP methods over those URLs. As this has taken hold in the mainstream developer market it's starting to also become the way APIs are designed internally. It’s moving away from big, heavy, service-oriented architectures and into light weight REST-based architectures that are equally distributed, equally valuable, but a lot less technical overhead.

As we progress towards simpler and simpler APIs, we start to open up new use cases. There's a way that normal people end up using API data that the API producers don't necessarily consider. An example of this would be something like, IFTTT (If This Then That), where I used to work, taking all of these really simple public APIs and letting people start connecting them together very easily.

If you take this model and apply it to internal APIs, we're soon going to see all these new simple APIs being put to work by end users using an IFTTT-like automation tool. Developers will be able to focus more on creating better APIs and less on any given integration leaving that up to the people closest to the task at hand.

API Economist: What type of API design use cases do you see?

John Sheehan: There are three really big use cases that normal people are interested in. One is sharing data. It’s taking it from one place and putting it in another, whether that's reposting or just taking something that you do on one service and posting it in another. Or, taking something that you do and then giving it to someone specifically, basically just repurposing of content for another context. That's sharing.

Archiving is really big. When something happens just keep a record of it. It's sort of like sharing, not person to person, but person to machine. At IFTTT we saw a lot of archiving - we had a channel for the Withings WiFi smart scale, so people would take their weight and automatically save it to a Google Spreadsheet. Now they have an automatically updating weight log that lets them do all sorts of analysis, regarding their weight. 

People like to do a lot of archiving for social services. One of the most popular IFTTT recipes is 'Save your Instagram Pictures to Dropbox.' People are getting a little bit more skeptical about social services now and how they want to handle their data, so they want to have it in two places. It's really common for people to take a picture that was uploaded on one service and save it into a file storage like SkyDrive, Box or Dropbox.

The biggest use case, by far, is notifications. Just tell me when something is new. Make it easy for me to determine, "Hey, I've got a new email that matches a certain search string" or "Hey, there's a new blog post or new article or new something from this website that matches some criteria that I really, really care about, and I want to know about it immediately."

What you can do is design the APIs to match these use cases. What a lot of APIs don't do is make it very easy to see what's new. The best thing you can do, hands down, to make your API more approachable to end user cases is to make it easy to see what is new.

One of the best technologies for this that actually has been in the news a lot lately is RSS. It has probably the most uptake amongst normal people. My wife who has never consumed an API in her life has a bunch of RSS feeds in Google Reader. She's essentially using APIs from all of those web sites to find out what is new. That's the use case she cares about the most, and RSS is the API to get that data.

The biggest thing about RSS is that it's designed by spec to give you the newest things first. It makes it very, very easy to find out what is new. If you take that concept of making it easy to see what is new and apply that to other API ideas, you can start to make notifications over other data more interesting.

API Economist: So, RSS for e-commerce and social media?

Exactly…instead of having to pull up a customer and then pull up all their orders to get that feed, you can have a global orders feed that has all the orders across all the system. Now the developer that wants to integrate any sort of action on a new order doesn't have to check every customer. They just get one single feed that tells them what is new on the orders.

An example from a social API would be what Facebook does with photos. It used to be if you wanted to find out when somebody uploaded a new picture, you would traverse all the albums on their account and then all the pictures in all the albums. That's a lot of requests. That could be thousands of requests just to find out when something is new. They recently added a new API endpoint that basically just gives you all the new photos for a person across all their albums. You don't have to do that traversal anymore.

API Economist: What’s your bottom line on API design?

John Sheehan: If I could impress anything upon people, when you're designing APIs, think about the notification scenario and exposing information across groups more easily to determine what is new so that people can get the alerts they want.

API Economist: What advice do you have for API providers that can help them drive adoption of their APIs?

John Sheehan: From a technical standpoint developers like simplicity. They tend to approach something like this: “I have a problem that needs to be solved. This API looks like it can solve it.” If you then get them over the gap of actually helping them solve their problem in a few minutes, you've won. You've got them onboard. They are not going down these technical rabbit holes trying to figure out “why doesn't this return what I'm expecting?” or “why doesn't this do what I'm expecting?” Simplicity will always help somebody solve that problem faster and that should be 100% the goal of your API.

At Twilio we spent a lot of time thinking about that first five minutes. Making your first API call, making your phone ring within a couple of minutes of signing up, giving you the proper examples to get started in just a couple of minutes. Let’s take the quick start tutorial. All it does is make your phone ring. Its sole purpose is to prove to you that the service works as quickly as possible. Anything that increases cognitive overhead in that first five minutes is a huge barrier to entry and will be a detriment to adoption.

If you ask yourself the question, “How can we make our API better for a developer?” you have to think about that first five minutes. Give it to a developer and see where they stumble in the first five minutes and solve those problems first. That's really low hanging fruit that will pay off huge dividends.

API Economist: When did you start working with REST APIs?

John Sheehan: Around the end of 2009, right after Twilio launched is when I really got into it. They had solved a problem that I was trying to solve and had no luck with the old fashioned way, doing it myself. And so I started playing with their API. Going back further, the first API integration I did was in 2002 or so with the Weather.com API for a sports league website I was building. I wanted the weather displayed on the site and I was calling their API to get the latest forecast and current conditions.

API Economist: What insights did you gain from your time at IFTTT?

John Sheehan: At IFTTT we had over 60 API integrations. That's a high number for most companies. You start to see a lot of different interpretations of the same specs, like OAuth or even HTTP and the problems those differences can cause.  

There was one API provider, a very high profile one, that would get slow at a very consistent time every other week for an hour. The response times would drop to 10 or 15 seconds. The calls would still work though. It would still finish, but it just took forever. It was very difficult for us to troubleshoot where that was. Is that in our code? Was that our servers? Was it our infrastructure provider? Was it the API? Was it the API's network provider? Slowness is very hard to pinpoint. Seeing that there was a lack of tools for figuring out those kinds of problems was probably the biggest thing that opened my eyes at IFTTT.

API Economist: It sounds like there is a need for forensic API tracing tools.

John Sheehan: Exactly. Once something gets on the wire you're mostly blind to it. That's very different from traditional software. In traditional software all of those wires are on your server talking to each other directly. You can see them very easily. But when you outsource large portions of your code to other providers you lose that visibility. People think of an API as a service dependency. I think of it as a code dependency. It is code that I didn't write that's running on someone else's server. Now I've got my app that's literally code on my server and code on someone else's server and I have no insight into those wires anymore. So how do I get insight into that? How do I see something going back and forth in these distributed systems?

This problem inspired me to start Runscope. And I will leave it at that for now. [Laughs].

API Economist: As a former Developer Evangelist, what advice would give API providers regarding developer outreach?

John Sheehan: Developer outreach at its finest focuses solely on enabling developers. Sales, marketing, business development and support are byproducts of that. Focus on helping developers to be successful at something important to them. The last thing somebody wants to hear when she is solving a problem is a sales pitch. Nobody wants to be given a solution to a problem they don't have or that doesn't quite fit. You don't care about anything else when you have a problem to solve. Help them solve their problems and it will pay off for you in the long run.

API Economist: Time for one of our favorite questions at The API Economist: what are some of your favorite mobile devices and mobile apps?

John Sheehan: I've become a huge Apple fan boy. For the longest time I was a Windows user. A couple years ago I switched over to a MacBook after I had been using an iPhone. My iPhone and I are essentially inseparable. You can ask my wife about that. [Laughs]. It is the computing device I probably use more than any other. Even when I'm at my computer, I'm frequently using my phone for smaller tasks like quick emails. I use it more than anything else.

I love my iPad too. Every time I'm using it I just hold it in my hands and think, "I can't believe this exists. This is the most amazing piece of technology ever created." I'm just stunned that it even exists. It blows my mind that I can fly from here to Amsterdam and use one device to watch hours of video, read books, write a blog post all on this thing that's a half inch thick and on a single charge. It's mind blowing to me that the iPad even exits.

My favorite apps would be Tweetbot for Twitter, which is boring because it's just a regular Twitter client, but it's better than the Twitter one. For App.net, I use an app called Riposte, which has some subtle but innovative UI touches. If you used App.net you should get this app. It's probably the best social stream app for any service.

I also really am horribly obsessed with Carcassonne on iOS. It’s easily the most beautiful app I’ve used.

API Economist: John, thanks for your time!

John Sheehan: My pleasure!