Investigating Web APIs on the World Wide Web

The world of services on the Web has traditionally been limited to “classical” Web services based on WSDL and SOAP. For quite some time now, though, services on the Web are increasingly marked by the domination of Web APIs, which includes, but is not limited to, RESTful services. Despite their popularity currently the development of Web APIs is not guided by standards, patterns, or guidelines. It is more an art than a science whereby Web APIs are more often than not solely described in HTML as part of a webpage rather than using an interface description language that could better support software development.

Lately, there is significant effort devoted to this “new kind” of services. A good deal of this work has focused on better understanding the architectural implications of the REST Representational State Transfer (REST) principles [1] as opposed to traditional Web services [2], or to better supporting their composition and invocation [3,4]. Most of this work, however, takes REST principles as a starting point despite the fact that on the Web many of the existing APIs do not strictly adhere to these constraints. Before we can make any significant improvement to current practices and technologies, it is thus necessary to reach a deeper understanding of existing Web APIs regarding for instance how they are actually described, what principles they follow and what drives their reuse. It is only in the light of these findings that we may then try and figure out how existing solutions, as opposed to ideal scenarios, could be enhanced and better supported. To this end we recently carried out a thorough analysis of large body of Web APIs [5]. This exercise has revealed interesting issues which I’d like to briefly highlight herein.

The analysis was carried about one year ago over more than 200 Web APIs randomly selected from Programmable Web. For each of these APIs we tracked general information (number of operations, number of mashups, etc), the type of Web APIs (RESTful, RPC oriented, or hybrid), information about the inputs and outputs (formats, parameter types, etc), invocation details and some further details such as the presence of example requests. The analysis highlighted some issues that came as no surprise such as the heterogeneity of Web APIs descriptions and the lack of detail in some of them. But most importantly, the analysis helped us identify some results which we believe are remarkable:

  1. The quality of the descriptions leaves a lot to be desired. For instance, only 28% of the APIs clearly state the data-type of their parameters, and only 60% state the HTTP method to be used (gladly 83% of the APIs provide examples of requests).
  2. Only about 32% of the Web APIs appear to be truly RESTful (48% are RPC-style and about 20% are  hybrid).
  3. The fact that a service follows REST principles does not seem to have any impact (positive or negative) in its reuse in mashups.
  4. The output format still clearly favours XML (85% of the APIs can provide the results in XML) as opposed to JSON (supported by 42%). Indeed as you may guess several services support both.
  5. More than 80% of the Web APIs require some form of authentication (in some cases only data modification operations require it).

Although certain aspects require further investigation, this analysis has already highlighted many issues that are indeed of particular concern for further automating or better supporting common service manipulation tasks such as their discovery, their composition or their invocation. Not surprisingly, using Web APIs requires a lot of manual work and yet they are particularly popular these days. The appealing features derived from applying REST principles have often been argued to be behind this popularity but our study indicates that (at least when we carried it out) there was no clear indication that this is actually the case and in fact most Web APIs don’t follow these principles. Is it their simplicity from a technology perspective then? Is it that Web APIs are a rather natural means for exposing functionality already existing behind Web sites? Is it just a matter of freedom?

Perhaps the reason for the popularity of Web APIs is a mix of all these factors. Although the answer may well not be just related to technological aspects, I believe reaching further insights in this respect may be of particular relevance for the future of Web APIs, and probably the world of services in general.  Hopefully we will soon have more to tell in this regard.

The entire analysis with more detailed explanations of the methodology followed, the terminology used and the decisions adopted can be found here.

Some References

[1] Fielding, Roy Thomas (2000), Architectural Styles and the Design of Network-based Software Architectures, Doctoral dissertation, University of California, Irvine

[2] Pautasso, Cesare; Zimmermann, Olaf; Leymann, Frank (2008), RESTful Web Services vs. Big Web Services: Making the Right Architectural Decision, 17th International World Wide Web Conference (WWW2008) (Beijing, China)

[3] Pautasso, Cesare (2009), RESTful Web service composition with BPEL for REST, Data & Knowledge Engineering, Vol. 68, No. 9. (September 2009), pp. 851-866.
[4] Curbera, Francisco; Duftler, Matthew; Khalaf, Rania; Lovell, Douglas (2007), Bite: Workflow Composition for the Web, Proceedings of the 5th international conference on Service-Oriented Computing, September 17-20, 2007, Vienna, Austria.
[5] Maleshkova, Maria; Pedrinaci, Carlos; and Domingue, John (2010), Investigating Web APIs on the World Wide Web, European Conference on Web Services (ECOWS), Ayia Napa, Cyprus