Designing a RESTful api for YaST

As we move YaST to the Web, we also want to give it a web-like architecture offering its functionality for widespread consumption.

So we started off by separating the client frontend (user interface) from the server backend (functionality) and settling for a RESTful api between them. We also decided on using Ruby on Rails (usually just named 'Rails') to implement both layers. Besides its widespread adoption outside of OpenSUSE, Rails is used in a couple of company projects like

• SUSE Studio
• OpenSUSE buildservice (frontend)
• OpenFATE

Rails comes as a natural choice as we can build upon a rich knowledge base with developers sitting (literally) next door. Rails was also one of the early adopters for RESTful and offers a nice encapsulation layer named ActiveResource.

ActiveResource ...

ActiveResource gives you an object-oriented interface for resources, hiding all the HTTP protocol or serialization issues from you.

Following the model-view-controller principle of Rails, resources are separated into their model, behaviour, and visualization. The resource model is expressed as a ruby class below app/models, the resource behaviour and basic lifecycle operations (create-read-update-delete) is implemented in a Controller class living below app/controllers, and app/views hosts the visualization files.

A device resource would be modelled as app/models/device.rb, implemented as app/controllers/devices_controller.rb, and viewed through app/views/devices. (Note that controllers are always plural, even for singleton resources.)

... and routing

To make resource accessible from the outside, routing information must be placed inside config/routes.rb. Routing assigns a URL to the resource, making it accessible from the outside and uniquely identifying it.

For the device resource, a map.resources :device call inside config/routes.rb would publish the resource as /devices, effectively offering a set of URLs and HTTP methods:

HTTP verb	URL	action	used for
GET	/devices	index	display a list of all devices
POST	/devices	create	create a new device
GET	/devices/1	show	display a specific device
PUT	/devices/1	update	update a specific device
DELETE	/devices/1	destroy	delete a specific device

(taken from RailsGuides)

Limitations of the routing scheme

This scheme of identifying resources via URLs works well for a limited number of resources which all have models below app/models and controllers inside app/controllers.

But it is hard to follow if you allow resources as plugins, which you cannot control and prevent them stomping on another plugins feet. Imagine a printer plugin offering devices and a network plugin doing the same. The /devices URL cannot uniquely identify the resource, a simple

map.resources :devices

routing is not sufficient.

Pluggable resources

To make pluggable resources work, you have to centralize the routing and restrain from defining URLs. This also matches well with the RESTful principle of hypertext driven APIs and narrowing the URL exposure window.

A resource plugin for the rest-service of YaST is identified by the interface it offers and the controller implementing it.

This is done with a config file in yaml format. It specifies the interface and controller as

 interface: org.opensuse.yast.printer.devices
 controller: yast/printer/devices

in a .yml file below config/resources. A resource plugin can offer any number of resources this way.

Interfaces are qualified names like org.opensuse.yast.system.language or org.opensuse.yast.system.users. Ideally, these names should match the prefix for the PolicyKit action controlling access to the resource.

Implementation details

To minimize name clashes, the controller implementing the resource must be inside a namespace. Rails supports this by allowing subdirectories below app/controllers and automatically wrapping controller classes within modules matching the directory names.

Thus a network/device_controller and a printer/device_controller can live side-by-side an be accessed as Network::DeviceController and Printer::DeviceController. You could even nest more deeply like Org::Opensuse::Yast::System::Users, but let's not overdo it.

Rails resource routing puts another restriction on URLs: Rails manages routes as a set of maps, describing URLs as a combination of controller (implementing the resource) and action. Thus the file name and path of the controller define the URL, you cannot name them independently. So the controller: value in the resource description directly affects the resulting URL. Different resource plugins should use different namespaces to effectively prevent name clashes.

Querying the rest-service

The rest-service only publishes a single url, namely /resources, where clients can query for the URL offering a specific interface.

This URL follows RESTful principles since a HTTP GET request returns a list of all resources. Its also possible to request a specific output format, either by setting the Accept header of the http request or by appending .xml or .html.

You can easily try it out by starting the rest-service and pointing a browser to http://localhost:3000/resources.

To find the right resource URL, the client just needs to pass the interface name as ?interface=name and extract the reference from the result. Stay tuned for a future post exploring the client side in more detail.

A closer look at REST

YaST, the openSUSE installation and configuration tool, is about to get a web based user interface.

The interesting part of this is the proposed service-oriented architecture. REST, Representational State Transfer, is currently the holy grail for managing resources (objects) using http.
REST is not a protocol but an architectural style making best use of the properties of http. And the Internet is the best proof that this style performs and scales well with a distributed client/server architecture.

Roy Fielding, one of the authors of the http protocol, has written a complete doctoral dissertation on this topic. For the purpose of this blog entry, seeing it as a lightweight alternative to e.g. XML-RPC or SOAP is sufficient.

How does it relate to YaST ?

As Stefan has explained on his blog before, a web based YaST will be splitted into client and server parts. This forms a three tiered architecture, where the server runs on the managed system and exposes a REST-style API to access YaST functionality.

Getting this API right in terms of extensibility, flexibility or conformance to REST best practices is an important design goal.

Now whats good REST style ?

There is no fixed list of requirements for REST or a conformance testing tool. Designing a well behaved REST implementation is mostly about learning from others experience and follow common practice.

Browsing through popular REST related bookmarks at delicious.com gives me a much better signal-to-noise ratio than Google. And it pointed me to a couple of useful references for the do's and dont's when planning a REST-style architecture.

1. REST APIs must be hypertext-driven
2. Versioning REST Web Services
3. Common REST Mistakes

Hypertext style

The first is the most interesting (IMHO). Its about exposing only a very limited set of explicit URIs. This also prevents hard-coding them into the client but let the client query the server instead. So you start from a single URI and hop from there (thats what hypertext is all about!) to the right resource. Thats like moving from node to node in a state diagram.

Versioning in media types

When doing API versioning on the web, one might be tempted to do an all-or-nothing approach and embed a version specifier (like .../v1/...) into the URL. The second link above explains why this is a bad idea and comes up with a better one: media types. The client can tell the server how to format the response to an http request by setting the Accept field in the http header. Its like calling a function and telling it the expected return type.

And REST explicitly supports this style as it does not make any assumption on the representation of a resource. Client and server are free to agree on the actual format. Using XML for object serialization is a smart move as it allows for extensibility. Clients just pick the xml tags they need and ignore all others.

Stateless

REST, being based on http, stipulates a stateless protocol. Doing stateful REST is listed as one of the typical mistakes in implementing REST-style.
Considering the proposed YaST web architecture, this means keeping state (session) information in the client and not in the server.

More recommended reading

• How to Create a REST Protocol
• Explaining REST to Damien Katz
• Roger L. Costello: REST