RE: [obix-xml] Asynchronous Web Services Summary

Title: Asynchronous Web Services Summary

From  http://weblogs.cs.cornell.edu/AllThingsDistributed/archives/000388.html
Events, Eventing and Notifcation

Three web-service specifications have been published in the past 6 months, 
that deal with event notification, a fundamental paradigm in building 
distributed applications. HP published ws-events 
2.0 in July of last year, and in the past month we have seen the publication of 
ws-eventing 
(MS, et al) and ws-notifcation 
(IBM, et al). I have made my first past over these specs and here are my initial 
thoughts. Most of these are just notes trying to distinguish the different 
systems from each other. 

Event notification, and its big brother publish-subscribe, are technologies 
that are part of every distributed systems developers toolbox, and they been 
been for many years. The technology predates web-services by two decades and 
there is a wide range of free and commercial software systems available. Event 
notification and publish-subscribe are often used in inter-exchangeable manner, 
but me for they are different:

  
Event notification is a contract between two components where the 
  event-sink requests the a-synchronous delivery of specific events from an 
  event-source.

  
Publish-subscribe is a message (event) routing paradigm, where 
  messages are being delivered to receivers, not based on specific addresses but 
  on a set of criteria related to message metadata or to the message content. 
  The publishers are in general unaware of the receivers, and the selective 
  forwarding and delivery is performed by extensive middleware systems. 

I know this is a very simplistic view, but I am trying to make a clear 
distinction between the two technologies. Event based programming is an 
essential tool in building a-synchronous applications, which helps to build 
scalable systems in the distributed and in the non-distributed case. One can see 
event notification as a restricted form of the publish-subscribe paradigm, where 
it appears as if there is a direct interaction between the publisher and the 
subscriber. This simple form is not a bad thing as a large collection of 
applications can be build using the simple notification pattern, and they do not 
need the overhead of a full blown publish-subscribe system. Of course a simple 
event based notification system can be used as a building block in a more 
elaborate pub-sub system.

There is a class of middleware systems, that includes broker technology to 
provide routing functionality, yet still claim to be event notification system. 
One of the litmus tests I often use for making the distinction between the two 
paradigms is whether there are dedicated interface specifications for 
publishers, and for the broker management system. In a simple event notification 
system for example there are no facilities to indicate that events need to be 
persisted, that there may be causal relations between events, how to handle 
periodic events with temporal expiration, etc.

If we have a look at these three specifications that all describe very 
different event systems, we see the following:

ws-eventing

ws-eventing is minimalist interface to an event system. It builds on 
the ws-addressing specification and relies on the wsa:EndpointReference 
construct and specifically uses the wsa:ReferenceProperties to 
identify the subscriptions. There are only a few interactions defined: 
subscribe, renew and unsubscribe, to which the event-source 
needs to respond. These requests do not need to be generated by the service that 
will consume the events, it can be done by a 3rd party that controls the 
source-sink associations. The subscriptions are leased, and need to be 
periodically renewed. When a subscription ends, the event-source needs to 
generate a subscription-end message, which is send to an endpoint specified in 
the original subscription.

The specification only deals with setting up and tearing down of the 
subscriptions, but does not specify what format the events should be in. If any 
reference properties were specified in the subscription request, then to will 
show up again as properties in the wsa:To section of the notification. Also the 
format of the event filtering specification, if used, is completely free.

ws-events

ws-events is a more extensive specification of an event notification 
system. It does not use ws-addressing, and is http specific. It has the 
subscribe/renew/unsubscribe interaction we already saw at ws-eventing, although 
they are named subscribe/extend/cancel. The notification message is encapsulated 
in a notification envelope.

An important part of ws-events appears to be the event-type discovery 
mechanism. A subscriber can query the system for the available types, for the 
definition of specific types and for which of these types have been 
instantiated. It can also subscribe to be notified when new types become 
available. Subscriptions are made to particular event types and there is no 
facility for the specification of subscription filters that would work over meta 
data or the event content.

The events can be delivered a-synchronously to a URL specified at subscribe 
time or they can be retrieved synchronously through a pull mechanism. In the 
pull operation the request can contain constraints such as all 
messages-since-messageID, messages-since-date, or message-range. The result of a 
pull operation is a notification list.

ws-notification

ws-notification is the more extensive of the three. But even the 70 
pages that the specs now runs are actually on the low side, I expect that this 
specification will easily grow to 150-200 once more detailed examples of the 
different scenarios are added. The spec uses some of the idiom from the ws-resource 
frameworkk but is relatively self contained, and I would not be surprised if 
some of the idioms get replaced over time to make the spec completely 
independent. At 30,000 feet a ws-notifcation based system has a 
remarkable resemblance to the Corba Notification Service, but this is not 
surprising as there are only a few general accepted pub-sub/eventing interface 
strategies.

According to my earlier classification ws-notification describes a full-blown 
publish-subscribe system, not just an event notification interface. It clearly 
identifies publishers, brokers and subscribers, even promises support for 
federated brokers (although there is no specific mention in the spec on the 
intra-broker interaction), and allows for complex interaction between the 
different components (e.g. on-demand activation of publishers). The abstractions 
are chosen such that direction interaction between notification producers and 
consumers is possible, or can be accomplished with the help of a broker. 

ws-notification uses a topic based publishing mechanism. This means that 
notifications are organized into topics and these topics can be organized into 
hierarchies where a parent topic is the aggregation of all child topics. A 
subscription to /system/motherboard will receive notifications published to 
/system/motherboard/fans/CPUfan. 

A subscription request issued by a consumer can contain 3 constraints:

  
A topic expression. This expresses which topics you are interested 
  in. There is a specific topic path expression language you can use to specify 
  the topics.

  
A precondition query. This is an Boolean query that must evaluate 
  to true when executed over the properties of the notification publishers. The 
  language is independent of the spec and could be XPath. You can use this to 
  select a subset of the notification publishers for your purposes. For example 
  in a datacenter control system you could use this to select all quad processor 
  machines.

  
A selector query. This is a Boolean expression that is run over the 
  notification message, and must evaluate to true if the individual message is 
  to be delivered to the consumer. Can be any selection language.

For example a subscription could request to receive notifications to be that 
are posted to both the /system/network/tcp-connections and 
/notifications/alarms/critical topics, with a precondition of  
'server type == Tier #1' and a selector of "#syn-sends > 
30,000". Of course these constraints require the knowledge of the topic 
spaces in use, the properties of the producers and the  structure and 
content of the messages.

This spec provides for notifications to be delivered 'raw', similar to 
ws-eventing or encapsulated in a notification message, similar to ws-events. The 
interaction is always a-synchronous, even though there is a facility to retrieve 
the last notification produced for a particular subscription from the producer. 
A consumer can pause and resume a subscription, but there is no leased renewal 
based mechanism, nor have I been able to find the equivalent of an unsubscribe 
interface.

Overall the spec reads OK, but I feel that the second part of the spec, 
section 7 and higher which deals with the interfaces and the messages, can 
benefit from a more coherent discussion, and good examples. This part feels as 
if it was rushed to the printer.

Some final remarks

ws-eventing and ws-notification describe interfaces to systems that appear to 
be at opposite end of the spectrum. ws-eventing describes a simple mechanism for 
a-synchronous event delivery, ws-notifcation describes the interface to a system 
that has to potential be a full-blown pub-sub system. I see little overlap 
between the two systems, asa they are likely to target different audiences. 
ws-events seems to be caught in the middle of this, and the question is whether 
it will survive the corporate politics, and whether it should. The participation 
of HP in ws-notifcation could mean that ws-events might be replaced.

I do feel that these specs are very preliminary. They will change 
significantly before they are cast in stone. The feedback cycle with workshops 
that both groups have started should be interesting. I am looking forward to the 
moment when interface issues such as flow-control on notification delivery will 
be discussed. I will go to the ws-eventing 
workshop this week at Tibco but have to miss next weeks ws-notifcation 
workshop at HP, as there is a ws-transaction feedback workshop at the same 
time.

I do have some criticisms on both specs but will keep them to myself until 
these workshops are over and i have had some direct feedback.

One issue that I have not touched on at all is the interaction between these 
specs and WSDL. You should read Steve Vinoski's integration 
column in the March/April issue of IEEE Internet Computing for more insights in 
that issue, and for his views on some of these specs.

  
-----Original Message-----
From: Doug Ransom 
  [mailto:]
Sent: Tuesday, September 21, 2004 
  10:10 PM
To: OBIX XML
Subject: [obix-xml] Asynchronous Web 
  Services Summary

  
I think key questions to be answered: 
  
    
are there use cases for request-response 
    services, where the service response may not be immediate? 
    
are there use cases for one-way web 
    services.  If so, for subscribing and unsubscribing 
    programmatically? 

  
I think choosing between  WS-this or WS-that 
  is not important at this time - only deciding if asychronous stuff is 
  on. 

  
The most common web service today is 
  request/response, where the SOAP request piggybacks on an HTTP Request and the 
  Response piggybacks on the HTTP response.  These are basically 
  synchronous, although to scale, responses are often processed 
  asychronously.

  
A common asychronous scheme is still 
  request/response, but the response may not be immediate (and therefore not 
  delivered on an HTTP response),  A request could be delivered via HTTP 
  POST or SMTP (or BEEP, or whatever protocol you like) message to a service, 
  the service might reply 10 minutes later with an HTTP Post, SMTP message etc. 
  to the sender (or a URI specified by the sender).  These can help both 
  clients and servers scale well and have lightweight 
implementations.

  
In terms of designing interfaces, the same 
  interface (as specified in WSDL) could be used for either approach, and 
  vendors could choose which to implement.  A specification of which 
  protocols are supported for which OBIX interfaces may be warranted (i.e. the 
  discovery service should always be supported through the standard HTTP 
  binding, the historian always through SMTP).  

  
A one-way web service may be useful for some 
  applications (i.e. data push of new information), and might very well be 
  useful in obix (i.e. receiving a message every half hour about the status of 
  something).  The configuration of the address data is being sent to may 
  happen out-of-band (i.e. configuring a peice of equipment with a config file 
  or user interface).  It is nice and simple for implementors.

  
The WS-Eventing and WS-Notifications essentially 
  provide subscription interfaces to allow programmatic subscription to  
  one way web services, and provide a filtering mechansim as well, so you can 
  subscribe to only data from the service if it meets various 
  criteria.   WS-Notifications provide more advanced features as well, 
  like subscribing to topics and notification brokers.  WS-Eventing is very 
  simple.  One could create their own interface instead, but there is 
  likely no implementation benefit from rolling ones own. 

  
I don't understand biztalk like systems very well, 
  but I think they primarily work on coordinating one-way messages between 
  several actors according the rules of a process calculus (pi-calculus) 
  expressed in a language like BP4WELS http://www.oasis-open.org/apps/org/workgroup/wsbpel/.  This is useful if an process is complicated, 
  requiring the delivery and reception of messages from several sources in order 
  to sucessfully complete.  I think coordination activities and whether to 
  use biztalk vs Java or C# to implement such coordination is an implementation 
  issue.  We might find BP4WELS or some other process calculus (like UML 
  synchronization diagrams, petri nets etc.) a convenient way to specify such 
  coordination of several asychronous messages. 

  
WS-Reliability and WS-Reliable Messaging provide 
  for guaranteed delivery of messages.  An application sends the message 
  and delivery is guaranteed eventually (therefore asychronously).  
  Implementations typically use  middleware like MSMQ. I don't know if we 
  can leave whether to implement this  to implementors of obix, or if we 
  need to specify when/how they are used and make them optional parts of the 
  spec. 

  
I have not had a chance to investigate OPC. 
  

  
Doug Ransom 
Energy Analytics Domain Expert 
Power Measurement 
2195 Keating Cross Road 
Saanichton, BC, Canada  V8M 2A5 
Tel: (250) 652-7100  ext. 7558 
Fax: (250) 652-0411 
E-Mail: 
  mailto: 
Website: 
  http://www.pwrm.com 
  
ION®  
  smart energy everywhere™