Sunil:
I
think I can answer all these... see inline:
-----Original Message-----
From: Sunil Kunisetty
[mailto:]
Sent: Wednesday, November 12,
2003 12:15 AM
To: Jacques Durand
Cc: Iwasa;
Subject: Re: [wsrm] Proposed resolution
for Rel 50
1) The semantics
of ExpiryTime in overloaded.
In Rel 50, we
are saying that ExpiryTime is nothing but
RejectMeIfYouReceiveMeAfterThisTime, i.e., this is to
reject it it was delayed at the transport
layer and nothing
to do with it's life span
once received.
However, in Rel 52, for the termination
conditions t2 and t4,
we are using max of
ExpiryTime of individual messages to figure out
the GroupExpiryTime if the latter is not
present.
[Jacques Durand] no, you misunderstood: in t2,
MaxIdleTime decides of the group termination. The use of max(ExpiryTime)
is only for removal of the group state which
was
Sure, I understand it is for removal of the group. For my
examples, I was not
distinguishing between the group termination and
group removal case.
My point if it is just used at the time of receive, why is this
parameter ever
used there after, whether it is for removal or
termination? Essentially when
you are trying to compute the
group removal life span, you are using
the max of all
message's expiry times. So essentially you are using the
expiryTime
as a message's life span thingy in here, which contradicts your earlier
usage and assumption.
[Jacques Durand] Sunil: no
contradiction: Removal of the state (i.e. message IDs) of either groups or
singletons, is indeed based on ExpiryTime (or GroupExpiryTime when
present), but will NEVER affect the outcome of processing a future message
once it has been accepted, which is the basic principle behind the proposed
ExpiryTime. This removal is just a convenience in managing RMP state (yet
a huge one): we could keep these IDs forever in RMP instead, that will NOT
produce a different outcome for future messages. So I should have said that
this removal is RECOMMENDED. This removal will make dup check and persistent
store more manageable. If you don;'t remove, you'll just do more useless dup
checks.
still needed a while for dup elimination. (It
happens that for t1, both events are decided by GroupExpiryTime...). Same
for t4, where max(ExpiryTime) would decide only when to remove the state
(only affects dup elimination). In t4, I propose group termination to be
effective as soon as a message is out of order (though Tom Rutt proposes
something less drastic).
...
Assume somehow the Receiver receives the same message (Msg3) at 5.00 pm
with a different ExpiryTime say 6.00 pm.
Obviously this won't be discarded, rather
accepted. [Note that we had Rel 38 that says Tiemstamps have to be the
same during retries -First of all, Timestamp
is gone, and secondly, we don't say any
thing
about ExpiryTime during Retries, so Retries can have different
ExpiryTimes].
[Jacques Durand] well, why would duplicates
have different ExpiryTimes? we normally expect duplicates to be exactly
same, though I agree that the formal definition of
duplicates
Not
necessarily. Duplicates may or rather must have the same Timestamp, but not
necessarily the same expiryTime.
For ex., a msg. sent at 12 noon has a expiry time of 4.00 pm. For
some reason,
the Sender never receives an Ack for that msg. inspite
of (re)trying it until 4.00 pm.
So if the Sender has to continue his
retries after 4.00 pm, he obviously has to change
the expiryTime, if
not, his retries after 4.00 pm will never be successful.
[Jacques
Durand] I think we need in our TC to clearly decide about this: in
my understanding, retries means resending exactly identical messages. No
change in ExpiryTime. in other words, the ExpiryTime should be greater
than the number of retries * retryinterval. (if not, some Sender should stop
resending when message expires). In other words, ExpiryTime is a countdown
starting from the very first time the message is sent/generated. Retries don't
reset the counter: they are part of the effort in getting the message arrive
before ExpiryTime.
is just same IDs. If tehre is tampering wiith
duplicates, that becomes a security /
integrity
And when a retried Msg 3 arrives at
5.00 pm with ExpiryTime, it will rejected as
a
duplicate as the previous Msg3 still exists in the persistence cache.
So essentially there is a difference in
behavior of duplicates depending upon whether
it is Singleton, Group-UnOrdered or Group Ordered.
[Jacques Durand] the only (expected) "app-visible" difference, is
as you said that Msg3 will never be delivered to app, because the group is
ordered and M2 has not arrived before GroupExpiryTime. In all cases,
retries of Msg3 would be considered as duplicates until GroupExpiryTime
expires. After that, they would be rejected. For singletons, as there is
no
So are we saying
that even for group-unordered cases, all message IDs of that group
are kept until that group is removed? If so, atleast this problem
will be solved for
the group unordered case (though the issue still
persists for Singleton messages),
but then won't it defeat the whole
purpose of group-unordered optimization it was
suppose to add?
Unordered-groups are potentially going to very very long and it is
not advisable to keep the Message IDs for expired messages.
[Jacques Durand] You are right that groups, whether ordered
or not, can be very long. But precisely, the IDs of grouped messages can be
stored very efficiently, as intervals of integers, and the nice thing is that
the less we discard of them, the smaller the number of intervals... again this
is just a RECOMMENDATION, as it facilitates management of persistence and dup
check, but no logical impact on processing outcome of future
messages.
My main point is what warrants the change when we don't seem to have
any issues
with the old behavior of ExpiryTime (TTL one) as I listed
below.
[Jacques Durand] teh issue with "time at which mesg should be
delivered to app" , (besides tha additional burden on RMP : 2-checks, and
additional failure cases, although that should not be the sole decision
factor) is that we give some app semantics to this expiration
time. If a Purchase Order is valid only until January 31, 2004, should we
put an ExpiryTime to this date? But we may still want a Vendor application to
still receive such a message and ack it (at app level), even if its content
has expired application-wise. So it is hard to justify an application-level
semantics for such value. It should be up to the application layer to decide
about the validity of the mesage content based on dates. The proposal here is
to just consider ExpiryTime like a transport QoS, like we do for guaranteed
delivery which is a contract RMP-to-RMP.
-Sunil
GroupExpiryTime, the limit of ID persistence
is ExpiryTime. In all cases, the contract is fulfilled: never duplicates
would be delivered. (granted, for singletons we might get rid of Mesg3
state a bit earlier than if in a group, as GroupExpiryTime is usually equal or later than
ExpiryTime. But that does not affect semantics: just an internal
optimization issue for dup check.)
These inconsistencies that trouble me.
So lets take a step back and see what are the issues with the
actual TimeToLive semantics.
1) Periodic checking for Msg expires: As I said on the call, we
could solve this by having the
checks
only twice. Once during "acceptance" stage and other during "make it
available" stage.
2) Sender won't know whether a Msg. was really made it available
to the User.
As we agreed at the F2F
(on thursday), the way to solve this is by saying that
Sender will only assume that a msg. was
really made it available is when
i) It's not expired
AND
ii) It was acked
AND
iii) For a Group
ordered or un-ordered case, the Group is not terminated
AND
iii) And in a
group-ordered case, all preceding messages (the ones with lower seq. nos.)
are acked.
Note that just saying getting an
ack doesn't always mean that the msg was made
available as they are cases such
as Group being expired, Persistence store failure etc..
Since we don't have async fault
notifications, we either assume rules such as above
or use Poll for all the reply
pattern cases as Tom suggested.
As I said on the call the new proposal does make implementations
simpler, but I'm just
"little" bit worried about the
inconsistencies.
-Sunil
Iwasa wrote:
I like this
proposal. I would
like to reply some questions raised in thetelecon today, which is "What
is the fundamentalfunction for ExpiryTime?". The fundamental function for
ExpiryTime, I believe, isto enable receiver check duplicate message(s)
even ifit removes old message information. When we don't want to enforce receiver
keepall previously received message inforamtion forever,we need
ExpiryTime. And it should not be changedwhen the message is
re-sent. In this
case, receiver have to keep old messageinformation until at least the
ExpiryTime.So it is possible to check duplicate when theduplicate
message arrived before it's ExpiryTime. And after the ExpiryTime has passed,
receivermay remove the message information.In that case, receiver do not
have to do duplicatecheck for the message, since receiver just needto
send back Fault message to let the senderknow about expiration of the
message. That
would be a critical reason we need ExpiryTimeas mandatory
element, And I see
Sunil's concern, which is meaningful for brokensequence use case. Let me
clarify the use case thatSunil mentioned. Use case1: Sender
sent four messages with ordered
delivery: GroupId=x.y.z
SequeceNumber=0
GroupId=x.y.z
SequeceNumber=1
GroupId=x.y.z
SequeceNumber=2
GroupId=x.y.z SequeceNumber=3 Receiver received
three messages:
GroupId=x.y.z
SequeceNumber=0
GroupId=x.y.z
SequeceNumber=1
GroupId=x.y.z SequeceNumber=3 When receiver did
not received SeqNum=2 message forever, receiver have
to keep SeqNum=3 message forever, even if the
ExpiryTime is expired, if the message arrived at
receiver by the ExpiryTime. I believe this is Sunil's concern,
isn't it? How bout
the following resolution: Resolution 1:- Adopt Jacques's proposal for semantics of
ExpiryTime as is.- We clarify special error case only for
OrderedDelivery like: In case of one or more message(s)
was/were missing in a sequence, and the next message of the
missing message was expired, the receiver may remove all the
following messages. This resolution doesn't change the
semantics of ExpiryTimethat Jacques proposed, but solves problem Sunil
raised. I do see
problems if we change the sematics of ExpiryTimeitself, since it would
be affect to non ordered messages.The use case is: Use case 2: 1. Sender sent
one message without ordered delivery, and
the ExpiryTime = 12.00.00 2. Receiver received the message
at 11:59:59. And sent back Ack to
the sender. 3. Before the receiver give it to the
application, time was changed to
12.00.01. And receiver considered it
was expired, and removed it.I don't believe this makes sence, and it
would behappen when we allow to remove any messagewhich was expired even
after the receiver receivedit in time, and sent back Ack.
In conclution, I believe
resolution1 abovesolves all problems mentioned. Thanks, Iwasa
----- Original Message
-----
From:
Jacques Durand
To: ''
Sent: Saturday, November
08, 2003 12:20 PM
Subject: [wsrm] Proposed
resolution for Rel 50
Rel 50: Semantics of
ExpiryTime:
-------------------------------
Proposal:
ExpiryTime indicates the
ultimate date after which the receiver RMP will
not accept a message. In
order to be accepted by an RMP, a message
must have schema-valid RM headers,
and must not have expired.
Only accepted messages are subject to
further processing by the RMP.
When receiving a message
after its ExpiryTime date, an RMP MUST NOT accept the
message.
Once
a message has been accepted by the RMP, the fact that the message
expires while
being processed by the RMP MUST NOT affect its processing, nor
its final availability
status to the receiver application.
Such expired messages must still be delivered to
the application, assuming
they satisfy other reliability requirements.
NOTE:
Although ExpiryTime will
not affect further the processing of accepted messages,
it is useful for an RMP to
remember it in order to avoid unnecessary duplicate
checks
with
future messages, when "at most once" delivery is
required.
Given the above definition of ExpiryTime, it is impossible
that a received message M
accepted by the RMP, is a duplicate
of past messages that are expired at the time M
was
accepted.
