Re: [EXT] [cti] TAXII Update From F2F Meeting

All,

I just wanted to make one last call for objections to the following changes (breaking and non-breaking) as described in the email below.  If you only have a few minutes to review, then please review the top section called
 "breaking changes".

Bret

From:  <> on behalf of Bret Jordan <>

Sent: Monday, February 5, 2018 6:29 PM

To: 

Subject: [EXT] [cti] TAXII Update From F2F Meeting

 

All,

This TAXII update is a bit long, but it contains elements we
 need to to review:

During the face to face, we discussed the following TAXII topics
 that are each documented in the Github Issue tracker (https://github.com/oasis-tcs/cti-taxii2/issues).
 The consensus from the room is that we should address these, as noted in the issue tracker. We would like to ask for a call for objections on implementing these changes.  

If the TC approves these changes, we would like to release
 a TAXII 2.1 CSD-01 as soon as possible, so organizations can start coding to these changes.  The current TAXII 2.1 document can be found here:
https://docs.google.com/document/d/1EsiWY7TGqt9yH6QUXv4c-opXSr3wR0TDMt8Q0yJjpoo/edit

Issues discussed at F2F - Backwards Breaking
 Changes

The issues in this section represent changes that are incompatible
 with the TAXII 2.0 specification and are considered “backwards breaking”.  While our goal is to have no backwards breaking changes between one release and the next, in this case we have found a number of issues that require incompatible changes in TAXII 2.1.

Issue 18

TAXII
 Discovery URL Collides With Existing Product URLs #18

During interoperability testing, the CTI TC learned that the
 /taxii/ URL conflicts with existing product URLs. The conflict prevents effective implementation of TAXII2 for existing products, which is undesirable.

The proposed solution is to change the DiscoveryURL from /taxii/
 to /taxii2/. While this change breaks backward compatibility, the estimated impact is small. Servers supporting both TAXII 2.0 and TAXII 2.1 can easily support both URLs through aliasing. Servers that can not use the /taxii/ URL can now use the /taxii2/ URL.
 Clients that support both TAXII 2.0 and TAXII 2.1 may have to try multiple URLs for discovery to succeed.

This change will also help distinguish TAXII 2.

Issue 23

Item
 Based Pagination is Unusable for Rapidly Changing Datasets  #23

When a resource changes faster than a client can page through
 it, pagination becomes unpredictable.

For instance, imagine a client that pages through a result
 set at a rate of one page every second. Each second, the server adds one page of content to the head of the resource, and removes one page of content from the tail of the resource each second. The client will effectively receive half of the dataset when paging
 through, while thinking they received the whole resource

Time

Resource State

Client Requests Page

Client Receives

0

Page 0: AAAA
Page 1: BBBB
Page 2: CCCC

0

AAAA

1

Page 0: BBBB
Page 1: CCCC
Page 2: DDDD

1

CCCC

2

Page 0: CCCC
Page 1: DDDD
Page 2: EEEE

2

EEEE

We discussed this at the F2F and the general consensus in the
 room (not unanimity) was that this is a problem and we need to fix it. Further, a proposal was put forward to fix this by using sorting of the UUIDs and add an id_after URL parameter.  No decision was made to make the change yet. The consensus was more research
 and testing needs to be done to verify that this will actually make things better.

Issue 29

Need
 to change the media type for TAXII per OASIS / IETF / IANA #29

We discussed this at the F2F and the consensus is that we need
 to make a change. There is an open question about using a single media type to represent both STIX and TAXII or keep using two. While several people in the room believed that using a single media type is probably the best, we would need to address the various
 version problems that can come about because STIX will rev faster than TAXII.  So more work and research needs to be done here.

Issue 36

Manifest
 Resource Cannot Accurately Specify All Media Types and Version Combinations #36

For each object, the manifest resource specifies a list of
 versions and a list of media types. However, not all version and media type combinations are necessarily valid. For instance, a Collection may contain an object where v1 is available as STIX 2.0, and v2 is available as both STIX 2.0 and STIX 2.1. The manifest
 resource cannot accurately specify this condition.

The proposed change is to:

Reduce the scope of each manifest resource from "one manifest for all versions
 of an object" to "one manifest per object version and media type"

Modify the Manifest resource to permit only a single value for object version
 and media type.

This change breaks backward compatibility, and the impact is
 estimated to be moderate. While a large portion of the specification will change in a backward incompatible way (Section
 5.6), the STIX2/TAXII2 Preferred program does not currently define tests forconsider this area of the specification, and
 few implementations have implemented this area of the spec.     

We discussed this at the F2F and the consensus in the room
 was that we should make the proposed change to the manifest resource. 

Issues discussed at F2F - Non-Backwards Breaking
 Changes

The issues in this section are all backwards compatible with
 the TAXII 2.0 specification.

Issue 21

Object
 by ID Resource Responses Can be Huge and Needs Pagination Support  #21

The Object resource can contain a significant number of object
 versions, which become unwieldy to manage in a single request/response pair. Without a mechanism to manage highly-versioned objects, effective transport is significantly limited.

The proposed change is to add pagination to this resource in
 section 5.5. This change is backward compatible.

Issue 28

Discovery
 Resource Does not Clearly Articulate API Root Allowable Values #28

During interoperability testing, implementers discovered that
 the specification does not clearly specify the syntax for URL values. Implementations were evenly split between fully qualified URLs (e.g.,
https://example.com/api/1/)
 and URL paths (e.g., /api/1/).

At the Jan F2F in Salt Lake City, the consensus of the group
 was that both "Fully Qualified" URLs and "Absolute Path" URLs should be permitted, and clarifying text should be added to the specification. In large part, determining fully qualified URLs of hosted services is seen as prohibitively complex to implement. This
 is a backward compatible change.

The dissenting opinion is that the HTTP URL syntax is clearly
 defined, and that it requires the "fully qualified" form. The complexity of implementing fully qualified URLs is seen by some as overstated.

Issue 30

Manifest
 Resources Cannot Accurately Specify All Media Type and Version Combinations #30

This issue is superseded by issue 36

Issue 31

Object
 Sort Order is Confusing and Surprising to Some #31

Objects in TAXII are returned in an order of "oldest first".
 This is surprising to some, and the amount of surprise could be reduced by adding clarifying text.

While this was discussed at the F2F and the consensus in the
 room was to add clarifying text, no change has been proposed yet.

Issue 39

Statements
 About Sorting Appear to Conflict But on Closer Reading Do Not - Needs Clarification #39

Section 3.3, which discusses sorting, is worded such that many
 first-time readers incorrectly conclude that the specification makes incompatible statements about sorting rules.

The proposed change is that the wording be clarified so that
 first-time readers do not make this conclusion. Normative changes are not proposed.

We discussed this at the F2F and wordsmithed a proposed change
 in section
 3.3.  Please review the proposed text changes.

Issues in final review

The issues in this section are all backwards compatible with
 the TAXII 2.0 specification.

In addition to what was discussed at the F2F, the following
 issues are in final review (they have been discussed previously on working calls). If you have any comments or feedback on the proposed changes, please let us know.

Issue 8

max_content_length
 example value is not exemplary #8

Please review the suggested text and example changes is
Section
 4.2.1

Issue 9

Get
 STIX Object by ID does not describe why a bundle is required #9

Please review the suggested text changes in
section
 5.3

Issue 17

Add
 text about TLS 1.3 0-rtt #17

Please review the suggested text changes in
section
 8.2.2

Issue 20

Manifests
 Can Advertise Media Types Beyond Collection Capabilities #20

Please review the suggested text changes in
section
 5.6.1 as it contains a new normative statement.

Issue 25

Version
 Filter is Underspecified WIth Returnable Content #25

The Version filter text specifies that when absent from a request,
 the most recent version should be returned. The text does not specify whether other versions are permitted.

The proposed change is to clarify that only the current version
 should be returned, and that other versions are prohibited.

Please review the suggested text changes in
section
 3.5.1

Issue 26

Version
 keyword 'all' should not be allowed when using multiple version parameters #26

Please review the suggested text changes in
section
 3.5.1

Issue 34

Version
 filter description only talks about object(s) #34

Please review the suggested text changes in
section
 3.5.1

Issue 35

added_after
 needs clarifying text #35

Please review the suggested text changes in
section
 3.5

Bret