[Rets-dev] Contemplations on RETS 2.0

Thu Aug 2 15:22:27 CDT 2007

Matt, I'd like to address some of your concerns. I do need some 
clarification on some of the terms that you use before I can properly 
respond.

Please see below:

Matt Lavallee wrote:
> Hi everyone,
>  
> Being out at Inman discussing the new standard with colleages and 
> processing the myriad changes that have gone into the standard over 
> the past five months, I have some concerns over complexities in the 
> standard that may introduce "disparity loopholes" among systems.  I'm 
> going to list some of these below and hope that the community and 
> stakeholders can explain why certain parts of the new standard are the 
> way they are...
It sounds like it was an interesting discussion. Most of the changes 
over the last five months have been on the representation of standard 
schemas. I think we agree that where we are now suggests revisiting the 
query language to make sure that it is still the best choice.

What is a disparity loophole? Is it a concern about interoperability 
between systems?
>  
> 1. Exclusivity between the "Resource" model and the standards-defined 
> "Payload" (i.e., output) model.  Is there any particular reason that 
> these exist independently, and is it reasonable for us to consider 
> merging the two.
>
What do you mean 'exclusivity between the "Resource" model...' What is 
your meaning of exclusive?

Resources are tightly coupled to schemas, although one resource can have 
more than one schema. Think of the schemas as views of a resource. Also 
consider the Vocabulary as an alias to searchable fields in the 
DataDictionary. Resource [1..n] Vocabulary. Resource [1..n] 
OutputFormats (Schemas are used to generate the output formats except 
where the output is delimited or delimited-encoded).

> 2. As anyone who has attended the Payload meetings knows, I am not a 
> big fan of the Vocabulary or DataDictionary structures.  In fact, as 
> time has progressed, I've deepened my position to feeling that these 
> are bottleneck, meta-meta loopholes that will only serve to harm 
> implementation of the standard (as a true, universal standard).  Given 
> that each of these is extensible and overrideable, it's conceivable 
> that any "compliant" system could be entirely foreign when compared to 
> any other compliant system because of liberal [ab]use of the 
> meta-extensibility model.  If #1 (above) can be satisfied, is there 
> continued need for these two structures outside of custom 
> payloads/resources?  To that end, is it necessary that they be related 
> to the existing (standards-defined) payloads at all?
>
I am not clear on what of the Vocabulary (stuff you can search on) and 
the DataDictionary (all the stuff there is) you don't like.

What does 'meta-meta' mean?

What do you mean when you say that the Vocabulary and DataDictionary are 
extensible and overrideable?

They are are extensible but they are not overrideable. Any local 
additions are in a new namespace.

What does meta-extensibility mean to you?

What in #1 needs to be satisfied?

Both Vocabulary and DataDictionary are needed both with and without 
custom output documents and additional resources. They do not need to be 
related to the schemas for a particular resource, although I expect that 
they will be related in some form in a meaningful implementation.

Both XML instance documents and Delimited streams are permitted as the 
response to a search request.

Please refer to examples 9.2 and 9.3 of the Service document for simple 
examples of a search request.

In those examples, the well-known vocabulary for Residential is used.

We could create a simple vocabulary, for the purposes of illustration 
let's define it as two fields with no grouping and no required. The 
vocabulary would look like
<Vocabulary name="ResidentialLocal">
<Fields>
<Field>
<MetaEntryId>
2445342
</MetaEntryId>
<Name>
ListingID
</Name>
<DataType>
Character
</DataType>
</Field>
<Field>
<MetaEntryId>
776362621
</MetaEntryId>
<Name>
PostalCode
</Name>
<DataType>
Character
</DataType>
</Field>
</Fields>
</Vocabulary>

The MetaEntryId relates the element of this vocabulary to a specific 
element in the DataDictionary. The Name is the value used in the 
formation of the WHERE clause in the RQL.

As a requestor of this system, I know that it must provide at least the 
Well-known vocabulary, see Requirement R0025 and R0026. It will have a 
form similar to that of my example above. I may not be permitted to 
access all the elements, based on the local business rules, but I do 
know that it will have the well-known vocabulary and I can then 
determine why I am not permitted to use the well-known vocabulary to 
access records by contacting the system operator. I can discover using 
the metadata action that there is a second vocabulary - the trivial one 
I created above.
> 3. Extensibility and replaceability of the standard payloads:  They 
> have been used in defense of the uber-standard model quite a bit, 
> given that if a certain payload does not meet the needs of the local 
> system it is open to ad-hoc extension.  While this flexibility is 
> well-intended, it does give rise to the likelihood of remarkably 
> dissimilar systems (to this I point to REIL and the use of additional 
> RETS 1.x Resources to define common things like cities).  I would 
> propose that, in order to be compliant, every system MUST support the 
> standard payloads (as defined by the standard, with the standard's 
> data types), and MAY provide alternative payloads that include local 
> redefinition.  To this end, all of the fields in the /standard/ 
> schema would need to be closed to disallow extension.
>
What do you mean when you say that the standard schemas are extensible 
and replaceable? The standard schemas are extensible, using other 
namespaces to add elements that are not part of the standard. These 
namespaces are outside the standard schema, although they are delivered 
as part of the instance document. Vendors may choose to ignore the 
information in namespaces other than the standard - based on the 
application they are building.

The standard schema are not replaceable. See requirement R0104.

There is no ability to extend the schema in an ad-hoc manner. They 
require an extension namespace. See the example of an extension schema here
http://www.rets.org/cms/files/extendedProperties.xsd
and the instance document here
http://www.rets.org/cms/files/extendedPropertiesInstance.xml

I don't understand what you mean by the reference to REIL. Can you 
expand on this?

I think that enforcing standard schemas for MLS systems is a good idea. 
See R0104.

I don't understand why the [sic] fields (elements) of the schema need to 
be closed. What does closed mean to you?

If you deliver instance documents from standard schemas, the namespace 
is closed and you can safely ignore the extensions.

> I realize the points above may be a radical shift to rigidity, and I 
> have no intent of devaluing the work that went into making the 
> standard broadly applicable.  However, the limited pain imposed by 
> forcing /some/ data homogeny for the sake of a reliable 
> transport would be greatly outweighed by the pain felt by the masses 
> (for years to come) as the standard is consumed.  I earnestly hope to 
> avoid the death-by-a-thousand-papercuts that has plagued RETS 1 adoption.
How are the data and the transport related? In RETS1 and RETS2 there is 
no dependency between the two. What is your meaning here?

I don't follow your argument  about the 'forcing some data homogeny...'. 
Can you please provide more details?

I'd like to avoid papercuts or other impediments to adoption. I'd like 
to understand your concerns better to see if I can answer them or if 
they merit a re-visiting of the standard to revise or clarify the standard.

Paul
>  
> -Matt
> ------------------------------------------------------------------------
>
> _______________________________________________
> Rets-dev mailing list
> Rets-dev at rets.org
> http://lists.rets.org/mailman/listinfo/rets-dev
>   

-- 
Paul Stusiak
Falcon Technologies Corp.