[Rets-dev] Contemplations on RETS 2.0

[Matt Lavallee]

On Friday, August 03, 2007 at 7:50 AM, Steve Clarke wrote:

> Obviously, I disagree with Matt that we should essentially 
> rely 100% on standard payloads and that this would render our 
> metadata obsolete.
> Sure, this argument holds water, but to me it's more like a 
> "devils advocate" argument that is intended to get a reaction 
> (like mine) of "hey, wait a minute, we can't do that!"

Why not?

> From a practical standpoint, we have MLS systems that will 
> natively deviate significantly from the standard payload 
> structure (and even more so from the strict data typing).

Who are they, and what are their phone numbers.  Seriously, this has been
the entire purpose of having open discourse in the Payloads committee with
MLS involvement.  If there are "significant" deviations from what has been
outlined, those parties need to be involved.  I have to raise an eye of
skepticism, here, because the number of vendors and MLSes validating the
payload has been fairly high thusfar...

>  It is unrealistic to think that every MLS will modify their
> data to fit the mold of the standard payloads.

There is no reason they have to remodel internally to deliver RETS.  As has
been demo'ed at more than one RETS meeting, mapping tools needn't be so
complex that they require countless hours of consulting.  As I surveyed and
pointed out at the last interim meeting, virtually all MLSes are using MSSQL
or Oracle backends, which provide infinite (simple) possibilities with
respect to reformatting the data in a presentable (even XML) way.

>  It is also 
> unrealistic to think that MLS vendors will absorb the cost to 
> implement complex mapping and data translation layers (plus 
> the tools to support the mappings and the support overhead to 
> maintain the mappings as their native systems change) solely 
> for the purpose of being able to better fit the standard payloads.

A number of MLS vendors have been deeply involved in the standard payload
definition, and none have raised any issue with the current standard.  In
fact, many changes have been given deference for implementation complexity.

> ...especially when the RETS standard does provide for the 
> native MLS data to be represented, described and queried in 
> its native form by using the resource metadata.

"Native MLS data" is a dubious term - what you're really saying is a data
structure that does not conform to the standard, and is therefore
non-standard.  The purpose of the robust payload format is to account for
the widest constraint possible, and to use nullability to emit more limited
data sets.

> Beyond that, the ability for a client to specify what fields
> they desire and retrieve them in a compact payload are
> benefits that will make it even more difficult for the
> standard to mandate a 100% standard payloads solution.

Field selectability is a feature of the soon-to-be-proposed RQLX.  The
deficiencies in RQL have been cited several times on- and off-list, and the
current "instance formatted output" concept of RQL is essentially dead on
arrival.

> Finally, we still have the fundamental problem of what to do 
> when the native resources do not even match up to the 
> standard payloads (e.g. An MLS with multiple "Residential" 
> property types).

Paul requested this information back in Rockville, and the Payloads
committee selected a set of would-be all-encompassing Property Types at
Westlake.  I suggest you review the current payloads and comment as
appropriate as to which types are not properly covered.  This data has been
requested by Paul since (at least) April.

> The standard does not describe what the 
> server needs to do to support that situation.

Worst case scenario: use "Other" and include the local type's name in the
otherdescription field.  Per the spec.

>  I suspect that 
> if the specification requires that the server MUST somehow 
> combine the query, perform multiple searches and merge the 
> results from multiple property types there would be some 
> resistance.  But without providing some way to relate the 
> native resources to the payloads (and allowing the client to 
> query against a native resource) it count render unreliable
> results.

To me, the truly native implementation should NOT be visible to the
consumer.  It shouldn't have to be.  If there are data structures that exist
entirely outside of the standard payloads, custom payloads should be used.
At least in this case, no presumption of a standard would affect the client.

-Matt