LLSD for Parameters (was re: [sldev] Plugin API)

Ryan Williams rdw at lindenlab.com
Fri Mar 2 14:50:17 PST 2007


As a disclaimer, I really have no experience with plugin apis, I was
just suggesting LLSD because it is the data interchange format of the
future, useful for interoperating between parts that are loosely coupled.

Tim Shephard wrote:
> I was thinking on the email proposed by Ryan regarding LLSD and it has
> some interesting implications.
> 
> If we were to use LLSD for parameter passing, one useful area might be
> (I stress the word, might, as I have significant concerns about this
> approach) for registering services.
> 
> For example, we might have UIServices, RenderingServices, and
> MessageServices.
> 
> Here is what a C api might roughly look like (so far, only 6 top level
> functions):
> 
> //Get a list of the services (Ie: UI Services, Message Services,
> //Rendering Servers) this plugin wishes to subscribe to
> // Included in the LLSD object will be function pointers to
> // callbacks that the client will call into the plugin.
> LLSD *SLP_GetServiceSubscriptions();
> 
> //Tell the client it's trying to subscribe to a no longer supported
> // service and version tuple.  This would be at a coarse grain level.
> void SLP_InvalidateServiceSubscription(const char *ServiceLabel);
> 
> //Initialization / Shutdown callbacks
> //called when the client loads /unloads the plugin.
> //XML Manifest file would be more suited
> SLPError SLP_Load();
> SLPError SLP_Unload();
> 
> // called when the client starts up /shuts down
> SLPError SLP_ClientStartup();
> SLPError SLP_ClientShutDown();
> 
> 
> Given that services from the client themselves (on the assumption that
> it's a rapidly evolving client) will need to be versioned, I was
> thinking we may want to use the LLSD object to register services with
> the client.

I really like LLSD and I think it is the perfect solution to a bunch of
problems.  I don't think you can store function pointers in LLSD though,
unless you want to get into some horrible horrible casting.  Hmmm... I
don't know if plugin registration absolutely needs function pointers.

Or you could use the LLSD for everything but the function pointers, and
then also pass along a list of function pointers...  I dunno, just
thinking out loud.

> For example (this is just to give you an idea.  This would be highly
> evolvable, which is the whole point of using LLSD objects):
> 
> There would be a part of the plugin api:
> 
> LLSD *SLP_GetServiceSubscriptions();
> 
> SLP_GetServiceSubscriptions could return an
> 
> Services Low    Trusted  Zerocoded
> {
> UIService Variable
>       {
>              ServiceData  Single
>               { Version         S32}
>               { ServiceLabel Variable 2}
>               {  XMLSpec     Variable 2}   //UI Xml Spec
>               {  MenuName  Variable 2}   //There could be a 'plugin' menu
>               {  Show           LLFUNC}    //call back when show is
> called, has a signature not documented here
>               {  .. etc .. }
> 
>               Events Variable
>               {       EventType                       S8
> }  //eg, setAction,
>               {       EventFunc                       LLFUNC        }
> //callback function
>                    .. etc
>       }
> 
> MessageService Variable
> // Dynamic structure for MessageService
> // Probably something that specifies message types and
> // associated callbacks
> 
> RenderingService Variable
> // etc
> ..
> 
> }

So I think I created some confusion by conflating LLSD with the message
system.  They have similar properties, but the fundamental difference is
that LLSD is just a wire format combined with an in-memory api.  The
message system is more specific and limited.  I believe the acronym
stands for "Linden Lab Data System" in some sort of oddly-ordered
universe.  We are in the process of replacing the on-the-wire message
system format with an LLSD-based system (see Zero's blog post here:
http://blog.secondlife.com/2006/12/21/a-big-change-youll-barely-notice/).

So what you have up there looks like a message system message, not LLSD.
 As LLSD you would notate it more like:

{
'uiservices': {
'servicedata':{'version': <int>,
               'servicelabel': <string>,
               'xmlspec':      <string>,
               'menuname':     <string>,
               'show':         <some sort of function reference>,
               ....etc....}
'events':[
{'eventtype':<int>, 'eventfunc': <some sort of function reference>}]
}

This notation doesn't correspond to any actual parseable specification,
because the actual LLSD on-the-wire formats are either annoyingly
verbose or binary.  This json syntax is the closest human-readable
syntax we use (and I'd be happy if we had an LLSD formatter/parser for
it).  Nor can we enforce any sort of data structure using LLSD because
it is dynamically-typed like Python.  However, this is actually a good
thing because you can deal more gracefully with failures.

Anyway, this little pedantic data system tale doesn't really change
anything about your concept here.

> This has four aspects, two I think is relatively straightforward, the
> other two are concerns.
> 
> The first aspect is versioning.  Right now the versioning in the LLSD
> objects are done in different parts with different nomenclature.
> Ideally, we could develop a more  consistent approach not just in
> verbiage but also handling revision tracking of the message protocol
> itself so that future tools for dealing with versioning on a finer
> grained scale will be more successful.

True.  One question is whether the LLSD objects themselves contain their
own version number, or if we have a meta-object that describes the
versions of all the objects it knows about.

> The second issue is how flexible the LLSD protocol is.   Looking
> through message_template, I couldn't find examples of realtively
> complex hierarchical types.   It would be useful to urge LL to XMLify
> this.

Take a look in llsd.h.  It supports arbitrary hierarchies, and it
serializes to XML.

> The third aspect are function pointers as a LLFUNC type which is a
> pointer.  This will have a different size on different platforms, and
> isn't entirely in the spirit of the LLSD object which is mostly used
> for RPC / serialization .. though there is some interesting discussion
> there to be had for using the message spec for services in other
> plugins which work over the wire.

I guess I talked about this before.  I don't think we should try to
wrangle LLSD to do something platform/implementation-specific like carry
function pointers.  A side channel would probably be better.

> The fourth aspect is related to the second in that we do not really
> need to encode and we're using this message spec for communicating the
> structure of the Service specifications, on the theory that they
> change rapidly and we'll need dynamic typing.  Perhaps we're over
> building if we do this.
> 
> The alternative approach is to use LLSD like type C structs.
> 
> I still think we need some kind of specification for what the structs
> will look like
> (we could use the message protocol) and we still need to allow for
> dynamic typing because of changing features (adding and deprecating
> data fields, LLFUNCS, etc), however, instead of returning LLSD class
> objects, we'd return a C version which is similar but without the C++
> features of the full on LLSD object.
> 
> The advantage of that approach is that we'd be able to deal with C++
> ABI issues and we'd be able to share concept, code, and grammar
> between the two types, LLSD and CLLSD.   The Plugin SDK would have
> helper funcs for creating these CLLSD objects.

I know nothing about C/C++ interoperation, but I like the idea of C LLSD
 bindings.  Part of the fun of C++ LLSD is how syntactically sugared it
is, but it's certainly possible to make a nice C api.  Actually that
would make a nice self-contained project if someone is interested in it.

-RYaN

> Anyhow, let me know your thoughts.
> 
> For those who don't like the idea, alternative header files are your
> best answer!  This idea should inspire better ideas only, not empty
> critique :)
>
> For those who like the idea, suggestions on alternative nomenclature
> and function grammar that clearly evokes purpose.  Ideas for parameter
> signatures would be useful.   Expanding the API is fine, however,
> keeping it as simple as possible .. ((especially the bits that are
> likely to be stable) is critical for Plugin API success.
> 
> Cheers,
> 
> Tim.



More information about the SLDev mailing list