From hraban at 0brg.net  Sun Aug 14 12:24:02 2011
From: hraban at 0brg.net (Hraban Luyat)
Date: Sun, 14 Aug 2011 14:24:02 +0200
Subject: [cl-json-devel] Guidance requested: how to dispatch members to
	external decoders?
Message-ID: <CAK273yJi8S-CRyRG=1VzeQx-UKkXY1NhNSOZ7SUmX2yvBMz8+Q@mail.gmail.com>

Hello,

As far as I understand the decoder API it is mostly geared towards
defining the decoder for an entire snippet in one place. I would like
to know what the recommended way is to separate the definition of the
decoder over separate modules.

The semantics of my incoming messages are thus:

object:
  - "type": string denoting the type
  - "payload": type-specific payload

I want to create a decoder that only extracts the type and uses that
to determine which decoder to send the payload to. Then it continues
with whatever lisp object the decoder returned.

What I thought would be appropriate is to create a generic function;

(defgeneric json->data (type payload))

and then simply register decoders as follows:

(defmethod json->data ((type (eq :foo)) payload)
  "Decode message of type foo."
  ...)

(defmethod json->data ((type (eq :bar)) payload)
  "Decode message of type bar."
  ...)

But now I am not really sure how to glue this together. What would you
recommend? Is this the right frame of mind at all or should I take a
totally different approach?

Thanks!

Hraban



From rpgoldman at sift.info  Sun Aug 14 16:19:36 2011
From: rpgoldman at sift.info (Robert P. Goldman)
Date: Sun, 14 Aug 2011 11:19:36 -0500
Subject: [cl-json-devel]
	=?utf-8?q?Guidance_requested=3A_how_to_dispatch_m?=
	=?utf-8?q?embers_to=09external_decoders=3F?=
In-Reply-To: <CAK273yJi8S-CRyRG=1VzeQx-UKkXY1NhNSOZ7SUmX2yvBMz8+Q@mail.gmail.com>
References: <CAK273yJi8S-CRyRG=1VzeQx-UKkXY1NhNSOZ7SUmX2yvBMz8+Q@mail.gmail.com>
Message-ID: <4b6ef696-df99-4bb4-b0bf-57cf059935bc@email.android.com>

Is it necessary to do this relatively complex thing, or can you simply decode the JSON object in a standard way and then decode the JSON object into your own object?

Yes, slightly less efficient, but more respectful of the API and likely better abstraction....
-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Hraban Luyat <hraban at 0brg.net> wrote:

Hello,

As far as I understand the decoder API it is mostly geared towards
defining the decoder for an entire snippet in one place. I would like
to know what the recommended way is to separate the definition of the
decoder over separate modules.

The semantics of my incoming messages are thus:

object:
- "type": string denoting the type
- "payload": type-specific payload

I want to create a decoder that only extracts the type and uses that
to determine which decoder to send the payload to. Then it continues
with whatever lisp object the decoder returned.

What I thought would be appropriate is to create a generic function;

(defgeneric json->data (type payload))

and then simply register decoders as follows:

(defmethod json->data ((type (eq :foo)) payload)
"Decode message of type foo."
...)

(defmethod json->data ((type (eq :bar)) payload)
"Decode message of type bar."
...)

But now I am not really sure how to glue this together. What would you
recommend? Is this the right frame of mind at all or should I take a
totally different approach?

Thanks!

Hraban

_____________________________________________

cl-json-devel mailing list
cl-json-devel at common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/cl-json-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-json-devel/attachments/20110814/1a115b2e/attachment.html>

From hraban at 0brg.net  Sun Aug 14 16:46:25 2011
From: hraban at 0brg.net (Hraban Luyat)
Date: Sun, 14 Aug 2011 18:46:25 +0200
Subject: [cl-json-devel] Guidance requested: how to dispatch members to
 external decoders?
In-Reply-To: <4b6ef696-df99-4bb4-b0bf-57cf059935bc@email.android.com>
References: <CAK273yJi8S-CRyRG=1VzeQx-UKkXY1NhNSOZ7SUmX2yvBMz8+Q@mail.gmail.com>
	<4b6ef696-df99-4bb4-b0bf-57cf059935bc@email.android.com>
Message-ID: <CAK273yJSkC5tymKDHCU8NmNXLUiC7Nou9S4BoTk9wh6amWL6PA@mail.gmail.com>

Hi Robert,

Strictly necessary, no. However, I actually think it is a better
abstraction than decoding the whole object with the default decoder.
While that seems to make the external module JSON agnostic, it will
still have to deal with the expected output of cl-json, making it a
rather empty abstraction advantage. However, by dispatching directly,
the general module acts purely as a proxy and does not impose any
restrictions on the structure of the payload element (which comes down
to more efficiency, as you mentioned).

The encoder actually allows for this pretty elegantly, I like that API
a lot. The general module sets up an object environment with
json:with->object, invokes json:as-object-member and passes the stream
to the external module. The latter, at this point, only needs to send
one json element over the stream and has no business with the wrapping
structure. Just like the general module has no business with the
semantics of the payload. Combining this with generic functions makes
for a very sober encoding framework.

Would it not be a useful feature if the API allowed for this? I am
interested in hearing your opinion(s) about it. Of course, there is a
difference between "should" and "will" be implemented, but that does
not make it an irrelevant discussion. :)

Greetings,

Hraban

2011/8/14 Robert P. Goldman <rpgoldman at sift.info>:
> Is it necessary to do this relatively complex thing, or can you simply
> decode the JSON object in a standard way and then decode the JSON object
> into your own object?
>
> Yes, slightly less efficient, but more respectful of the API and likely
> better abstraction....
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>
> Hraban Luyat <hraban at 0brg.net> wrote:
>>
>> Hello,
>>
>> As far as I understand the decoder API it is mostly geared towards
>> defining the decoder for an entire snippet in one place. I would like
>> to know what the recommended way is to separate the definition of the
>> decoder over separate modules.
>>
>> The semantics of my incoming messages are thus:
>>
>> object:
>>   - "type": string denoting the type
>>   - "payload": type-specific payload
>>
>> I want to create a decoder that only extracts the type and uses that
>> to determine which decoder to send the payload to. Then it continues
>> with whatever lisp object the decoder returned.
>>
>> What I thought would be appropriate is to create a generic function;
>>
>> (defgeneric json->data (type payload))
>>
>> and then simply register decoders as follows:
>>
>> (defmethod json->data ((type (eq :foo)) payload)
>>   "Decode
>>  message
>> of type foo."
>>   ...)
>>
>> (defmethod json->data ((type (eq :bar)) payload)
>>   "Decode message of type bar."
>>   ...)
>>
>> But now I am not really sure how to glue this together. What would you
>> recommend? Is this the right frame of mind at all or should I take a
>> totally different approach?
>>
>> Thanks!
>>
>> Hraban
>>
>> ________________________________
>> cl-json-devel mailing list
>> cl-json-devel at common-lisp.net
>> http://lists.common-lisp.net/cgi-bin/mailman/listinfo/cl-json-devel
>



From boris.smilga at gmail.com  Sun Aug 14 22:29:13 2011
From: boris.smilga at gmail.com (Boris Smilga)
Date: Mon, 15 Aug 2011 02:29:13 +0400
Subject: [cl-json-devel] Guidance requested: how to dispatch members to
	external decoders?
In-Reply-To: <CAK273yJi8S-CRyRG=1VzeQx-UKkXY1NhNSOZ7SUmX2yvBMz8+Q@mail.gmail.com>
References: <CAK273yJi8S-CRyRG=1VzeQx-UKkXY1NhNSOZ7SUmX2yvBMz8+Q@mail.gmail.com>
Message-ID: <00D95B14-3D36-4861-9B96-3074CBC6DDCE@gmail.com>

On 14 Aug 2011, at 16:24, Hraban Luyat wrote:

> The semantics of my incoming messages are thus:
>
> object:
>   - "type": string denoting the type
>   - "payload": type-specific payload
>
> I want to create a decoder that only extracts the type and uses that
> to determine which decoder to send the payload to. Then it continues
> with whatever lisp object the decoder returned.
>
> What I thought would be appropriate is to create a generic function;
>
> (defgeneric json->data (type payload))
>
> and then simply register decoders as follows:
>
> (defmethod json->data ((type (eq :foo)) payload)
>   "Decode message of type foo."
>   ...)
>
> (defmethod json->data ((type (eq :bar)) payload)
>   "Decode message of type bar."
>   ...)
>
> But now I am not really sure how to glue this together. What would you
> recommend? Is this the right frame of mind at all or should I take a
> totally different approach?

Why, yes, you can certainly make it that way.

Re. gluing it together: there is some relevant reading in CL-JSON  
User Guide under http://common-lisp.net/project/cl-json/#DECODER- 
CUSTOMIZATION .  To apply the API to your case, you'd have to define  
a dynamic variable which would store the type while the decoder is  
waiting for the payload to arrive, and another, which would store the  
payload while the decoder is waiting for the end of the object:

   (defvar *payload-type* nil)

   (defvar *payload* nil)

Then you customize your decoder to decode top-level JSON {} objects  
in the following way:

   (let ((default-decoder (json:current-decoder)))
     (json:bind-custom-vars
         (;; Initialize variables in the dynamic scope of the object:
          :beginning-of-object
           (lambda () (setq *payload-type* nil *payload* nil))
          ;; The handler for key sets the internal decoder and the
          ;; handler for value depending on the key:
          :object-key
           (lambda (json-id)
             (let ((lisp-id
                    (json:safe-json-intern
                      (funcall json:*json-identifier-name-to-lisp*
                               json-id))))
               (case lisp-id
                 ;; If the key is "type", the value is decoded in the
                 ;; standard way and stored in *payload-type*:
                 ((:type)
                  (json:set-custom-vars
                    :internal-decoder
                     default-decoder
                    :object-value
                     (lambda (value) (setq *payload-type* value))))
                 ;; If the key is "payload", the value is decoded by
                 ;; your type-specific decoder and stored in *payload*:
                 ((:payload)
                  (json:set-custom-vars
                    :internal-decoder
                     (lambda (stream)
                       (json->data *payload-type* stream))
                    :object-value
                     (lambda (value) (setq *payload* value)))))))
          ;; The value of the whole object is the value decoded from
          ;; its "payload" field:
          :end-of-object
           (lambda () *payload*)
          ;; This you need only if your type+payload objects may be
          ;; nested:
          :object-scope
           '(*payload* *payload-type*))
       (decode-json)))

For clarity, I've left out any safety checks, which you might want to  
put in here or there.  Also, this code includes a tacit assumption  
that, in your objects, the "type" field always precedes the "payload"  
field.  If you cannot guarantee this, you'll have to provide a json- 
 >data method for type NIL which maybe decodes to some intermediate  
representation, and add code to the value handler for "type" fields  
to finalize such semi-parsed values if found in *payload*.

Hope this helps; feel free to ask me for clarifications if you find  
some portion of the code to be too obscure.

Sincerely,
  - B. Smilga.




From hraban at 0brg.net  Mon Aug 15 19:20:09 2011
From: hraban at 0brg.net (Hraban Luyat)
Date: Mon, 15 Aug 2011 21:20:09 +0200
Subject: [cl-json-devel] Guidance requested: how to dispatch members to
 external decoders?
In-Reply-To: <00D95B14-3D36-4861-9B96-3074CBC6DDCE@gmail.com>
References: <CAK273yJi8S-CRyRG=1VzeQx-UKkXY1NhNSOZ7SUmX2yvBMz8+Q@mail.gmail.com>
	<00D95B14-3D36-4861-9B96-3074CBC6DDCE@gmail.com>
Message-ID: <CAK273y+_VxwV2sxYP2VKDQft45PZ-y9oBGLckbVFYv3oKefxFA@mail.gmail.com>

Aha! :internal-decoder! Perfect, exactly what I need. Thank you very
much for your extensive explanation.

Also, I will change the spec to send the data as an array instead of
an object to enforce order.

Thanks again.

Sincerely,

Hraban Luyat

2011/8/15 Boris Smilga <boris.smilga at gmail.com>:
> On 14 Aug 2011, at 16:24, Hraban Luyat wrote:
>
>> The semantics of my incoming messages are thus:
>>
>> object:
>> ?- "type": string denoting the type
>> ?- "payload": type-specific payload
>>
>> I want to create a decoder that only extracts the type and uses that
>> to determine which decoder to send the payload to. Then it continues
>> with whatever lisp object the decoder returned.
>>
>> What I thought would be appropriate is to create a generic function;
>>
>> (defgeneric json->data (type payload))
>>
>> and then simply register decoders as follows:
>>
>> (defmethod json->data ((type (eq :foo)) payload)
>> ?"Decode message of type foo."
>> ?...)
>>
>> (defmethod json->data ((type (eq :bar)) payload)
>> ?"Decode message of type bar."
>> ?...)
>>
>> But now I am not really sure how to glue this together. What would you
>> recommend? Is this the right frame of mind at all or should I take a
>> totally different approach?
>
> Why, yes, you can certainly make it that way.
>
> Re. gluing it together: there is some relevant reading in CL-JSON User Guide
> under http://common-lisp.net/project/cl-json/#DECODER-CUSTOMIZATION . ?To
> apply the API to your case, you'd have to define a dynamic variable which
> would store the type while the decoder is waiting for the payload to arrive,
> and another, which would store the payload while the decoder is waiting for
> the end of the object:
>
> ?(defvar *payload-type* nil)
>
> ?(defvar *payload* nil)
>
> Then you customize your decoder to decode top-level JSON {} objects in the
> following way:
>
> ?(let ((default-decoder (json:current-decoder)))
> ? ?(json:bind-custom-vars
> ? ? ? ?(;; Initialize variables in the dynamic scope of the object:
> ? ? ? ? :beginning-of-object
> ? ? ? ? ?(lambda () (setq *payload-type* nil *payload* nil))
> ? ? ? ? ;; The handler for key sets the internal decoder and the
> ? ? ? ? ;; handler for value depending on the key:
> ? ? ? ? :object-key
> ? ? ? ? ?(lambda (json-id)
> ? ? ? ? ? ?(let ((lisp-id
> ? ? ? ? ? ? ? ? ? (json:safe-json-intern
> ? ? ? ? ? ? ? ? ? ? (funcall json:*json-identifier-name-to-lisp*
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?json-id))))
> ? ? ? ? ? ? ?(case lisp-id
> ? ? ? ? ? ? ? ?;; If the key is "type", the value is decoded in the
> ? ? ? ? ? ? ? ?;; standard way and stored in *payload-type*:
> ? ? ? ? ? ? ? ?((:type)
> ? ? ? ? ? ? ? ? (json:set-custom-vars
> ? ? ? ? ? ? ? ? ? :internal-decoder
> ? ? ? ? ? ? ? ? ? ?default-decoder
> ? ? ? ? ? ? ? ? ? :object-value
> ? ? ? ? ? ? ? ? ? ?(lambda (value) (setq *payload-type* value))))
> ? ? ? ? ? ? ? ?;; If the key is "payload", the value is decoded by
> ? ? ? ? ? ? ? ?;; your type-specific decoder and stored in *payload*:
> ? ? ? ? ? ? ? ?((:payload)
> ? ? ? ? ? ? ? ? (json:set-custom-vars
> ? ? ? ? ? ? ? ? ? :internal-decoder
> ? ? ? ? ? ? ? ? ? ?(lambda (stream)
> ? ? ? ? ? ? ? ? ? ? ?(json->data *payload-type* stream))
> ? ? ? ? ? ? ? ? ? :object-value
> ? ? ? ? ? ? ? ? ? ?(lambda (value) (setq *payload* value)))))))
> ? ? ? ? ;; The value of the whole object is the value decoded from
> ? ? ? ? ;; its "payload" field:
> ? ? ? ? :end-of-object
> ? ? ? ? ?(lambda () *payload*)
> ? ? ? ? ;; This you need only if your type+payload objects may be
> ? ? ? ? ;; nested:
> ? ? ? ? :object-scope
> ? ? ? ? ?'(*payload* *payload-type*))
> ? ? ?(decode-json)))
>
> For clarity, I've left out any safety checks, which you might want to put in
> here or there. ?Also, this code includes a tacit assumption that, in your
> objects, the "type" field always precedes the "payload" field. ?If you
> cannot guarantee this, you'll have to provide a json->data method for type
> NIL which maybe decodes to some intermediate representation, and add code to
> the value handler for "type" fields to finalize such semi-parsed values if
> found in *payload*.
>
> Hope this helps; feel free to ask me for clarifications if you find some
> portion of the code to be too obscure.
>
> Sincerely,
> ?- B. Smilga.
>
>
> _______________________________________________
> cl-json-devel mailing list
> cl-json-devel at common-lisp.net
> http://lists.common-lisp.net/cgi-bin/mailman/listinfo/cl-json-devel
>