[cl-json-devel] proposed: improvements to decoder customization

[Hans Hübner]

On Mon, Aug 4, 2008 at 05:47, Henrik Hjelte <henrik at evahjelte.com> wrote:
> On Mon, Aug 4, 2008 at 1:58 AM, Hans Hübner <hans at huebner.org> wrote:
>
>> I don't quite like the serialization mechanism that cl-json provides.
>> The attempt to map from a Lisp datatype to a certain json structure is
>> necessarily imperfect because no 1:1 relationship exists, and the
>> requirement to first make up a data structure and then call
>> encode-json to convert it to a json string is wasteful.
>
> In principle yes. But to defend the current way, it make a clear division of
> concerns, separating the code for formating output from the code for logic.

I don't quite understand what you mean by that.  My concern with the
current mechanism is that it is not transparent:  As there is no 1:1
mapping between native Lisp datatypes and json structures, the
application needs to be aware of the desired JSON structure format
anyway.  One would have to have kind of a "JSON Schema" that is
external to the application and describes the format in an abstract
way if one really wanted to keep the format specification outside of
the application code.  I am not sure if that would be the right
direction, though.  The idea of JSON is to keep things simple.

>> A streaming serialization API is more useful, as one has more control
>> over the json format that is being generated and the need to make up a
>> data structure that can uniquely be mapped to json structures is
>> removed.
>
> I agree to a large degree, but in many situations it is nice to
> have a simple to use format and API. What is simple is of course up
> to debate. But in the sample below, I think cl-jsons current encoder
> could save some code lines. But, why not have both, I am sure
> there are lots of situations were your idea will be better?

I am not advocating that the current API should be removed.  I just
don't like it and would have to have more control over the format
without having to first create a data structure that accidentially
serializes to what I want to have.  Even with a specially crafted
structure, I would have to make explicit calls to specialized JSON
functions to select how I want lists to be serialized.  A streaming
API would be unambiguous in that respect, as it does not iterate over
data structures itself.

I am a big fan of saving code lines, but in general, terseness can't
overrule correctness and ambiguity should be avoided.

> In a way this is the reverse situation of the decoder issue that Boris
> has.
>
>> It is inspired by CXML's streaming serialization which I
>> find very easy and straightforward to use.
>
> It is the SAX thing that is used for serailization that is the inspiration?
> CXML also has a klacks parser which is inspired from the Java
> Streaming XML API, and I know there is a project jettison that
> implements JSON for this API.

The CXML XML serializer is inspired by SAX.

> A cool feature would be if a json parser/serializer could be used as a
> simple drop in replacement for an existing xml parser/geneneratior,
> there are a few to choose from.

I'm not sure if you would not end up with a half-baked solution that
works either inefficient or restricts both the XML schemata and JSON
structures that you can create interchangeably.  I would leave such a
plug-compatible layer up to the application programmer to write, if
desired.

> ;;------------------------------------------------------------------------------
> (defmethod handle-object ((handler json-news-archive-handler)
>                          (channel rss-channel))
>  (with-json-response ()
>    (with-object-element ("months")
>      (with-json-array ()
>        (dolist (month (sort (rss-channel-archived-months channel)
>                              (lambda (a b)
>                                (if (= (first a) (first b))
>                                    (> (second a) (second b))
>                                    (> (first a) (first b))))))
>          (with-json-array ()
>            (encode-array-element (first month))
>            (encode-array-element (second month))))))))
>
> ;;------------------------------------------------------------------------------
> ;; If I am not wrong, this is how it would look in the
> ;; current cl-json implementation.
>
> (defmethod handle-object ((handler json-news-archive-handler)
>                          (channel rss-channel))
>  (encode-json-response ;; or similar
>    (with-object-element ("months")
>      (sort (rss-channel-archived-months channel)
>                              (lambda (a b)
>                                (if (= (first a) (first b))
>                                    (> (second a) (second b))
>                                    (> (first a) (first b))))))))

I am not sure how an ENCODE-JSON-RESPONSE function would decide
whether a list should be serialized as an object or as an array.  This
is the basic problem that the streaming API solves.  I think that no
ENCODE-JSON-RESPONSE function exists that matches the requirements of
every application, and as such it would be better to provide the
building blocks that can be used to universally generate any JSON
format that exists.

-Hans