[cl-openid-devel] missing packages in shelf and trivial-utf8 bug

[Maciek Pasternacki]

Hello,

On Mon, 2008-07-14 at 22:45 +1000, Simon Cusack wrote:
> Thanks for cl-openid I want to add openid logins to a website I have
> written in lisp so this has been great to play with, it looks
> excellent.

Please remember it's only prototype at the moment; after implementation
of OpenID Provider prototype it will undergo major refactoring into a
usable library.  It's probably not usable at all yet.  If you're willing
to help in testing this beast, it would be great (RP can use some
testing with different providers).

> I had to add the following libs to shelf.lisp to get it to load
> cleanly (cl-librarian made this part really painless, kudos).

Thanks!  I was tired myself by downloading endless dependencies, so I
started the Librarian.  However, it does not automatically discover
dependencies, like asdf-install, yet, and that's where your problem
comes from.

> (babel tarball-repo :source "asdf-install:babel")
> (alexandria darcs-repo :source "http://common-lisp.net/project/alexandria/darcs/alexandria/")
> (trivial-features tarball-repo :source "asdf-install:trivial-features")

I added this to the Librarian (since it's within Hunchentoot's
dependencies that are only included by shelf.lisp -- CFFI introduced
those recently, and I still have version without those dependencies).  I
added the main darcs repos of all projects -- Librarian prefers those.
Just darcs pull in 'lib/' subdirectory, and no changes to shelf.lisp
are needed.

> I then tried to login using an openid that I have and got hit by an
> error in UTF-8-BYTES-TO-STRING.
> 
> I tracked it down to one of the (UTF-8-BYTES-TO-STRING array :start
> start :end colon) calls in PARSE-KV.
> 
> I did the http-request call at the REPL and then saved the body from
> the response into *test-body*.
[...]
> Calling (utf-8-bytes-to-string *test-body*) results in
> "Invalid byte at start of character: 0x84".

As Anton wrote, it's already fixed in trivial-utf-8 repo.

> I'm not sure what the correct approach here is, I read the openid spec
> and the above seems ok to me.  Is there a reason that trivial-utf8 was
> used instead of flexi-streams and cl-ppcre?

Efficiency (trivial-utf-8 is small and fast) and keeping direct
dependency list as little as possible.  CL-OpenID is supposed to be
usable with different HTTP server and client libraries, so I decided not
to use flexi-streams even though it comes with Hunchentoot and Drakma --
it is known to be quite slow, and introduces yet another dependency (on
trivial-gray-streams).  Flexi-streams do recoding strings via Gray
streams CLOS machinery, which just has to be much slower than simply
operating on array of bytes -- more flexible, too, but CL-OpenID doesn't
need this flexibility.

As for CL-PPCRE:SPLIT, for similar reasons, I prefer to use
SPLIT-SEQUENCE when possible.

Also, your solution conses more than one with TRIVIAL-UTF-8: first a new
string is created by FLEXI-STREAMS (which itself would cons a bit
internally), then a new string for every line, and then a new string for
every key and value.  My solution works directly on octet array and
creates only strings that are actually needed: those for keys and
values.

Regards and thanks for feedback,
Maciej.

-- 
-><- Maciej 'japhy' Pasternacki -><- http://www.pasternacki.net/ -><-