[cl-openid-devel] missing packages in shelf and trivial-utf8 bug

[Simon Cusack]

Hi,

Thanks for cl-openid I want to add openid logins to a website I have
written in lisp so this has been great to play with, it looks
excellent.

I had to add the following libs to shelf.lisp to get it to load
cleanly (cl-librarian made this part really painless, kudos).

(babel tarball-repo :source "asdf-install:babel")
(alexandria darcs-repo :source
"http://common-lisp.net/project/alexandria/darcs/alexandria/")
(trivial-features tarball-repo :source "asdf-install:trivial-features")

I then fired up hunchentoot and (after changing the realm) added the
required handler using;

 (hunchentoot:start-server :port 4242)
 (push (openid-ht-dispatcher "/cl-openid" "http://lizard.tasak.gda.pl:4242/") hunchentoot:*dispatch-table*)

I then tried to login using an openid that I have and got hit by an
error in UTF-8-BYTES-TO-STRING.

I tracked it down to one of the (UTF-8-BYTES-TO-STRING array :start
start :end colon) calls in PARSE-KV.

I did the http-request call at the REPL and then saved the body from
the response into *test-body*.

(defparameter *test-body* #(97 115 115 111 99 95 104 97 110 100 108 101
58 123 72 77 65 67 45 83 72 65 50 53 54 125 123 52 56 55 98 52 49 50 56
125 123 67 89 99 54 121 81 61 61 125 10 97 115 115 111 99 95 116 121
112 101 58 72 77 65 67 45 83 72 65 50 53 54 10 100 104 95 115 101 114
118 101 114 95 112 117 98 108 105 99 58 65 73 78 57 88 113 100 109 51
116 52 100 67 109 55 108 50 88 121 87 114 68 68 78 102 109 70 72 78 88
76 73 120 121 57 80 83 83 73 69 81 116 47 97 118 57 99 83 87 67 68 56
69 115 81 72 88 111 82 43 90 114 114 102 110 115 67 101 77 121 102 109
113 89 72 78 111 104 65 85 115 109 43 112 71 88 65 120 87 79 69 100 122
116 102 66 90 112 105 116 68 49 88 101 80 53 90 52 65 107 111 117 84 86
103 84 65 53 86 56 118 49 77 100 109 87 117 89 110 52 106 97 79 85 100
75 99 90 88 119 103 79 101 122 111 50 69 71 100 78 73 71 75 102 104 80
50 89 66 115 67 102 71 47 109 89 74 47 81 55 65 43 10 101 110 99 95 109
97 99 95 107 101 121 58 85 74 54 99 49 118 71 86 105 66 68 106 77 115
100 121 83 50 80 55 51 104 119 47 118 106 55 86 111 55 67 119 113 122
80 99 55 108 78 109 110 52 73 61 10 101 120 112 105 114 101 115 95 105
110 58 49 50 48 57 54 48 48 10 110 115 58 104 116 116 112 58 47 47 115
112 101 99 115 46 111 112 101 110 105 100 46 110 101 116 47 97 117 116
104 47 50 46 48 10 115 101 115 115 105 111 110 95 116 121 112 101 58 68
72 45 83 72 65 50 53 54 10))

Calling (utf-8-bytes-to-string *test-body*) results in
"Invalid byte at start of character: 0x84".

However (flexi-streams:octets-to-string
*test-body* :external-format :utf-8) results in;

"assoc_handle:{HMAC-SHA256}{487b4128}{CYc6yQ==}
assoc_type:HMAC-SHA256
dh_server_public:AIN9Xqdm3t4dCm7l2XyWrDDNfmFHNXLIxy9PSSIEQt/av9cSWCD8EsQHXoR+ZrrfnsCeMyfmqYHNohAUsm+pGXAxWOEdztfBZpitD1XeP5Z4AkouTVgTA5V8v1MdmWuYn4jaOUdKcZXwgOezo2EGdNIGKfhP2YBsCfG/mYJ/Q7A+
enc_mac_key:UJ6c1vGViBDjMsdyS2P73hw/vj7Vo7CwqzPc7lNmn4I=
expires_in:1209600
ns:http://specs.openid.net/auth/2.0
session_type:DH-SHA256
"

So I'm guessing it is a bug in trivial-utf8.

When I changed PARSE-KV to use flexi-streams instead like so;


(defun parse-kv (array)
  "Parse key-value form message passed as an (unsigned-byte 8) array into alist.

OpenID Authentication 2.0 4.1.1.  Key-Value Form Encoding."

  (loop for line in (cl-ppcre:split #\NewLine (flexi-streams:octets-to-string array :external-format :utf-8))
        as  colon = (position #\: line)
        collect (cons (subseq line 0 colon)
                      (subseq line (1+ colon)))))

And it worked beautifully, woohoo!

I'm not sure what the correct approach here is, I read the openid spec
and the above seems ok to me.  Is there a reason that trivial-utf8 was
used instead of flexi-streams and cl-ppcre?

regards, sim.