[cl-openid-devel] missing packages in shelf and trivial-utf8 bug

[Anton Vodonosov]

on Monday, July 14, 2008, 6:11:27 PM Maciek wrote:

>> I'm not sure what the correct approach here is, I read the openid spec
>> and the above seems ok to me.  Is there a reason that trivial-utf8 was
>> used instead of flexi-streams and cl-ppcre?

> Efficiency (trivial-utf-8 is small and fast) and keeping direct
> dependency list as little as possible.  CL-OpenID is supposed to be
> usable with different HTTP server and client libraries, so I decided not
> to use flexi-streams even though it comes with Hunchentoot and Drakma --
> it is known to be quite slow, and introduces yet another dependency (on
> trivial-gray-streams).  Flexi-streams do recoding strings via Gray
> streams CLOS machinery, which just has to be much slower than simply
> operating on array of bytes -- more flexible, too, but CL-OpenID doesn't
> need this flexibility.

Flexi-streams are not slow. Some work was invested into optimization
recently. Take a look:

(defparameter *test-body*
  #(97 115 115 111 99 95 104 97 110 100 108 101
    58 123 72 77 65 67 45 83 72 65 50 53 54 125 123 52 56 55 98 52 49 50 56
    125 123 67 89 99 54 121 81 61 61 125 10 97 115 115 111 99 95 116 121
    112 101 58 72 77 65 67 45 83 72 65 50 53 54 10 100 104 95 115 101 114
    118 101 114 95 112 117 98 108 105 99 58 65 73 78 57 88 113 100 109 51
    116 52 100 67 109 55 108 50 88 121 87 114 68 68 78 102 109 70 72 78 88
    76 73 120 121 57 80 83 83 73 69 81 116 47 97 118 57 99 83 87 67 68 56
    69 115 81 72 88 111 82 43 90 114 114 102 110 115 67 101 77 121 102 109
    113 89 72 78 111 104 65 85 115 109 43 112 71 88 65 120 87 79 69 100 122
    116 102 66 90 112 105 116 68 49 88 101 80 53 90 52 65 107 111 117 84 86
    103 84 65 53 86 56 118 49 77 100 109 87 117 89 110 52 106 97 79 85 100
    75 99 90 88 119 103 79 101 122 111 50 69 71 100 78 73 71 75 102 104 80
    50 89 66 115 67 102 71 47 109 89 74 47 81 55 65 43 10 101 110 99 95 109
    97 99 95 107 101 121 58 85 74 54 99 49 118 71 86 105 66 68 106 77 115
    100 121 83 50 80 55 51 104 119 47 118 106 55 86 111 55 67 119 113 122
    80 99 55 108 78 109 110 52 73 61 10 101 120 112 105 114 101 115 95 105
    110 58 49 50 48 57 54 48 48 10 110 115 58 104 116 116 112 58 47 47 115
    112 101 99 115 46 111 112 101 110 105 100 46 110 101 116 47 97 117 116
    104 47 50 46 48 10 115 101 115 115 105 111 110 95 116 121 112 101 58 68
    72 45 83 72 65 50 53 54 10))

(time
 (dotimes (i 100000)
  (utf-8-bytes-to-string *test-body*)))

Evaluation took:
  4.813 seconds of real time
  4.8125 seconds of user run time
  0.0 seconds of system run time
  [Run times include 0.454 seconds GC run time.]
  0 calls to %EVAL
  0 page faults and
  528,785,288 bytes consed.

(defparameter *utf8-ex-fmt* (flexi-streams:make-external-format :utf-8))

(time
 (dotimes (i 100000)
   (flexi-streams:octets-to-string *test-body* 
                                   :external-format *utf8-ex-fmt*)))

Evaluation took:
  4.109 seconds of real time
  4.109375 seconds of user run time
  0.0 seconds of system run time
  [Run times include 0.078 seconds GC run time.]
  0 calls to %EVAL
  0 page faults and
  159,986,344 bytes consed.

This is SBCL with flexi-streams-1.0.3 and the latest trivial-utf-8.

- Anton