[flexi-streams-devel] flexi-streams 1.0.3 and CMUCL 19e

[Eugene Ossintsev]

Hi,

I was trying to compile flexi-streams 1.0.3 with CMUCL 19e and it stopped with 
the following message in ascii.lisp :

--------------------------------------------------------
Type-error in KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER:
    65533 is not of type (UNSIGNED-BYTE 8)
    [Condition of type TYPE-ERROR]
--------------------------------------------------------

As I see, in MAKE-DECODING-TABLE the function MAKE-ARRAY has
:element-type 'char-code-integer.


And CHAR-CODE-INTEGER is defined as

--------------------------------------------------------
(deftype char-code-integer ()
   "The subtype of integers which can be returned by the function CHAR-CODE."
   '(integer 0 #.(1- char-code-limit)))
--------------------------------------------------------

But in CMUCL CHAR-CODE-LIMIT is 256, that is not enough to hold 65533.


If I modify mapping.lisp the following way (see also the attached patch):

--------------------------------------------------------
  (deftype char-code-integer ()
    "The subtype of integers which can be returned by the function CHAR-CODE."
-  '(integer 0 #.(1- char-code-limit)))
+  #-:cmu '(integer 0 #.(1- char-code-limit))
+  #+:cmu '(integer 0 65533))
--------------------------------------------------------


it's compiled with CMUCL successfully and a simple Hunchentoot web application 
of mine runs without any problem.

But if I try to perform the tests from flexi-streams/test I get a bunch of 
successful results

--------------------------------------------------------
FLEXI-STREAMS-TEST> (run-tests)
Test (using COPY-STREAM) "kafka_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [INPUT]
  --> (:UTF-8 :EOL-STYLE :LF) [OUTPUT].

Test (using COPY-STREAM) "kafka_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [IO]
  --> (:UTF-8 :EOL-STYLE :LF) [OUTPUT].

Test (using COPY-STREAM) "kafka_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [INPUT]
  --> (:UTF-8 :EOL-STYLE :LF) [IO].

   [[ skipped; too many to put them all here ]]
--------------------------------------------------------


followed by some failures (all with the hebrew* files):

--------------------------------------------------------
Test (using COPY-STREAM) "hebrew_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [INPUT]
  --> (:UTF-8 :EOL-STYLE :LF) [OUTPUT].
  Test failed!!!

   [[ skipped ]]

Test (using COPY-STREAM) "hebrew_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [IO]
  --> (:UTF-8 :EOL-STYLE :CRLF) [IO].
  Test failed!!!

Test (using COPY-STREAM) "hebrew_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [INPUT]
  --> (:ISO-8859-8 :EOL-STYLE :LF) [OUTPUT].
--------------------------------------------------------


and eventually, hello the debugger!

--------------------------------------------------------
#\å (code NIL) is not in this encoding.
    [Condition of type EXTERNAL-FORMAT-ENCODING-ERROR]

Restarts:
  0: [ABORT] Return to SLIME's top level.
  1: [ABORT] Return to Top-Level.

Backtrace:
   0: (FLEXI-STREAMS::SIGNAL-ENCODING-ERROR #<FLEXI-STREAMS::FLEXI-8-BIT-FORMAT 
(:ISO-8859-8 :EOL-STYLE :LF) {5A537DA5}> "~S (code ~A) is not in this encoding." 
#\å NIL)
   1: (FLEXI-STREAMS::WRITE-SEQUENCE* #<#1=unavailable-arg> #<#1#> #<#1#> #<#1#> 
...)
   2: (TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE #<#1=unavailable-arg> #<#1#> 
#<#1#> #<#1#> ...)
   3: (LISP::WRITE-LINE* ":åèÐÔ êÐÕ ÝÙÞéÔ êÐ ÝÙÔÜÐ ÐèÑ êÙéÐèÑ 	Ð  1" 
#<FLEXI-OUTPUT-STREAM {59392C6D}> 0 41)
   4: (COPY-STREAM #<Stream for file 
"/home/eugene/lisp/flexi-streams-1.0.3/test/hebrew_utf8_lf.txt"> 
#<FLEXI-STREAMS::FLEXI-UTF-8-FORMAT (:UTF-8 :EOL-STYLE :LF) {5A537D75}> #<Stream 
for file "/tmp/odd-streams-test/hebrew_latin8_lf.txt"> 
#<FLEXI-STREAMS::FLEXI-8-BIT-FORMAT (:ISO-8859-8 :EOL-STYLE :LF) {5A537DA5}>)
   5: (COPY-FILE 
#P"/home/eugene/lisp/flexi-streams-1.0.3/test/hebrew_utf8_lf.txt" 
#<FLEXI-STREAMS::FLEXI-UTF-8-FORMAT (:UTF-8 :EOL-STYLE :LF) {5A537D75}> 
#P"/tmp/odd-streams-test/hebrew_latin8_lf.txt" 
#<FLEXI-STREAMS::FLEXI-8-BIT-FORMAT (:ISO-8859-8 :EOL-STYLE :LF) {5A537DA5}> ...)
   6: (COMPARE-FILES "hebrew_utf8_lf.txt" #<FLEXI-STREAMS::FLEXI-UTF-8-FORMAT 
(:UTF-8 :EOL-STYLE :LF) {5A537D75}> "hebrew_latin8_lf.txt" 
#<FLEXI-STREAMS::FLEXI-8-BIT-FORMAT (:ISO-8859-8 :EOL-STYLE :LF) {5A537DA5}>)
   7: (RUN-TESTS)
  --more--
--------------------------------------------------------



I don't know if it's the same or similar for the other Common Lisp 
implementations, but that patch allows flexi-streams at least to be compiled 
with CMUCL.

I know it's far from being perfect. I hope there are much more experienced 
people here, who know how to fix it.



- Eugene
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mapping.lisp.diff
Type: text/x-diff
Size: 453 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/flexi-streams-devel/attachments/20080715/401d428c/attachment.diff>