[pg-cvs] pg.lisp UTF8 SBCL

[Johan Ur Riise]

Hello, this is my first bug report on pg.lisp

Hmm, seems I registered on mailing list pg-cvs rather than pg-devel, 
http://common-lisp.net/mailman/listinfo/pg-devel lets me register on pg-cvs..

Context: I want to use unicode for the Postgresql database.

The Postgresql version is 8.1, from Ubuntu.
pg.lisp is from cvs

The common lisp is SBCL 0.9.8

I have, with psql \l-command:
autotest=> \l
        List of databases
   Name    |  Owner   | Encoding
-----------+----------+----------
 autotest  | postgres | UTF8
 postgres  | postgres | UTF8
 template0 | postgres | UTF8
 template1 | postgres | UTF8
(4 rows)

I put "UTF8" in *pg-client-encoding* 
(setf *pg-client-encoding* "UTF8")

With 
(pg-connect "autotest" "autotest" :password "autotest")
I get this:

Illegal :UTF-8 character starting at byte position 0.
   [Condition of type SB-IMPL::INVALID-UTF8-CONTINUATION-BYTE]

Restarts:
  0: [USE-VALUE] Supply a replacement string designator.
  1: [ABORT-REQUEST] Abort handling SLIME request.
  2: [TERMINATE-THREAD] Terminate this thread (#<THREAD "repl-thread" {AF55261}>)

Backtrace:
  0: (SB-IMPL::DECODING-ERROR #(231 48 140 144) 0 1 :UTF-8 SB-IMPL::INVALID-UTF8-CONTINUATION-BYTE 1)
      Locals:
        SB-DEBUG::ARG-0 = 6
        SB-DEBUG::ARG-1 = #(231 48 140 144)
        SB-DEBUG::ARG-2 = 0
        SB-DEBUG::ARG-3 = 1
        SB-DEBUG::ARG-4 = :UTF-8
        SB-DEBUG::ARG-5 = SB-IMPL::INVALID-UTF8-CONTINUATION-BYTE
        SB-DEBUG::ARG-6 = 1
  1: (SB-IMPL::BYTES-PER-UTF8-CHARACTER-AREF #<unavailable argument> #<unavailable argument> #<unavailable argument>)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-2 = :<NOT-AVAILABLE>
  2: (SB-IMPL::UTF8->STRING-AREF #(231 48 140 144) 0 4)
      Locals:
        SB-DEBUG::ARG-0 = 3
        SB-DEBUG::ARG-1 = #(231 48 140 144)
        SB-DEBUG::ARG-2 = 0
        SB-DEBUG::ARG-3 = 4
  3: ((SB-PCL::FAST-METHOD POSTGRESQL::READ-STRING-FROM-PACKET (POSTGRESQL::PG-PACKET INTEGER)) #<unavailable argument> #<unavailable argument> #<POSTGRESQL::PG-PACKET type: R length: 12 position: 8 {AF65829}> 4)
      Locals:
        SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
        SB-DEBUG::ARG-2 = #<POSTGRESQL::PG-PACKET type: R length: 12 position: 8 {AF65829}>
        SB-DEBUG::ARG-3 = 4
  4: (POSTGRESQL::PG-CONNECT/V3 "autotest" "autotest" :HOST "localhost" :PORT 5432 :PASSWORD "autotest")
  5: (PG-CONNECT #<unavailable argument> #<unavailable argument> :HOST #<unavailable argument> :PORT #<unavailable argument> :PASSWORD #<unavailable argument>)
  6: (SB-INT:EVAL-IN-LEXENV (PG-CONNECT "autotest" "autotest" :PASSWORD "autotest") #<NULL-LEXENV>)
  7: (SWANK::EVAL-REGION "(pg-connect \"autotest\" \"autotest\" :password \"autotest\")

I fooled around a little, and found that this is an R-packet handled in #'pg-connect/v3
in the #\R and (5) case, where "salt" is read with read-string-from-packet.

So I fixed the read-string-from-packet method to use conversion from latin1 in case
this was an #\R packet. Probably not the right place to do it, but it seems to work.

jur at lark:/usr/local/lib/common-lisp/systems/pg-cvs$ diff -u v3-protocol.lisp.orig v3-protocol.lisp
--- v3-protocol.lisp.orig       2006-08-27 20:27:51.000000000 +0200
+++ v3-protocol.lisp    2006-08-27 22:05:13.000000000 +0200
@@ -276,8 +276,12 @@
              length))
     (let* ((octects (read-octets-from-packet packet
                                             length))
-          (string (convert-string-from-bytes octects)))
-       string)))
+          (string (convert-string-from-bytes octects 
+                                             (if (eql #\R (pg-packet-type packet)) 
+                                                 (implementation-name-for-encoding "LATIN1")
+                                                 (implementation-name-for-encoding *pg-client-encoding*)))))
+      string)
+    ))
 
 (defmethod read-octets-from-packet ((packet pg-packet) (length integer))
   (let ((result (make-array length :element-type '(unsigned-byte 8))))


That is it works for SBCL, with CLISP I get PG[5]> (setf *pg-client-encoding* "UTF8")
"UTF8"
PG[6]> (pg-connect "autotest" "autotest" :password "autotest")

*** - STRING=: argument #<ENCODING CHARSET:ISO-8859-1 :UNIX> should be a string, a symbol or a
      character
which I have no answer for.


-- 
Hilsen
Johan Ur Riise