From edi at agharta.de  Fri Oct  6 12:02:44 2006
From: edi at agharta.de (Edi Weitz)
Date: Fri, 06 Oct 2006 14:02:44 +0200
Subject: [drakma-devel] New Chunga version 0.2.0
Message-ID: <upsd5irdn.fsf@agharta.de>

ChangeLog:

  Version 0.2.0
  2006-10-06
  Only wrap inner stream with flexi stream if really needed

Download:

  http://weitz.de/files/chunga.tar.gz

Cheers,
Edi.


From edi at agharta.de  Fri Oct  6 23:53:09 2006
From: edi at agharta.de (Edi Weitz)
Date: Sat, 07 Oct 2006 01:53:09 +0200
Subject: [drakma-devel] New version 0.5.1
Message-ID: <ud59580ii.fsf@agharta.de>

ChangeLog:

  Version 0.5.1
  2006-10-07
  Take Content-Encoding header into account (due to a bug report by Gregory Tod)

Download:

  http://weitz.de/files/drakma.tar.gz

Cheers,
Edi.


From edi at agharta.de  Sun Oct  8 21:57:54 2006
From: edi at agharta.de (Edi Weitz)
Date: Sun, 08 Oct 2006 23:57:54 +0200
Subject: [drakma-devel] New version 0.5.2
Message-ID: <uk63a4gil.fsf@agharta.de>

ChangeLog:

  Version 0.5.2
  2006-10-08
  Adher to user-provided content length if FORM-DATA is true

Download:

  http://weitz.de/files/drakma.tar.gz

Cheers,
Edi.


From edi at agharta.de  Wed Oct 11 14:02:20 2006
From: edi at agharta.de (Edi Weitz)
Date: Wed, 11 Oct 2006 16:02:20 +0200
Subject: [drakma-devel] New version 0.5.3 (Was: [Lispweb] Announcement:
	Drakma)
In-Reply-To: <87r6xf3v2g.fsf@freemail.hu> (Magnus Henoch's message of "Wed,
	11 Oct 2006 14:17:59 +0200")
References: <ufydw0wyp.fsf@agharta.de> <87r6xf3v2g.fsf@freemail.hu>
Message-ID: <u4pub54sz.fsf_-_@agharta.de>

On Wed, 11 Oct 2006 14:17:59 +0200, Magnus Henoch <mange at freemail.hu> wrote:

> I tested it with CLISP (CVS version, slightly post-2.40).  I used
> the examples on the web page; it worked fine until the binary
> example.  In line 121 of request.lisp, READ-SEQUENCE for some reason
> extracts characters from STREAM, which it then fails to put into the
> byte array RESULT:
>
> SYSTEM::STORE: #\G does not fit into #(0 0 0 0 0 0 0 0 0 0 ...), bad type
>    [Condition of type SIMPLE-TYPE-ERROR]
>
> The TEXTP parameter to READ-BODY is NIL, and
> (flexi-streams:flexi-stream-element-type stream) returns CHARACTER.

Hmm, CLISP certainly tries hard to be different from the pack... :)

Anyway, please try the new version 0.5.3 and see if it makes a
difference.

Thanks for the report,
Edi.


From vodonosov at mail.ru  Tue Oct 24 03:05:04 2006
From: vodonosov at mail.ru (Anton Vodonosov)
Date: Tue, 24 Oct 2006 06:05:04 +0300
Subject: [drakma-devel] Error when chunked transfer encoding is used on clisp
Message-ID: <FBENJDGGPLPHJJCNFLMJCEPJCCAA.vodonosov@mail.ru>

Hi!

I've decided to repost this messsage here,
because the message I've posted to lispweb
mailing list contains stack backtrace
produced by slime. But, as it turned out,
slime debugger doesn't work well with clisp.

The problem is that the http-request function
fails when it's used for server with chunker
transfer encoding.

For example, all the following calls fail:

(http-request "http://www.phpsecurepages.com/test/test.php")
(http-request "http://bcheck.scanit.be/bcheck/")
(http-request "http://google.com")
(http-request "http://weitz.de/drakma/")

whereas following cals work fine:

(http-request "http://zappa.com/favicon.ico")
(http-request "http://www.jalat.com/blogs/lisp?id=3")
(http-request "http://www.lispworks.com/")
(http-request "http://meme.b9.com/login.html")

All the failed calls use cunked transfer
encoding; all successful - don't use chunked
transfer encoding.

The error is always the same:
SYSTEM::STORE: #\< does not fit into #(0 0 0 0 0 0 0 0 0 0 ...), bad type

See backtrace bellow.

Also, consider an example:

;; in order to use the
;; chunked-stream-input-stream
;; function
(in-package :chunga)

(setq *stream*
      (drakma:http-request "http://weitz.de/drakma/"
                           :want-stream t))

*stream*
	=> #<FLEXI-IO-STREAM #x1A223C11>

(flexi-streams:flexi-stream-stream *stream*)
	=> #<CHUNKED-IO-STREAM #x1A221BBD>

(chunked-stream-input-stream (flexi-streams:flexi-stream-stream *stream*))
	=> #<FLEXI-IO-STREAM #x1A223C4D>

(flexi-streams:flexi-stream-stream (chunked-stream-input-stream
(flexi-streams:flexi-stream-stream *stream*)))
	=> #<IO UNBUFFERED SOCKET-STREAM (UNSIGNED-BYTE 8) weitz.de:80>

As far as I understand, chunked streams expect
underlying stream to be binary.

But in this case, chunked input stream has
flexi stream as underlying stream.

-----------------------------------------
Backtrace for the error signaled by
(http-request "http://weitz.de/drakma/")
-----------------------------------------
The error itself:


#<SIMPLE-TYPE-ERROR #x1A22990D>:  standard object
 type: SIMPLE-TYPE-ERROR
0 [$DATUM]:  #\<
1 [$EXPECTED-TYPE]:  (UNSIGNED-BYTE 8)
2 [$FORMAT-CONTROL]:
"~S: ~S does not fit into ~S, bad type
"
3 [$FORMAT-ARGUMENTS]:  (SYSTEM::STORE #\< #<ARRAY (UNSIGNED-BYTE 8) (3566)
#x1A228AE5>)

-------------------------------------------
Backtrace:


...
<11> #<SYSTEM-FUNCTION INVOKE-DEBUGGER> 1
frame binding variables (~ = dynamically):
  | ~ SYSTEM::*PRIN-STREAM* <--> #<UNBOUND>
frame binding variables (~ = dynamically):
  | ~ *PRINT-READABLY* <--> NIL
frame binding variables (~ = dynamically):
  | ~ *PRINT-ESCAPE* <--> T
- #<SIMPLE-TYPE-ERROR #x1A230399>
-
#(0 0 0 0 0 ...)
<12> #<SYSTEM-FUNCTION SYSTEM::STORE> 3
- #\<
- 0
-
#(8 #<SYSTEM-FUNCTION SYSTEM::VECTOR-INIT> #<SYSTEM-FUNCTION
SYSTEM::VECTOR-UPD> #<SYSTEM-FUNCTION SYSTEM::VECTOR-ENDTEST>
  #<SYSTEM-FUNCTION SYSTEM::VECTOR-FE-INIT> #<SYSTEM-FUNCTION
SYSTEM::VECTOR-FE-UPD>
  #<SYSTEM-FUNCTION SYSTEM::VECTOR-FE-ENDTEST> #<SYSTEM-FUNCTION AREF>
#<SYSTEM-FUNCTION SYSTEM::STORE>
  #<SYSTEM-FUNCTION IDENTITY> #<SYSTEM-FUNCTION SYSTEM::VECTOR-LENGTH>
  #<COMPILED-FUNCTION #:|67 91 (MAPC #'(LAMBDA # #) '(1 2 4 ...))-5-1|>
#<SYSTEM-FUNCTION AREF>
  #<SYSTEM-FUNCTION SYSTEM::STORE> #<SYSTEM-FUNCTION
SYSTEM::VECTOR-INIT-START>
  #<SYSTEM-FUNCTION SYSTEM::VECTOR-FE-INIT-END>)
- 3566
- 0
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D>
-
#(0 0 0 0 0 ...)
<13> #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE>
- #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE>
- (UNSIGNED-BYTE 8)
- CHARACTER
- #<COMPILED-FUNCTION READ-SEQUENCE>
- 3566
- 0
- (:START 0 :END 3566)
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D>
-
#(0 0 0 0 0 ...)
<14> #<COMPILED-FUNCTION READ-SEQUENCE>
- #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>
- 3566
- #<COMPILED-FUNCTION CHUNGA::GET-CHUNK-SIZE>
-
#(#(#(NIL #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>)
#<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D>)
  #<COMPILED-FUNCTION CHUNGA::ADD-EXTENSIONS>)
frame binding variables (~ = dynamically):
  | ~ CHUNGA::*CURRENT-ERROR-FUNCTION* <--> #<COMPILED-FUNCTION #:|86 143
(DEFMETHOD FILL-BUFFER (#) ...)-9-1-1-1|>
- #(#(NIL #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>)
#<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D>)
- #(NIL #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>)
- #<COMPILED-FUNCTION #:|86 143 (DEFMETHOD FILL-BUFFER (#) ...)-9-1-1|>
- #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>
<15> #<COMPILED-FUNCTION #:|86 143 (DEFMETHOD FILL-BUFFER (#) ...)-9-1-1|>
<16> #<STANDARD-GENERIC-FUNCTION CHUNGA::FILL-BUFFER>
- #<COMPILED-FUNCTION #:|145 156 (DEFMETHOD STREAM-READ-BYTE (#)
...)-10-1-1|>
- #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>
<17> #<COMPILED-FUNCTION #:|145 156 (DEFMETHOD STREAM-READ-BYTE (#)
...)-10-1-1|>
<18> #<STANDARD-GENERIC-FUNCTION GRAY:STREAM-READ-BYTE>
- NIL
- NIL
- #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>
<19> #<SYSTEM-FUNCTION READ-BYTE>
- #<COMPILED-FUNCTION FLEXI-STREAMS::READ-BYTE*>
- NIL
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>
<20> #<COMPILED-FUNCTION FLEXI-STREAMS::READ-BYTE*>
- #<COMPILED-FUNCTION FLEXI-STREAMS::READ-CHAR-8-BIT>
-
#(0 1 2 3 4 5 ... 253 254 255)
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>
<21> #<COMPILED-FUNCTION FLEXI-STREAMS::READ-CHAR-8-BIT>
- #<COMPILED-FUNCTION FLEXI-STREAMS::GET-CHAR-CODE>
<22> #<COMPILED-FUNCTION FLEXI-STREAMS::GET-CHAR-CODE>
- #<COMPILED-FUNCTION FLEXI-STREAMS::GET-CHAR-CODE>
- :LF
-
#(#(NIL #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>)
  #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x1A22CB99>
:ISO-8859-1)
- #(NIL #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>)
- #<COMPILED-FUNCTION #:|221 275 (DEFMETHOD STREAM-READ-CHAR (#)
...)-11-1-1|>
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>
<23> #<COMPILED-FUNCTION #:|221 275 (DEFMETHOD STREAM-READ-CHAR (#)
...)-11-1-1|>
<24> #<STANDARD-GENERIC-FUNCTION GRAY:STREAM-READ-CHAR>
- NIL
- 0
- #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE
START ...) ...)-13-1-1-1|>
- 8192
- 0
- "^@^@^@^@^@..."
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>
<25> #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (#
SEQUENCE START ...) ...)-13-1-1-1|>
- #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE
START ...) ...)-13-1-1-1|>
- #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE
START ...) ...)-13-1-1|>
- NIL
- 8192
- 0
- "^@^@^@^@^@..."
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>
<26> #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (#
SEQUENCE START ...) ...)-13-1-1|>
<27> #<STANDARD-GENERIC-FUNCTION TRIVIAL-GRAY-STREAMS:STREAM-READ-SEQUENCE>
- 8192
- 0
-
#<COMPILED-FUNCTION
  #:|30 105 (PROGN (DEFMETHOD GRAY:STREAM-READ-BYTE-SEQUENCE # ...)
(DEFMETHOD GRAY:STREAM-WRITE-BYTE-SEQUENCE # ...) ...)-11-3-1-1|>
- (0 8192)
- "^@^@^@^@^@..."
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>
<28>
#<COMPILED-FUNCTION
  #:|30 105 (PROGN (DEFMETHOD GRAY:STREAM-READ-BYTE-SEQUENCE # ...)
(DEFMETHOD GRAY:STREAM-WRITE-BYTE-SEQUENCE # ...) ...)-11-3-1-1|>
<29> #<STANDARD-GENERIC-FUNCTION GRAY:STREAM-READ-CHAR-SEQUENCE>
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>
- "^@^@^@^@^@..."
- 8192
- 0
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>
- "^@^@^@^@^@..."
<30> #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE>
- #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE>
- CHARACTER
- CHARACTER
- #<COMPILED-FUNCTION READ-SEQUENCE>
- #<UNBOUND>
- #<UNBOUND>
- NIL
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>
- "^@^@^@^@^@
<31> #<COMPILED-FUNCTION READ-SEQUENCE>
- NIL
- 0
- ""
- "^@^@^@^@^@
- T
- CHARACTER
- NIL
-
#(NIL
  ((:DATE . "Tue, 24 Oct 2006 02:51:03 GMT") (:SERVER . "Apache")
(:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked")
   (:CONTENT-TYPE . "text/html; charset=iso-8859-1")))
- #<COMPILED-FUNCTION DRAKMA::READ-BODY>
- #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x1A22CB99>
- T
-
((:DATE . "Tue, 24 Oct 2006 02:51:03 GMT") (:SERVER . "Apache") (:CONNECTION
. "close") (:TRANSFER-ENCODING . "chunked")
 (:CONTENT-TYPE . "text/html; charset=iso-8859-1"))
- #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>
<32> #<COMPILED-FUNCTION DRAKMA::READ-BODY>
- NIL
- #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x1A22CB99>
- NIL
-
((:DATE . "Tue, 24 Oct 2006 02:51:03 GMT") (:SERVER . "Apache") (:CONNECTION
. "close") (:TRANSFER-ENCODING . "chunked")
 (:CONTENT-TYPE . "text/html; charset=iso-8859-1"))
- 200
- :HTTP/1.1
- #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-FINISH-REQUEST>
- NIL
- NIL
<33> #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-FINISH-REQUEST>
- #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-FINISH-REQUEST>
- #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-WRITE-HEADER>
-
#(#(#(NIL #<URI http://weitz.de/drakma/> NIL :HTTP/1.1 :GET NIL NIL NIL 5
(:GET :HEAD) NIL :LATIN-1 :LATIN-1 NIL NIL)
    #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> T)
  #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-WRITE-HTTP-LINE>)
unwind-protect frame
-
#(#(NIL #<URI http://weitz.de/drakma/> NIL :HTTP/1.1 :GET NIL NIL NIL 5
(:GET :HEAD) NIL :LATIN-1 :LATIN-1 NIL NIL)
  #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> T)
frame binding variables (~ = dynamically):
  | ~ FLEXI-STREAMS:*DEFAULT-EOL-STYLE* <--> :LF
- NIL
- :LF
compiled block frame for HTTP-REQUEST
- NIL
- #(NIL #<URI http://weitz.de/drakma/> NIL :HTTP/1.1 :GET NIL NIL NIL 5
(:GET :HEAD) NIL :LATIN-1 :LATIN-1 NIL NIL)
- #<COMPILED-FUNCTION HTTP-REQUEST>
- NIL
- #<UNBOUND>
- #<UNBOUND>
- #<UNBOUND>
- #<UNBOUND>
- T
- NIL
- #<UNBOUND>
- #<UNBOUND>
- #<UNBOUND>
- #<UNBOUND>
- NIL
- NIL
- "*/*"
- :DRAKMA
- NIL
- #<UNBOUND>
- NIL
- NIL
- "application/x-www-form-urlencoded"
- NIL
- NIL
- #<UNBOUND>
- #<UNBOUND>
- #<UNBOUND>
- NIL
- "http://weitz.de/drakma/"
<34> #<COMPILED-FUNCTION HTTP-REQUEST>
- #<COMPILED-FUNCTION HTTP-REQUEST>
EVAL frame for form (HTTP-REQUEST "http://weitz.de/drakma/")
frame binding environments
  VAR_ENV <--> NIL
  FUN_ENV <--> NIL
  BLOCK_ENV <--> NIL
  GO_ENV <--> NIL
  DECL_ENV <--> ((DECLARATION OPTIMIZE DECLARATION))
<35> ...

Regards,
-Anton



From edi at agharta.de  Thu Oct 26 01:29:31 2006
From: edi at agharta.de (Edi Weitz)
Date: Thu, 26 Oct 2006 03:29:31 +0200
Subject: [drakma-devel] New Chunga version 0.2.1 (Was: Error when chunked
 transfer encoding is used on clisp)
In-Reply-To: <FBENJDGGPLPHJJCNFLMJCEPJCCAA.vodonosov@mail.ru> (Anton
	Vodonosov's message of "Tue, 24 Oct 2006 06:05:04 +0300")
References: <FBENJDGGPLPHJJCNFLMJCEPJCCAA.vodonosov@mail.ru>
Message-ID: <uy7r3j24k.fsf@agharta.de>

On Tue, 24 Oct 2006 06:05:04 +0300, "Anton Vodonosov" <vodonosov at mail.ru> wrote:

> The problem is that the http-request function fails when it's used
> for server with chunker transfer encoding.

Thanks for the report.  It seems CLISP again needs special treatment
here.  Please try with the new Chunga release.

Cheers,
Edi.


From vodonosov at mail.ru  Thu Oct 26 03:46:55 2006
From: vodonosov at mail.ru (Anton Vodonosov)
Date: Thu, 26 Oct 2006 06:46:55 +0300
Subject: [drakma-devel] drakma works with clisp (Was: New Chunga version
	0.2.1 (Was: Error when chunked transfer encoding is used on clisp))
In-Reply-To: <uy7r3j24k.fsf@agharta.de>
Message-ID: <FBENJDGGPLPHJJCNFLMJMEAECDAA.vodonosov@mail.ru>

Drakma works well on clisp with the chunga 0.2.1

I've tried all the examples from http://weitz.de/drakma/
except for https and authorization.

-Anton

-----Original Message-----
From: Edi Weitz [mailto:edi at agharta.de]
Sent: Thursday, October 26, 2006 4:30 AM
To: Anton Vodonosov
Cc: drakma-devel at common-lisp.net; drakma-announce at common-lisp.net
Subject: New Chunga version 0.2.1 (Was: Error when chunked transfer encoding
is used on clisp)


On Tue, 24 Oct 2006 06:05:04 +0300, "Anton Vodonosov" <vodonosov at mail.ru>
wrote:

> The problem is that the http-request function fails when it's used
> for server with chunker transfer encoding.

Thanks for the report.  It seems CLISP again needs special treatment
here.  Please try with the new Chunga release.

Cheers,
Edi.