From vodonosov at mail.ru Fri Nov 9 00:27:50 2007 From: vodonosov at mail.ru (Anton Vodonosov) Date: Fri, 09 Nov 2007 03:27:50 +0300 Subject: [flexi-streams-devel] Re: New release 0.13.0 (Was: New release 0.11.1) In-Reply-To: References: Message-ID: Edi Weitz: > > On Sun, 29 Apr 2007 22:43:44 +0400, Vodonosov Anton wrote: > > > I have a new version, with better performance (see attached diff of > > output.lisp against 0.11.2). > > I've finally found some time to think about this. I've now released a > new version which is based on your ideas but which (I hope) is a bit > more elegant and maybe even a bit faster for the case where characters > are output individually. > > This new release also fixes a glaring bug in STREAM-WRITE-BYTE. > > Thanks a lot for your patch, > Edi. > > PS: And the next time please send a patch without TAB characters... :) > Hi. Glad you committed it. Sorry about tabs. You may consider setting emacs variable indent-tabs-mode to nil in your sources. Few days ago I've tried to measure performance improvement gained by buffering in stream-write-sequence. It turned out that in real word scenarios no improvement at all, because underlying stream always provides necessary buffering. For example, to notice changes in performance when working with file, we must use unbuffered file stream, which is of course very uncommon (moreover, in case of normal, buffered, file stream performance is slightly worse because we do some additional work on our side; but you must do lot of output to notice this). Therefore, I've decided to test it with network stream. I've setup hunchentoot and configured simple easy-handler that just return a 10k string. Also, I've changed the make-socket-stream function to create unbuffered socket (:buffering :none in sbcl). On the other host the was drakma performing a hundred of requests to this handler. I've tested this setup with two versions of flexi streams: 0.11.0 - without buffering and 0.13.1 - with buffering. It is surprising (for me) that in this case there was no performance difference too. Using sniffer (ethreal) I've discovered that even in case if we do not do any buffering on sbcl side, buffering is anyway performed - by TCP implementation. When we do lot of write-byte it sends first packet with data length = 1, but all subsequent packets are of size > 1k. Looks like they have some micro timeout after our write-byte to see whether we will do write-byte again. Maybe this buffering in stream-write-sequence is unnecessary... Best regards, -Anton P.S. There is a little test for stream-line-column function in the attach. -------------- next part -------------- A non-text attachment was scrubbed... Name: flexi.diff Type: application/octet-stream Size: 3297 bytes Desc: not available URL: From edi at agharta.de Fri Nov 9 07:45:09 2007 From: edi at agharta.de (Edi Weitz) Date: Fri, 09 Nov 2007 08:45:09 +0100 Subject: [flexi-streams-devel] Re: New release 0.13.0 In-Reply-To: (Anton Vodonosov's message of "Fri, 09 Nov 2007 03:27:50 +0300") References: Message-ID: On Fri, 09 Nov 2007 03:27:50 +0300, Anton Vodonosov wrote: > Sorry about tabs. You may consider setting emacs variable > indent-tabs-mode to nil in your sources. I have that, but the tabs were in your patch. Emacs won't automatically remove tabs introduced from patches, you'd have to do that automatically... > Few days ago I've tried to measure performance improvement gained by > buffering in stream-write-sequence. It turned out that in real word > scenarios no improvement at all, because underlying stream always > provides necessary buffering. > > For example, to notice changes in performance when working with > file, we must use unbuffered file stream, which is of course very > uncommon (moreover, in case of normal, buffered, file stream > performance is slightly worse because we do some additional work on > our side; but you must do lot of output to notice this). > > Therefore, I've decided to test it with network stream. I've setup > hunchentoot and configured simple easy-handler that just return a > 10k string. Also, I've changed the make-socket-stream function to > create unbuffered socket (:buffering :none in sbcl). > > On the other host the was drakma performing a hundred of requests to > this handler. > > I've tested this setup with two versions of flexi streams: 0.11.0 - > without buffering and 0.13.1 - with buffering. It is surprising (for > me) that in this case there was no performance difference too. > > Using sniffer (ethreal) I've discovered that even in case if we do > not do any buffering on sbcl side, buffering is anyway performed - > by TCP implementation. When we do lot of write-byte it sends first > packet with data length = 1, but all subsequent packets are of size > > 1k. Looks like they have some micro timeout after our write-byte > to see whether we will do write-byte again. > > Maybe this buffering in stream-write-sequence is unnecessary... Hmmm.... ISTR that I did some simple tests with your patch and that there was a noticeable difference, but I don't have the results anymore. Anyway, I think I'll leave it like it is for now. > P.S. There is a little test for stream-line-column function in the > attach. Thanks. Cheers, Edi. From vodonosov at mail.ru Fri Nov 9 09:15:49 2007 From: vodonosov at mail.ru (Anton Vodonosov) Date: Fri, 09 Nov 2007 12:15:49 +0300 Subject: [flexi-streams-devel] Re[2]: New release 0.13.0 In-Reply-To: References: Message-ID: Edi Weitz : > > On Fri, 09 Nov 2007 03:27:50 +0300, Anton Vodonosov wrote: > > > Sorry about tabs. You may consider setting emacs variable > > indent-tabs-mode to nil in your sources. > > I have that, but the tabs were in your patch. Emacs won't > automatically remove tabs introduced from patches, you'd have to do > that automatically... I mean to add ;;; -*- ... indent-tabs-mode: nil -*- at the top of source files. This may decrease probability that people will introduce tabs into patches. Best regards, -Anton From edi at agharta.de Fri Nov 9 11:53:23 2007 From: edi at agharta.de (Edi Weitz) Date: Fri, 09 Nov 2007 12:53:23 +0100 Subject: [flexi-streams-devel] Re: New release 0.13.0 In-Reply-To: (Anton Vodonosov's message of "Fri, 09 Nov 2007 12:15:49 +0300") References: Message-ID: On Fri, 09 Nov 2007 12:15:49 +0300, Anton Vodonosov wrote: > I mean to add ;;; -*- ... indent-tabs-mode: nil -*- > at the top of source files. This may decrease probability that > people will introduce tabs into patches. Ah, I see, didn't know that. Thanks, Edi. From vodonosov at mail.ru Sat Nov 10 05:18:34 2007 From: vodonosov at mail.ru (Anton Vodonosov) Date: Sat, 10 Nov 2007 08:18:34 +0300 Subject: [flexi-streams-devel] Re[2]: New release 0.13.0 In-Reply-To: References: Message-ID: Edi Weitz : > > > Few days ago I've tried to measure performance improvement gained by > > buffering in stream-write-sequence. It turned out that in real word > > scenarios no improvement at all, because underlying stream always > > provides necessary buffering. > > > > For example, to notice changes in performance when working with > > file, we must use unbuffered file stream, which is of course very > > uncommon (moreover, in case of normal, buffered, file stream > > performance is slightly worse because we do some additional work on > > our side; but you must do lot of output to notice this). > > > > Therefore, I've decided to test it with network stream. I've setup > > hunchentoot and configured simple easy-handler that just return a > > 10k string. Also, I've changed the make-socket-stream function to > > create unbuffered socket (:buffering :none in sbcl). > > > > On the other host the was drakma performing a hundred of requests to > > this handler. > > > > I've tested this setup with two versions of flexi streams: 0.11.0 - > > without buffering and 0.13.1 - with buffering. It is surprising (for > > me) that in this case there was no performance difference too. > > > > Using sniffer (ethreal) I've discovered that even in case if we do > > not do any buffering on sbcl side, buffering is anyway performed - > > by TCP implementation. When we do lot of write-byte it sends first > > packet with data length = 1, but all subsequent packets are of size > > > 1k. Looks like they have some micro timeout after our write-byte > > to see whether we will do write-byte again. > > > > Maybe this buffering in stream-write-sequence is unnecessary... > > Hmmm.... > > ISTR that I did some simple tests with your patch and that there was a > noticeable difference, but I don't have the results anymore. Anyway, > I think I'll leave it like it is for now. Looks like slow network connection between hosts was the bottleneck in my test. I've tried with drakma and hunchentoot on the same computer and indeed, there is a difference, recent flexi-streams is a little faster. This difference doesn't depend on whether socket stream is buffered or no. Therefore I conclude that performance is saved by avoiding internal machinery of write-byte (access to internal slots of stream object, etc. in both flexi-stream and sbcl socket stream). I was confused because thought that multiple write-byte on socket will result multiple packets to be sent, that would made write-byte on underlying socket stream extremely expensive, and expected radical performance improvement when individual write-byte are avoided. Best regards, -Anton