[mel-base-devel] Maintenance and future development status

Tue Dec 22 12:58:53 UTC 2009

Am 22.12.2009 um 02:11 schrieb Ian Eslick:

> How actively is mel-base being used and maintained?  Any plans for further development, particularly of the IMAP module?

I still maintain it patching bugs and did some enhancements like using FLEXI-STREAMS (See current release on CLiki and darcs repository http://www.crispylogics.com/opensource/repos/mel-base). Please use the repository on www.crispylogics.com not the one at common-lisp.net (which is obsolete and should be better deleted - but I don't seem to be able to login anymore).

> I got mel-base running under ClozureCL last night and found that there were a number of interesting things missing from IMAP support and a few bugs I had to work around.  It was fairly simple to get it working with cl+ssl, per a prior question.  This was a custom hack for ccl though.

That's good news. :-)

> I'd love to adopt mel-base for a project, but I'd like to change quite a bit including proper support for cl+ssl, use of usocket, bordeaux-threads, etc instead of embedded portability layers,

When I created mel-base I wanted to have as little dependencies as possible. Nowadays the situation and the general quality of cross platforms libs like usocket or bordeaux-threads is a lot better so certainly support this step. The use of usocket and cl+ssl is obvious, I currently don't use any locking or threading primitives so there was no demand for bordeaux-threads. If something in this direction is needed BT will be certainly my choose too.

> IMAP search and IMAP idle support and so on.  I'm wiling to fork my own version to add these features, but wanted to see what the maintainer's feelings and thoughts were before I did any significant surgery.   I'd certainly rather not be the primary maintainer going forward, but am willing to do so if you are not able to invest the time to make and test some significant changes and additions.

If you're willing to use darcs, you can clone can just get your own repository were I can pull any changes from. I want to see mel-base to move in that direction too, so I can see were I can help implementing and/or testing things, answering questions and maintaining the main branch. I would want to see those changes in the main branch.

I'm also open to allow pushing patches to an unstable repository hosted on one of my servers if you need or want that.

Some notes on the changes itself:

1) I think USOCKET can straight forwardly replace all calls to MEL.NETWORK:MAKE-CONNECTION.
2) Converting most I/O to binary
Mel-base traditionally used character based I/O and faithful I/O. Since this is not a good idea for mails, I started converting it to binary I/O were possible. One of the last things I did (release 0.9-2) was changing the element-type of the IMAP connection stream to '(unsigned-byte 8).

One problem remains though: The IMAP commands are still sent using character based I/O. I know two possible solutions for this:

1) Convert all command handling code to work with byte vectors (ugly)
2) Use bivalent streams (prefer native, fallback to FLEXI-STREAMS)

The first solution might to some look like the most sane one. But if you think about it a bit further, one would rebuild many things already available in CL. (decoding strings, PEEK-CHAR). This solution would be a lot of work with probably little payback.

So I would prefer the latter one, since it is just so much more comforable (think debugging!). Bivalent streams are no problem for SBCL and Lispworks. The question remains what external-format is used to read characters from a bivalent stream. LispWorks' bivalent streams read characters of binary streams as if CODE-CHAR has been used to convert them. The external format is always Latin-1 which is ok for the us-ascii based command coding. SBCL allows an explicit external-format for its bivalent streams. USOCKET doesn't set it on SOCKET-CONNECT (there is a commentary ";;###FIXME: The above line probably needs an :external-format"). It should work for us-ascii too, but an explicit settign would perhaps be better. ClozureCLs network streams are bivalent too - there is actually no such distinction. The character encoding can be configured by CCL:*DEFAULT-SOCKET-CHARACTER-ENCODING and is by default Latin-1. So bivalent streams are quite well supported on current CLs. If an implementation doesn't support bivalent streams it would be easy to just use FLEXI-STREAMS for them. I would not want to use FLEXI-STREAMS for all implementations.

3) Replace MEL.ENVIRONMENT functions

GETPID: ? (perhaps just CFFI/UFFI based?)
GETHOSTNAME: USOCKET::GET-HOST-NAME (why is it internal?)

4)
MEL.UNIX:
- DIRECTORY-CONTENTS-CHANGED-P: ?
MEL.FILESYSTEM:

Obvious candidates are probably CL-FAD or something like that. But actually MEL.FILESYSTEM is highly optimized to scan directories as fast as possible. This is really important for decent performance of Maildir-Foldes. There is a portable fallback implementation in the code. I think this part perhaps isn't worth the effort to replace it through another library.

5) MEL.GRAY-STREAM
Could be replaced by TRIVIAL-GRAY-STREAMS - but I think it isn't very important since gray streams will not really change much. I think this would be just a cosmetic change. (though probably producing new bugs)

6) CL+SSL
I already said CL+SSL is perhaps quite obvious to use. Actually I would prefer to use LWs and ACLs native apis were possible, because its a simple out-of-the-box solution that is supported by the vendors. So I would feel better if CL+SSL would be used as a fallback implementation. Hunchentoot does the same btw.

so far!

ciao,
Jochen

-- 
Jochen Schmidt 
CRISPYLOGICS
Uhlandstr. 9, 90408 Nuremberg

Fon +49 (0)911 517 999 82
Fax +49 (0)911 517 999 83

mailto:(format nil "~(~36r@~36r.~36r~)" 870180 1680085828711918828 16438) http://www.crispylogics.com