clasp next steps ? speed-up ?

[Christian Schafmeister]

Marcus,

Here’s a bit of history on Clasp development.

(1) I wrote an S-expression walking Common Lisp interpreter and discovered that it was too slow for my needs.
(2) I wrote a simple compiler that translated Common Lisp into LLVM-IR and I was able to get Common Lisp running but it was about 100x slower than SBCL.   It runs CLOS.
(3) I integrated a new compiler (Cleavir) that Robert Strandh is actively developing that makes extensive use of CLOS and performs many Common Lisp level optimization like escape analysis and it allows me to implement aggressive inlining. Robert is also adding a sophisticated type inference algorithm that will eliminate runtime type checks and dead code.

Cleavir will be the only compiler that users will interact with. The interpreter and bootstrapping compiler are just there for bootstrapping from a C++ compiler.  

Unfortunately, Cleavir is not finished yet, but Robert is working on it and I am working on integrating his changes into Clasp.   If you want to wait a few months until it is more complete and performance improves and things stabilize that would be fine with me. You may also decide that it’s just too raw and uncertain and avoid it all together - also fine. 

The Cleavir compiler reuses most of the bootstrapping compiler code.

DWARF is the debugging format used by the GCC and Clang compilers.  It’s the state of the art for C++/C debugging.  I’m adapting it for Common Lisp.  What it will allow is debugging of C++/Common Lisp code at the same time.  It will also enable profiling of both C++ and Common Lisp code together.

I’m guessing your handle is kr_. Check out irccloud.com — it is the IRC client that I use.

But it’s great to use the mailing list - I think I’ve got it figured out now.


On Jun 24, 2015, at 1:26 AM, Markus Krummenacker <kr at AI.SRI.COM> wrote:

> Christian Schafmeister writes:
>> Hey Markus,
>> 
>> You will have a better debugging experience if you use Slime.
>> 
>> We should talk and I’ll give you a better update as to where Clasp is wrt optimization and debugging.  It’s kind of a complex picture that is changing rapidly.  In a nutshell:
>> 
>> 1) I’m incorporating inlining and a new compiler as we speak and this should produce a large improvement in performance.
> 
> hi chris,
> 
> so having 2 compilers floating around does create the puzzle of where
> exactly improvements should be made.  i should probably pick something
> to do that will survive the transition from one compiler to the
> other...
> 
> 
> 
>> 2) The debugging experience is mixed and should be improved
>> soon. I’m incorporating a new compiler but it generates no
>> debugging information at the moment.  I’m adding DWARF debugging
>> support so that we can debug using gdb or lldb.
> 
> i know nothing about dwarves, but if such debug info could be used on
> the lisp side as well, maybe i should study that a bit.  (though it
> will probably take me quite a bit of time to digest.  i don't want to
> prevent other people from investigating this as well.)
> 
> 
>> 3) On freenode there is a #clasp chatroom - could you drop in and I can update you on where things are?
> 
> quite frankly, i have no idea how to use irc chat.  i did use firefox
> to visit the freenode website, and i seem to have joined #clasp, but i
> don't know how i would type anything,  it does not do anything for me,
> and i don't see any activity.  maybe it is also getting late on the
> east coast.
> 
> anyways, my cultural preference generally is towards "traditional"
> email and mailing-lists, that are properly archived, and where
> communication is asynchronous.
> 
> i would think explanations regarding what glue holds clasp together
> these days would be of broad interest on the mailing list too, instead
> of just the presumably ephemeral irc channel.
> 
> -- 
> --
> Regards
> Markus Krummenacker
> 
> 
> 
> 
>> Best,
>> 
>> .Chris.
>> 
>> 
>> 
>> On Jun 14, 2015, at 5:17 PM, Markus Krummenacker <kr at AI.SRI.COM> wrote:
>> 
>>> after a few more hickups, i finally succeeded in building clasp, on
>>> linux.
>>> 
>>> one thing i wonder is how to get a stack backtrace, if there is a
>>> break.  just interacting at the prompt just seems to return to the
>>> prompt, flushing the error, like this:
>>> 
>>> 
>>>> (+ 2 'b)
>>> 
>>> Debugger received error of type: SIMPLE-TYPE-ERROR
>>> In lisp_errorUnexpectedType ../../src/core/foundation.cc line 163
>>> COMMON-LISP:TYPE-ERROR :initializers (:DATUM B :EXPECTED-TYPE COMMON-LISP:NUMBER  )
>>> 
>>> Error flushed.
>>>> 
>>> 
>>> 
>>> that'll make it difficult to get to the bottom of various issues.
>>> 
>>> 
>>> also, drmeister, you wanted me to look into speeding up clasp.
>>> but i'll need some pointers on how to get started on this.
>>> 
>>> it would be good if there were a rough "how to optimize" guide
>>> available.  is there a worked out example for me to look at ?  which
>>> function/file ?
>>> 
>>> and could you please assign one of those open tasks to me ?  it will
>>> probably take some time for me to chew on, but i'd prefer to work on
>>> something that doesn't duplicate what other people might be working on.
>>> 
>>> thanks.
>>> 
>>> -- 
>>> --
>>> Regards
>>> Markus Krummenacker
>>> 
>>