[Ecls-list] ECL and profiling

[Juan Jose Garcia-Ripoll]

On Sat, Sep 15, 2012 at 7:36 PM, Matthew Mondor <mm_lists at pulsar-zone.net>wrote:

> At http://ecls.sourceforge.net/ecl/The-profiler.html#The-profiler are
> listed a few functions, which despite building ECL with --with-profile
> appear to be existing yet unbound symbols.
>

This belongs to the old manual, which is declared obsolete. The currently
linked manual is the only valid one
http://ecls.sourceforge.net/new-manual/index.html


> I also see in contrib/ a profile tool ported from SBCL, but
> unfortunately the SBCL documentation I can find seems to address
> another profiler (sb-prof).


Well, SBCL has diverged since then, acquiring some tools to keep cross
references and statistical profiling that unfortunately ECL doesn't have.


> If I look at the source, the profiler appears to only be able to profile
> specified functions, when I'd like
> to have more complete profiling.
>

The Lisp profiler is a bit invasive. There is no way for ECL to do full
profiling by itself because it cannot intercept the function calls which
appear in three forms: C calls, compiled Lisp calls and interpreted
function calls. That is too complicated. The profiler is just intended to
be used to get statistics of a finite subset of code, to see how it
performs.


> I guess that since ECL compiles to C it'd also be possible to use gprof
> and other C profiling tools, though.  But is there another recommended
> way?
>

I have found OS X's statistical profiler to be the most useful tool. gprof
tends to be very confused by ECL's deeply nested tree of function calls.
"shark" instead just interrogates the program every few milliseconds or
some smaller unit to know what the program is doing and gets very accurate
reports. It also helps that one has a GUI to hide unwanted functions or
whole libraries from the report.


> I have resumed a bit work on my httpd (now named Crow), and since
> yesterday it also hosts my personal site.  The server's hardware is
> modest compared to my development system (although Crow runs fine and is
> not slower than the previous js-written HTTPd which hosted my site for
> years), I'm at a stage where profiling would be nice to have, instead
> of blindly optimizing every function.
>

I believe that a C profiler will be most useful.


> Optimizing is also a way to get better aquainted with ECL.  I noticed
> yesterday how rewriting URL-ENCODE differently increased its speed
> 50-fold (it previously used REDUCE (with custom LAMBDA), CONCATENATE,
> MAP (with custom LAMBDA), FORMAT, as well as a CHARACTER-VALID-P
> function which was passed a dynamic variable as its table, all of which
> resulting in a tremendous amount of function calls for simple string
> processing).  It was elegant to read and fast to write, which was fine
> for early interactive development, but really had to be rewritten for
> production use (it now expands to efficient mostly inline C).
>

I have been incorporating compiler macros for many sequence functions over
the years and this seems to work quite well, but not all of them are
covered. In any case REDUCE+CONCATENATE is not a very friendly couple in
terms of resources: consing is to be avoided whenever possible, specially
high-level consing which cannot be optimized away.


> I also discovered that CHAR<= couldn't be inlined as <= can, such that
> a macro expanding to CHAR-CODE and <= resulted in a significant speed
> improvement to test if a character is within an expected range (see
>
> http://cvs.pulsar-zone.net/cgi-bin/cvsweb.cgi/mmondor/mmsoftware/cl/server/character.lisp.diff?r1=1.4;r2=1.5
> ).
> It's possible that the compiler only needs some small modification to
> allow inlining CHAR* functions, but I've not looked at that yet.


The CHAR<= function uses the C library and cannot be inlined: its purpose
is to compare characters according to some collation order, which of course
has nothing to do with CHAR-CODE.

Juanjo

-- 
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/ecl-devel/attachments/20120915/aa668831/attachment.html>