[Ecls-list] Help needed, really

[Gabriel Dos Reis]

Hi,

Happy New Year everybody!


The message below is an ackward way of starting the new year, but
it seems there is no way around it.  So, I'll try to be brief.
I have a hectic schedule ahead of me but I'll try to follow the
discussion, to the extent I can and also depending on whether it leads
to anything that would be useful for building OpenAxiom with ECL.

Matthew Mondor wrote:

| On Thu, 30 Dec 2010 10:42:43 +0100
| Juan Jose Garcia-Ripoll <juanjose.garciaripoll at googlemail.com> wrote:
| 
| Sorry that I didn't read the previous discussions, which most probably
| happened on another list (and it's unclear to me why OpenAxiom requires
| that implementation-dependent low-level information; why it cannot use
| an FFI library or standard means such as CL min/max constants such as
| MOST-POSITIVE-FIXNUM, MOST-NEGATIVE-FIXNUM to obtain the information it
| needs); I'll simply suggest ideas about what I know, hoping they
| weren't discussed or dismissed already.  You may cross-post or quote
| from this message as wanted.


Most of the discussion happened in private between Juanjo and me,
starting with his reporting that OpenAxiom could not build with ECL on
macintel 64-bit.   I was able to reproduce the problem and explained to
him what was happening.  I'm going to summarize the explanation, and 
also give a brief explanation of how and why *FEATURES* came into the
picture.  Yes, Juanjo suggested that I look at integer sizes.  I'll
also address that point, as I did in private when he made the
suggestion.  Then, I'll conclude.  There is a potential for fruitful
technical discussion and an opportunity for ECL to make itself more
useful, but after having tried several times and having been called many
things, I don't see that happening soon, so I'll just put the issue to
rest on the platforms that are affected.  This is a fresh new year and
I'm not planning on starting it the way I ended the last one :-)

Juanjo made a good effort at summarizing the discussion.  But, I do not
feel that his summary adequately captures the problem and what I have
been suggesting. 

* Background

OpenAxiom needs a Lisp system that it can extend with its own native
libraries (e.g. C++ libraries.)  Because of this extension requirement,
OpenAxiom has to get some `binary compatibility' information from the
Lisp system in order to construct adequate compiler and linker flags for
its own (e.g. OpenAxiom's) libraries that it uses to extend the base
Lisp.  You may think of it as the ABI, but it is not the full ABI.
Usually, this not much of a problem for systems that support only on
kind of ABI interface.  However, it is an issue on platforms that
provide "compatibility" modes, e.g. Windows 64-bit making it possible to
compile a program as a 32-bit binary or a 64-bit binary, or macintel
offering 32-bit binary or 64-binary or universal binary formats.  Of
course, those of us who had tried SPARC v8 in 90s are familiar with the
issue -- sometimes called "multilib".

Please note that the `binary information' I'm talking about here is
about a program binary flavor or a library flavor, NOT whether a C
integer type has size 32-bit or 64-bit.  For example, the C `long' integer
type is 32-bit wide on many ilp32 systems and llp64 systems, but 64-bit
on lp64 unix systems.  Similarly the C `long long' integer type is
64-bit wide on almost all systems.  So the suggestion of looking C
integer types is a non-solution, despite being frequently suggested.
The closest that could give some information is the width of `void*',
but, again, that does not give the `binary information' in question for
a value 32-bit or 64-bit does say whether libecl is only 32-bit or
64-bit or universal on macintels for example (where, I as said earlier
Juanjo first experienced the problem with OpenAxiom.)


* The problem

As stated above, OpenAxiom needs an indication about the binary flavor
of the Lisp system.  From experiments, I noticed that the five systems
supported by OpenAxiom (SBCL, ECL, CLISP, GCL, Clozure CL) give some
hints on their *FEATURES* -- yes, including ECL (but more on this below)!

As DS also pointed out, x86-64, etc., are common tokens to refer to 
a class of binary flavor.  So, OpenAxiom was looking for that and
decides (albeit imperfectly at times) whether to build 32-bit library or
64-bit library.  What happens on macintel 64-bit with ECL is that ECL
puts out I686 instead there -- even when it is clearly a 64-bit binary
program.  OpenAxiom then proceeded to build a 32bit library and tried to
link it with ECL, and patatra!
I explained that to Juanjo and suggested that ECL puts a more correct
token on *FEATURES*.   After that, we had had very long discussions.

* Summary

OpenAxiom is not insisting on the token appearing on ECL's *FEATURES* --
it just happens that it seems to be a common place where many free Lisp
systems put that kind of information.  As long as ECL provides a simple
and reliable way to retrieve that information, I'm perfectly willing to
put up with one more of its eccentricities.

At the moment, since the discussion is stall and it does not appear that
ECL is willing to make itself more useful (at least as far as OpenAxiom
is concerned) on these multilib-platforms, and after reading indication
that ECL intentionally put misleading information regarding platforms on
*FEATURES*, I've disabled attempts to build OpenAxiom with ECL on know
problematic platforms.

| > I sincerely have no idea how to provide this ABI stuff. The only pieces of
| > information that ECL has are C integer sizes (EXT:C-UINT-MAX,
| > EXT:C-ULONG-MAX, etc) If someone may infer the appropriate ABI name and
| > compiler flags from it, then it is ok, but I do not know how to get that.
| 
| Is GNU binutils/objdump capable of distinguishing the various ABIs
| properly from a given object file (i.e. especially in the cases you
| mentioned where there may be multiple 32-bit and/or 64-bit options on
| the same host)?

I believe GNU `file' utility will attempt to report the binary flavor.
I do not believe that ECL needs or has to copy or emulate GNU
binutils/objdump to make the binary information available.
For example, ECL links against GMP library (either system-installed or
custom).  During configuration time, ECL could very well store that
binary using a simple test program that links against GMP.  GMP is just
an example, ECL uses other libraries.  
Anyway, I do no believe that the suggestion for ECL to make available
its binary flavor is equivalent to building and maintaining a database
of all imaginable ABIs and all combinations of processors.  But, I'm not
going to press the point.  It is a design decision -- not a technical one.
And ECL maintainers have every right to dismiss any design decision they
don't like.  We just hope that the dismissal is given a technical spin.


I just found that at least Ruby interpreter will report that binary
information available to its users through

     ruby -v

On my macintel, it reports

   ruby 1.8.7 (2009-06-12 patchlevel 174) [universal-darwin10.0]


I would also like to recognize that DS made excellent points about
dependency on the compiler flavour since ECL targets the C language and
relies on C compilers doing the right thing.

It has been suggested that OpenAxiom just takes the exact ECL CFLAGS and
LDFLAGS.   That piece of information is both too much and too brittle.
We must keep in mind that the beauty of linking against a C library (be
it GMP or C library, etc.) is that one does not need to know the exact
flags used to compile it.  Only its binary flavor is sufficient.

Finally, instead of requiring every ECL user to conceive of her or his
own brittle way to get the information that ECL possesses, it sounds to
me to me to be a good abstraction for ECL to provide a reliable way to
get hold on it, once.  If there is a bug, we know there is only one
place to fix.

Thanks for your time,

-- Gaby