[Ecls-list] Mingw32 build issues with libffi (libffi issue solved, patch needed, now sockets fails)

[Marko Kocić]

Hi Juan,

Seems like my firts mail was rejected from list because of attachment.
This is going to be a long mail with lots of things.

First, about the system. Its Win7 64 bit, with 32 bit mingw and msys.
I also installed gmp, gc-7.2.alpha4 and libffi-3.0.9 from
http://sourceware.org/libffi/ Each one of them is built using
./configure && make && make install. gmp, and gc put its libs in
c:/msys/1.0/local/lib and headers in c:/msys/1.0/local/include, which
are standard msys/mingw install folders. From msys shell those are
mapped to /usr/local.

When installed libffi put its libs to c:/msys/1.0/local/lib and
headers in c:/msys/1.0/local/lib/libffi-3.0.9/include. The last one is
not a standard location, so I had to put it to CFLAGS in order to be
detected by ECL configure. Other system libs (gmp and gc) are also
detected properly.

I used the following configure options.
./configure CFLAGS="-Ic:/msys/1.0/local/include
-Ic:/msys/1.0/local/lib/libffi-3.0.9/include"
LDFLAGS="-Lc:/msys/1.0/local/lib" --prefix=c:/opt/ecl --enable-threads
--enable-unicode --with-system-gmp -enable-boehm=system && make 2>&1 |
tee -a build.log

After configure step, it reports that libffi is detected and build
continues, but fails with the following error:
>
> if [ -f CROSS-COMPILER ]; then \
>                touch ecl_min.exe; \
>        else \
>                g++ -Lc:/msys/1.0/local/lib  -lffi -o ecl_min.exe
> cinit.o c/all_symbols.o -L./ libeclmin.a -leclatomic
> -lgmp -lgc -lm -lws2_32;\
>        fi
> Info: resolving _GC_dont_gc by linking to __imp__GC_dont_gc (auto-import)
> Info: resolving _GC_oom_fn by linking to __imp__GC_oom_fn (auto-import)
> Info: resolving _GC_no_dls by linking to __imp__GC_no_dls (auto-import)
> Info: resolving _GC_all_interior_pointers by linking to
> __imp__GC_all_interior_pointers (auto-import)
> Info: resolving _GC_time_limit by linking to __imp__GC_time_limit (auto-import)
> Info: resolving _GC_push_other_roots by linking to
> __imp__GC_push_other_roots (auto-import)
> Info: resolving _GC_start_call_back by linking to
> __imp__GC_start_call_back (auto-import)
> Info: resolving _GC_java_finalization by linking to
> __imp__GC_java_finalization (auto-import)
> Info: resolving _GC_print_stats by linking to __imp__GC_print_stats
> (auto-importc:/mingw32/bin/../lib/gcc/mingw32/4.5.1/
> ../../../../mingw32/bin/ld.exe: warning: auto-importing has been
> activated without --enable-auto-import specified on the
>  command line.
> This should work unless it involves constant data structures
> referencing symbols from auto-imported DLLs.
> libeclmin.a(ffi.o):tmp.c:(.text+0x169a): undefined reference to `ffi_prep_cif'
> libeclmin.a(ffi.o):tmp.c:(.text+0x17da): undefined reference to `ffi_call'
> libeclmin.a(ffi.o):tmp.c:(.text+0x1ad4): undefined reference to
> `ffi_prep_closure'
> libeclmin.a(ffi.o):tmp.c:(.data+0x20): undefined reference to `ffi_type_sint8'
> libeclmin.a(ffi.o):tmp.c:(.data+0x24): undefined reference to `ffi_type_uint8'
> libeclmin.a(ffi.o):tmp.c:(.data+0x28): undefined reference to `ffi_type_sint8'
> libeclmin.a(ffi.o):tmp.c:(.data+0x2c): undefined reference to `ffi_type_uint8'
> libeclmin.a(ffi.o):tmp.c:(.data+0x30): undefined reference to `ffi_type_sint16'
> libeclmin.a(ffi.o):tmp.c:(.data+0x34): undefined reference to `ffi_type_uint16'
> libeclmin.a(ffi.o):tmp.c:(.data+0x38): undefined reference to `ffi_type_sint32'
> libeclmin.a(ffi.o):tmp.c:(.data+0x3c): undefined reference to `ffi_type_uint32'
> libeclmin.a(ffi.o):tmp.c:(.data+0x40): undefined reference to `ffi_type_sint32'
> libeclmin.a(ffi.o):tmp.c:(.data+0x44): undefined reference to `ffi_type_uint32'
> libeclmin.a(ffi.o):tmp.c:(.data+0x48): undefined reference to `ffi_type_sint8'
> libeclmin.a(ffi.o):tmp.c:(.data+0x4c): undefined reference to `ffi_type_uint8'
> libeclmin.a(ffi.o):tmp.c:(.data+0x50): undefined reference to `ffi_type_sint16'
> libeclmin.a(ffi.o):tmp.c:(.data+0x54): undefined reference to `ffi_type_uint16'
> libeclmin.a(ffi.o):tmp.c:(.data+0x58): undefined reference to `ffi_type_sint32'
> libeclmin.a(ffi.o):tmp.c:(.data+0x5c): undefined reference to `ffi_type_uint32'
> libeclmin.a(ffi.o):tmp.c:(.data+0x60): undefined reference to `ffi_type_sint64'
> libeclmin.a(ffi.o):tmp.c:(.data+0x64): undefined reference to `ffi_type_uint64'
> libeclmin.a(ffi.o):tmp.c:(.data+0x68): undefined reference to `ffi_type_sint64'
> libeclmin.a(ffi.o):tmp.c:(.data+0x6c): undefined reference to `ffi_type_uint64'
> libeclmin.a(ffi.o):tmp.c:(.data+0x70): undefined reference to `ffi_type_pointer'
> libeclmin.a(ffi.o):tmp.c:(.data+0x74): undefined reference to `ffi_type_pointer'
> libeclmin.a(ffi.o):tmp.c:(.data+0x78): undefined reference to `ffi_type_pointer'
> libeclmin.a(ffi.o):tmp.c:(.data+0x7c): undefined reference to `ffi_type_float'
> libeclmin.a(ffi.o):tmp.c:(.data+0x80): undefined reference to `ffi_type_double'
> libeclmin.a(ffi.o):tmp.c:(.data+0x84): undefined reference to `ffi_type_void)
> '
> collect2: ld returned 1 exit status
> make[1]: *** [ecl_min.exe] Error 1
> make[1]: Leaving directory `/c/development/cvstree/ecl/build'
> make: *** [all] Error 2
>
> c:\development\cvstree\ecl>

If you take a look at the build/Makefile it has two variables defined:
LIBS	=    -lm -lws2_32
FASL_LIBS =  -lgmp -lgc
CORE_LIBS = -leclatomic
LDFLAGS	= -Lc:/msys/1.0/local/lib  -lffi

If you take alook at ecl_min.exe target it looks like this:
ecl_min$(EXE): $(LIBRARIES) .gdbinit libeclmin.a
	if [ -f CROSS-COMPILER ]; then \
		touch $@; \
	else \
		$(CC) $(LDFLAGS) -o $@ cinit.o c/all_symbols.o -L./ libeclmin.a
$(CORE_LIBS) $(FASL_LIBS) $(LIBS);\
	fi
and this is why linking fails.
When I manually add -lffi to either CORE_LIBS build succeeds.
CORE_LIB is generated by ./configure magic, and I don't know how to
fix that. Note that libffi is the ony library missing here.

As another coincidence, if you remember our email correspondence of
few days ago where I tried to build ecl with libffi on Gentoo linux,
maybe adding it to CORE_LIBS would be proper fix, and maybe linked
--as-needed flag was not linking it properly because it is not listed
as ecl_min dependency. I don't have access to that machine right now
to test this.

As for libffi headers location, it's the same on mingw and Gentoo
linux, so its a libffi standard location. Maybe other distros are
putting those headers on /usr/include, but that's not the rule. I
don't know how to portaby detect libffi headers location on mingw32.
CFLAGS seems reasonable to me.

As a side note, libffi is the ony library used by ecl that is not
bundled. gmp, gc and libatomic are bundled. Since ffi/cffi now
completly depends on libffi, it would make sense to do the same with
libffi too, to bundle it with ecl, and leave configure option wheater
to use bundled or system one. I'm not even sure if maintaining ecl
without libffi makes any sense anymore, since you explained that code
that you used before for ffi calls withoud libffi is deprecated.

Since mingw builds proved to be most fragile, it would make sense to
configure mingw build farm machine to use all "complicated" features
(unicode, threads, libffi) for compliance builds. Right now it uses
defaults only.


Another issues is with install target on mingw. When I tried to start
built and installed ecl.exe, I got the error box saying that
libffi-5.dll and libgc-1.dll are missing. When I copy them manually to
install folder ecl.exe start normally. This is because on windows dlls
are looked up from application current folder or from PATH environment
variable. This is not a problem when building ecl, or when invoking it
from inside msys shell, since msys adds c:/mysy/1.0/local/bin
directory to PATH, but when ecl is started from windows command
prompt, from slime or by doubleclicking to ecl.exe, the only place it
can look up those dll files is its own folder, since msys location is
not it standard windows path. The proper fix for mingw would probably
be to either statically link gc and libffi, or to copy those dll files
to install folder.

As for cxx and socet issue form previous mail, there is no reason why
I used cxx except that's what is title for mingw server in build farm.
I noted it here just as a note. It builds fine without cxx.

I hope this wall of text isn't too boring ;)

As another note, with ecl built this way, I get errors as soon as I
try to (require 'asdf), but let's leave that for later after build
issuse are resolved.

Regards,
Marko Kocić