[Ecls-list] Issue: ECL fails to compile under Chinese Windows

[Daniel Kochmański]

Hello,

I believe that I have succesfully found solution. Problem was recurrence
in si_make_encoding function. It was calling ext::make-encoding, but due
to bogus entry in symbols_list, this was pointing back to
si_make_encoding. I think problem was affecting not only Chinese, but
also other windows installations with custom code page.

Fix is pushed to develop and rc1 branches. I would be grateful, if you
could check if it works on your computer. (I've tested it on changed
encoding on windows, so I could reproduce error, and now it compiles
fine here).

Cheers,
Daniel

Daniel Kochmański writes:

> Hey,
>
> I'm very thankful for this detailed description. Really, well done! :-)
> It is very helpful to know, where problem and what potential solution
> is. I'm downloading now windows 7 trial version and will try to
> reproduce error.
>
> Best regards,
> Daniel
>
> Rujia Liu writes:
>
>> Dear all,
>>
>> I'm a Chinese developer who have used ECL for some time in 2013. Although I don't use it recently, I'd like to report an old issue (and our investigations) that I struggled with.
>>
>> Problem: ECL fails to compile under CHINESE windows XP and windows 7, due to ecl_min.exe crash.
>>
>> This issue is present in 13.5 (and also 15.3.7-rc1) but not in version 12.12.1. 
>>
>> I've discussed this with Juan Jose in private emails (thanks, Juanjo!) but we were both quite busy and didn't finish it.
>>
>> However, we have some important observations: 
>>
>> 1. Before 2012-12-16, the whole ECL is working on our machines (not only ecl-min) so something that changed after that is causing this problem.
>>
>> 2. I debugged with windbg, found that ecl_min.exe crashed in cl_boot, here:
>>
>> if (cl_fboundp(@'ext::make-encoding') != ECL_NIL) {
>>         maybe_fix_console_stream(cl_core.standard_input);
>>         maybe_fix_console_stream(cl_core.standard_output);
>>         maybe_fix_console_stream(cl_core.error_output);
>>     }
>>
>> (BTW: our encoding is CP936 and windbg showed the code successfully detected this, i.e. known_cp[i].name is "WINDOWS-CP936")
>>
>> In maybe_fix_console_stream(cl_object stream), there is a call:
>>
>>     si_stream_external_format_set(stream, external_format);
>>
>> Then I traced into it, and found it got here:
>>
>> #ifdef ECL_CLOS_STREAMS
>>         unlikely_if (ECL_INSTANCEP(stream)) {
>>                 FEerror("Cannot change external format of stream ~A", 1, stream);  <--- *** it got here. ***
>>         }
>> #endif
>>
>> and here:
>>
>>                         cl_object elt_type = ecl_stream_element_type(stream);
>>                         unlikely_if (elt_type != @'character' &&
>>                                      elt_type != @'base-char')
>>                                 FEerror("Cannot change external format"
>>                                         "of binary stream ~A", 1, stream); <-- *** and here. ***
>>                         set_stream_elt_type(stream, stream->stream.byte_size,
>>                                             stream->stream.flags, format);
>>
>> Then in set_stream_elt_type, it got into an INFINITE RECURSION:
>>
>> static int
>> parse_external_format(cl_object stream, cl_object format, int flags)
>> {
>>         if (format == @':default') {
>>                 format = ecl_symbol_value(@'ext::*default-external-format*');
>>         }
>>     if (CONSP(format)) {  <-- this is always true, so it recurses indefinitely
>>         flags = parse_external_format(stream, ECL_CONS_CDR(format), flags);
>>         format = ECL_CONS_CAR(format);
>>     }
>>
>> I don't know why format is such "infinite cons", maybe because of the FEerrors, the format variable is left in an abnormal state?
>>
>> 3. In ECL12.12.1, the condition below is not satisfied so the 'troublesome' maybe_fix_console_stream is not called at all (that's why ecl_min.exe does NOT crash under 12.12.1).
>>
>> if (cl_fboundp(@'ext::make-encoding') != ECL_NIL) {
>>         maybe_fix_console_stream(cl_core.standard_input);
>>         maybe_fix_console_stream(cl_core.standard_output);
>>         maybe_fix_console_stream(cl_core.error_output);
>>     }
>>
>> Actually, after I commented out these lines in git repository's master, everything seems to work fine.
>>
>> I've tested some programs, outputing chinese characters without any problem. But since I haven't tested a lot of program, I'm not sure whether there is any potential problem.
>>
>> Finally, my questions:
>>
>> 1. Is it possible to skip setting the external format? Actually our program will reset the encoding to utf-8, despite our windows defaults to CP936 so it really doesn't matter whether to set it in cl_boot. It's perfectly ok to set it to LATIN-1 instead. We tested ECL12.12.1, it depends on msvc/encodings directory (when I removed the director it asks for WINDOWS-CP936 file) but it works.
>>
>> 2. Since this issue affacts a lot of (potential) users, should we add it somewhere in the document or wrap these lines in a compiler directive so that user can choose to skip this part when compiling?
>>
>> BTW: I'm very happy to see that ECL has a new maintainer now. That's REALLY great!
>> ‍
>> Best regards,
>> Rujia Liu------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
>> by Intel and developed in partnership with Slashdot Media, is your hub for all
>> things parallel software development, from weekly thought leadership blogs to
>> news, videos, case studies, tutorials and more. Take a look and join the 
>> conversation now. http://goparallel.sourceforge.net/_______________________________________________
>> Ecls-list mailing list
>> Ecls-list at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/ecls-list

-- 
Daniel Kochmański | Poznań, Poland
;; aka jackdaniel

"Be the change that you wish to see in the world." - Mahatma Gandhi