[Antik-devel] Reader macro for sharp-m returns literal foreign-array (patch included)

[James Wright]

Hi Liam,

>> I have encountered the following problem a couple of times before,
>> although not as reproducibly as the memory corruption issue from my
>> previous message:  If I use the #m reader macro in a function
>> somewhere, dump an SBCL core image, load the image, and call the
>> function, I get memory corruption.  I have mostly dealt with it up
>> until now by not using the #m() notation in anything that I plan to
>> dump to a binary.
>
> This is odd.  There is a make-load-form for foreign-arrays, so I assumed
> this would work.  Example?

I'm working on trying to reproduce this, but I haven't had any luck so
far.  (This is why memory issues are such a joy).  However, I don't
think that SBCL uses make-load-form when dumping an image; my
understanding is that a dumped image is really just a copy of the raw
memory bytes.  I've attached a script with output that demonstrates
that SBCL seems to want a make-load-form method for writing a FASL but
not for writing a core image.

> My intent is that it be analogous to #, e.g. #(1.0d0 2.0d0) expands to a
> (CL) array.  I used to have it the way you want but I changed it as of
> 3e610bd4d5261 to be analogous to the regular array macro

Fair enough.  My version does not evaluate its arguments, so in that
sense it is the same as #( or #2a(, but on the other hand it does
return a form instead of a literal value, so in that sense it is less
like #(.

> (in fact, #m should
> make a CL array if grid:*default-grid-type* is set accordingly; I'm still
> not sure about the wisdom of doing that).

It is nice to be able to switch seamlessly between CL and foreign
arrays.  One funny situation the current version leads to is that
evaluating the string printed by the REPL for a foreign array can give
you a CL array back:

  ANTIK-USER> (setf *default-grid-type* 'array)
  ARRAY
  ANTIK-USER> (make-foreign-array 'double-float :initial-contents '(1d0 2d0))
  #m(1.000000000000000d0 2.000000000000000d0)
  ANTIK-USER> #m(1.000000000000000d0 2.000000000000000d0)
  #(1d0 2d0)

> By the way, do you know about grid:grid?  Maybe that's what you want.  It's
> a function, so it evaluates its arguments
> (grid:grid pi pi pi)
> #m(3.141592653589793d0 3.141592653589793d0 3.141592653589793d0)

I didn't know about that; it looks like it will simplify my code.  Thanks!

> I think I'd like to figure out why #m can't be saved before I consider
> changing this back.

I think it's analogous to the problem with foreign pointers in global
vars: if you have an actual literal foreign array in your code, then
that is what the image will contain when you dump core, so it'll come
back as an uninitialized foreign pointer; but if you have a form for
creating a foreign array in your code, then the creation form (or
rather the code that it compiles down to, of course) is what will get
dumped.  I'll spend a bit more time next week and see if I can cough
up a failing example.

At any rate, it's an easy problem for end users to avoid, so I don't
want to come across as badgering.  But I do think it's a bit of a
stability landmine to be aware of for people that need to dump
binaries instead of being able to run from fasls; needing a binary is
my typical gsll use case, since I use gsll on a torque cluster.

Thanks!
      James
-------------- next part --------------
tanky:~/src/toys% sh dump-differs-from-compile.sh
===== Load then dump =====
This is SBCL 1.0.48.19, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
[undoing binding stack and other enclosing state... done]
[saving current Lisp image into foo.core:
writing 3512 bytes from the read-only space at 0x01000000
writing 2256 bytes from the static space at 0x01100000
writing 32034816 bytes from the dynamic space at 0x09000000
done]
===== Load then compile =====
This is SBCL 1.0.48.19, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.

; compiling file "/home/james/src/toys/foo.lisp" (written 11 DEC 2011 02:04:24 PM):
; compiling (IN-PACKAGE #:CL-USER)
; compiling (DEFSTRUCT FOO ...)
; compiling (DEFUN MAIN ...)
; file: /home/james/src/toys/foo.lisp
; in: DEFUN MAIN
;     (LET ((MY-FOO #S(FOO :BAR 'BAR-VAL :BAZ 'BAZ-VAL)))
;       MY-FOO)
; ==>
;   #S(FOO :BAR 'BAR-VAL :BAZ 'BAZ-VAL)
; 
; caught ERROR:
;   don't know how to dump #S(FOO :BAR 'BAR-VAL :BAZ 'BAZ-VAL) (default MAKE-LOAD-FORM method called).
; 
; note: deleting unreachable code
; 
; note: deleting unreachable code
; 
; compilation unit finished
;   caught 1 ERROR condition
;   printed 2 notes


; /home/james/src/toys/foo.fasl written
; compilation finished in 0:00:00.011
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dump-differs-from-compile.sh
Type: application/x-sh
Size: 466 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/antik-devel/attachments/20111211/24e596d7/attachment.sh>