<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hello,<div class=""><br class=""></div><div class="">Someone on the LISP HUG Mailing list was recently trying to allocate a 4GB array and found that it was failing. I was recently thinking about such problems wishing there were sparse arrays in CL; I ended up making something called ’Spaces’ which potentially solve the problem.</div><div class=""><br class=""></div><div class="">Below is a letter to HUG that I sent to someone outlining my solution to the problem.</div><div class=""><br class=""></div><div class="">I thought some of you might find it useful as well, or might want to discuss this data structure, which I think is new, or at least I have never read nor heard of it before I invented it a couple of months ago.</div><div class=""><br class=""></div><div class="">Burton<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">Begin forwarded message:</div><br class="Apple-interchange-newline"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">From: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">Burton Samograd <<a href="mailto:busfactor1@icloud.com" class="">busfactor1@icloud.com</a>><br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Subject: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class=""><b class="">Re: Creating large arrays</b><br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Date: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">July 18, 2019 at 2:45:14 PM MDT<br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">To: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class=""><a href="mailto:lisp@gmx.com" class="">lisp@gmx.com</a><br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Cc: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class=""><a href="mailto:lisp-hug@lispworks.com" class="">lisp-hug@lispworks.com</a><br class=""></span></div><br class=""><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class=""><div class="">Are there other fast-access data structures with no such limitations, on<br class="">LispWorks?<br class=""><br class=""><br class="">Thank you again,<br class=""><br class="">Priyadarshan<br class=""></div></div></blockquote></div><br class=""><div class="">I have created something called ’N-Dimensional Sparse Spaces’ as a solution to the large array allocation problem. </div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><img class="spotlight" alt="No photo description available." aria-busy="true" apple-inline="yes" id="B58D6DF5-CC08-4DE3-9322-02D326DB1332" src="cid:6D3944FF-54B8-4C31-9A73-D3F5D36C2270"></div><div class=""><br class=""></div><div class="">This initial implementation is one to replace a set of random points within an N-Dimensional Sparse Grid addressed using integers, hence the use of #’= in #’make-space.</div><div class=""><br class=""></div><div class="">It is more generallized using other equals functions such as #’eql (so you can use symbols, and hence make an addressable symbol space), #’equals and #equalp letting you use S-Expressions as your addresses.</div><div class=""><br class=""></div><div class="">I have a more general implementation somewhere in my code, but the above image is the code I initially wrote that can be springboarded off of to create your own implementation.</div><div class=""><br class=""></div><div class="">One trick I have thought of is to, after you have completed your work with the space, is to compile it to a more suituable data structure for read-only access.</div><div class=""><br class=""></div><div class="">The sparce space is for editing and loading of the data you want to work with. A complied space is for reading data from the space, giving up generality.</div><div class=""><br class=""></div><div class="">Feel free to write me to discuss this data structure further if you have any questions or comments on its usefulness and generality and applicability to your problem.</div><div class=""><br class=""></div><div class="">Burton Samograd</div><div class="">BusFactor1 Inc.</div></div></div></blockquote></div><br class=""></div></body></html>