Erlang style processes in Common Lisp

[Stelian Ionescu]

> On Mon, Aug 3, 2015 at 3:36 PM, Attila Lendvai <attila at lendvai.name> wrote:
> >>> How might we get equivalent cheap ephemeral processes into a
> >>> contemporary Common Lisp implementation?
> >>
> >> In short, you need to write from scratch a new CL implementation. Current ones are not designed with the Erlang constraints in mind.
> >
> > well, Nikodemus had some plans for green threads for SBCL and it
> > didn't sound like a rewrite.

There are two main reasons why Erlang(the VM, not so much the language IMO) is so effective in its niche:

1) Per-process(green thread) GC heap and no sharing between processes except through very narrow and explicit primitives. This makes Erlang code pretty safe for concurrency as long as you're not seeking to satisfy hard real-time constraints.

2) It has a bytecode-interpreting VM in which each instruction is a safepoint and all I/O is non-blocking, basically implementing scheduler preemption in user space. This allow Erlang watcher processes to randomly kill worker processes and restart them. Being an interpreter is crucial here, because as soon as they tried compiling to native code(HiPE) they had problems with tight loops in pure Erlang code that would not yield to the scheduler and thus starved other processes. Almost nobody uses HiPE.
Erlang code can still block if it calls foreign code or it performs some syscalls that can't be avoided to block, but that's why they try to implement everything in pure Erlang - they switched the default SSL implementation to one written by them, a few years ago, because they had issues with OpenSSL.

So again, unless you're willing to implement something along those lines, you could have Erlang-style concurrency - which is still useful I guess - but you wouldn't achieve the rock-solid robustness for which the Erlang VM is so famous(except for inter-process message queues that are unbounded by default so you can easily use all memory if you're not careful).

-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib