From luke at bluetail.com  Mon Aug  2 07:51:01 2004
From: luke at bluetail.com (Luke Gorrie)
Date: Mon, 02 Aug 2004 09:51:01 +0200
Subject: [movitz-devel] Bits and pieces
Message-ID: <lhvfg2t1ui.fsf@dodo.bluetail.com>

Ahoy,

It's probably more fun to talk about Movitz on the mailing list than
hassling Frode directly, so -

I want to make Erlang operating systems using Movitz as the kernel.
For fun I'm hoping to write some very-small operating systems in the
style of Per Brinch Hansen's "Architecture of Concurrent Programming"
book. (I can highly recommend that book for inspiration in writing
small-as-in-feasible operating systems, not that I've done so yet.
Cheap on bookfinder.com.)

Anyway. I've got a working compiler from Erlang to Lisp and I've used
it to write a "hello world" ErlangOS. But it currently only handles
the sequential subset of Erlang and that's not going to impress my
friends. So I need to add support for concurrency.

I think a nice way to do this would be to have Lisp-level coroutines
as the building block. It looks like this is mostly written in Movitz
but I'm not sure if it's finished (couldn't figure out how to use it).
I'm looking at functions like COPY-CONTROL-STACK,
CLONE-RUN-TIME-CONTEXT, and SWITCH-TO-CONTEXT.

Is that stuff usable? If not, what needs to be done?

Ideally I would like coroutines to have variable-size stacks (growing
and shrinking on demand), but that's not so important - I'll take
anything I can get :-)


I'd also like to do some VGA hacks. Is there any example code?

Cheers,
Luke





From luke at bluetail.com  Mon Aug  2 08:14:09 2004
From: luke at bluetail.com (Luke Gorrie)
Date: Mon, 02 Aug 2004 10:14:09 +0200
Subject: [movitz-devel] Hardware requirements?
Message-ID: <lhpt6at0ry.fsf@dodo.bluetail.com>

Another thing,

Is it right that Movitz should be able to boot and do basic VGA stuff
on basically any computer, include modern laptops?

I can't find my floppy drive to test this.





From ffjeld at common-lisp.net  Mon Aug  2 09:48:29 2004
From: ffjeld at common-lisp.net (Frode Vatvedt Fjeld)
Date: Mon, 02 Aug 2004 11:48:29 +0200
Subject: [movitz-devel] Re: Hardware requirements?
References: <lhpt6at0ry.fsf@dodo.bluetail.com>
Message-ID: <2hy8kxsweq.fsf@vserver.cs.uit.no>

Luke Gorrie <luke at bluetail.com> writes:

> Is it right that Movitz should be able to boot and do basic VGA stuff
> on basically any computer, include modern laptops?

Yes, if by "computer" you mean "x86 PC" :-)

> I can't find my floppy drive to test this.

There is of course the bootloader issue. The simple, built-in (floppy
only) bootloder seems to work on many PCs (such as Robert Strandhs
laptop in Oslo, for one random example), but then again I seem to
remember some report that it didn't work on some toshiba laptop. The
Grub loader is probably safer, it seems they managed to break Grub for
386 again, so it's not entirely rock solid either.

-- 
Frode Vatvedt Fjeld




From ffjeld at common-lisp.net  Mon Aug  2 10:05:49 2004
From: ffjeld at common-lisp.net (Frode Vatvedt Fjeld)
Date: Mon, 02 Aug 2004 12:05:49 +0200
Subject: [movitz-devel] Re: Bits and pieces
References: <lhvfg2t1ui.fsf@dodo.bluetail.com>
Message-ID: <2hu0vlsvlu.fsf@vserver.cs.uit.no>

Luke Gorrie <luke at bluetail.com> writes:

> Anyway. I've got a working compiler from Erlang to Lisp and I've
> used it to write a "hello world" ErlangOS. But it currently only
> handles the sequential subset of Erlang and that's not going to
> impress my friends. So I need to add support for concurrency.

Yes, this is something I'm hoping to be able to do in the
not-too-distant future too (a few months).

> I think a nice way to do this would be to have Lisp-level coroutines
> as the building block. It looks like this is mostly written in
> Movitz but I'm not sure if it's finished (couldn't figure out how to
> use it).  I'm looking at functions like COPY-CONTROL-STACK,
> CLONE-RUN-TIME-CONTEXT, and SWITCH-TO-CONTEXT.
>
> Is that stuff usable? If not, what needs to be done?

It's not usabel as-is. Copy-control-stack I wrote quite recently, as a
debugging tool actually, but it'll clearly become a concurrency
primitive also. Clone-run-time-context and switch-to-context I wrote
quite a while ago trying to get some threads going, but I stopped that
work for some reasone I don't remember. This means they are incomplete
and possibly a bit outdated wrt. the changes in the run-time-context
since then.

If you want to try and make this stuff work properly for threads or
coroutines or whatever, please do :) It probably requires you to
figure out how the bits and pices of the run-time-context and
associated structures fit together, but there's nothing really
complicated in there, I believe. I'll of course answer any questions
here or on #lisp.

> Ideally I would like coroutines to have variable-size stacks
> (growing and shrinking on demand), but that's not so important -
> I'll take anything I can get :-)

This should also be no problem to implement, and it's really unrelated
to the concurrency stuff. Well, one issue is from where does one
allocate the stack.. When there's one thread like today there's no
such issue, but later one there's got to be some better global memory
management.

> I'd also like to do some VGA hacks. Is there any example code?

There's losp/x86-pc/{vga,textmode}.lisp. The stuff in vga.lisp mostly
just fiddles with the textmode registers, but I guess the graphcis
stuff isn't much different.


Did everyone try (x86-pc:set-textmode +vga-state-90x60+) yet, or
+vga-state-40x25+ for the visually challenged? :)

Luke: I suspect (setf vga-state) is what needs to be slightly improved
(or even just give it the right parameters) to have graphics.

Oh, and isn't it possible to have the VGA textmode be 132 wide? Does
anyone know how to figure out e.g. +vga-state-132x60+ ?

-- 
Frode Vatvedt Fjeld




From ffjeld at common-lisp.net  Mon Aug  9 10:19:27 2004
From: ffjeld at common-lisp.net (Frode Vatvedt Fjeld)
Date: Mon, 09 Aug 2004 12:19:27 +0200
Subject: [movitz-devel] Re: The reactions of gc to hard-drive reads
References: <20040615060358.GA11052@home.cvip.net>
	<2hoenkbfes.fsf@vserver.cs.uit.no>
	<20040618061938.GA12965@home.cvip.net>
Message-ID: <2hsmawlikw.fsf@vserver.cs.uit.no>

Elliott <ejohnson at kvpt.org> writes:

> Yep I poped in there and adjusted the value.  Changing 16000 to
> 65537 on line 192 of los0-gc.lisp allows for 256 sectors to be read
> or writen.  I can now handle 131kb of data to and from disk, plus
> run out of memory a lot faster. :) I'll if I can raise the amount of
> memory seen by los0

Just thought I'd mention that this has been fixed for a few weeks
now. That is, los0-gc is able to work with semi-spaces of practically
any size, and the default action of the los0 image should be to split
the machine's available memory in two and do stop-and-copy between
these parts. (Although still there's no decent code to detect memory
other than the ancient bios method, which is restricted to 64 MB, but
this should be trivial to fix once the good detection mechanism
identified). And allcating buffers of a few megabytes should therefore
be unproblematic now.

-- 
Frode Vatvedt Fjeld




From jcrippen at gmail.com  Fri Aug 13 06:45:09 2004
From: jcrippen at gmail.com (James Crippen)
Date: Thu, 12 Aug 2004 22:45:09 -0800
Subject: [movitz-devel] Segmentation
Message-ID: <caf8a68a04081223456a983ff8@mail.gmail.com>

Is segmentation in use anywhere in Movitz, or is it just talked about
in the manual? And if so, where's the code that sets it up?



From ffjeld at common-lisp.net  Fri Aug 13 09:07:29 2004
From: ffjeld at common-lisp.net (Frode Vatvedt Fjeld)
Date: Fri, 13 Aug 2004 11:07:29 +0200
Subject: [movitz-devel] Re: Segmentation
References: <caf8a68a04081223456a983ff8@mail.gmail.com>
Message-ID: <2hisbn1k4u.fsf@vserver.cs.uit.no>

James Crippen <jcrippen at gmail.com> writes:

> Is segmentation in use anywhere in Movitz, or is it just talked
> about in the manual? And if so, where's the code that sets it up?

It's used, but not to a very great extent so far. That is,
thread-local storage is referenced through a designated segment
selector, but so far there's only one thread, and it's segment
selector (in FS) is the same as DS.  DS however is not quite "flat
model", it's starting at physical address 1M. GS (I think it was) is
used to access (memory mapped hardware) below 1M, e.g. by giving a
true physicalp argument to memref-int.

The basic execution environment is set up in the very early boot-up
code that gets generated in make-toplevel-funobj in image.lisp.

-- 
Frode Vatvedt Fjeld




From jcrippen at gmail.com  Sun Aug 15 23:03:16 2004
From: jcrippen at gmail.com (James Crippen)
Date: Sun, 15 Aug 2004 15:03:16 -0800
Subject: [movitz-devel] ATA reference
Message-ID: <caf8a68a0408151603ad3577a@mail.gmail.com>

Where can I find a reference for ATA programming so I can understand
harddisk.lisp?



From peter.minten at wanadoo.nl  Mon Aug 16 07:35:48 2004
From: peter.minten at wanadoo.nl (Peter Minten)
Date: Mon, 16 Aug 2004 09:35:48 +0200
Subject: [movitz-devel] ATA reference
In-Reply-To: <caf8a68a0408151603ad3577a@mail.gmail.com>
References: <caf8a68a0408151603ad3577a@mail.gmail.com>
Message-ID: <200408160935.49796.peter.minten@wanadoo.nl>

On Monday 16 August 2004 01:03, James Crippen wrote:
> Where can I find a reference for ATA programming so I can understand
> harddisk.lisp?

The reference I used when making the file was ATAPI-6 
(http://www.t13.org/project/d1410r3a.pdf).

Greetings,

Peter



From jcrippen at gmail.com  Wed Aug 18 08:30:46 2004
From: jcrippen at gmail.com (James Crippen)
Date: Wed, 18 Aug 2004 00:30:46 -0800
Subject: [movitz-devel] with-inline-assembly-case
Message-ID: <caf8a68a040818013048d40422@mail.gmail.com>

What is the purpose of with-inline-assembly-case? I understand the
idea of the inline assembly directive, but I don't get why it needed
to be folded into a case statement.

'james



From axcd at libero.it  Wed Aug 18 11:45:46 2004
From: axcd at libero.it (Alessio Stalla)
Date: Wed, 18 Aug 2004 13:45:46 +0200
Subject: [movitz-devel] ext2 fs support
Message-ID: <I2N4OA$879180A567F4A0F1A3FE0104078F244F@libero.it>

I'm writing a module for los0 that aims to support the ext2 fs. So far it can only read regular files (i.e. no symlinks and no writing at all) and is quite slow, but it seems to work (I've tested it only on a bochs disk image formatted with an old version of ext2, but there should be no problems with newer versions). I've written also a sort of common file system interface so that different filesystems can be supported trasparently but it's actually quite a random thing, a real os would need something way better!

I don't expect adding writing support to be too easy because things change a bit from version to version, plus the docs I've found aren't that clear... We'll see. For now, I hope that what I've written can be useful.

Files attached:
partitions.lisp -- reading partition tables (no support for extended partitions)
fs.lisp           -- common fs interface
ext2fs.lisp     -- obvious... :)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: partitions.lisp
Type: application/octet-stream
Size: 1248 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/movitz-devel/attachments/20040818/9b1c8422/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fs.lisp
Type: application/octet-stream
Size: 2346 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/movitz-devel/attachments/20040818/9b1c8422/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ext2fs.lisp
Type: application/octet-stream
Size: 15902 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/movitz-devel/attachments/20040818/9b1c8422/attachment-0002.obj>

From axcd at libero.it  Wed Aug 18 12:30:26 2004
From: axcd at libero.it (Alessio Stalla)
Date: Wed, 18 Aug 2004 14:30:26 +0200
Subject: [movitz-devel] bug report
Message-ID: <I2N6QQ$5B5B9E5BF50E5A4A41208BE5F3F12BCF@libero.it>

I forgot to mention that I think I have discovered two bugs in movitz:

1) the way setq is implemented in muerte, it always assigns values to special variables, so that e.g.

       (let ((x 5)) (setq x 6))

doesn't do what one might expect. If I've understood correctly how the thing works, the function eval-setq in muerte/eval.lisp should be corrected in:

(defun eval-setq (form env)
  (do* ((p (cdr form) (cddr p))
        (value nil))
      ((null p) value)
    (assert (cdr p) (form)
      "Odd number of arguments to setq: ~W" form)
    (setf value
      (let ((local-var (find (car p) env :key #'car))
	    (form-value (eval-form (cadr p) env)))
        (if local-var
         (setf (cdr local-var) form-value)
         (set (car p) form-value))))))

A similar correction should be applied to eval-setf.

2) This is probably harder to notice: under certain conditions, labels and flet do not properly capture bindings: I was trying to movitz-compile this defun

(defun my-test-labels (x)
  (labels ((p () (print x))
           (q (y) (list x y)))
        (p)))

but it gave me this error:

The slot MOVITZ::REFERENCE-SLOT is unbound in the object
#<MOVITZ::BORROWED-BINDING name: X {4913F8FD}>

after tinkering a while with this piece of code I found that the bug is probably caused by q being defined but not called/used in some way, in fact if I remove its definition or call it inside the labels it compiles without problems. Now this may not be a real bug because actually defining a function without using it hasn't much sense, nevertheless people should know of its existence because it made me waste a lot of time trying to figure out what caused it :(

Alessio




From ffjeld at common-lisp.net  Wed Aug 18 19:43:19 2004
From: ffjeld at common-lisp.net (Frode Vatvedt Fjeld)
Date: Wed, 18 Aug 2004 21:43:19 +0200
Subject: [movitz-devel] Re: with-inline-assembly-case
References: <caf8a68a040818013048d40422@mail.gmail.com>
Message-ID: <2hhdr0z0yw.fsf@vserver.cs.uit.no>

James Crippen <jcrippen at gmail.com> writes:

> What is the purpose of with-inline-assembly-case? I understand the
> idea of the inline assembly directive, but I don't get why it needed
> to be folded into a case statement.

The with-inline-assembly-case is just a mechanism for weaving the
inline assembly into the compiler's code with a bit more fine-grained
control. The "case" refers to a compile-time dispatch on the current
"result-mode" of the compiler. Which is a reall broken concept to
begin with, but that's just me not really knowing how to write a
compiler.

Anyway, I'd simply suggest ignoring the with-inline-assembly-case
operator, if at all possible.

-- 
Frode Vatvedt Fjeld




From ffjeld at common-lisp.net  Wed Aug 18 19:51:48 2004
From: ffjeld at common-lisp.net (Frode Vatvedt Fjeld)
Date: Wed, 18 Aug 2004 21:51:48 +0200
Subject: [movitz-devel] Re: bug report
References: <I2N6QQ$5B5B9E5BF50E5A4A41208BE5F3F12BCF@libero.it>
Message-ID: <2hd61oz0kr.fsf@vserver.cs.uit.no>

"Alessio Stalla" <axcd at libero.it> writes:

> 1) the way setq is implemented in muerte, it always assigns values to
> special variables, so that e.g.
>
>        (let ((x 5)) (setq x 6))

Right, lexical variables in eval was probably added much later than
setq/setf, and I've never really written so extensive forms at the
REPL that this issue ever came up. I'll fix it right away.

> 2) This is probably harder to notice: under certain conditions, labels
> and flet do not properly capture bindings: I was trying to
> movitz-compile this defun
>
> (defun my-test-labels (x)
>   (labels ((p () (print x))
>            (q (y) (list x y)))
>         (p)))
>
> but it gave me this error:
>
> The slot MOVITZ::REFERENCE-SLOT is unbound in the object
> #<MOVITZ::BORROWED-BINDING name: X {4913F8FD}>

This is touching some of the hairier corners of the compiler, so don't
be too surprised to find compiler bugs when using labels etc. I'll
look into this too, shortly.

Thanks for the bug-reports.

-- 
Frode Vatvedt Fjeld




From ffjeld at common-lisp.net  Thu Aug 19 00:23:02 2004
From: ffjeld at common-lisp.net (Frode Vatvedt Fjeld)
Date: Thu, 19 Aug 2004 02:23:02 +0200
Subject: [movitz-devel] Re: bug report
References: <I2N6QQ$5B5B9E5BF50E5A4A41208BE5F3F12BCF@libero.it>
Message-ID: <2h8yccyo0p.fsf@vserver.cs.uit.no>

Both bugs reported should be fixed now. Do let me know if there are
further problems.

-- 
Frode Vatvedt Fjeld




From ffjeld at common-lisp.net  Thu Aug 19 00:30:14 2004
From: ffjeld at common-lisp.net (Frode Vatvedt Fjeld)
Date: Thu, 19 Aug 2004 02:30:14 +0200
Subject: [movitz-devel] Re: ext2 fs support
References: <I2N4OA$879180A567F4A0F1A3FE0104078F244F@libero.it>
Message-ID: <2h4qn0ynop.fsf@vserver.cs.uit.no>

I checked the files into CVS. Thanks!

-- 
Frode Vatvedt Fjeld




From jcrippen at gmail.com  Tue Aug 24 07:27:02 2004
From: jcrippen at gmail.com (James Crippen)
Date: Mon, 23 Aug 2004 23:27:02 -0800
Subject: [movitz-devel] Procfs support for non-Allegro lisps?
Message-ID: <caf8a68a04082400274a6a2740@mail.gmail.com>

Is there any procfs support in the works out there for non-Allegro
lisps? I note that the procfs-image.lisp file uses a few functions
from a PROCFS package that I presume is Allegro-specific. What would
it take to get this working in other Lisps? It seems like it should be
some not-overly-complicated file accessing after some pid lookup,
right?

Also, I'm working on implementing timers and RTC support. I intend to
use this to implement some support for things like Unix's sleep(3) and
alarm(2). That should fix the problems with things like the busy-wait
loops in harddisk.lisp.

'james



From ffjeld at common-lisp.net  Fri Aug 27 12:24:00 2004
From: ffjeld at common-lisp.net (Frode Vatvedt Fjeld)
Date: Fri, 27 Aug 2004 14:24:00 +0200
Subject: [movitz-devel] Re: Procfs support for non-Allegro lisps?
References: <caf8a68a04082400274a6a2740@mail.gmail.com>
Message-ID: <2hd61cwyzj.fsf@vserver.cs.uit.no>

James Crippen <jcrippen at gmail.com> writes:

> Is there any procfs support in the works out there for non-Allegro
> lisps? I note that the procfs-image.lisp file uses a few functions
> from a PROCFS package that I presume is Allegro-specific. What would
> it take to get this working in other Lisps? It seems like it should
> be some not-overly-complicated file accessing after some pid lookup,
> right?

It's actually not Allegro-specific, it is if anything FreeBSD (or just
*BSD, I don't know) specific. I think someone on #lisp made a minimal
effort of porting it to linux, but I think it wasn't 100% trivial, I
don't remember what came of it. Shouldn't be too difficult,
though. The code is in the infunix package that I uploaded at
http://www.cs.uit.no/~frodef/sw/ if you want to have a look. Note
however that to be actually useful, the procfs stuff requires some
sort of hacked bochs (or perhaps qemu can also be hacked) to be
useful. I've modified bochs to print out the locations of the emulated
address-space and the variables that hold the emulated CPU registers
etc. (and this info is read in when creating a bochs-image instance,
which is a specialized procfs-image) so as to get interactive probing
of the emulated system (and this is a _very_ nice debugging tool).

> Also, I'm working on implementing timers and RTC support. I intend
> to use this to implement some support for things like Unix's
> sleep(3) and alarm(2). That should fix the problems with things like
> the busy-wait loops in harddisk.lisp.

I suppose one very simple approach to the "wait for interrupt
response" problem would be to do e.g.

  (do-send-harddisk-request device ...)
  (setf (exception-handler (irq-exception-vector device))
    (lambda ()
       (do-acknowledge-response device ..)))

i.e. simply to install the continuation (as a lambda) as the interrupt
handler. The same can be done for the timers. This scheme of course
can't deal with multiple outstanding requests etc. but might be
appropriate for very simple (MoKA) architectures. Just a thought.

-- 
Frode Vatvedt Fjeld