From metawilm at gmail.com  Sun Aug  2 21:32:21 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Sun, 2 Aug 2009 23:32:21 +0200
Subject: [clpython-devel] Fwd: mixed Lisp/Python source code
Message-ID: <f6bc9b490908021432g1140cea7k2de9beba53cec9ed@mail.gmail.com>

Hi everyone,

I just posted the following on comp.lang.lisp; it's about a new
feature (in development) to support mixed Lisp/Python syntax.
Hope you find it interesting.

http://groups.google.nl/group/comp.lang.lisp/browse_frm/thread/1861e201b6eabd33/c13b7ec4af5301ad
(copied below; I've shortened the irrelevant long path names in the output)

- Willem

- - -
On Aug 2, 9:10 pm, Oxide Scrubber <jharri... at hatlop.de> wrote:
> vippstar wrote:
> > With CL macros you can have any embedded language expanding to lisp
> > code, but the embedded language can't have a language feature that CL
> > does not, that's the only limitation.

> Balderdash. Any language feature that can be implemented by a Turing
> machine can be included, using a suitable macro to implement it.

And don't forget reader macros: they gives you _complete_ control over
syntax. Python and Ruby source code can be supported directly, with
the right readtable. CLPython defines a readtable that dispatches
every input character to the Python parser, and Python source files
are then compiled using basically:

(let ((*readtable* *python-parsing-readtable*))
  (compile-file "source.py"))

Even more interesting is a mixed-syntax Python/Lisp readtable. It's in
development, here is a preview:

clpython(213): (.parser::enter-mixed-lisp-python-syntax)
; The mixed Lisp/Python syntax mode is now enabled; Lispy *readtable*
is now set.
clpython(214): range(100)[98:2:-2]
#(98 96 94 92 90 88 86 84 82 80 ...)
clpython(215): import os
#<module `os'
   :src #P"/System/Library/Frameworks/.../python2.5/os.py"
   :binary #P"/Users/willem/.fasl/.../2.5/lib/python2.5/os.fasl"
   :src-time 3400632679 :bin-time 3457618461 @ #x121964e2>
clpython(216): os.urandom(1)
"M"
clpython(217): (funcall ~os.urandom 1)
"?"
clpython(218): (format nil "OS is a ~A, and its source file is ~A"
                   (type-of ~os) (module-src-pathname ~os))
"OS is a module, and its source file is /System/..../python2.5/os.py"

Compiling and running a mixed-syntax file, using the normal compile-
file and load functions:

- - - foo.lispy - - -
(in-package :cl-user)
(clpython.parser::enter-mixed-lisp-python-syntax)

def pyfact(x):
  if x <= 1:
    return 1
  else:
    return x * lispfact(x-1)

(defun lispfact (x)
  (if (<= x 1)
      1
    (* x (clpython:py-call ~pyfact (1- x)))))

(trace lispfact)
print pyfact(10)
- - -

clpython(203): :cf
;;; Compiling file foo.lispy
; The mixed Lisp/Python syntax mode is now enabled; Lispy *readtable*
is now set.
;;; Writing fasl file foo.fasl
;;; Fasl write complete
clpython(204): :ld
; Fast loading foo.fasl
 0[2]: (lispfact 9)
   1[2]: (lispfact 7)
     2[2]: (lispfact 5)
       3[2]: (lispfact 3)
         4[2]: (lispfact 1)
         4[2]: returned 1
       3[2]: returned 6
     2[2]: returned 120
   1[2]: returned 5040
 0[2]: returned 362880
3628800

> With the right set of macros you could probably give your embedded language
> first-class continuations, for instance.

Good guess... CLPython has a Python CPS convertor, to support
generator expressions.

> The macros will essentially compile it to CL the way GCC compiles C to assembly.

Yeah, Common Lisp makes an excellent implementation language for
Python, and probably Ruby too. It's almost cheating.

- Willem



From jdn at math.carleton.ca  Fri Aug  7 23:20:03 2009
From: jdn at math.carleton.ca (Jason Nielsen)
Date: Fri, 7 Aug 2009 19:20:03 -0400 (EDT)
Subject: [clpython-devel] Build error
Message-ID: <alpine.DEB.2.00.0908071918400.25503@viper.gateway.2wire.net>

I've just git pulled the latest version of cl-python and am getting the 
following build error:

<snip>

; compiling (YACC:DEFINE-PARSER *CL-YACC-PYTHON-PARSER* ...)
; compiling (DEFMETHOD LEXER-EOF-TOKEN ...)
; compiling (DEFMETHOD PARSE-FORM-WITH-YACC ...); compilation aborted 
because of fatal error:
;   READ failure in COMPILE-FILE:
;     SB-INT:SIMPLE-READER-PACKAGE-ERROR at 2660 (line 59, column 62) on 
#<SB-SYS:FD-STREAM for "file 
/home/jdn/work/lisp/cl-python/parser/grammar-clyacc.lisp" {1005DE4541}>:
;       The symbol "YACC-EOF-SYMBOL" is not external in the YACC package.
; compilation aborted after 0:00:10.417

WARNING:
    COMPILE-FILE warned while performing #<COMPILE-OP NIL {1003130FA1}> on
    #<CL-SOURCE-FILE "grammar-clyacc" {100351D501}>.
unhandled ASDF:COMPILE-FAILED in thread #<SB-THREAD:THREAD "initial 
thread" RUNNING {10029DDD71}>:
   erred while invoking #<COMPILE-OP NIL {1003130FA1}> on
   #<CL-SOURCE-FILE "grammar-clyacc" {100351D501}>

0: (SB-DEBUG::MAP-BACKTRACE #<CLOSURE (LAMBDA #) {1003F3F679}>)[:EXTERNAL]
1: (BACKTRACE 128 #<SYNONYM-STREAM :SYMBOL SB-SYS:*STDERR* {1000172DD1}>)
2: (SB-DEBUG::DEBUGGER-DISABLED-HOOK
     #<ASDF:COMPILE-FAILED {1003E2B3C1}>
     #<unavailable argument>)
3: (INVOKE-DEBUGGER #<ASDF:COMPILE-FAILED {1003E2B3C1}>)
4: (ERROR ASDF:COMPILE-FAILED)[:EXTERNAL]
5: ((SB-PCL::FAST-METHOD ASDF:PERFORM (ASDF:COMPILE-OP 
ASDF:CL-SOURCE-FILE))

</snip>

Cheers,
Jason



From metawilm at gmail.com  Sat Aug  8 16:51:52 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Sat, 8 Aug 2009 18:51:52 +0200
Subject: [clpython-devel] Build error
In-Reply-To: <alpine.DEB.2.00.0908071918400.25503@viper.gateway.2wire.net>
References: <alpine.DEB.2.00.0908071918400.25503@viper.gateway.2wire.net>
Message-ID: <f6bc9b490908080951k5511fdedre3656cd1ad9eac50@mail.gmail.com>

On Sat, Aug 8, 2009 at 1:20 AM, Jason Nielsen<jdn at math.carleton.ca> wrote:
> I've just git pulled the latest version of cl-python and am getting the
> following build error:

Jason, thanks for still regularly checking the build. It's fixed now.

- Willem



From jdn at math.carleton.ca  Sat Aug  8 17:09:56 2009
From: jdn at math.carleton.ca (Jason Nielsen)
Date: Sat, 8 Aug 2009 13:09:56 -0400 (EDT)
Subject: [clpython-devel] Build error
In-Reply-To: <f6bc9b490908080951k5511fdedre3656cd1ad9eac50@mail.gmail.com>
References: <alpine.DEB.2.00.0908071918400.25503@viper.gateway.2wire.net>
	<f6bc9b490908080951k5511fdedre3656cd1ad9eac50@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.0908081304330.7401@viper.gateway.2wire.net>

On Sat, 8 Aug 2009, Willem Broekema wrote:

> On Sat, Aug 8, 2009 at 1:20 AM, Jason Nielsen<jdn at math.carleton.ca> wrote:
>> I've just git pulled the latest version of cl-python and am getting the
>> following build error:
>
> Jason, thanks for still regularly checking the build. It's fixed now.
>
> - Willem

Thanks for the fix.  The build gets a little further along now but I still 
get the following error:

<snip>
; compiling file "/home/jdn/work/lisp/cl-python/core/metaclass.lisp" 
(written 07 AUG 2009 07:15:27 PM):
; compiling (IN-PACKAGE :CLPYTHON); compilation aborted because of fatal 
error:
;   READ failure in COMPILE-FILE:
;     SB-INT:SIMPLE-READER-PACKAGE-ERROR at 2925 (line 111, column 30) on 
#<SB-SYS:FD-STREAM for "file 
/home/jdn/work/lisp/cl-python/core/metaclass.lisp" {10052551F1}>:
;       The symbol "DEFINE-HASH-TABLE-TEST" is not external in the SB-EXT 
package.
; compilation aborted after 0:00:00.005

WARNING:
    COMPILE-FILE warned while performing #<COMPILE-OP NIL {100333ED31}> on
    #<CL-SOURCE-FILE "metaclass" {1003340401}>.
unhandled ASDF:COMPILE-FAILED in thread #<SB-THREAD:THREAD "initial 
thread" RUNNING {10029DDD71}>:
   erred while invoking #<COMPILE-OP NIL {100333ED31}> on
   #<CL-SOURCE-FILE "metaclass" {1003340401}>

0: (SB-DEBUG::MAP-BACKTRACE #<CLOSURE (LAMBDA #) {10052D0CF9}>)[:EXTERNAL]
1: (BACKTRACE 128 #<SYNONYM-STREAM :SYMBOL SB-SYS:*STDERR* {1000172DD1}>)
2: (SB-DEBUG::DEBUGGER-DISABLED-HOOK
     #<ASDF:COMPILE-FAILED {10052C4A51}>
     #<unavailable argument>)
3: (INVOKE-DEBUGGER #<ASDF:COMPILE-FAILED {10052C4A51}>)
4: (ERROR ASDF:COMPILE-FAILED)[:EXTERNAL]
5: ((SB-PCL::FAST-METHOD ASDF:PERFORM (ASDF:COMPILE-OP 
ASDF:CL-SOURCE-FILE))
     #<unavailable argument>
     #<unavailable argument>
     #<ASDF:COMPILE-OP NIL {100333ED31}>
     #<ASDF:CL-SOURCE-FILE "metaclass" {1003340401}>)
</snip>

It has been on my to-do list to look deeper into the cl-python internals 
but I haven't got around to it.  Sadly there are not enough hours in a 
day.  Keep up the good work as cl-python is pretty cool.

Jason



From metawilm at gmail.com  Sat Aug  8 22:04:47 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Sun, 9 Aug 2009 00:04:47 +0200
Subject: [clpython-devel] Build error
In-Reply-To: <alpine.DEB.2.00.0908081304330.7401@viper.gateway.2wire.net>
References: <alpine.DEB.2.00.0908071918400.25503@viper.gateway.2wire.net>
	<f6bc9b490908080951k5511fdedre3656cd1ad9eac50@mail.gmail.com>
	<alpine.DEB.2.00.0908081304330.7401@viper.gateway.2wire.net>
Message-ID: <f6bc9b490908081504p1db0fc87j5b55c72382a31d25@mail.gmail.com>

On Sat, Aug 8, 2009 at 7:09 PM, Jason Nielsen<jdn at math.carleton.ca> wrote:
> Thanks for the fix. ?The build gets a little further along now but I still
> get the following error:
>
> ; ? ? ? The symbol "DEFINE-HASH-TABLE-TEST" is not external in the SB-EXT
> package.

On May 21, 2009, SBCL changed it's custom hash table hash/test
interface, and I assume you are using an earlier SBCL version. I only
aim to support the most recent version of each implementation and
library, unless there are compelling arguments to support a particular
older version.

> It has been on my to-do list to look deeper into the cl-python internals
> but I haven't got around to it.  Sadly there are not enough hours in a
> day.  Keep up the good work as cl-python is pretty cool.

Thanks for your continued interest, and let me know if there are
things that are blocking you from using CLPython for some task it
looks suitable for...

- Willem



From luismbo at gmail.com  Mon Aug 17 17:23:37 2009
From: luismbo at gmail.com (=?ISO-8859-1?Q?Lu=EDs_Oliveira?=)
Date: Mon, 17 Aug 2009 18:23:37 +0100
Subject: [clpython-devel] Running pybench with clpython
Message-ID: <391f79580908171023v5eed49deya9fc146618192b03@mail.gmail.com>

Hello,

I'm trying to run pybench with clpython. It fails for me pretty early
by not finding the 'platform' module. Has anyone tried to do this
before?

Thanks in advance,

-- 
Lu?s Oliveira
http://student.dei.uc.pt/~lmoliv/



From metawilm at gmail.com  Tue Aug 18 22:00:26 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Wed, 19 Aug 2009 00:00:26 +0200
Subject: [clpython-devel] Running pybench with clpython
In-Reply-To: <391f79580908171023v5eed49deya9fc146618192b03@mail.gmail.com>
References: <391f79580908171023v5eed49deya9fc146618192b03@mail.gmail.com>
Message-ID: <f6bc9b490908181500n7f7dd703i913e41b10c5038ae@mail.gmail.com>

Hello Lu?s, welcome to the list.

On Mon, Aug 17, 2009 at 7:23 PM, Lu?s Oliveira<luismbo at gmail.com> wrote:
> I'm trying to run pybench with clpython. It fails for me pretty early
> by not finding the 'platform' module. Has anyone tried to do this
> before?

Make sure that you have set cl-user::*clpython-module-search-paths* to
include the path to the Python 2.5 (or so) libs, and that the path
ends in a slash. (I'm working on removing the silly slash
requirement.) Then it should be able to find module "platform".

Importing the file will still probably fail though, because it
references "sys.executable" which is not defined yet, and beyond that
there's probably more things missing. Maybe you can hack up the
missing things as you encounter them and see how far you can get?

- Willem



From luismbo at gmail.com  Tue Aug 18 22:07:03 2009
From: luismbo at gmail.com (=?ISO-8859-1?Q?Lu=EDs_Oliveira?=)
Date: Tue, 18 Aug 2009 23:07:03 +0100
Subject: [clpython-devel] Running pybench with clpython
In-Reply-To: <f6bc9b490908181500n7f7dd703i913e41b10c5038ae@mail.gmail.com>
References: <391f79580908171023v5eed49deya9fc146618192b03@mail.gmail.com> 
	<f6bc9b490908181500n7f7dd703i913e41b10c5038ae@mail.gmail.com>
Message-ID: <391f79580908181507j61680208k9c81e26ea92b89f8@mail.gmail.com>

On Tue, Aug 18, 2009 at 11:00 PM, Willem Broekema<metawilm at gmail.com> wrote:
> Make sure that you have set cl-user::*clpython-module-search-paths* to
> include the path to the Python 2.5 (or so) libs, and that the path
> ends in a slash. (I'm working on removing the silly slash
> requirement.) Then it should be able to find module "platform".

Ah, I see. I was trying to do that using sys.path.append() and it
wasn't working.


> Importing the file will still probably fail though, because it
> references "sys.executable" which is not defined yet, and beyond that
> there's probably more things missing. Maybe you can hack up the
> missing things as you encounter them and see how far you can get?

Will do, thanks for the hints.

-- 
Lu?s Oliveira
http://student.dei.uc.pt/~lmoliv/



From metawilm at gmail.com  Tue Aug 18 22:20:27 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Wed, 19 Aug 2009 00:20:27 +0200
Subject: [clpython-devel] Running pybench with clpython
In-Reply-To: <391f79580908181507j61680208k9c81e26ea92b89f8@mail.gmail.com>
References: <391f79580908171023v5eed49deya9fc146618192b03@mail.gmail.com>
	<f6bc9b490908181500n7f7dd703i913e41b10c5038ae@mail.gmail.com>
	<391f79580908181507j61680208k9c81e26ea92b89f8@mail.gmail.com>
Message-ID: <f6bc9b490908181520y273d7efchd4e48fce4bc8b398@mail.gmail.com>

On Wed, Aug 19, 2009 at 12:07 AM, Lu?s Oliveira<luismbo at gmail.com> wrote:
> Ah, I see. I was trying to do that using sys.path.append() and it
> wasn't working.

The items in sys.path and *clpython-module-search-paths* are combined,
so putting it in either should work. But if you found it didn't, I'll
have to look at it.

- Willem