From attila.lendvai at gmail.com  Tue Nov  2 13:08:46 2010
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Tue, 2 Nov 2010 14:08:46 +0100
Subject: [cl-rdbms-devel] Fwd: hu,
	dwim.rdbms + oracle 10g express + utf8 + input/read problem
In-Reply-To: <vmime.4ccf4ae1.1b67.5855c77767988a1@gleanon.com>
References: <vmime.4ccf4ae1.1b67.5855c77767988a1@gleanon.com>
Message-ID: <AANLkTinK-vMyqzdeNukbexyWPmKvXJr++L-KdiyJJ92+@mail.gmail.com>

as a first step i'm forwarding this to the list, maybe someone has a
quick idea what's going wrong...

once i have some time, i'll also look into the situation.

- attila


---------- Forwarded message ----------
From: Marshall McLuhan <marshall.mcluhan at gmx.net>
Date: Tue, Nov 2, 2010 at 00:18
Subject: hu,dwim.rdbms + oracle 10g express + utf8 + input/read problem
To: attila.lendvai at gmail.com


Hi

using sbcl + hu.dwim.rdbms.oracle and wanting UTF I have a hard time
getting correct input:

When using slime it crashes and the debugger starts whenever I ask for
character data e.g. (execute [select "SYMTAG_KEY" symtag])
?where column symtag_key is of type varchar2 or nvarchar2. (tried both)

Maybe you have some quick pointers regarding hu.dwim.rdbms, oracle and utf
or should I just avoid the trouble for now and go with ascii?

When starting sbcl from the shell without emacs/slime I get empty
strings like this:
(Well now while in emacs preparing this mail some empty spaces show up
as asian characters others as empty boxes)

* (with-symdb (execute [select "SYMTAG_KEY" symtag]))
23:14 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database
#<ORACLE {100434A001}>
23:14 0????&nbs p;????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
23:14 0?????????? RDBMS DEBUG?? Connecting in transaction
#<#<STANDARD-CLASS NIL {1004CCC1A1}> :begin-executed-p #t
{1003BFB6F1}>
23:14 0?????????? RDBMS DEBUG?? Logging on in transaction
#<#<STANDARD-CLASS NIL {1004CCC1A1}> :begin-executed-p #t
{1003BFB6F1}>
23:14 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY
FROM symtag"
23:14 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at inde x 0
23:14 0?????????? RDBMS DEBUG?? Fetched: "???"
23:14 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
23:14 0?????????? RDBMS DEBUG?? Fetched: "????????"
23:14 0?????????? RDBMS DEBUG?? About to COMMIT transaction
#<#<STANDARD-CLASS NIL {1004CCC1A1}> :begin-executed-p #t
{1003BFB6F1}>
23:14 0?????????? RDBMS DEBUG?? Cleaning up Oracle transaction
#<#<STANDARD-CLASS NIL {1004CCC1A1}> :begin-executed-p #t
{1003BFB6F1}> to database #<ORACLE {100434A001}>
23:14 0?????????? RDBMS DEBUG?? Calling logoff in transac tion
#<#<STANDARD-CLASS NIL {1004CCC1A1}> :begin-executed-p #t
{1003BFB6F1}>
23:14 0?????????? RDBMS DEBUG?? Freeing environment handle of
transaction #<#<STANDARD-CLASS NIL {1004CCC1A1}> :begin-executed-p #t
{1003BFB6F1}>
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {1003CAF34F}>
* (setq a *)

;???? (SETQ SYMARENA-ORACLE::A *)
;
; caught WARNING:
;?? undefined variable: A
;
; compilation unit finished
;?? Undefined variable:
;???? A
;?? caught 1 WARNING condition

#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {1003CAF34F}>
* (elt a 0)

#<(AND (VECTOR T 1) (NOT SIMPLE-ARRAY)) {1003CAF46F}>
* (elt a 1)

#<(AND (VECTOR T 1) (NOT SIMPLE-ARRAY)) {1003CB4C6F}>
* (elt (elt a 1) 0)

"????????& quot;
* (length a)

2
* (length (elt (elt a 1) 0))

8
* (length (elt (elt a 0) 0))

3
*



Here are the relevant settings as I see it:


*** 1. SBCL

$ sb-impl::*default-external-format*
:UTF-16

$ (with-database *symdb*
???? (with-transaction
?????? (let ((db (hu.dwim.rdbms::database-of *transaction*)))
??? ????? (lst
??? ?????? (hu.dwim.rdbms.oracle::connection-encoding-of db)
??? ?????? (hu.dwim.rdbms::encoding-of db)))))
(:UTF-16 :UTF-16)


**** 2. Oracle 10 Express - sqlplus

SQLPLUS> select symtag_key from symtag;

SYMTAG_KEY
----------------------
foo
chrismay

SQLPLUS>? describe symtag;
?Name??? ??? ??? ??? ??? ?? Null???? Type
?----------------------------------------- -------- ----------------------------
?SYMTAG_KEY??? ??? ??? ??? ?? NOT NULL NVARCHAR2(22)
?SYMTAG_CREATION_DATE??? ??? ??? ?? NOT NULL TIMESTAMP(6) WITH TIME ZONE

** 2.1 http://localhost:8080/apex - about database

National Language Parametersort by this column??? Value
NLS_CALENDAR??? GREGORIAN
NLS_CHARACTERSET??? AL32UTF8
NLS_COMP??? BINARY
NLS_CURRENCY??? $
NLS_DATE_FORMAT??? DD-MON-RR
NLS_DATE_LANGUAGE??? AMERICAN
NLS_DUAL_CURRENCY??? $
NLS_ISO_CURRENCY??? AMERICA
NLS_LANGUAGE??? AMERICAN
NLS_LENGTH_SEMANTICS??? BYTE
NLS_NCHAR_CHARACTERSET??? AL16UTF16
NLS_NCHAR_CONV_EXCP??? FALSE
NLS_NUMERIC_CHARACTERS??? .,
NLS_SORT??? BINARY
NLS_TERRITORY??? AMERICA
NLS_TIMESTAMP_FORMAT??? DD-MON-RR HH.MI.SSXFF AM
NLS_TIMESTAMP_TZ_FORMAT??? DD-MON-RR HH.MI.SSXFF AM TZR
NLS_TIME_FORMAT??? HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT??? HH.MI.SSXFF AM TZR

*** 3. Shell

chris at catch22:~$ locale
LANG=de_DE.UTF-8
LC_CTYPE=de_DE.UTF-8
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=
chris at catch22:~$


Regards,
chris

!DSPAM:4ccf4ae148588202753979!



From marshall.mcluhan at gmx.net  Tue Nov  2 21:39:03 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Tue, 2 Nov 2010 22:39:03 +0100
Subject: [cl-rdbms-devel] hu,
	dwim.rdbms + oracle 10g express + utf8 + input/read problem - 2
Message-ID: <vmime.4cd084f7.1b67.5397576ff4d0c1@gleanon.com>


Hi

Why does hu.dwim.rdbms.oralce not support utf-8 when it comes to connection encodings?
( database encodings supported are utf-8 and us-ascii )
Could that be related to my problem? Doesn't make sense except maybe this was written for cmucl? (a wild guess?)
I can't test this easily because I don't have a 32bit unix installed.

Maybe that's important: I entered the data (symtag_key varchar2, nvarchar2 datatype) into the tables with sqlplus.
But I got the same results (empty spaces/boxes in strings with sbcl started from the shell doing [select "SYMTAG_KEY" from symtag])
entering the data with hu.dwim.rdbms's insert-record function if I did not miss something.

Do you know of a hu.dwim.rdbms.oracle + oracle setup that actually works?
If so what operating system and oracle product is part of this setup?


Sqlplus has no problems showing me the data in both cases:
1. entering the data with sqlplus
2. entering the data with using hu.dwim.rdbms's insert-record

SQL> select "SYMTAG_KEY" from symtag2;

SYMTAG_KEY
----------------------
foo
foobar

SQL> select "SYMTAG_KEY" from symtag;

SYMTAG_KEY
--------------------------------------------------------------------------------
foo
foobar
foo11


If we can't solve this soon I will have to go with ascii or dig into the source code myself.
Have you any hints for the latter?



Regards,
chris



*** 0. no utf-8 connection encoding

** hu.dwim.rdbms/source/oralce/backend.lisp
.. (starts line 97)
??? (rdbms.debug "Connecting in transaction ~A" transaction)
??? (oci-call (oci:env-create (environment-handle-pointer transaction)
????????????????????????????? (logior
?????????????????????????????? (ecase (connection-encoding-of (database-of *transaction*))
???????????????????????????????? (:ascii 0)
???????????????????????????????? (:utf-16 oci:+utf-16+))
?????????????????????????????? *default-oci-flags*)
????????????????????????????? null null null null 0 null))
..


** hu.dwim.rdbms/source/oracle/database.lisp
.. (starts at line 9)
(def (class* e) oracle (database)
? ((connection-encoding
??? :utf-16
??? :type (member :ascii :utf-16))))
..



*** 1. All my settings are set to utf-8 except for the network encoding of hu.dwim.rdbms

chris at catch22:~$ locale
LANG=de_DE.UTF-8
LC_CTYPE=de_DE.UTF-8
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=


* sb-impl::*default-external-format*

:UTF-8
* (get-encodings)
21:54 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1004C49001}>
21:54 0?????????? RDBMS DEBUG?? About to COMMIT transaction #<#<STANDARD-CLASS NIL {1004C970B1}> :begin-executed-p #f {10039EB251}>
(("connection-encoding" . :UTF-16) ("database-encoding" . :UTF-8))
* 




*** 2. Fooling around with the NLS_LANG environment variable and sqlplus

AMERICAN_AMERICA.AL32UTF8 doesn't work with sqlplus but AMERICAN_AMERICA.AL32UTF8 does

chris at catch22:~$ sqlplus-connect.sh 
Error 19 initializing SQL*Plus
Invalid NLS character set for this OS environment
chris at catch22:~$ $NLS_LANG
bash: AMERICAN_AMERICA.AL16UTF16: Kommando nicht gefunden.
chris at catch22:~$ export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
chris at catch22:~$ sqlplus-connect.sh 

SQL*Plus: Release 10.2.0.5.0 - Production on Tue Nov 2 20:24:10 2010

Copyright (c) 1982, 2010, Oracle.? All Rights Reserved.


Connected to:
Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production

SQL>




*** 3. Oracle Environment Variables - bashrc snippet

export ORACLE_HOME=~/usr/instantclient_10_2
export XE=/usr/lib/oracle/xe/app/oracle/product/10.2.0/server
export ORACLE_SID=XE
export LD_LIBRARY_PATH=~/lib:/lib:/usr/lib:/usr/local/lib:~/usr/instantclient_10_2:$XE/lib
export PATH=~/bin:~/usr/instantclient_10_2:$XE/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/bin/X11:/usr/games? 
export ORACLE_USERID='chris/woirgend'
export ORACLE_DSN='dbi:Oracle:XE'
export TNS_ADMIN=~/usr/instantclient_10_2/network/admin
export NLS_LANG=AMERICAN_AMERICA.AL32UTF8




*** 4. SBCL console - What kind of characters are we dealing with?
* (select-symtag2)
20:58 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1004C49001}>
20:58 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
20:58 0?????????? RDBMS DEBUG?? Connecting in transaction #<#<STANDARD-CLASS NIL {1004C970B1}> :begin-executed-p #t {100370D6A1}>
20:58 0?????????? RDBMS DEBUG?? Logging on in transaction #<#<STANDARD-CLASS NIL {1004C970B1}> :begin-executed-p #t {100370D6A1}>
20:58 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
20:58 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
20:58 0?????????? RDBMS DEBUG?? Fetched: "???"
20:58 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
20:58 0?????????? RDBMS DEBUG?? Fetched: "??????"
20:58 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
20:58 0?????????? RDBMS DEBUG?? Fetched: "?????"
20:58 0?????????? RDBMS DEBUG?? About to COMMIT transaction #<#<STANDARD-CLASS NIL {1004C970B1}> :begin-executed-p #t {100370D6A1}>
20:58 0?????????? RDBMS DEBUG?? Cleaning up Oracle transaction #<#<STANDARD-CLASS NIL {1004C970B1}> :begin-executed-p #t {100370D6A1}> to database #<ORACLE {1004C49001}>
20:58 0?????????? RDBMS DEBUG?? Calling logoff in transaction #<#<STANDARD-CLASS NIL {1004C970B1}> :begin-executed-p #t {100370D6A1}>
20:58 0?????????? RDBMS DEBUG?? Freeing environment handle of transaction #<#<STANDARD-CLASS NIL {1004C970B1}> :begin-executed-p #t {100370D6A1}>
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {100395D23F}>
* (length *)

3
* (setq a **)

;???? (SETQ SYMARENA-ORACLE::A **)
; 
; caught WARNING:
;?? undefined variable: A
; 
; compilation unit finished
;?? Undefined variable:
;???? A
;?? caught 1 WARNING condition

#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {100395D23F}>
* (elt (elt a 0) 0)

"???"
* (setq b *)

;???? (SETQ SYMARENA-ORACLE::B *)
; 
; caught WARNING:
;?? undefined variable: B
; 
; compilation unit finished
;?? Undefined variable:
;???? B
;?? caught 1 WARNING condition

"???"
* b

"???"
* (elt b 0)

#\U6600
* #\U6600

#\U6600
* (elt b 1)

#\U6F00
*

** 3.1. emacs - scratch buffer - elisp

?U6FFO - doesn't work in the emacs scratch buffer so I copied the char from sbcl to the emacs scratch buffer after expressing it like this in sbcl #\U6FF0
and got the following:

(char-charset ??)
chinese-big5-2
?


!DSPAM:4cd084f748589401417091!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101102/51088008/attachment.html>

From marshall.mcluhan at gmx.net  Tue Nov  2 22:14:40 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Tue, 2 Nov 2010 23:14:40 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms + oracle + utf + input/read problem
	- 3
Message-ID: <vmime.4cd08d50.1b67.241d6f478749e51@gleanon.com>


Do you know if it is a setup problem or a software bug?
I have some time on my hands therefore if you could guide me a little I may be able enough to get things done. I'm eager to learn too.

Regards,
chris


!DSPAM:4cd08d5048581671511048!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101102/7393dbf5/attachment.html>

From marshall.mcluhan at gmx.net  Tue Nov  2 23:19:09 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Wed, 3 Nov 2010 00:19:09 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-4
Message-ID: <vmime.4cd09c6d.1b67.462eae3722af9ce1@gleanon.com>


My current thinking is that it is a software bug because using hu.dwim.rdbms's insert-record function to insert string data
and reading it afterwards with sqlplus works.

Therefore the software bug should have something to do with reading string data from the oracle database server with hu.dwim.rdbms.

Regards,
chris




*** Fooling around with the idea that I need to make xterm capable of unicode without success. (

(just boxes and asian chars insteat of empty spaces, also tried rxvt-unicode ...)

xterm +u8 -fn '-Misc-Fixed-Medium-R-SemiCondensed--13-120-75-75-C-60-ISO10646-1'


chris at catch22:~$ sbcl --eval "(progn (asdf::load-system 'symarena-oracle) (in-package "SYMORA"))"

* (select-symtag)
--TIME MARK 2010-11-02--
23:35 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1003F0E741}>
23:35 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
23:35 0?????????? RDBMS DEBUG?? Connecting in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:35 0?????????? RDBMS DEBUG?? Logging on in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:35 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
23:35 0?????????? RDBMS DEBUG?? Fetching "??????????????
???????????????????????????????????????????????? ????? 14;??????????" from buffer at index 0
23:35 0?????????? RDBMS DEBUG?? Fetched: "????? 14;??? 14;"
23:35 0?????????? RDBMS DEBUG?? Fetching "??????????????
???????????????????????????????????????????????? ????? 14;??????????" from buffer at index 0
23:35 0?????????? RDBMS DEBUG?? Fetched: "????? 14;??? 14;????
???????????????????????????????????????????????? ??"
23:35 0?????????? RDBMS DEBUG?? Fetching "??????????????
???????????????????????????????????????????????? ????? 14;??????????" from buffer at index 0
23:35 0?????????? RDBMS DEBUG?? Fetched: "????? 14;??? 14;??
??????????????????????????????????????????????? ??
???????????????????????????????????????????????? "
23:35 0?????????? RDBMS DEBUG?? Fetching "??????????????
???????????????????????????????????????????????? ????? 14;??????????" from buffer at index 0
23:35 0?????????? RDBMS DEBUG?? Fetched: "????? 14;??? 14;????
???????????????????????????????????????????????? ??????"
23:35 0?????????? RDBMS DEBUG?? About to COMMIT transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:35 0?????????? RDBMS DEBUG?? Cleaning up Oracle transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}> to database #<ORACLE {1004C7A001}>
23:35 0?????????? RDBMS DEBUG?? Calling logoff in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:35 0?????????? RDBMS DEBUG?? Freeing environment handle of transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {10034F65BF}>
* (select-symtag2)
23:35 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1004C7A001}>
23:35 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
23:35 0?????????? RDBMS DEBUG?? Connecting in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:35 0?????????? RDBMS DEBUG?? Logging on in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:35 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
23:35 0?????????? RDBMS DEBUG?? Fetching "??????????????
???????????????????????????????????????????????? ????? 14;??????????" from buffer at index 0
23:35 0?????????? RDBMS DEBUG?? Fetched: "????? 14;??? 14;"
23:35 0?????????? RDBMS DEBUG?? Fetching "??????????????
???????????????????????????????????????????????? ????? 14;??????????" from buffer at index 0
23:35 0?????????? RDBMS DEBUG?? Fetched: "????? 14;??? 14;????
???????????????????????????????????????????????? ??"
23:35 0?????????? RDBMS DEBUG?? Fetching "??????????????
???????????????????????????????????????????????? ????? 14;??????????" from buffer at index 0
23:35 0?????????? RDBMS DEBUG?? Fetched: "????? 14;??? 14;??
??????????????????????????????????????????????? ??
???????????????????????????????????????????????? "
23:35 0?????????? RDBMS DEBUG?? Fetching "??????????????
???????????????????????????????????????????????? ????? 14;??????????" from buffer at index 0
23:35 0?????????? RDBMS DEBUG?? Fetched: "????? 14;??? 14;????
???????????????????????????????????????????????? ??????"
23:35 0?????????? RDBMS DEBUG?? About to COMMIT transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:35 0?????????? RDBMS DEBUG?? Cleaning up Oracle transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}> to database #<ORACLE {1004C7A001}>
23:35 0?????????? RDBMS DEBUG?? Calling logoff in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:35 0?????????? RDBMS DEBUG?? Freeing environment handle of transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {100393E23F}>
*

chris at catch22:~$ xterm -u8 -fn '-Misc-Fixed-Medium-R-SemiCondensed--13-120-75-75-C-60-ISO10646-1'


* (select-symtag)
--TIME MARK 2010-11-02--
23:36 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1003F0E741}>
23:36 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
23:36 0?????????? RDBMS DEBUG?? Connecting in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:36 0?????????? RDBMS DEBUG?? Logging on in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:36 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
23:36 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:36 0?????????? RDBMS DEBUG?? Fetched: "???"
23:36 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:36 0?????????? RDBMS DEBUG?? Fetched: "??????"
23:36 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:36 0?????????? RDBMS DEBUG?? Fetched: "?????"
23:36 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:36 0?????????? RDBMS DEBUG?? Fetched: "????????"
23:36 0?????????? RDBMS DEBUG?? About to COMMIT transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:36 0?????????? RDBMS DEBUG?? Cleaning up Oracle transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}> to database #<ORACLE {1004C7A001}>
23:36 0?????????? RDBMS DEBUG?? Calling logoff in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:36 0?????????? RDBMS DEBUG?? Freeing environment handle of transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {10034F65BF}>
* (select-symtag2)
23:36 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1004C7A001}>
23:36 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
23:36 0?????????? RDBMS DEBUG?? Connecting in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:36 0?????????? RDBMS DEBUG?? Logging on in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:36 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
23:36 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:36 0?????????? RDBMS DEBUG?? Fetched: "???"
23:36 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:36 0?????????? RDBMS DEBUG?? Fetched: "??????"
23:36 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:36 0?????????? RDBMS DEBUG?? Fetched: "?????"
23:36 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:36 0?????????? RDBMS DEBUG?? Fetched: "????????"
23:36 0?????????? RDBMS DEBUG?? About to COMMIT transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:36 0?????????? RDBMS DEBUG?? Cleaning up Oracle transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}> to database #<ORACLE {1004C7A001}>
23:36 0?????????? RDBMS DEBUG?? Calling logoff in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:36 0?????????? RDBMS DEBUG?? Freeing environment handle of transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {100393E23F}>
*



chris at catch22:~$ rxvt-unicode
chris at catch22:~$ sbcl --eval "(progn (asdf::load-system 'symarena-oracle) (in-package "SYMORA"))

* (select-symtag)
--TIME MARK 2010-11-02--
23:38 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1003F0E741}>
23:38 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
23:38 0?????????? RDBMS DEBUG?? Connecting in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:38 0?????????? RDBMS DEBUG?? Logging on in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:38 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
23:38 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:38 0?????????? RDBMS DEBUG?? Fetched: "???"
23:38 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:38 0?????????? RDBMS DEBUG?? Fetched: "??????"
23:38 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:38 0?????????? RDBMS DEBUG?? Fetched: "?????"
23:38 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:38 0?????????? RDBMS DEBUG?? Fetched: "????????"
23:38 0?????????? RDBMS DEBUG?? About to COMMIT transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:38 0?????????? RDBMS DEBUG?? Cleaning up Oracle transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}> to database #<ORACLE {1004C7A001}>
23:38 0?????????? RDBMS DEBUG?? Calling logoff in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
23:38 0?????????? RDBMS DEBUG?? Freeing environment handle of transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {1004C7A021}>
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {10034F65BF}>
* (select-symtag2)
23:38 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1004C7A001}>
23:38 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
23:38 0?????????? RDBMS DEBUG?? Connecting in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:38 0?????????? RDBMS DEBUG?? Logging on in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:38 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
23:38 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:38 0?????????? RDBMS DEBUG?? Fetched: "???"
23:38 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:38 0?????????? RDBMS DEBUG?? Fetched: "??????"
23:38 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:38 0?????????? RDBMS DEBUG?? Fetched: "?????"
23:38 0?????????? RDBMS DEBUG?? Fetching "??????????????????????????" from buffer at index 0
23:38 0?????????? RDBMS DEBUG?? Fetched: "????????"
23:38 0?????????? RDBMS DEBUG?? About to COMMIT transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:38 0?????????? RDBMS DEBUG?? Cleaning up Oracle transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}> to database #<ORACLE {1004C7A001}>
23:38 0?????????? RDBMS DEBUG?? Calling logoff in transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
23:38 0?????????? RDBMS DEBUG?? Freeing environment handle of transaction #<#<STANDARD-CLASS NIL {1004CC6BC1}> :begin-executed-p #t {10036EE6A1}>
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {100393E23F}>
* 
?


!DSPAM:4cd09c6e48582529512900!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101103/62988021/attachment.html>

From marshall.mcluhan at gmx.net  Wed Nov  3 12:26:33 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Wed, 3 Nov 2010 13:26:33 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms+oracle+utf+read-from-server-problem-5
Message-ID: <vmime.4cd154f9.1b67.7d78442762c25c71@gleanon.com>


Looking at the source and debug output now. How can I find out more about the converter.
For example the function typemap-oci-to-lisp is somewhat mysterious to me. Where is it defined?


ORACLE> (describe #'typemap-oci-to-lisp)
#<CLOSURE (LAMBDA (SB-KERNEL:INSTANCE)) {1003A97B79}>
? [compiled closure]

Lambda-list: (INSTANCE)
Derived type: (FUNCTION (T) (VALUES T &OPTIONAL))
Source file: SYS:SRC;CODE;TARGET-DEFSTRUCT.LISP
; No value
ORACLE> 


chris at catch22:~/code/lisp-tp$ ls hu-dwim/
hu.dwim.asdf? hu.dwim.common? hu.dwim.common-lisp? hu.dwim.def? hu.dwim.defclass-star? hu.dwim.logger? hu.dwim.rdbms? hu.dwim.syntax-sugar? hu.dwim.util? hu.dwim.walker? local-time

chris at catch22:~/code/lisp-tp$ grep -R "typemap-oci-to-lisp" hu-dwim
hu-dwim/hu.dwim.rdbms/source/oracle/backend.lisp:?????????????????????? (converter (typemap-oci-to-lisp (typemap-of column-descriptor))))


*** source/oracle/backend.lisp

(def function fetch-column-value (column-descriptor row-index)
? (rdbms.debug "Fetching ~S from buffer at index ~D" (name-of column-descriptor) row-index)
? (aprog1 (let* ((indicator (cffi:mem-aref (indicators-of column-descriptor) :short row-index)))
??????????? (if (= indicator -1)
??????????????? :null
??????????????? (let* ((buffer (buffer-of column-descriptor))
?????????????????????? (size (size-of column-descriptor))
?????????????????????? (converter (typemap-oci-to-lisp (typemap-of column-descriptor))))
????????????????? #+nil
??? ??? ? (rdbms.dribble "Buffer:~%~A"
???????????????????????????????? (dump-c-byte-array buffer (* size +number-of-buffered-rows+)))
????????????????? (rdbms.dribble "Convert from ~D, size is ~D, content:~%~A"
???????????????????????????????? (typemap-external-type (typemap-of column-descriptor)) size
???????????????????????????????? (dump-c-byte-array (cffi:inc-pointer buffer (* row-index size))
??????????????????????????????????????????????????? size))

????????????????? (funcall converter
?????????????????????????? (cffi:inc-pointer buffer (* row-index size))
?????????????????????????? size))))
??? (rdbms.debug "Fetched: ~S" it)))


*** sbcl/shell

* (setf (hu.dwim.rdbms::log-level 'rdbms) 0)
* (select-symtag)
12:20 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1002AB8041}>
12:20 0???????????? SQL DRIBBLE SELECT SYMTAG_KEY FROM symtag
12:20 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
12:20 0?????????? RDBMS DEBUG?? Connecting in transaction #<#<STANDARD-CLASS NIL {10030AFD91}> :begin-executed-p #t {1003D1DE11}>
12:20 0?????????? RDBMS DEBUG?? Logging on in transaction #<#<STANDARD-CLASS NIL {10030AFD91}> :begin-executed-p #t {1003D1DE11}>
12:20 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
12:20 0?????????? RDBMS DRIBBLE Statement is allocated
12:20 0?????????? RDBMS DRIBBLE Retrieving column: name="??????????", type=1, size=44
12:20 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
12:20 0?????????? RDBMS DRIBBLE Convert from 5, size is 90, content:
66 00 6F 00 6F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
12:20 0?????????? RDBMS DEBUG?? Fetched: "???"
12:20 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
12:20 0?????????? RDBMS DRIBBLE Convert from 5, size is 90, content:
66 00 6F 00 6F 00 31 00 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
12:20 0?????????? RDBMS DEBUG?? Fetched: "?????"
12:20 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
12:20 0?????????? RDBMS DRIBBLE Convert from 5, size is 90, content:
66 00 6F 00 6F 00 62 00 61 00 72 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
12:20 0?????????? RDBMS DEBUG?? Fetched: "??????"
12:20 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
12:20 0?????????? RDBMS DRIBBLE Convert from 5, size is 90, content:
66 00 6F 00 6F 00 62 00 61 00 72 00 33 00 33 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
12:20 0?????????? RDBMS DEBUG?? Fetched: "????????"
12:20 0?????????? RDBMS DEBUG?? About to COMMIT transaction #<#<STANDARD-CLASS NIL {10030AFD91}> :begin-executed-p #t {1003D1DE11}>
12:20 0?????????? RDBMS DEBUG?? Cleaning up Oracle transaction #<#<STANDARD-CLASS NIL {10030AFD91}> :begin-executed-p #t {1003D1DE11}> to database #<ORACLE {1002AB8041}>
12:20 0?????????? RDBMS DEBUG?? Calling logoff in transaction #<#<STANDARD-CLASS NIL {10030AFD91}> :begin-executed-p #t {1003D1DE11}>
12:20 0?????????? RDBMS DEBUG?? Freeing environment handle of transaction #<#<STANDARD-CLASS NIL {10030AFD91}> :begin-executed-p #t {1003D1DE11}>
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {1003DCCCBF}>
* 


66 00 6F 00 6F 00??? ??? ??? ??? ??? equals "foo"

66 00 6F 00 6F 00 31 00 31 00??? ??? ??? ??? equals "foo11"

66 00 6F 00 6F 00 62 00 61 00 72 00??? ??? ??? equals "foobar"

66 00 6F 00 6F 00 62 00 61 00 72 00 33 00 33 00??? ??? equals "foobar33"




*** sqlplus

SQL> select "SYMTAG_KEY" from symtag;

SYMTAG_KEY
--------------------------------------------------------------------------------
foo
foobar
foo11
foobar33

SQL>




Regards,
chris


!DSPAM:4cd154f948581496711765!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101103/1eb7b2eb/attachment.html>

From marshall.mcluhan at gmx.net  Wed Nov  3 13:15:41 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Wed, 3 Nov 2010 14:15:41 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-6 - trailing
	zero byte problem for ascii characters using utf-8?
Message-ID: <vmime.4cd1607d.1b67.3f622117436a6d01@gleanon.com>


The converter doesn't seem to remove the second byte which is zero. ( 66 00 6F 00 6F 00 equals "foo" )
As far as I know UTF-8 uses only one byte per ascii character. How do I get this thing to remove the zero byte?



12:20 0?????????? RDBMS DRIBBLE Convert from 5, size is 90, content:
66 00 6F 00 6F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 


hex??? binary??? dec
0??? 0000??? 0
1??? 0001 ??? 1
2??? 0010 ??? 2
3??? 0011 ??? 3
4??? 0100 ??? 4
5??? 0101 ??? 5
6??? 0110 ??? 6
7??? 0111 ??? 7
8??? 1000 ??? 8

9??? 1001 ??? 9
A??? 1010 ??? 10
B??? 1011 ??? 11
C??? 1100 ??? 12
D??? 1101 ??? 13
E??? 1110 ??? 14
F??? 1111 ??? 15



*** How it should be:

ORACLE> #B01100110? ;? HEX: 66
102
ORACLE> (char-code #\f)
??? ??? ?? 
102



*** How it is

* (select-symtag2)
13:34 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1002AB8041}>
13:34 0???????????? SQL DRIBBLE SELECT SYMTAG_KEY FROM symtag
13:34 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
13:34 0?????????? RDBMS DEBUG?? Connecting in transaction #<#<STANDARD-CLASS NIL {10030AFD91}> :begin-executed-p #t {1003E4FDA1}>
13:34 0?????????? RDBMS DEBUG?? Logging on in transaction #<#<STANDARD-CLASS NIL {10030AFD91}> :begin-executed-p #t {1003E4FDA1}>
13:34 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
13:34 0?????????? RDBMS DRIBBLE Statement is allocated
13:34 0?????????? RDBMS DRIBBLE Retrieving column: name="??????????", type=1, size=44
13:34 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
13:34 0?????????? RDBMS DRIBBLE Convert from 5, size is 90, content:
66 00 6F 00 6F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
..

#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {1003F01CBF}>
* (setq x *)

#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {1003F01CBF}>
* (elt (elt x 0) 0)

"???"
* (elt * 0)

#\U6600?? ; this should be the character #\f
* (setq y *)

#\U6600
* (char-code y)

26112 ; this should be the char-code 102
* 


!DSPAM:4cd1607e48581541520698!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101103/92bce6de/attachment.html>

From attila.lendvai at gmail.com  Wed Nov  3 13:15:49 2010
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Wed, 3 Nov 2010 14:15:49 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-6 - trailing
 zero byte problem for ascii characters using utf-8?
In-Reply-To: <vmime.4cd1607d.1b67.3f622117436a6d01@gleanon.com>
References: <vmime.4cd1607d.1b67.3f622117436a6d01@gleanon.com>
Message-ID: <AANLkTindFz8AnrA76QwWkHW5tiKkhh9Z7k3az1tL=8dp@mail.gmail.com>

> The converter doesn't seem to remove the second byte which is zero. ( 66 00
> 6F 00 6F 00 equals "foo" )
> As far as I know UTF-8 uses only one byte per ascii character. How do I get
> this thing to remove the zero byte?
>
>
>
> 12:20 0?????????? RDBMS DRIBBLE Convert from 5, size is 90, content:
> 66 00 6F 00 6F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00


this data is utf-16, babel can deal with that, too.

i'm looking into installing oracle now, so that i can test it myself...

-- 
?attila



From marshall.mcluhan at gmx.net  Wed Nov  3 13:43:15 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Wed, 3 Nov 2010 14:43:15 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-7
Message-ID: <vmime.4cd166f3.1b67.59bb350711965e91@gleanon.com>


> > The converter doesn't seem to remove the second byte which is zero. ( 66 00
> > 6F 00 6F 00 equals "foo" )
> > As far as I know UTF-8 uses only one byte per ascii character. How do I get
> > this thing to remove the zero byte?
> >
> >
> >
> > 12:20 0?????????? RDBMS DRIBBLE Convert from 5, size is 90, content:
> > 66 00 6F 00 6F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 
> 
> this data is utf-16, babel can deal with that, too.
> 
> i'm looking into installing oracle now, so that i can test it myself...
>

Hm, the connection encoding is UTF-16. SBCL doesn't support UTF-16. Therefore I have to convert
the UTF-16 string with babel to UTF-8 if babel does such a thing.
Sounds promising. But, if that is right, why isn't this done by default.
Is there a lisp that uses UTF-16? I will try this later, have to eat and thank you.

> -- 
>? attila

chris


!DSPAM:4cd166f348585239786034!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101103/da7889c2/attachment.html>

From attila.lendvai at gmail.com  Wed Nov  3 13:47:24 2010
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Wed, 3 Nov 2010 14:47:24 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-7
In-Reply-To: <vmime.4cd166f3.1b67.59bb350711965e91@gleanon.com>
References: <vmime.4cd166f3.1b67.59bb350711965e91@gleanon.com>
Message-ID: <AANLkTikRYtdbbYmEbwAzb=ifa2A5mhLAs9Z_2DcDxKmB@mail.gmail.com>

>> > 12:20 0?????????? RDBMS DRIBBLE Convert from 5, size is 90, content:
>> > 66 00 6F 00 6F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> > 00
>> > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> > 00
>> > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> > 00
>> > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>
>>
>> this data is utf-16, babel can deal with that, too.
>>
>> i'm looking into installing oracle now, so that i can test it myself...
>>
>
> Hm, the connection encoding is UTF-16. SBCL doesn't support UTF-16.
> Therefore I have to convert
> the UTF-16 string with babel to UTF-8 if babel does such a thing.


reading this makes me think that you don't have a clear model of lisp
strings/unicode/encodings.

i don't remember reading this article, but a quick glance looks
promising: http://www.joelonsoftware.com/articles/Unicode.html

in short: you need to convert between lisp strings and byte arrays,
sending/receiving byte arrays to/from the database. the encoding
needed for OCI, oracle's C interface, is utf-16.

utf-8 is nowhere in the picture (if not the encoding emacs/slime uses
to communicate with the cl process).

-- 
?attila



From marshall.mcluhan at gmx.net  Wed Nov  3 18:52:19 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Wed, 3 Nov 2010 19:52:19 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-8
Message-ID: <vmime.4cd1af63.1b67.73a96ff778f5c121@gleanon.com>


?

>>> > 12:20 0?????????? RDBMS DRIBBLE Convert from 5, size is 90, content:
>>> > 66 00 6F 00 6F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> > 00
>>> > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> > 00
>>> > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> > 00
>>> > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>
>>>
>>> this data is utf-16, babel can deal with that, too.
>>>
>>> i'm looking into installing oracle now, so that i can test it myself...
>>>
>>
>> Hm, the connection encoding is UTF-16. SBCL doesn't support UTF-16.
>> Therefore I have to convert
>> the UTF-16 string with babel to UTF-8 if babel does such a thing.
>
>
>reading this makes me think that you don't have a clear model of lisp
>strings/unicode/encodings.

I probably don't, yes.

>
>i don't remember reading this article, but a quick glance looks
>promising: http://www.joelonsoftware.com/articles/Unicode.html
>

Interesting read, I saw this once but didn't read it. But I did now.

>in short: you need to convert between lisp strings and byte arrays,
>sending/receiving byte arrays to/from the database. the encoding
>needed for OCI, oracle's C interface, is utf-16.
>

Is getting strings in utf-16 at this point a feature or a bug?
Asked another way: Am I supposed to convert utf-16 strings to lisp strings or should that already be in there?

If it were not a bug I would get a byte array instead of a utf-16 string?
Error messages don't show up correctly either e.g. for violations of unique constraints?

On the other hand, this being a feature would makes sense when thinking about where the data is supposed to be seen,
the major web browsers. Assuming the major web browsers support utf-16.

But then, how do I go about manipulation utf-16 string data with sbcl.
Do I have to take a harder look at babel or flexi-streams?


>utf-8 is nowhere in the picture (if not the encoding emacs/slime uses
>to communicate with the cl process).
>

Isn't sbcl able to use utf-8 to represent its lisp strings?




Babel isn't able to convert between utf-16 and utf-8, yet? Confusion reigns ...

ORACLE> (let ((octet-array (make-array 6 :element-type '(unsigned-byte 8)
??? ??? ??? ??? ?????? :initial-contents (vector #X66 #X00 #X6F #X00 #X6F #X00))))
??? ? (babel:octets-to-string octet-array :encoding :utf-8))
"f@^o@^o^@"

ORACLE>


Regards,
chris

>-- 
> attila
>
>
?

?


!DSPAM:4cd1af6448581812115720!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101103/52265aea/attachment.html>

From marshall.mcluhan at gmx.net  Wed Nov  3 22:42:13 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Wed, 3 Nov 2010 23:42:13 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-9 - Just an
	endianness / byte-order problem?
Message-ID: <vmime.4cd1e545.1b67.599c1e7318924b1@gleanon.com>


We get big endian but need little endian? How do we best change to little endian?

* (select-symtag)
23:11 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1002AB8041}>
..
23:11 0?????????? RDBMS DRIBBLE Retrieving column: name="??????????", type=1, size=44
23:11 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
23:11 0?????????? RDBMS DRIBBLE Convert from 5, size is 90, content:
66 00 6F 00 6F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
23:11 0?????????? RDBMS DEBUG?? Fetched: "???"
..
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {100358C16F}>

* (defparameter foostring (elt (elt * 0) 0))

FOOSTRING
* (length foostring)

3
* (lst (elt foostring 0) (elt foostring 1) (elt foostring 2))

(#\U6600 #\U6F00 #\U6F00)
* (coerce '(#\U0066 #\U006F #\U006F) 'string)

"foo"
* foostring

"???"
* 



Regards,
chris


!DSPAM:4cd1e54648587535314996!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101103/2be26ff3/attachment.html>

From attila.lendvai at gmail.com  Wed Nov  3 23:26:47 2010
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Thu, 4 Nov 2010 00:26:47 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-8
In-Reply-To: <vmime.4cd1af63.1b67.73a96ff778f5c121@gleanon.com>
References: <vmime.4cd1af63.1b67.73a96ff778f5c121@gleanon.com>
Message-ID: <AANLkTi=WK3YZdthZTMVr61oto4fJ3N3m6in6xsKz9+b1@mail.gmail.com>

>>in short: you need to convert between lisp strings and byte arrays,
>>sending/receiving byte arrays to/from the database. the encoding
>>needed for OCI, oracle's C interface, is utf-16.
>>
>
> Is getting strings in utf-16 at this point a feature or a bug?


you don't get utf-16 strings. you get lisp strings (which consists of
characters, and whose encoding is implementation detail), which have
been constructed in a bogus way. so, yes, it's a bug (or
misconfiguration).

but setting up oracle is still a pita, my efforts have failed so far.
will play with it a bit more tomorrow.


> Asked another way: Am I supposed to convert utf-16 strings to lisp strings
> or should that already be in there?


the byte array -> lisp strings conversion should have been done
properly. (it's been done in a bogus way).


> On the other hand, this being a feature would makes sense when thinking
> about where the data is supposed to be seen,
> the major web browsers. Assuming the major web browsers support utf-16.


again, the API of hu.dwim.rfbms should return lisp strings, which have
no encoding (besides the implementation detail you can't see without
looking at the sources of your lisp vm).


> But then, how do I go about manipulation utf-16 string data with sbcl.
> Do I have to take a harder look at babel or flexi-streams?


hu.dwim.rdbms should have decoded utf-16 into lisp strings properly.
you, as a user of it, have nothing to do. as a developer, it's a bug
and/or misconfiguration to be found.


>>utf-8 is nowhere in the picture (if not the encoding emacs/slime uses
>>to communicate with the cl process).
>>
>
> Isn't sbcl able to use utf-8 to represent its lisp strings?


it is able to *export* its strings into byte arrays using various
encodings. but it's a very different thing from using utf-8 to
represent lisp strings.


> Babel isn't able to convert between utf-16 and utf-8, yet? Confusion reigns
> ...
> ORACLE> (let ((octet-array (make-array 6 :element-type '(unsigned-byte 8)
> ??? ??? ??? ??? ?????? :initial-contents (vector #X66 #X00 #X6F #X00 #X6F
> #X00))))
> ??? ? (babel:octets-to-string octet-array :encoding :utf-8))
> "f@^o@^o^@"
>


babel can do these, and only these conversions:

byte vector -> lisp string
lisp string -> byte vector

in the above you feed in an utf-16 byte vector and tell babel to
decode it as utf-8.

-- 
?attila



From attila.lendvai at gmail.com  Wed Nov  3 23:33:13 2010
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Thu, 4 Nov 2010 00:33:13 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-8
In-Reply-To: <AANLkTi=WK3YZdthZTMVr61oto4fJ3N3m6in6xsKz9+b1@mail.gmail.com>
References: <vmime.4cd1af63.1b67.73a96ff778f5c121@gleanon.com>
	<AANLkTi=WK3YZdthZTMVr61oto4fJ3N3m6in6xsKz9+b1@mail.gmail.com>
Message-ID: <AANLkTimvCHD=df9-bk8JMsYLBwe-aON7iyoARYrc-wb6@mail.gmail.com>

oh, and one more thing: there's a pending patch to babel which
introduces utf-16 encoding with explicit byte order... but it's not in
the repo yet. i've dropped a mail asking what's the situation with it.

reading your other mail, i think it's a byte order issue.

-- 
?attila



From marshall.mcluhan at gmx.net  Thu Nov  4 12:23:55 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Thu, 4 Nov 2010 13:23:55 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-10 - Is it babel?
Message-ID: <vmime.4cd2a5db.1b67.38301ad75e321941@gleanon.com>


Tried a dirty patch so that I can move on until the proper patch arrives.
Strange thing is, it doesn't work when it comes to hu.dwim.rdbms.oracle?
Don't like second guessing you but maybe babel isn't doing this?


*** Before the patch:

BABEL> (let ((octet-array-utf-16le (make-array 6 :element-type '(unsigned-byte 8)
??? ??? ??? ??? ??? ?????? :initial-contents (vector #X66 #X00 #X6F #X00 #X6F #X00))))
??? ?(babel:octets-to-string octet-array-utf-16le :encoding :utf-16))
"???"
BABEL> 


*** After the path:

BABEL> (let ((octet-array-utf-16le (make-array 6 :element-type '(unsigned-byte 8)
??? ??? ??? ??? ??? ?????? :initial-contents (vector #X66 #X00 #X6F #X00 #X6F #X00))))
??? ?(babel:octets-to-string octet-array-utf-16le :encoding :utf-16))
"foo"
BABEL>



$ cat babel-src-enc-unicode.lisp.diff 
548c548
<?????????????????????? (t #+little-endian nil)))))
---
>?????????????????????? (t #+little-endian t)))))
602c602
<????????????????????? (t #+little-endian nil)))))
---
>????????????????????? (t #+little-endian t)))))



But after the patch the trouble with hu.dwim.rdbms.oracle stays the same:

$ rm -r .cache/common-lisp/sbcl-1.0.37-linux-x86-64/home/chris/code/lisp-tp/babel/
$ sbcl --eval "(progn (asdf::load-system 'symarena-oracle) (in-package "SYMORA"))"

..

* (select-symtag)
--TIME MARK 2010-11-04--
12:51 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1003B5A741}>
12:51 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
..
12:51 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
12:51 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
12:51 0?????????? RDBMS DEBUG?? Fetched: "???"
12:51 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
..
12:51 0?????????? RDBMS DEBUG?? Freeing environment handle of transaction #<#<STANDARD-CLASS NIL {1004F711B1}> :begin-executed-p #t {1003286641}>
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {100415A8DF}>
* (defparameter foostr (elt (elt * 0) 0))

FOOSTR
* (list (elt foostr 0) (elt foostr 1) (elt foostr 2))

(#\U6600 #\U6F00 #\U6F00)
* foostr

"???"
* 

?

?

?

Regards,

chris


!DSPAM:4cd2a5dc48581985511769!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101104/1d35d953/attachment.html>

From attila.lendvai at gmail.com  Thu Nov  4 15:06:10 2010
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Thu, 4 Nov 2010 16:06:10 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-8
In-Reply-To: <AANLkTimvCHD=df9-bk8JMsYLBwe-aON7iyoARYrc-wb6@mail.gmail.com>
References: <vmime.4cd1af63.1b67.73a96ff778f5c121@gleanon.com>
	<AANLkTi=WK3YZdthZTMVr61oto4fJ3N3m6in6xsKz9+b1@mail.gmail.com>
	<AANLkTimvCHD=df9-bk8JMsYLBwe-aON7iyoARYrc-wb6@mail.gmail.com>
Message-ID: <AANLkTikcfEn3zD9NV-Pn-o=rcRa6kKggrbSeV0eauRPD@mail.gmail.com>

> oh, and one more thing: there's a pending patch to babel which

fyi, that pending patch has been pushed to babel. now it supports utf-16le/be

-- 
?attila



From marshall.mcluhan at gmx.net  Thu Nov  4 16:36:52 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Thu, 4 Nov 2010 17:36:52 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-11 - Still
	trouble even with new babel?
Message-ID: <vmime.4cd2e125.1b67.1a376ac72c432ed1@gleanon.com>



For me it seems iolib is the only system using babel directly?
Can't see iolib using the octects-to-string function, only for dns but maybe I didn't look hard enough.
Or it uses another babel function to do this?

Do I have to turn on the right byte-ordering somewhere, just for the time being?


Regards,
chris


BABEL> (let ((octet-array-utf-16le (make-array 6 :element-type '(unsigned-byte 8)
??? ??? ??? ??? ??? ?????? :initial-contents (vector #X66 #X00 #X6F #X00 #X6F #X00))))
??? ?(babel:octets-to-string octet-array-utf-16le :encoding :utf-16))
"???"
BABEL>


* (select-symtag)
--TIME MARK 2010-11-04--
17:03 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1003BAE741}>
17:03 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
..
17:03 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
17:03 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
17:03 0?????????? RDBMS DEBUG?? Fetched: "???"
17:03 0?????????? RDBMS DEBUG?? Fetching "??????????" from buffer at index 0
..
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {100403E8DF}>
* (defparameter foostr (elt (elt * 0) 0))

FOOSTR
* (lst (elt foostr 0) (elt foostr 1) (elt foostr 2))

(#\U6600 #\U6F00 #\U6F00)
* foostr

"???"
* 


!DSPAM:4cd2e12548585927441065!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101104/650b09f0/attachment.html>

From attila.lendvai at gmail.com  Thu Nov  4 16:45:23 2010
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Thu, 4 Nov 2010 17:45:23 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-11 - Still
 trouble even with new babel?
In-Reply-To: <vmime.4cd2e125.1b67.1a376ac72c432ed1@gleanon.com>
References: <vmime.4cd2e125.1b67.1a376ac72c432ed1@gleanon.com>
Message-ID: <AANLkTikKUaxZfOO6YDXzggVTm_C04f8b5F3mjHfXxGXE@mail.gmail.com>

> For me it seems iolib is the only system using babel directly?
> Can't see iolib using the octects-to-string function, only for dns but maybe
> I didn't look hard enough.
> Or it uses another babel function to do this?
>
> Do I have to turn on the right byte-ordering somewhere, just for the time
> being?


ouch: this is in oracle/cffi-util.lisp

(def function oci-string-to-lisp (pointer &optional size)
  #+nil
  (cffi:foreign-string-to-lisp pointer :count size
                               :encoding (connection-encoding-of
(database-of *transaction*)))
  ;; the above doesn't work, because babel thinks the encoding is
  ;; invalid and returns question marks only.  Perhaps Babel doesn't
  ;; understand the endianness?  Need to investigate.
  (coerce (iter (for i from 0 by 2)
		(when size (while (< i size)))
		(let ((code (cffi:mem-ref pointer :short i)))
		  (until (zerop code))
		  (collect (code-char code))))
	  'string))

try to re-enable the #+nil-ed line and drop the rest...

this change is nasty, it should have deserved a KLUDGE comment, or
maybe even better would have been a runtime (warn "description of the
issue")...

David, if you're reading this then update your coding standards... in
the long term you'll also benefit from it, believe me... ;)

-- 
?attila



From marshall.mcluhan at gmx.net  Thu Nov  4 16:51:42 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Thu, 4 Nov 2010 17:51:42 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems-12 - ops,
	babel does it
Message-ID: <vmime.4cd2e49e.1b67.1d91a1b7440ff561@gleanon.com>


And hu.dwim.rdbms does use babel directly too.

So now we just need to patch hu.dwim.rdbms?

?

?

Regards,

chris

?

BABEL> (let ((octet-array-utf-16le (make-array 6 :element-type '(unsigned-byte 8)
??? ??? ??? ??? ??? ?????? :initial-contents (vector #X66 #X00 #X6F #X00 #X6F #X00))))
??? ?(babel:octets-to-string octet-array-utf-16le :encoding :utf-16le))
"foo"


!DSPAM:4cd2e49e48581293472516!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101104/e0255a1f/attachment.html>

From marshall.mcluhan at gmx.net  Thu Nov  4 17:11:09 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Thu, 4 Nov 2010 18:11:09 +0100
Subject: [cl-rdbms-devel] hu.dwim.rdbms.oracle-utf-problems
Message-ID: <vmime.4cd2e92d.1b67.4b27afa71a7cfcf1@gleanon.com>


Solved. Thank you

?

--------

chris.

?

?

* (select-symtag)
--TIME MARK 2010-11-04--
17:46 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {1003212741}>
17:46 0?????????? RDBMS DEBUG?? Executing "SELECT SYMTAG_KEY FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
17:46 0?????????? RDBMS DEBUG?? Connecting in transaction #<#<STANDARD-CLASS NIL {1005497791}> :begin-executed-p #t {100501B101}>
17:46 0?????????? RDBMS DEBUG?? Logging on in transaction #<#<STANDARD-CLASS NIL {1005497791}> :begin-executed-p #t {100501B101}>
17:46 0?????????? RDBMS DEBUG?? Preparing command: "SELECT SYMTAG_KEY FROM symtag"
17:46 0?????????? RDBMS DEBUG?? Fetching "SYMTAG_KEY" from buffer at index 0
17:46 0?????????? RDBMS DEBUG?? Fetched: "foo"
17:46 0?????????? RDBMS DEBUG?? Fetching "SYMTAG_KEY" from buffer at index 0
17:46 0?????????? RDBMS DEBUG?? Fetched: "foo11"
17:46 0?????????? RDBMS DEBUG?? Fetching "SYMTAG_KEY" from buffer at index 0
17:46 0?????????? RDBMS DEBUG?? Fetched: "foobar"
17:46 0?????????? RDBMS DEBUG?? Fetching "SYMTAG_KEY" from buffer at index 0
17:46 0?????????? RDBMS DEBUG?? Fetched: "foobar33"
17:46 0?????????? RDBMS DEBUG?? About to COMMIT transaction #<#<STANDARD-CLASS NIL {1005497791}> :begin-executed-p #t {100501B101}>
17:46 0?????????? RDBMS DEBUG?? Cleaning up Oracle transaction #<#<STANDARD-CLASS NIL {1005497791}> :begin-executed-p #t {100501B101}> to database #<ORACLE {100501B001}>
17:46 0?????????? RDBMS DEBUG?? Calling logoff in transaction #<#<STANDARD-CLASS NIL {1005497791}> :begin-executed-p #t {100501B101}>
17:46 0?????????? RDBMS DEBUG?? Freeing environment handle of transaction #<#<STANDARD-CLASS NIL {1005497791}> :begin-executed-p #t {100501B101}>
#<(AND (VECTOR T 8) (NOT SIMPLE-ARRAY)) {1003A38F1F}>
* 


!DSPAM:4cd2e92d48581444717246!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101104/aeb6e27f/attachment.html>

From marshall.mcluhan at gmx.net  Thu Nov  4 21:15:38 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Thu, 4 Nov 2010 22:15:38 +0100
Subject: [cl-rdbms-devel]
	hu.dwim.rdbms.oracle-timestamp-with-timezone-local-time-bug
Message-ID: <vmime.4cd3227a.1b67.1dd98497338fd581@gleanon.com>



How do I handle such thing most efficiently? Should I report them at all?

Did change the following in hu.dwim.rdbms/source/oracle/conversion.lisp
(dunno if this does more harm than good without unit tests but atleast the error is gone)


(def function make-timezone (hours minutes)
? (let ((offset-in-sec (* (+ (* 60 hours) minutes) 60)))
??? (if (and (= minutes 0)
???????????? (= hours 0))
??????? +utc-zone+
??????? (local-time::make-timezone
??? ?:subzones `((,offset-in-sec nil "anonymous" nil nil))
??? ?:name "anonymous"
??? ?:loaded t))))

to??? ?

(def function make-timezone-old (hours minutes)
? (let ((offset-in-sec (* (+ (* 60 hours) minutes) 60)))
??? (if (and (= minutes 0)
???????????? (= hours 0))
??????? +utc-zone+
??? (local-time::make-timezone
??? ?:subzones (make-array 1 :initial-contents
??? ??? ??? ?????? (list (local-time::make-subzone
??? ??? ??? ??? ????? :offset offset-in-sec)))
??? ?:name "anonymous"
??? ?:loaded t))))


to get rid of:??? ?

SYMORA> (with-symdb (execute [select * symtag]))
--TIME MARK 2010-11-04--
21:34 0?????????? RDBMS DEBUG?? About to BEGIN transaction in database #<ORACLE {100DD2F9B1}>
21:34 0?????????? RDBMS DEBUG?? Executing "SELECT * FROM symtag"
STYLE-WARNING: Undefined alien: "OCIGetInstantClientPackageName"
21:34 0?????????? RDBMS DEBUG?? Connecting in transaction #<#<STANDARD-CLASS NIL {10033B8C31}> :begin-executed-p #t {1002DFA061}>
21:34 0?????????? RDBMS DEBUG?? Logging on in transaction #<#<STANDARD-CLASS NIL {10033B8C31}> :begin-executed-p #t {1002DFA061}>
21:34 0?????????? RDBMS DEBUG?? Preparing command: "SELECT * FROM symtag"
21:34 0?????????? RDBMS DEBUG?? Fetching "SYMTAG_KEY" from buffer at index 0
21:34 0?????????? RDBMS DEBUG?? Fetched: "foo"
21:34 0?????????? RDBMS DEBUG?? Fetching "SYMTAG_CREATION_DATE" from buffer at index 0
21:34 0?????????? RDBMS DEBUG?? Fetched: :NULL
21:34 0?????????? RDBMS DEBUG?? Fetching "SYMTAG_KEY" from buffer at index 0
21:34 0?????????? RDBMS DEBUG?? Fetched: "foobar"
21:34 0?????????? RDBMS DEBUG?? Fetching "SYMTAG_CREATION_DATE" from buffer at index 0
21:34 0?????????? RDBMS DEBUG?? Fetched: :NULL
21:34 0?????????? RDBMS DEBUG?? Fetching "SYMTAG_KEY" from buffer at index 0
21:34 0?????????? RDBMS DEBUG?? Fetched: "foo11"
21:34 0?????????? RDBMS DEBUG?? Fetching "SYMTAG_CREATION_DATE" from buffer at index 0


--- *sbcl debug buffer*

The value ((3600 NIL "anonymous" NIL NIL))
is not of type
? (SIMPLE-ARRAY T (*)).
?? [Condition of type TYPE-ERROR]

Restarts:
?0: [TERMINATE-TRANSACTION] return (values) from the WITH-TRANSACTION block executing the current terminal action :COMMIT
?1: [COMMIT-TRANSACTION] mark transaction for commit only and return (values) from the WITH-TRANSACTION block
?2: [ROLLBACK-TRANSACTION] mark transaction for rollback only and return (values) from the WITH-TRANSACTION block
?3: [RESTART-TRANSACTION] rollback the transaction by unwinding the stack and restart the WITH-TRANSACTION block in a new database transaction
?4: [RETRY] Retry SLIME REPL evaluation request.
?5: [*ABORT] Return to SLIME's top level.
?--more--

Backtrace:
? 0: (LOCAL-TIME::MAKE-TIMEZONE)[:EXTERNAL]
????? Locals:
??????? SB-DEBUG::ARG-0 = 6
? 1: (HU.DWIM.RDBMS.ORACLE::LOCAL-TIME-FROM-TIMESTAMP-TZ #.(SB-SYS:INT-SAP #X006918C0) #<unavailable argument>)
????? Locals:
??????? SB-DEBUG::ARG-0 = #.(SB-SYS:INT-SAP #X006918C0)
??????? SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
? 2: (HU.DWIM.RDBMS.ORACLE::FETCH-COLUMN-VALUE #<HU.DWIM.RDBMS.ORACLE::COLUMN-DESCRIPTOR {100B9E6761}> 0)
????? Locals:
??????? SB-DEBUG::ARG-0 = #<HU.DWIM.RDBMS.ORACLE::COLUMN-DESCRIPTOR {100B9E6761}>
??????? SB-DEBUG::ARG-1 = 0
? 3: (CURRENT-ROW #<HU.DWIM.RDBMS.ORACLE::ORACLE-SEQUENTIAL-ACCESS-CURSOR {100B9E6E71}>)[:EXTERNAL]
? 4: (FOR-EACH-ROW #<CLOSURE (LAMBDA #) {100B9E8979}> #<HU.DWIM.RDBMS.ORACLE::ORACLE-SEQUENTIAL-ACCESS-CURSOR {100B9E6E71}>)[:EXTERNAL]
? 5: (COLLECT-ROWS #<HU.DWIM.RDBMS.ORACLE::ORACLE-SEQUENTIAL-ACCESS-CURSOR {100B9E6E71}>)[:EXTERNAL]
? 6: (HU.DWIM.RDBMS.ORACLE::EXECUTE-PREPARED-STATEMENT ..)[:EXTERNAL]
? 7: ((SB-PCL::FAST-METHOD HU.DWIM.RDBMS::EXECUTE-COMMAND (ORACLE HU.DWIM.RDBMS.ORACLE::ORACLE-TRANSACTION STRING)) ..)[:EXTERNAL]
? 8: ((LAMBDA (SB-PCL::.PV. SB-PCL::.NEXT-METHOD-CALL. SB-PCL::.ARG0. SB-PCL::.ARG1. SB-PCL::.ARG2. SB-INT:&MORE SB-PCL::.DFUN-MORE-CONTEXT. SB-PCL::.DFUN-MORE-COUNT.)) ..)
? 9: (HU.DWIM.RDBMS::CALL-WITH-TRANSACTION* #<FUNCTION (LABELS HU.DWIM.RDBMS::WITH-TRANSACTION*-BODY) {10034E8619}>)[:EXTERNAL]
?10: ((LAMBDA ()))
?11: (SB-INT:SIMPLE-EVAL-IN-LEXENV (WITH-SYMDB (EXECUTE (SQL #))) #<NULL-LEXENV>)
?--more--
?


!DSPAM:4cd3227b48581611617599!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101104/76a167bf/attachment.html>

From marshall.mcluhan at gmx.net  Fri Nov  5 12:29:37 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Fri, 5 Nov 2010 13:29:37 +0100
Subject: [cl-rdbms-devel] Convenience over extensibility? Or: How to build
	on top of hu.dwim.rdbms?
Message-ID: <vmime.4cd3f8b1.1b67.746e108765ff0f11@gleanon.com>


Hi

While the sql reader syntax e.g. [select * table] is convenient for humans
it doesn't allow for being used by other functions or does it?

In my opinion such a sql reader syntax is nice to have but not being able to extend the software
through the use of functions is disastrous in my eyes. I have the feeling that we create two camps here, one the developers
and the other the users. Personally I don't like that distinction and would much prefer to make the possibility of extending
the software through the use of functions as easy as possible for the so called "user".

With that in mind I find myself using internal function and classes from the hu.dwim.rdbms package
( e.g. sql-and, sql-sequence-nextval-column, sql-identifier, sql-all-columns, sql-= )
and writing wrapper functions for it e.g.

(defun sqlid (name)
? (hu.dwim.rdbms::sql-identifier :name (string-upcase name)))

(defun sqleq (left right)
? (hu.dwim.rdbms::sql-= left right))

(defun sqlallcol ()
? (lst (hu.dwim.rdbms::sql-all-columns)))

(defun sqlselect-single (tbl keycol-id keycol-val)
? (select (sqlallcol) (lst tbl)
??? ? :where (sqleq (sqlid keycol-id) keycol-val)))
? 
(defun seqnext-literal (name)
? (make-instance
?? 'hu.dwim.rdbms::sql-sequence-nextval-column
?? :name (string-upcase name)))


What do you think about what I just said. Did I go wrong somewhere?
And if the above is really an issue, any ideas as how to handle it?

Right now I did create another system and package called hu.dwim.rdbms.oracle.ext for things like the above.
(The names above are not carved into stone tablets. I often change my mind before I settle for certain names and yes,
?often the wrappers seem unnecessary but I don't want to change the package definition of hu.dwim.rdbms
?and/or prefer shorter names.)


Regards,
chris


!DSPAM:4cd3f8b248581688513886!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101105/b5b2e9e2/attachment.html>

From attila.lendvai at gmail.com  Fri Nov  5 12:40:04 2010
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Fri, 5 Nov 2010 13:40:04 +0100
Subject: [cl-rdbms-devel] Convenience over extensibility? Or: How to
 build on top of hu.dwim.rdbms?
In-Reply-To: <vmime.4cd3f8b1.1b67.746e108765ff0f11@gleanon.com>
References: <vmime.4cd3f8b1.1b67.746e108765ff0f11@gleanon.com>
Message-ID: <AANLkTindcrxaadRak0-Pz1bYDD3fNc-r18Xuc5s0cc63@mail.gmail.com>

> While the sql reader syntax e.g. [select * table] is convenient for humans
> it doesn't allow for being used by other functions or does it?


(read-from-string "") as a last resort, otherwise it reads into a CLOS
based AST anyways and that you can construct programmatically.

note: the SQL reader and AST stuff is on the TODO to be moved into
hu.dwim.quasi-quote and rebased on its infrastructure. but it's not
high priority, and wouldn't change anything conceptually...


> In my opinion such a sql reader syntax is nice to have but not being able to
> extend the software
> through the use of functions is disastrous in my eyes. I have the feeling
> that we create two camps here, one the developers
> and the other the users. Personally I don't like that distinction and would


it's a library for developers, there are no two camps. it's only developers...


> much prefer to make the possibility of extending
> the software through the use of functions as easy as possible for the so
> called "user".
>
> With that in mind I find myself using internal function and classes from the
> hu.dwim.rdbms package
> ( e.g. sql-and, sql-sequence-nextval-column, sql-identifier,
> sql-all-columns, sql-= )
> and writing wrapper functions for it e.g.


we didn't put too much effort into deciding what should be exported
and be part of a long-time API, and what isn't. but while (not) doing
that we were on the conservative side...


> What do you think about what I just said. Did I go wrong somewhere?
> And if the above is really an issue, any ideas as how to handle it?


define your own convenience functions? if they seem to be generally
useful, propose an extension (but to spare some time, the naming
convention you use in your functions will not go though :)


> Right now I did create another system and package called
> hu.dwim.rdbms.oracle.ext for things like the above.
> (The names above are not carved into stone tablets. I often change my mind
> before I settle for certain names and yes,
> ?often the wrappers seem unnecessary but I don't want to change the package
> definition of hu.dwim.rdbms
> ?and/or prefer shorter names.)


that's a good compromise and leaves you the freedom to use
names/shortcuts you prefer. even if such utils were provided by the
main lib, it would not please everyone... so the way to go is to
provide a preferably complete skeleton of functionality using
descriptive and sometimes long names, and let people do their
shortcuts if the need be.

we stick to long names (and use fuzzy completion in slime).

-- 
?attila



From marshall.mcluhan at gmx.net  Fri Nov  5 13:44:55 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Fri, 5 Nov 2010 14:44:55 +0100
Subject: [cl-rdbms-devel] Convenience over extensibility? Or: How to build
	on top of hu.dwim.rdbms?
Message-ID: <vmime.4cd40a57.1b67.a8343772fc239a1@gleanon.com>


?

>> While the sql reader syntax e.g. [select * table] is convenient for humans
>> it doesn't allow for being used by other functions or does it?
>
>
>(read-from-string "") as a last resort, otherwise it reads into a CLOS
>based AST anyways and that you can construct programmatically.
>
>note: the SQL reader and AST stuff is on the TODO to be moved into
>hu.dwim.quasi-quote and rebased on its infrastructure. but it's not
>high priority, and wouldn't change anything conceptually...
>
>
>> In my opinion such a sql reader syntax is nice to have but not being able to
>> extend the software
>> through the use of functions is disastrous in my eyes. I have the feeling
>> that we create two camps here, one the developers
>> and the other the users. Personally I don't like that distinction and would
>
>
>it's a library for developers, there are no two camps. it's only developers....
>
>

I made a rash judgement here, sorry.

>> much prefer to make the possibility of extending
>> the software through the use of functions as easy as possible for the so
>> called "user".
>>
>> With that in mind I find myself using internal function and classes from the
>> hu.dwim.rdbms package
>> ( e.g. sql-and, sql-sequence-nextval-column, sql-identifier,
>> sql-all-columns, sql-= )
>> and writing wrapper functions for it e.g.
>
>
>we didn't put too much effort into deciding what should be exported
>and be part of a long-time API, and what isn't. but while (not) doing
>that we were on the conservative side...
>

Maybe we should separate the sql reader syntax "stuff" from the rest, so we have one more package to maintain? :)

Please make sure that you expose a "relative" low level interface that can be accessed programmatically
and is as expressive as necessary so that a great variety of extension can be build on top of it.
I find it somewhat puzzling that the sql reader syntax was not build on top of such an interface.
After all, the sql reader syntax should be just an extension?

If I think about I often work in one package despite the fact that two would be warranted just to get things done.
Maintaining packages is hassle. So maybe not so puzzling after all ...


>
>> What do you think about what I just said. Did I go wrong somewhere?
>> And if the above is really an issue, any ideas as how to handle it?
>
>
>define your own convenience functions? if they seem to be generally
>useful, propose an extension (but to spare some time, the naming
>convention you use in your functions will not go though :)
>

Yes, I will stick to my convenience functions and adapt to your future changes.

As for the naming convention I'm more than willing to compromise and there are many ways for dealing with this
e.g. emacs abbrev mode, graham's abbrev macro or just inline functions for personal use.


Don't worry, I never expected my naming convention to go through.

>
>> Right now I did create another system and package called
>> hu.dwim.rdbms.oracle.ext for things like the above.
>> (The names above are not carved into stone tablets. I often change my mind
>> before I settle for certain names and yes,
>>? often the wrappers seem unnecessary but I don't want to change the package
>> definition of hu.dwim.rdbms
>>? and/or prefer shorter names.)
>
>
>that's a good compromise and leaves you the freedom to use
>names/shortcuts you prefer. even if such utils were provided by the
>main lib, it would not please everyone... so the way to go is to
>provide a preferably complete skeleton of functionality using
>descriptive and sometimes long names, and let people do their
>shortcuts if the need be.
>
>we stick to long names (and use fuzzy completion in slime).
>
>-- 
> attila
>
>

?--
?chris


?


!DSPAM:4cd40a5848581824115346!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101105/66e31178/attachment.html>

From marshall.mcluhan at gmx.net  Fri Nov  5 14:43:25 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Fri, 5 Nov 2010 15:43:25 +0100
Subject: [cl-rdbms-devel] Convenience over extensibility? Or: How to build
	on top of hu.dwim.rdbms?
Message-ID: <vmime.4cd4180d.1b67.687e0d672ba1e531@gleanon.com>


>we stick to long names (and use fuzzy completion in slime).

I will try this fuzzy completion thing but I'm sceptical. Shorter names may still give you an advantage when
it comes to reading code. :)

You may argue that the win in clarity is well worth it. I would would argue that while
it is true that short names can easily obfuscate code it's not necessarily so.
But I don't want to get into this debate. When working with other people I would always adapt.

Just want to add that I'm grateful for your software, thank you.


Regards,
chris


!DSPAM:4cd4180e48581103713428!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101105/3e7c6d74/attachment.html>

From attila.lendvai at gmail.com  Fri Nov  5 14:56:35 2010
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Fri, 5 Nov 2010 15:56:35 +0100
Subject: [cl-rdbms-devel] Convenience over extensibility? Or: How to
 build on top of hu.dwim.rdbms?
In-Reply-To: <vmime.4cd4180d.1b67.687e0d672ba1e531@gleanon.com>
References: <vmime.4cd4180d.1b67.687e0d672ba1e531@gleanon.com>
Message-ID: <AANLkTinqPX6FoDZepDfzw9ekcybTf6swPhchPN8EKCXF@mail.gmail.com>

>>we stick to long names (and use fuzzy completion in slime).
>
> I will try this fuzzy completion thing but I'm sceptical. Shorter names may
> still give you an advantage when
> it comes to reading code. :)


i suggest a bit more meditation on this. and if you still haven't
changed your mind, then run experiments that include returning to code
you wrote several months/years ago... :)

sometimes it happens that i accidentally implement something twice
because of patch organization problems and/or not having an unlimited
memory. to my delight, i more often than not name stuff exactly the
same way in both implementations which are quite distant in time.

it also means that when i happen to look for something, i'll grep for
at least a part of the name i would give it. this strategy usually
very cheaply gets me to what i'm looking for. even years later...

all in all, having long and descriptive names is not a question for me
anymore, especially with slime's fuzzy completion.


> Just want to add that I'm grateful for your software, thank you.


we're glad you find it useful!

-- 
?attila



From marshall.mcluhan at gmx.net  Fri Nov  5 16:20:15 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Fri, 5 Nov 2010 17:20:15 +0100
Subject: [cl-rdbms-devel] Convenience over extensibility? Or: How to build
	on top of hu.dwim.rdbms?
Message-ID: <vmime.4cd42ebf.1b67.3a7a5e5737c7a1c1@gleanon.com>


>>>we stick to long names (and use fuzzy completion in slime).
>>
>> I will try this fuzzy completion thing but I'm sceptical. Shorter names may
>> still give you an advantage when
>> it comes to reading code. :)
>
>
>i suggest a bit more meditation on this. and if you still haven't
>changed your mind, then run experiments that include returning to code
>you wrote several months/years ago... :)

Writing code in such a way that you can come back to it after years and understand it is certainly a skill.
A skill I don't have much experience with. I will encounter this challenge soon enough, though.

>
>sometimes it happens that i accidentally implement something twice
>because of patch organization problems and/or not having an unlimited
>memory. to my delight, i more often than not name stuff exactly the
>same way in both implementations which are quite distant in time.
>

As for patching I don't have much experience either. I have never really collaborated with others when writing code.
I'm using bazaar tentatively to sync my stuff between desktop and laptop but that's about it.
(well, and since recently some kind of production server)

>it also means that when i happen to look for something, i'll grep for
>at least a part of the name i would give it. this strategy usually
>very cheaply gets me to what i'm looking for. even years later...
>

Global grep certainly benefits from long descriptive names but it can, as I see it,
lead to weak files/directories and systems/packages structures.

A grep of a sub-directory should do the trick even for short names.
(short names may also appear in conjunction with package names)
Ensuring that meaning is distributed equally on all levels (text/code, files/directories, systems/packages)
should make short names more practical. I have to admit that I enjoy working with packages a lot, even
as to go so far as to shadowing symbols so that I can reuse them as often as I see fit.

Fictional example:

(defpackage "CURL"
?? (:export "GET"))

(defpackage "FOO"
?? (:use "CURL")
?? (:shadow "GET"))

(in-package "FOO")

(defun get (arg)
? (do-additional-stuff (curl:get arg)))

I'm also fond of using :shadowing-import-from.
? 
For me calling the same thing (or almost the same) with the same words is just to relieving.

>all in all, having long and descriptive names is not a question for me
>anymore, especially with slime's fuzzy completion.
>

I'm afraid I have to learn this the hard way even if it is just for finding the right balance between short and long.

Writing down what some abbreviations stand for is also something I'm willing to do.
(readme file, documentation string, comments)
Also, in my opinion, the more local a name the shorter it can be.

Enough of my strange customs. :) Wish you a nice weekend.


( P.S. So now you have it, hopefully not a debate. Let's make sure this stays dialogue.
? (and just in case you have doubts, I'm still willing to adapt))

>
>> Just want to add that I'm grateful for your software, thank you.
>
>
>we're glad you find it useful!
>
>-- 
> attila
>
>













?


!DSPAM:4cd42ec048584894726250!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101105/bbb324cd/attachment.html>

From marshall.mcluhan at gmx.net  Fri Nov  5 19:11:26 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Fri, 5 Nov 2010 20:11:26 +0100
Subject: [cl-rdbms-devel] Convenience over extensibility? Or: How to build
	on top of hu.dwim.rdbms?
Message-ID: <vmime.4cd456de.1b67.7e3b26475d4dff51@gleanon.com>


To me, in the end, it is more like an art than a science. ( I guess this can be generally agreed upon )

I, like you, I suspect, see the need for complexity to be saved, like a precious resource,
so that it is not being used up before it is really needed.
For me this also includes starting with short names / using short names where possible.
(UIGNI - You ain't gonna need it)

But that doesn't mean that I'm not willing to use long names where I see fit. The opposite is more likely,
I see the possibility of using long names in contrast to short names as an asset.

It's just that I want to keep things as simple as possible for as long as possible,
so that it can grow gracefully. (KISS - Keep It Simple Stupid)

Nonetheless, I'm still willing to ignore my feelings in this regard in order to be able to work with other people and
their sense of order and complexity.

Although, I have to admit, it is kind of a neat idea (albeit not a necessity) to be able to follow my own conventions
in the packages/systems I'm responsible for except for the public interfaces, these would follow traditional conventions.

That, of course, would only work when starting from scratch.
When editing others people work the old style has to be preserved, that, even I'm able too see :)

Whatever ...

Regards,
chris


!DSPAM:4cd456de48582047418725!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101105/25a072eb/attachment.html>

From marshall.mcluhan at gmx.net  Mon Nov  8 12:16:53 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Mon, 8 Nov 2010 13:16:53 +0100
Subject: [cl-rdbms-devel] Convenience over extensibility? Or: How to build
	on top of hu.dwim.rdbms?
Message-ID: <vmime.4cd7ea36.1b67.781a55378d08de1@gleanon.com>


One more critic.

Constantly you have to come up with names that describe things that can't be (easily) described objectively
and therefore make hardly more sense to an outsider than short names.

I see unit tests as more import than long names (and shorter because of the use of short names)
even if they seem redundant at times. They make a great documenation.
A documentation that can test itself and describes code most accurately.
In my opinion, the letters you save with short names should go into unit tests.

That said, short names have their drawbacks, of course, and it's also a personal question. (who you came from, genetics etc)

?

?

Regards,

chris


!DSPAM:4cd7ea3648581477279777!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101108/ea41cc32/attachment.html>

From marshall.mcluhan at gmx.net  Mon Nov  8 12:56:59 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Mon, 8 Nov 2010 13:56:59 +0100
Subject: [cl-rdbms-devel] Convenience over extensibility? Or: How to build
	on top of hu.dwim.rdbms?
Message-ID: <vmime.4cd7f39b.1b67.5ef8db2723fbdd71@gleanon.com>


After all, you derived the long names from the "mechanics" of the code.
And now you expect an outsider to go the opposite direction, deriving the "mechanics" of the code from the long names?
I don't think this is going to work for an outsider. Long names may even confuse him because he has to
"see through" them in order to be able to understand the "mechanics".

Maybe you don't expect an outsider to understand the "mechanics" of the code by reading your long names and
merely want to aid that process of understanding. How well that is accomplished through the use of long names, is, of course,
still open for question. :)

Regards,
chris
?


!DSPAM:4cd7f39b48584559617140!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101108/16117517/attachment.html>

From attila.lendvai at gmail.com  Mon Nov  8 13:22:22 2010
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Mon, 8 Nov 2010 14:22:22 +0100
Subject: [cl-rdbms-devel] Convenience over extensibility? Or: How to
 build on top of hu.dwim.rdbms?
In-Reply-To: <vmime.4cd7f39b.1b67.5ef8db2723fbdd71@gleanon.com>
References: <vmime.4cd7f39b.1b67.5ef8db2723fbdd71@gleanon.com>
Message-ID: <AANLkTikJrQ-em2crUVcLmVx0a+5wuAB1Ay8+z9d_XCfm@mail.gmail.com>

> Maybe you don't expect an outsider to understand the "mechanics" of the code
> by reading your long names and
> merely want to aid that process of understanding. How well that is
> accomplished through the use of long names, is, of course,
> still open for question. :)


well, it certainly helps the process of understanding to have one less
indirection (e.g. not having to make the extra step to get from the
abbreviated short name to its original long form).

and don't forget that the frequency the name is "used" plays a very
important role regarding the decision about long/short names. you can
easily convince me about short names of frequently used constructs
(bind, let, ...), because even we prefer short names for them. but
these names are only a tiny portion of all the names introduced in the
codebase...

the extremum on this scale, namely writing code with abbreviated short
names of infrequently used constructs is basically what code
obfuscater algorithms do.

random thought re names: once a flat-text codebase has been parsed
into a graph, then all the human given names could be dropped because
they are redundant. the machine can execute the semantics encoded the
same way without identities having human parsable character strings.

conclusion: names should/could have much less important role regarding
identity management and restoring a graph from flat-text, and much
more important role in helping human understanding of such a graph.
e.g. having a long and a short names for the same identity; the
ability to have personalized names (e.g. let shown as a special
graphical element in my editor instead of using a three letter word);
etc...

i have a lot more thought on this (partly because we worked at
intentsoft.com), but it's a bit out of scope on this list... and
writing code has much more utility than talking about writing code...
:)

but reading your mails makes me think that you are already getting far
ahead of most people when it comes to giving good names to
abstractions. spending brain cycles to think about the problem is
already more than what most people do.

-- 
?attila



From marshall.mcluhan at gmx.net  Mon Nov  8 15:55:24 2010
From: marshall.mcluhan at gmx.net (=?windows-1252?Q?Marshall_McLuhan?=)
Date: Mon, 8 Nov 2010 16:55:24 +0100
Subject: [cl-rdbms-devel] Cvenience over extensibility? Or: How to build on
	top of hu.dwim.rdbms?
Message-ID: <vmime.4cd81d6c.1b67.2f2ea817201d7e01@gleanon.com>


>> Maybe you don't expect an outsider to understand the "mechanics" of the code
>> by reading your long names and
>> merely want to aid that process of understanding. How well that is
>> accomplished through the use of long names, is, of course,
>> still open for question. :)
>
>
>well, it certainly helps the process of understanding to have one less
>indirection (e.g. not having to make the extra step to get from the
>abbreviated short name to its original long form).
>

Agreed.


Nonetheless, in an (my) ideal world you would be able to derive the long form from the abbreviated form by:

??? ???? 1. knowing the outlines of your territory (files, dirs, packages, systems, context of all kinds)
??? ???? 2. looking at and hopefully understanding the "mechanics" of the code


Admittedly that isn't cheap, although it seems doable for an outsider. (eventually we are all outsiders)

All in the name of circumventing names that are hard to understand. (for outsiders)

The more I think about it I come to the conclusion that excessive use of long names are a way of "getting more intimate"
with your code. That helps you personally (or a small group) over the short-, midterm (maybe even long-term or
the rest of your life) but harms others.

In the end, it seems to me, it all comes down to not having any standards and a practically infinite number
of possible meanings. (costs of sharing meaning, transaction costs)

>and don't forget that the frequency the name is "used" plays a very
>important role regarding the decision about long/short names. you can
>easily convince me about short names of frequently used constructs
>(bind, let, ...), because even we prefer short names for them. but
>these names are only a tiny portion of all the names introduced in the
>codebase...
>

That makes sense to me.
Sounds like you are making a case for using long names as an asset by having the contrast between long and short.

And the costs of sharing meaning about a name that gets used frequently are easier to justify.

Again, opens the question as to how much long names are capable of reducing costs of sharing meaning.
Could they even increase the costs? I certainly made that case. Seems a little crazy from this perspective.

Surely the more words you use to describe something the easier it gets to share what it actually means?
Does purpose come into play now? Maybe I should stop here.

>
>the extremum on this scale, namely writing code with abbreviated short
>names of infrequently used constructs is basically what code
>obfuscater algorithms do.
>

Right, only context is missing in this picture.

Hm, the costs of sharing meaning often seem to be exceptionally low when it comes to context.


>
>random thought re names: once a flat-text codebase has been parsed
>into a graph, then all the human given names could be dropped because
>they are redundant. the machine can execute the semantics encoded the
>same way without identities having human parsable character strings.
>
>conclusion: names should/could have much less important role regarding
>identity management and restoring a graph from flat-text, and much
>more important role in helping human understanding of such a graph.
>e.g. having a long and a short names for the same identity; the
>ability to have personalized names (e.g. let shown as a special
>graphical element in my editor instead of using a three letter word);
>etc...
>

Absolutely

>
>i have a lot more thought on this (partly because we worked at
>intentsoft.com), but it's a bit out of scope on this list... and
>

Impressive, a testimony from the New York Times. Yes, this is awfully off topic.

As a side note: The relative low level interface of hu.dwim.rdbms I was talking about,
on top of which as many extensions as possible can be build, seems to be already there.
It's just that it isn't properly exposed, in my eyes at least.

>
>writing code has much more utility than talking about writing code...
>:)
>

Hard to tell. On average you are probably right.

>
>but reading your mails makes me think that you are already getting far
>ahead of most people when it comes to giving good names to
>abstractions. spending brain cycles to think about the problem is
>already more than what most people do.
>

I take this as an compliment. (normally I refuse compliments, tells me something)



Regards,
chris

>
>-- 
> attila
>
>

?


!DSPAM:4cd81d6e48581330461277!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cl-rdbms-devel/attachments/20101108/be6d6b18/attachment.html>

From attila.lendvai at gmail.com  Fri Nov 26 08:50:41 2010
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Fri, 26 Nov 2010 09:50:41 +0100
Subject: [cl-rdbms-devel]
	hu.dwim.rdbms.oracle-timestamp-with-timezone-local-time-bug
In-Reply-To: <vmime.4cd3227a.1b67.1dd98497338fd581@gleanon.com>
References: <vmime.4cd3227a.1b67.1dd98497338fd581@gleanon.com>
Message-ID: <AANLkTimtRNG4BG_7AGKaZu9owfD_6PFwLnHkOSqmXKSi@mail.gmail.com>

hi!

> How do I handle such thing most efficiently? Should I report them at all?

definitely, all bug reports are very welcome!

for the record, i've starred your mail, but i've failed to install
oracle in the 2 hours i'm willing to invest into oracle for now.

hopefully others could come up with suggestions, or oracle makes sure
that their monster can be installed on a 64bit ubuntu with reasonable
effort.

-- 
?attila