From lam at tuxfamily.org Thu Apr 19 20:16:07 2007 From: lam at tuxfamily.org (Nicolas Lamirault) Date: Thu, 19 Apr 2007 22:16:07 +0200 Subject: [cl-migrations-devel] first test ... Message-ID: <87d520ce7s.fsf@no-log.org> hello, i'm trying your cl-migrations tool. it seems very interesting ... i've some remarks : 1 - for cl-migrations' newbie (like me), it could be interesting to write in your example page (on cl.net) that we could change the *migration-dir* : CL-USER> (setf cl-migrations:*migration-dir* "/home/nicolas/tmp/") 2 - i try to creates the 1-.... file : CL-USER> (cl-migrations:generate "add-employees-table") junk in string "foo.lisp" [Condition of type SB-INT:SIMPLE-PARSE-ERROR] Restarts: 0: [ABORT] Return to SLIME's top level. 1: [ABORT] Exit debugger, returning to top level. Backtrace: 0: (PARSE-INTEGER "foo.lisp") 1: (CL-MIGRATIONS::GET-LATEST-MIGRATION) 2: (CL-MIGRATIONS:GENERATE "add-employees-table") 3: (SB-INT:SIMPLE-EVAL-IN-LEXENV (CL-MIGRATIONS:GENERATE "add-employees-table") #) the error comes from that i've got a /home/nicolas/tmp/foo.lisp files and from this method (parse-integer) : (defun get-migration-number (file) (parse-integer (subseq (file-namestring file) 0 (search "-" (file-namestring file))))) i don't really how correct this ? throw an error ? return nil ? ... 3- try to creates the database model : CL-USER> (cl-migrations:generate "add-employees-table") "1-add-employees-table.lisp" CL-USER> (cl-migrations:migrate) WARNING: Exiting - .migrate.conf not found in the home directory A database error occurred: NIL / NIL NIL is not a database. [Condition of type CLSQL-SYS:SQL-DATABASE-ERROR] Restarts: 0: [ABORT] Return to SLIME's top level. 1: [ABORT] Exit debugger, returning to top level. Backtrace: 0: (CLSQL-SYS::SIGNAL-NO-DATABASE-ERROR NIL) 1: (CLSQL-SYS::SIGNAL-NO-DATABASE-ERROR NIL) 2: (CLSQL-SYS:SELECT CL-MIGRATIONS::VERSION :FROM "schema_info" :FLATP T :FIELD-NAMES NIL) 3: (CL-MIGRATIONS::GET-DB-VERSION) 4: (CL-MIGRATIONS:MIGRATE :VERSION NIL) 5: (SB-INT:SIMPLE-EVAL-IN-LEXENV (CL-MIGRATIONS:MIGRATE) #) 6: (SWANK::EVAL-REGION "(cl-migrations:migrate) " T) i think i've got a problem with (read-specs) 4- creates the database model with default configuration CL-USER> (cl-migrations:generate "add-employees-table") Setting up migrations directory: "/home/nicolas/tmp/migrations/" The path #P"/home/nicolas/tmp/migrations/1-add-employees-table.lisp" does not exist. [Condition of type SB-INT:SIMPLE-FILE-ERROR] Restarts: 0: [ABORT] Return to SLIME's top level. 1: [ABORT] Exit debugger, returning to top level. i've got this on my .migrate.conf file : ((:mysql ("localhost" "test" "user" "user")) (:migration-dir ("/home/nicolas/tmp/migrations/"))) in the "generate" method, i think this could correct the problem : (ensure-directories-exist (directory-namestring #p"/home/nicolas/tmp/migrations/1-add-employees-table.lisp")) 5- Database model CL-USER> (ensure-directories-exist (directory-namestring #p"/home/nicolas/tmp/migrations/1-add-employees-table.lisp")) "/home/nicolas/tmp/migrations/" T CL-USER> (cl-migrations:generate "add-employees-table") "1-add-employees-table.lisp" CL-USER> (cl-migrations:migrate) Setting up migrations directory: "/home/nicolas/tmp/migrations/" ; loading system definition from ; /home/nicolas/.sbcl/systems/clsql-mysql.asd into # ; registering # as CLSQL-MYSQL ; loading system definition from ; /home/nicolas/.sbcl/systems/clsql-uffi.asd into # ; registering # as CLSQL-UFFI Schema table doesn't exist, creating. Database ready. Migration #1: EXEC: "create table employees (id serial, name varchar(100))" EXEC: "create table products (id serial, name varchar(100))" While accessing database # with expression "UPDATE SCHEMA_INFO SET VERSION = 1 WHERE (VERSION = 0)": Error 1146 / Table 'test.SCHEMA_INFO' doesn't exist has occurred. [Condition of type CLSQL-SYS:SQL-DATABASE-DATA-ERROR] Restarts: 0: [ABORT] Return to SLIME's top level. 1: [ABORT] Exit debugger, returning to top level. Backtrace: 0: ((SB-PCL::FAST-METHOD CLSQL-SYS:DATABASE-EXECUTE-COMMAND (T CLSQL-MYSQL:MYSQL-DATABASE)) # # "UPDATE SCHEMA_INFO SET VERSION = 1 WHERE (VERSION = 0)" #) 1: ((LAMBDA (SB-PCL::.PV-CELL. SB-PCL::.NEXT-METHOD-CALL. SB-PCL::.ARG0. SB-PCL::.ARG1.)) # # "UPDATE SCHEMA_INFO SET VERSION = 1 WHERE (VERSION = 0)" #) 2: ((SB-PCL::FAST-METHOD CLSQL-SYS:EXECUTE-COMMAND (STRING)) # # "UPDATE SCHEMA_INFO SET VERSION = 1 WHERE (VERSION = 0)" :DATABASE #) 3: ((SB-PCL::FAST-METHOD CLSQL-SYS:EXECUTE-COMMAND (CLSQL-SYS::%SQL-EXPRESSION)) # # # :DATABASE #) in the database : mysql> show tables ; +----------------+ | Tables_in_test | +----------------+ | employees | | products | | schema_info | +----------------+ 3 rows in set (0.00 sec) mysql> desc products ; +-------+---------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------+---------------------+------+-----+---------+----------------+ | id | bigint(20) unsigned | NO | PRI | NULL | auto_increment | | name | varchar(100) | YES | | NULL | | +-------+---------------------+------+-----+---------+----------------+ 2 rows in set (0.00 sec) mysql> desc employees ; +-------+---------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------+---------------------+------+-----+---------+----------------+ | id | bigint(20) unsigned | NO | PRI | NULL | auto_increment | | name | varchar(100) | YES | | NULL | | +-------+---------------------+------+-----+---------+----------------+ 2 rows in set (0.00 sec) mysql> UPDATE SCHEMA_INFO SET VERSION = 1 WHERE (VERSION = 0); ERROR 1146 (42S02): Table 'test.SCHEMA_INFO' doesn't exist mysql> UPDATE schema_info SET VERSION = 1 WHERE (VERSION = 0); Query OK, 1 row affected (0.01 sec) Rows matched: 1 Changed: 1 Warnings: 0 with mysql the problem comes from the table name. i've got a question about this tool and the Ror equivalent : what about def-view-class ? in ror, you creates the SQL model and you generate objects wrapper ? in some project, i use def-view-class and i generate the SQL tables from theses objects ... i thinks it's a great tool ... i'll use it in some of my projects -- Nicolas Lamirault From vamlists at gmail.com Fri Apr 20 09:46:53 2007 From: vamlists at gmail.com (Vamsee Kanakala) Date: Fri, 20 Apr 2007 15:16:53 +0530 Subject: [cl-migrations-devel] first test ... In-Reply-To: <87d520ce7s.fsf@no-log.org> References: <87d520ce7s.fsf@no-log.org> Message-ID: <46288C0D.6020401@gmail.com> Nicolas Lamirault wrote: > hello, > i'm trying your cl-migrations tool. > it seems very interesting ... > i've some remarks : Thank you. I'll try to address them one by one: > > 1 - for cl-migrations' newbie (like me), it could be interesting > to write in your example page (on cl.net) that we could change the > *migration-dir* : > > CL-USER> (setf cl-migrations:*migration-dir* "/home/nicolas/tmp/") > I am not sure if this is a good idea - I exported that var so that people can print it out for checking - it gets set along with the database config, so the appropriate way to set this would be by creating the config file properly and then running the generate/migrate functions. I am thinking of not exporting this variable anymore, but I will see if there's a way decouple database spec from directory setting and handle this properly. > 2 - i try to creates the 1-.... file : > > CL-USER> (cl-migrations:generate "add-employees-table") > junk in string "foo.lisp" > [Condition of type SB-INT:SIMPLE-PARSE-ERROR] > Looks like you have a regular file in the directory - it's always good to create a separate directory just for migrations. This happens because cl-migrations reads all the files in the directory and sorts them according to their migration numbers in the filenames. Anyways, I will try to fix this so that regular files (without numbers) get ignored. > > 3- try to creates the database model : > > CL-USER> (cl-migrations:generate "add-employees-table") > "1-add-employees-table.lisp" > > CL-USER> (cl-migrations:migrate) > WARNING: Exiting - .migrate.conf not found in the home directory > A database error occurred: NIL / NIL > NIL is not a database. > [Condition of type CLSQL-SYS:SQL-DATABASE-ERROR] > Most probably because you set the *migration-dir* directly and didn't create the .migrate.conf. Creating this file should fix the problem. > > > 4- creates the database model with default configuration > > CL-USER> (cl-migrations:generate "add-employees-table") > > Setting up migrations directory: "/home/nicolas/tmp/migrations/" > > The path > #P"/home/nicolas/tmp/migrations/1-add-employees-table.lisp" > does not exist. > [Condition of type SB-INT:SIMPLE-FILE-ERROR] > > > in the "generate" method, i think this could correct the problem : > > (ensure-directories-exist > (directory-namestring #p"/home/nicolas/tmp/migrations/1-add-employees-table.lisp")) Thanks, will patch this into the next release. > > > 5- Database model > > > mysql> UPDATE SCHEMA_INFO SET VERSION = 1 WHERE (VERSION = 0); > ERROR 1146 (42S02): Table 'test.SCHEMA_INFO' doesn't exist > > > mysql> UPDATE schema_info SET VERSION = 1 WHERE (VERSION = 0); > Query OK, 1 row affected (0.01 sec) > Rows matched: 1 Changed: 1 Warnings: 0 > > > with mysql the problem comes from the table name. > Wow, I'm very surprised about this. Sorry, I mostly tested in PostgreSQL, so I missed this one. Isn't SQL supposed to be case-insensitive? Will fix this anyways. > > i've got a question about this tool and the Ror equivalent : what > about def-view-class ? > in ror, you creates the SQL model and you generate objects wrapper ? > in some project, i use def-view-class and i generate the SQL tables > from theses objects ... > Hmm. Actually it's the other way round. There's a part of RoR called ActiveRecord that looks at the table (created by migrations or by hand with sql) and then automagically sets up the object and it's attributes (field names as slots). So, there's no need for something like def-view-class of CLSQL. Perhaps we need something like ActiveRecord in CL. But I think this behavior can be easily retrofitted to CLSQL or Postmodern. > i thinks it's a great tool ... > i'll use it in some of my projects > > Thanks a lot for the inputs. I will be a lot more careful about the bugs now. Little busy now, will push out a new version over the weekend. Best regards, Vamsee. From lam at tuxfamily.org Fri Apr 20 11:06:03 2007 From: lam at tuxfamily.org (Nicolas Lamirault) Date: Fri, 20 Apr 2007 13:06:03 +0200 Subject: [cl-migrations-devel] first test ... In-Reply-To: <46288C0D.6020401@gmail.com> (Vamsee Kanakala's message of "Fri\, 20 Apr 2007 15\:16\:53 +0530") References: <87d520ce7s.fsf@no-log.org> <46288C0D.6020401@gmail.com> Message-ID: <87fy6vs3tw.fsf@no-log.org> hello, Vamsee Kanakala writes: > Nicolas Lamirault wrote: >> hello, >> i'm trying your cl-migrations tool. >> it seems very interesting ... >> i've some remarks : > > Thank you. I'll try to address them one by one: > >> >> 1 - for cl-migrations' newbie (like me), it could be interesting >> to write in your example page (on cl.net) that we could change the >> *migration-dir* : >> >> CL-USER> (setf cl-migrations:*migration-dir* "/home/nicolas/tmp/") >> > > I am not sure if this is a good idea - I exported that var so that > people can print it out for checking - it gets set along with the > database config, so the appropriate way to set this would be by creating > the config file properly and then running the generate/migrate > functions. I am thinking of not exporting this variable anymore, but I > will see if there's a way decouple database spec from directory setting > and handle this properly. > > >> 2 - i try to creates the 1-.... file : >> >> CL-USER> (cl-migrations:generate "add-employees-table") >> junk in string "foo.lisp" >> [Condition of type SB-INT:SIMPLE-PARSE-ERROR] >> > > Looks like you have a regular file in the directory - it's always good > to create a separate directory just for migrations. This happens because > cl-migrations reads all the files in the directory and sorts them > according to their migration numbers in the filenames. Anyways, I will > try to fix this so that regular files (without numbers) get ignored. > >> >> 3- try to creates the database model : >> >> CL-USER> (cl-migrations:generate "add-employees-table") >> "1-add-employees-table.lisp" >> >> CL-USER> (cl-migrations:migrate) >> WARNING: Exiting - .migrate.conf not found in the home directory >> A database error occurred: NIL / NIL >> NIL is not a database. >> [Condition of type CLSQL-SYS:SQL-DATABASE-ERROR] >> > > Most probably because you set the *migration-dir* directly and didn't > create the .migrate.conf. Creating this file should fix the problem. > >> >> >> 4- creates the database model with default configuration >> >> CL-USER> (cl-migrations:generate "add-employees-table") >> >> Setting up migrations directory: "/home/nicolas/tmp/migrations/" >> >> The path >> #P"/home/nicolas/tmp/migrations/1-add-employees-table.lisp" >> does not exist. >> [Condition of type SB-INT:SIMPLE-FILE-ERROR] >> >> >> in the "generate" method, i think this could correct the problem : >> >> (ensure-directories-exist >> (directory-namestring #p"/home/nicolas/tmp/migrations/1-add-employees-table.lisp")) > > Thanks, will patch this into the next release. > > >> >> >> 5- Database model >> >> >> mysql> UPDATE SCHEMA_INFO SET VERSION = 1 WHERE (VERSION = 0); >> ERROR 1146 (42S02): Table 'test.SCHEMA_INFO' doesn't exist >> >> >> mysql> UPDATE schema_info SET VERSION = 1 WHERE (VERSION = 0); >> Query OK, 1 row affected (0.01 sec) >> Rows matched: 1 Changed: 1 Warnings: 0 >> >> >> with mysql the problem comes from the table name. >> > > Wow, I'm very surprised about this. Sorry, I mostly tested in > PostgreSQL, so I missed this one. Isn't SQL supposed to be > case-insensitive? Will fix this anyways. > >> >> i've got a question about this tool and the Ror equivalent : what >> about def-view-class ? >> in ror, you creates the SQL model and you generate objects wrapper ? >> in some project, i use def-view-class and i generate the SQL tables >> from theses objects ... >> > > Hmm. Actually it's the other way round. There's a part of RoR called > ActiveRecord that looks at the table (created by migrations or by hand > with sql) and then automagically sets up the object and it's attributes > (field names as slots). So, there's no need for something like > def-view-class of CLSQL. Perhaps we need something like ActiveRecord in > CL. But I think this behavior can be easily retrofitted to CLSQL or > Postmodern. > >> i thinks it's a great tool ... >> i'll use it in some of my projects >> >> > > Thanks a lot for the inputs. I will be a lot more careful about the bugs > now. Little busy now, will push out a new version over the weekend. great ! thanks for your reply ... > Best regards, > Vamsee. > _______________________________________________ > cl-migrations-devel mailing list > cl-migrations-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/cl-migrations-devel > -- Nicolas Lamirault From sasha at arcocene.org Sat Apr 21 02:48:14 2007 From: sasha at arcocene.org (Sasha Kovar) Date: Fri, 20 Apr 2007 19:48:14 -0700 Subject: [cl-migrations-devel] Sort with n > 9 Message-ID: <20070421024814.GL84817@arcocene.org> Hi. Attached is a patch to compare-files which fixes a problem with the ordering of migration files. On my system (linux, SBCL 1.0.0) when I added my tenth migration, it sorted "10-" after "1-" and before "2-". I suspect you wanted numeric rather than lexicographic order :) Thanks very much for cl-migrations. That plus clsql plus clsql-pg-introspect equals one happy developer! Sasha From sasha at arcocene.org Sat Apr 21 02:48:52 2007 From: sasha at arcocene.org (Sasha Kovar) Date: Fri, 20 Apr 2007 19:48:52 -0700 Subject: [cl-migrations-devel] Opps - patch Message-ID: <20070421024851.GM84817@arcocene.org> I knew I was going to do that... -------------- next part -------------- A non-text attachment was scrubbed... Name: compare-files.patch Type: text/x-diff Size: 854 bytes Desc: not available URL: From vamlists at gmail.com Sun Apr 22 18:52:41 2007 From: vamlists at gmail.com (Vamsee Kanakala) Date: Mon, 23 Apr 2007 00:22:41 +0530 Subject: [cl-migrations-devel] Sort with n > 9 In-Reply-To: <20070421024814.GL84817@arcocene.org> References: <20070421024814.GL84817@arcocene.org> Message-ID: <462BAEF9.3040200@gmail.com> Sasha Kovar wrote: > Attached is a patch to compare-files which fixes a problem with the ordering > of migration files. Thanks so much for the patch. Fixed in 0.0.3. asdf-install should get the latest now. > Thanks very much for cl-migrations. That plus clsql plus > clsql-pg-introspect equals one happy developer! > Thanks for the feedback. I'm glad it's useful :) Regards, Vamsee. From vamlists at gmail.com Sun Apr 22 19:04:54 2007 From: vamlists at gmail.com (Vamsee Kanakala) Date: Mon, 23 Apr 2007 00:34:54 +0530 Subject: [cl-migrations-devel] first test ... In-Reply-To: <87d520ce7s.fsf@no-log.org> References: <87d520ce7s.fsf@no-log.org> Message-ID: <462BB1D6.7080802@gmail.com> Nicolas Lamirault wrote: > > 5- Database model > [snip] > > Migration #1: > EXEC: "create table employees (id serial, name varchar(100))" > EXEC: "create table products (id serial, name varchar(100))" > While accessing database # {AA87289}> > with expression "UPDATE SCHEMA_INFO SET VERSION = 1 WHERE (VERSION = > 0)": > Error 1146 / Table 'test.SCHEMA_INFO' doesn't exist > has occurred. > [Condition of type CLSQL-SYS:SQL-DATABASE-DATA-ERROR] > It looks like a known problem. According to the clsql guys, it's better to solve this problem at the server side: http://lists.b9.com/pipermail/clsql-help/2004-December/000377.html The relevant mysql link: http://dev.mysql.com/doc/refman/5.0/en/identifier-case-sensitivity.html Hope this solves your problem. I have fixed the other problems you have reported - however, I am no longer exporting the *migration-dir*, to make sure the database connection is configured properly before the migrations are run. Decoupling *migration-dir* and database config is a little tricky now, will perhaps solve this in the next version. Thanks so much for the feedback. Regards, Vamsee