Quantcast

Organizational Meeting Notes

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Organizational Meeting Notes

Joshua Ferraro-3
Hi all,

Here are the meeting notes from today's organizational meeting:

Present:

paul
hdl
owen
shedges
chris
kados
slef
thd
sanspach
russ
si (briefly) :-)

MEETING NOTES FOR JANUARY 30, 2006

PERL-ZOOM
        Paul still unable to get extended services and CQL working
        Paul will summarize difficulties encountered by posting to
                koha-devel and koha-zebra

PERL MODULES
        Chris will remove reliance on Date::Manip for proc-
        sensitive modules (like circ)

        Chris introduced us to Smart::Comments, a nice dev tool
        We agreed to look into using MARC::File::XML

MOD_PERL
        Chris will be cleaning up Koha code to allow us to use
                mod_perl and mod_perl2 of these

SERIALS
        Katipo is working with several clients to enhance serials
        they will build on existing serials code
        paul announced working serials routing in HEAD

DATABASE
        Paul discussed his database constraints introduced in HEAD
        requires Mysql >= 4.1

        we agreed to require innodb for 3.0

TEMPLATES
        Owen will continue working on prog
        Katipo will work on a new design for official templates

INSTALLER
        slef will commit his new MakeMaker-based installer soon
        discussed moving C4/ modules to a Koha:: namespace and
        uploading to CPAN

OTHER
        thd will continue looking at encoding issues for Koha 3.0
                as well as record length issues

If I left anything out, let me know.

Cheers,

--
Joshua Ferraro               VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE
President, Technology       migration, training, maintenance, support
LibLime                                Featuring Koha Open-Source ILS
[hidden email] |Full Demos at http://liblime.com/koha |1(888)KohaILS


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/koha-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

SQL Dtabase Flavour Alternate Routines and Identifier Quoting Proposal

Thomas D-2
SQL DATABASE IMPLEMENTATION COMAPIBILITY

On repeated occasions, I have expressed a concern about the hazards of tying
Koha code too much to a particular database implementation flavour.  No one
has ever seemed to oppose the idea of database neutrality where it can be
made to work.  I am not presuming that there is enough Koha development time
available in the 3.0 schedule to write and test code for multiple SQL
database implementations.  Furthermore, the value in providing code for
multiple SQL implementations in the near term may be minor relative to the
gain from using that same time for other work.

However, given that a significant amount of new code is being written and
that a significant amount of old code is being revised for Koha 3.0, I
propose that a coding guideline be adopted to minimise the work necessary
for wherever SQL code is being changed or newly introduced for 3.X.

The value here is not only for allowing provision for the functional
advantage of using other database implementation types and to promote
thinking without using MySQL blinders.  There is a larger value in marketing
to help obtain wider adoption of Koha.  Many people have well justified
prejudices against MySQL which will persist despite present and future
improvements in MySQL and despite using constraints or other features now in
MySQL.  Software adoption is seldom decided on a proper objective evaluation
of all the merits.  Decisions are informed by prejudices that need to be
addressed.  Where Koha is non-standards compliant or otherwise deficient, at
a minimum, provision for more easily enabling the future correction of those
problems ought to exist in 3.0.

There is already some incomplete code in C4::Context.pm to support the use
of alternate SQL database implementation types.  However, actual coding
practise has made this moot.  I propose that this deficiency be remedied
before the task of escaping a single implementation choice becomes too daunting.


NEUTRAL SYNTAX AND IDENTIFIER QUOTING

Sufficiently neutral syntax ought to also be adopted for SQL code to protect
against present and future incompatibilities and allow reuse between various
versions.  Quotation of identifiers is already needed to avoid
incompatibility of one Koha column name with reserved words in MySQL 5.
Maximising database neutrality in quoting is best preserved by using ANSI
quotes, however, using them in MySQL poses problems.  One cannot rely on
universal settings in my.conf where Koha may not be the only database in use
on the server.  SQL statements for MySQL would need to be prefaced by the
ANSI quotes setting statement to preserve code reuse but that would add
otherwise unnecessary ANSI quotes setting statements which would be
unrecognised in other database systems.

SET sql_mode='ANSI_QUOTES';

There had been a report of the identifier problem on the koha list and then
again on the koha-devel list.  See the problem reported at
http://lists.gnu.org/archive/html/koha-devel/2005-12/msg00001.html and the
solution that I suggested at
http://lists.gnu.org/archive/html/koha-devel/2005-12/msg00003.html .

A means of preserving SQL implementation neutrality and avoiding the special
problems with ANSI quoting is needed.


ALTERNATE SQL DATABASE IMPLEMENTATION SPECIFIC ROUTINES

Different routines for different SQL database implementations are needed to
provide for all variants without the peculiarities of one interfering with
another.  Even where a single ANSI format might be used, different routines
will avoid the special problem for using ANSI quotes with MySQL.  MySQL
backticks can be used instead of ANSI quotes in a special routine for MySQL.

A parameter is needed for setting the SQL database implementation type to
test against for determining which implementation routine to use.  Given the
legacy for Koha in MySQL, that ought to be the default, despite the logic of
defaulting to the ANSI standard otherwise.

Only one PostgresSQL routine would be needed if the Postgres 8 statements
could be successfully written without referencing the unavoidable schemas.
I never succeeded at that when I was rushing to have running code while
Postgres 8 was in beta.

C4::Context.pm has some rough code for reading  a db_scheme value if set in
koha.conf, however, values returned by db_scheme2dbi() for use in DBI would
be insufficient for distinguishing between different versions of the same
SQL database type, such as the distinction between Postgres 7 and 8.  I
propose creating a virtual config variable instead before db_scheme2dbi() is
called with its effect of flattening the value of db_scheme.


ONLY A PLACEHOLDER

Of course, as I have indicated, there is insufficient time for implementing
the actual different SQL code for using separate database implementation
types.  This guideline is for a placeholder to reduce the work required for
implementing SQL code apart form merely MySQL.


PROPOSED CODE FOR GUIDELINE

# Modification for Context.pm
        # Quick hack for allowing databases name in full text
        # from authorised value list
        if ( $1 eq "db_scheme" ) {
            $retval{sqlDBType} = $2; # add virtual config variable
            $value = db_scheme2dbi($2);
        } else {
            $value = $2;
        }
        $retval->{$1} = $value;



# At the top of any C4 modules or $KOHA/admin/
# scripts requiring SQL database access:
# Obtain a value for the SQL database implementation type (flavour)
$my $sqlDBType = C4::Context->config("sqlDBType");
my $nonMySQLDBType = $sqlDBType;
if ($nonMySQLDBType =~ /mysql/i) {$nonMySQLDBType = undef;}


# Alternate routines or routine placeholders for various SQL database types
if ($nonMySQLDBType) {

    if ($nonMySQLDBType eq 'postgres7') {
        # Postgres 7 specific code
    } elsif ($nonMySQLDBType eq 'postgres8') {
        # Postgres 8 specific code
    } elsif ($nonMySQLDBType eq 'oracle') {
        # Oracle specific code
    } else {
        # ANSI code
    }

} else {
    # MySQL specific code
}


CHANGE TO CODING STANDARDS AND GUIDELINES

If there is general agreement, I propose that the proper quoting of
identifiers and a preferred standard form for alternate SQL database
implementation types such as I have described should be included on
kohadocs.org along with the rational.  Koha Development Team. Koha coding
standards and guidelines for contributors.
http://www.kohadocs.org/codingguidelines.html .


Thomas D

---------------------------------------------
Alinto wishes you a happy new year 2006 http://www.alinto.com


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/koha-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQL Dtabase Flavour Alternate Routines and Identifier Quoting Proposal

Thomas D-2
Oops, I left out an enclosing if clause for one part of the code at the end
of this section.  This is one thing need to prevent failure in the present
usual case where db_scheme may not be included in the koha.conf.

# At the top of any C4 modules or $KOHA/admin/
# scripts requiring SQL database access:
# Obtain a value for the SQL database implementation type (flavour)
my $sqlDBType = C4::Context->config("sqlDBType");
my $nonMySQLDBType = $sqlDBType;
if ($nonMySQLDBType) {
    if ($nonMySQLDBType =~ /mysql/i) {$nonMySQLDBType = undef;}
}


Thomas D


Quoting Thomas D <[hidden email]> :
> ---------------- Beginning of the original message ------------------
>

[snip]

> PROPOSED CODE FOR GUIDELINE
>
> # Modification for Context.pm
>         # Quick hack for allowing databases name in full text
>         # from authorised value list
>         if ( $1 eq "db_scheme" ) {
>             $retval{sqlDBType} = $2; # add virtual config
> variable
>             $value = db_scheme2dbi($2);
>         } else {
>             $value = $2;
>         }
>         $retval->{$1} = $value;
>
>
>
> # At the top of any C4 modules or $KOHA/admin/
> # scripts requiring SQL database access:
> # Obtain a value for the SQL database implementation type
> (flavour)
> $my $sqlDBType = C4::Context->config("sqlDBType");
> my $nonMySQLDBType = $sqlDBType;
> if ($nonMySQLDBType =~ /mysql/i) {$nonMySQLDBType = undef;}
>
>
> # Alternate routines or routine placeholders for various SQL
> database types
> if ($nonMySQLDBType) {
>
>     if ($nonMySQLDBType eq 'postgres7') {
>         # Postgres 7 specific code
>     } elsif ($nonMySQLDBType eq 'postgres8') {
>         # Postgres 8 specific code
>     } elsif ($nonMySQLDBType eq 'oracle') {
>         # Oracle specific code
>     } else {
>         # ANSI code
>     }
>
> } else {
>     # MySQL specific code
> }
>
>

[snip]

---------------------------------------------
Alinto wishes you a happy new year 2006 http://www.alinto.com


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/koha-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQL Dtabase Flavour Alternate Routines and Identifier Quoting Proposal

Nathan Gray-3
In reply to this post by Thomas D-2
On Sat, Feb 04, 2006 at 03:34:25AM +0100, Thomas D wrote:
> # At the top of any C4 modules or $KOHA/admin/
> # scripts requiring SQL database access:
> # Obtain a value for the SQL database implementation type (flavour)
> $my $sqlDBType = C4::Context->config("sqlDBType");
> my $nonMySQLDBType = $sqlDBType;
> if ($nonMySQLDBType =~ /mysql/i) {$nonMySQLDBType = undef;}

The DBI has a place for this:

  $dbh->{Driver}{Name};  # MySQL, SQLite, Pg, Oracle

There might be a place for database version (PostgreSQL 7 vs 8), though
I have never needed it.

I am very interested in having Koha able to interact with databases
other than MySQL.  I cannot offer any time right now, but this
shortcoming has been nagging me for years.

A start like this, to at least keep new code from getting worse, and
maybe helping old code get better, is a nice incentive for me to start
thinking about it again.

-kolibrie



_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/koha-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SQL Dtabase Flavour Alternate Routines and Identifier Quoting Proposal

Simon Mitchell
In reply to this post by Thomas D-2
FYI

Hello, last year I was  playing  around with postgresql and have put some patches up on
http://www.saas.nsw.edu.au/koha_wiki/index.php?page=PostgreSQL
, but had not done any real testing.

Am looking forward to the release 3 , with all that sql moved to zebra, and might have another play.

I have found

1. MySQL searches and db names are not case sensitive.

2. that some  sql can be changed or rearranged to suite postgresql and
still work with mysql. So just changing to a more of  ANSI sql would be a good start.

3. I needed to change Context db connect string and added "db_scheme=Postgres" to /etc/koha.conf

--- koha-2.2.4/modules/C4/Context.pm 
+++ postgresql/modules/C4/Context.pm
@@ -408,7 +408,7 @@
my $db_host = $context->{"config"}{"hostname"};
my $db_user = $context->{"config"}{"user"};
my $db_passwd = $context->{"config"}{"pass"};
- return DBI->connect("DBI:$db_driver:$db_name:$db_host",
+ return DBI->connect("DBI:$db_driver:dbname=$db_name;host=$db_host",
$db_user, $db_passwd);
}


I recommend using postgresql 8.1
1.  new postgresql 8.0 and above is available for windows
2. 8.0 has a some sort problem with character set.

Regards,
Simon


On 04/02/06, Thomas D <[hidden email]> wrote:
Oops, I left out an enclosing if clause for one part of the code at the end
of this section.  This is one thing need to prevent failure in the present
usual case where db_scheme may not be included in the koha.conf.

# At the top of any C4 modules or $KOHA/admin/
# scripts requiring SQL database access:
# Obtain a value for the SQL database implementation type (flavour)
my $sqlDBType = C4::Context->config("sqlDBType");
my $nonMySQLDBType = $sqlDBType;
if ($nonMySQLDBType) {
    if ($nonMySQLDBType =~ /mysql/i) {$nonMySQLDBType = undef;}
}


Thomas D


Quoting Thomas D <[hidden email]> :
> ---------------- Beginning of the original message ------------------
>

[snip]

> PROPOSED CODE FOR GUIDELINE

>
> # Modification for Context.pm
>         # Quick hack for allowing databases name in full text
>         # from authorised value list
>         if ( $1 eq "db_scheme" ) {
>             $retval{sqlDBType} = $2; # add virtual config
> variable
>             $value = db_scheme2dbi($2);
>         } else {
>             $value = $2;
>         }
>         $retval->{$1} = $value;
>
>
>
> # At the top of any C4 modules or $KOHA/admin/
> # scripts requiring SQL database access:
> # Obtain a value for the SQL database implementation type
> (flavour)
> $my $sqlDBType = C4::Context->config("sqlDBType");
> my $nonMySQLDBType = $sqlDBType;
> if ($nonMySQLDBType =~ /mysql/i) {$nonMySQLDBType = undef;}
>
>
> # Alternate routines or routine placeholders for various SQL
> database types
> if ($nonMySQLDBType) {
>
>     if ($nonMySQLDBType eq 'postgres7') {
>         # Postgres 7 specific code
>     } elsif ($nonMySQLDBType eq 'postgres8') {
>         # Postgres 8 specific code
>     } elsif ($nonMySQLDBType eq 'oracle') {
>         # Oracle specific code
>     } else {
>         # ANSI code
>     }
>
> } else {
>     # MySQL specific code
> }
>
>

[snip]

---------------------------------------------
Alinto wishes you a happy new year 2006 http://www.alinto.com


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/koha-devel


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/koha-devel
Loading...