Merge lp:~jbaker/storm/oracle_support into lp:storm

Proposed by Jason Baker
Status: Needs review
Proposed branch: lp:~jbaker/storm/oracle_support
Merge into: lp:storm
Diff against target: 1573 lines (+1287/-39)
9 files modified
README (+38/-2)
oracle_test_setup.sql (+13/-0)
storm/databases/oracle.py (+590/-0)
storm/expr.py (+1/-1)
storm/variables.py (+2/-0)
tests/databases/base.py (+32/-31)
tests/databases/oracle.py (+372/-0)
tests/store/base.py (+5/-5)
tests/store/oracle.py (+234/-0)
To merge this branch: bzr merge lp:~jbaker/storm/oracle_support
Reviewer Review Type Date Requested Status
Jamu Kakar (community) Abstain
Storm Developers Pending
Review via email: mp+15442@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Jason Baker (jbaker) wrote :

This should be ready for merge. I've commented out the disconnection tests for now. Some parts of this are still somewhat ugly, but it's a good starting point.

Revision history for this message
Jamu Kakar (jkakar) wrote :

[1]

I've managed to get Oracle XE installed, but I'm having trouble
getting cx_Oracle in place, so I can't run the test suite yet. I'm
going to keep at it, but if you have any hints on how to get
cx_Oracle in place (on a 64-bit machine) I'd appreciate some help.
Ideally, packages for Ubuntu would be nice.

[2]

I've pushed a branch that adds some details about getting Oracle XE
installed on a 64-bit machine to the README file and makes some
cosmetic changes to make the code match the Storm coding style more
closely. Please merge:

lp:~jkakar/storm/oracle-support-tweaks

[3]

Why isn't the builtin reserved words functionality being used?

I've made a broad pass over the changeset and it looks really good.
I'd like to get the test suite running before I do a more detailed
pass.

Revision history for this message
Jason Baker (jbaker) wrote :

> [1]
>
> I've managed to get Oracle XE installed, but I'm having trouble
> getting cx_Oracle in place, so I can't run the test suite yet. I'm
> going to keep at it, but if you have any hints on how to get
> cx_Oracle in place (on a 64-bit machine) I'd appreciate some help.
> Ideally, packages for Ubuntu would be nice.

I found this blog post to be a good help: http://catherinedevlin.blogspot.com/2007/10/cxoracle-and-oracle-xe-on-ubuntu.html ... Although note that they're a little bit outdated. I don't believe it's necessary to install the XE patch.

If you can give me a specific error message, I might be able to help out.

> [2]
>
> I've pushed a branch that adds some details about getting Oracle XE
> installed on a 64-bit machine to the README file and makes some
> cosmetic changes to make the code match the Storm coding style more
> closely. Please merge:
>
> lp:~jkakar/storm/oracle-support-tweaks

I'll look at them. Thanks!

> [3]
>
> Why isn't the builtin reserved words functionality being used?

I don't recall the *exact* reason why I did that, but there was some issue with the reserved word keylist considering words to be reserved that weren't. Perhaps a better solution would be to update the reserved word list. It's a bit dated (SQL 1992).

Revision history for this message
Jason Baker (jbaker) wrote :

> lp:~jkakar/storm/oracle-support-tweaks

Erm.... This branch doesn't seem to exist. Unless of course I'm doing something wrong (which is possible).

Revision history for this message
Jamu Kakar (jkakar) wrote :

Oops, the branch was private by mistake. It should be visible now.

lp:~jbaker/storm/oracle_support updated
357. By Jason Baker

Merging in changes from trunk.

358. By Jason Baker

Merging some changes in from lp:~jkakar/storm/oracle-support-tweaks

359. By Jason Baker

Updating from trunk.

360. By Jason Baker

Making the oracle backend properly handle disconnection errors

Revision history for this message
Jason Baker (jbaker) wrote :

Where do we stand on this? I've just merged in the latest revisions from trunk and also added a bugfix that makes the disconnection tests now pass.

Revision history for this message
Jamu Kakar (jkakar) wrote :

I had spent some time looking at this a while ago. I got as far as
getting Oracle installed (which was hard on 64-bit) and then stopped
when I had to figure out how to compile cxOracle DB-API drivers. I
looked over the code, and it looks good, but want to actually be
able to run the test suite.

I've since installed a new disk in my machine, so I'll need to setup
Oracle et al. again to continue. I've been thinking that we really
need a buildbot, since getting Oracle in place is rather awkward and
most Storm developers probably won't want to do it.

Revision history for this message
Jamu Kakar (jkakar) wrote :

I haven't managed to get Oracle installed. I'm going to abstain
from reviewing this for now.

review: Abstain

Unmerged revisions

360. By Jason Baker

Making the oracle backend properly handle disconnection errors

359. By Jason Baker

Updating from trunk.

358. By Jason Baker

Merging some changes in from lp:~jkakar/storm/oracle-support-tweaks

357. By Jason Baker

Merging in changes from trunk.

356. By Jason Baker <jason@ubuntu>

Added instructions for getting started with Oracle.

355. By Jason Baker <jason@ubuntu>

Adding in a script to set up a test user in an oracle database.

354. By Jason Baker <jason@ubuntu>

Reverting the error handling back to the way it was for now. For whatever reason, this led to spurious error reports when running tests.

353. By Jason Baker <jason@ubuntu>

Temporarily commenting out the Disconnection tests until they can be fixed.

352. By Jason Baker <jason@ubuntu>

Adding an error description for cx_Oracle not being found

351. By Jason Baker <jason@ubuntu>

Merge from trunk.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'README'
2--- README 2009-10-08 07:23:43 +0000
3+++ README 2010-03-15 15:48:27 +0000
4@@ -95,8 +95,8 @@
5 same procedure will probably work without changes on a Debian system
6 and with minimal changes on a non-Debian-based linux distribution.
7 In order to run the test suite, and exercise all supported backends,
8-you will need to install MySQL and PostgreSQL, along with the
9-related Python database drivers:
10+you will need to install MySQL, PostgreSQL and Oracle XE, along with
11+the related Python database drivers:
12
13 $ sudo apt-get install python-mysqldb python-psycopg2 mysql-server \
14 postgresql build-essential
15@@ -110,6 +110,34 @@
16 asked to enter a password multiple times. Leave it blank in each
17 case.
18
19+To install Oracle on a 32-bit system you must first add the
20+following line to your /etc/apt/sources.list:
21+
22+ deb http://oss.oracle.com/debian unstable main non-free
23+
24+You can then install Oracle XE by running the following commands:
25+
26+ $ sudo wget http://oss.oracle.com/el4/RPM-GPG-KEY-oracle -O- | sudo apt-key add -
27+ $ sudo apt-get update
28+ $ sudo apt-get install oracle-xe
29+
30+Binaries for Oracle XE on 64-bit systems are not available, so
31+you'll need to install the 32-bit versions. Start by downloading
32+the packages:
33+
34+ $ wget -c http://oss.oracle.com/debian/dists/unstable/main/binary-i386/libaio_0.3.104-1_i386.deb http://oss.oracle.com/debian/dists/unstable/non-free/binary-i386/oracle-xe-universal_10.2.0.1-1.1_i386.deb
35+
36+Once you have the packages you need to install them manually:
37+
38+ $ sudo apt-get install libc6-i386 bc
39+ $ sudo dpkg -i --force-architecture libaio_0.3.104-1_i386.deb
40+ $ sudo dpkg -i --force-architecture oracle-xe-universal_10.2.0.1-1.1_i386.deb
41+
42+Finally, when the packages are installed, you can then run the
43+following line to configure the database:
44+
45+ $ sudo /etc/init.d/oracle-xe configure
46+
47 Setting up database users and access security
48 ---------------------------------------------
49
50@@ -145,6 +173,14 @@
51 mysql> GRANT ALL PRIVILEGES ON *.* TO 'YOUR_USERNAME'@'localhost'
52 IDENTIFIED BY '' WITH GRANT OPTION;
53
54+For Oracle, we've included a script to set up a test user. To run it,
55+do the following:
56+
57+ $ sqlplus sys/<sys_password>@localhost:1521
58+ SQL> @oracle_test_setup
59+
60+And follow the prompts to create a user.
61+
62 Creating test databases
63 -----------------------
64
65
66=== added file 'oracle_test_setup.sql'
67--- oracle_test_setup.sql 1970-01-01 00:00:00 +0000
68+++ oracle_test_setup.sql 2010-03-15 15:48:27 +0000
69@@ -0,0 +1,13 @@
70+-- Note that this is a script that is intended to be run to set up a
71+-- test database. You probably don't want to run this on a production
72+-- database!
73+
74+CREATE USER &&user IDENTIFIED BY &&password;
75+GRANT CONNECT TO &&user;
76+GRANT CREATE ANY TABLE TO &&user;
77+GRANT CREATE ANY TRIGGER TO &&user;
78+GRANT CREATE ANY SEQUENCE TO &&user;
79+GRANT DROP ANY SEQUENCE TO &&user;
80+GRANT DROP ANY TABLE TO &&user;
81+GRANT DROP ANY TRIGGER TO &&user;
82+GRANT UNLIMITED TABLESPACE TO &&user;
83\ No newline at end of file
84
85=== added file 'storm/databases/oracle.py'
86--- storm/databases/oracle.py 1970-01-01 00:00:00 +0000
87+++ storm/databases/oracle.py 2010-03-15 15:48:27 +0000
88@@ -0,0 +1,590 @@
89+# Copyright (c) 2008 Alfaiati
90+#
91+# Written by Gustavo Noronha <kov@alfaiati.net>
92+# Willi Langenberger <wlang@wu-wien.ac.at>
93+#
94+# This file is part of Storm Object Relational Mapper.
95+#
96+# Storm is free software; you can redistribute it and/or modify
97+# it under the terms of the GNU Lesser General Public License as
98+# published by the Free Software Foundation; either version 2.1 of
99+# the License, or (at your option) any later version.
100+#
101+# Storm is distributed in the hope that it will be useful,
102+# but WITHOUT ANY WARRANTY; without even the implied warranty of
103+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
104+# GNU Lesser General Public License for more details.
105+#
106+# You should have received a copy of the GNU Lesser General Public License
107+# along with this program. If not, see <http://www.gnu.org/licenses/>.
108+
109+import os
110+import sys
111+from decimal import Decimal
112+
113+from storm.databases import dummy
114+
115+# If no NLS_LANG is set, we'll use english with UTF-8. Note that this
116+# variable *must* be set before importing cx_Oracle.
117+if not os.environ.get("NLS_LANG", None):
118+ os.environ["NLS_LANG"] = 'american_america.utf8'
119+try:
120+ import cx_Oracle as oracle
121+except Exception, e:
122+ oracle = dummy
123+
124+from storm.tracer import trace
125+from storm.variables import Variable, RawStrVariable, PickleVariable
126+from storm.database import Database, Connection, Result
127+from storm.exceptions import (
128+ install_exceptions, ClosedError, DatabaseModuleError, DatabaseError,
129+ OperationalError)
130+from storm.info import get_cls_info, ClassAlias
131+from storm.expr import (
132+ Undef, SetExpr, Select, Insert, Alias, And, Eq, FuncExpr, SQLRaw,
133+ Le, Gt, Column, is_safe_token, Except, Expr, Sequence, SQLToken,
134+ COLUMN, COLUMN_NAME, COLUMN_PREFIX, TABLE, compile, compile_eq,
135+ compile_select, compile_insert, compile_set_expr, compile_sql_token,
136+ State)
137+
138+
139+install_exceptions(oracle)
140+compile = compile.create_child()
141+
142+
143+RESERVED = set(
144+"""ACCESS ADD ALL ALTER AND ANY AS ASC AUDIT BETWEEN BY CHAR
145+ CHECK CLUSTER COLUMN COMMENT COMPRESS CONNECT CREATE CURRENT
146+ DATE DECIMAL DEFAULT DELETE DESC DISTINCT DROP ELSE EXCLUSIVE
147+ EXISTS FILE FLOAT FOR FROM GRANT GROUP HAVING IDENTIFIED
148+ IMMEDIATE IN INCREMENT INDEX INITIAL INSERT INTEGER INTERSECT
149+ INTO IS LEVEL LIKE LOCK LONG MAXEXTENTS MINUS MLSLABEL MODE
150+ MODIFY NOAUDIT NOCOMPRESS NOT NOWAIT NULL NUMBER OF OFFLINE
151+ ON ONLINE OPTION OR ORDER PCTFREE PRIOR PRIVILEGES PUBLIC
152+ RAW RENAME RESOURCE REVOKE ROW ROWS SELECT SESSION
153+ SET SHARE SIZE SMALLINT START SUCCESSFUL SYNONYM SYSDATE
154+ TABLE THEN TO TRIGGER UID UNION UNIQUE UPDATE USER VALIDATE
155+ VALUES VARCHAR VARCHAR2 VIEW WHENEVER WHERE WITH""".split())
156+
157+
158+def alias_names():
159+ ct = 0
160+ while 1:
161+ yield "_%x" % ct
162+ ct += 1
163+
164+
165+@compile.when(type)
166+def compile_type(compile, expr, state):
167+ cls_info = get_cls_info(expr)
168+ table = compile(cls_info.table, state)
169+ if state.context is TABLE and issubclass(expr, ClassAlias):
170+ return "%s %s" % (compile(cls_info.cls, state), table)
171+ return table
172+
173+
174+@compile.when(Alias)
175+def compile_alias(compile, alias, state):
176+ name = compile(alias.name, state, token=True)
177+ if state.context is COLUMN or state.context is TABLE:
178+ return "%s %s" % (compile(alias.expr, state), name)
179+ return name
180+
181+
182+@compile.when(Sequence)
183+def compile_sequence_oracle(compile, sequence, state):
184+ return "%s.NEXTVAL" % sequence.name
185+
186+
187+class Minus(SetExpr):
188+ oper = " MINUS "
189+
190+
191+@compile.when(Except)
192+def compile_except_oracle(compile, expr, state):
193+ new_expr = Minus()
194+ new_expr.exprs = expr.exprs
195+ new_expr.all = expr.all
196+ new_expr.order_by = expr.order_by
197+ new_expr.limit = expr.limit
198+ new_expr.offset = expr.offset
199+ return compile_set_expr_oracle(compile, new_expr, state)
200+
201+
202+@compile.when(SetExpr)
203+def compile_set_expr_oracle(compile, expr, state):
204+ names = alias_names()
205+ if isinstance(expr, Minus):
206+ # Build new set expression without arguments (order_by, etc).
207+ new_expr = expr.__class__()
208+ new_expr.exprs = expr.exprs
209+ new_expr.all = expr.all
210+
211+ if expr.order_by is not Undef:
212+ # Make sure that state.aliases isn't None, since we want them to
213+ # compile our order_by statement below.
214+ no_aliases = state.aliases is None
215+ if no_aliases:
216+ state.push("aliases", {})
217+
218+ aliases = {}
219+ for subexpr in expr.exprs:
220+ if isinstance(subexpr, Select):
221+ columns = subexpr.columns
222+ if not isinstance(columns, (tuple, list)):
223+ columns = [columns]
224+ else:
225+ columns = list(columns)
226+ for i, column in enumerate(columns):
227+ if column not in aliases:
228+ if isinstance(column, Column):
229+ aliases[column] = columns[i] = Alias(
230+ column, name=names.next())
231+ elif isinstance(column, Alias):
232+ aliases[column.expr] = column
233+ subexpr.columns = columns
234+ aliases.update(state.aliases)
235+ state.aliases = aliases
236+ aliases = None
237+
238+ set_statement = SQLRaw("(%s)" % compile(expr.exprs, state,
239+ join=expr.oper))
240+
241+ if expr.order_by is not Undef:
242+ # Build order_by statement, using aliases.
243+ state.push("context", COLUMN_NAME)
244+ order_by_statement = SQLRaw(compile(expr.order_by, state))
245+ state.pop()
246+ else:
247+ order_by_statement = Undef
248+
249+ # Build wrapping select statement.
250+ select = Select(
251+ SQLRaw("*"), tables=Alias(set_statement, name=names.next()),
252+ limit=expr.limit, offset=expr.offset, order_by=order_by_statement)
253+ return compile_select(compile, select, state)
254+ return compile_set_expr(compile, expr, state)
255+
256+
257+@compile.when(Select)
258+def compile_select_oracle(compile, select, state):
259+ limit = select.limit
260+ offset = select.offset
261+ # Make sure limit is Undef'ed.
262+ select.offset = select.limit = Undef
263+
264+ names = alias_names()
265+
266+ if select.default_tables is Undef:
267+ select.default_tables = ["DUAL"]
268+
269+ if select.order_by is not Undef:
270+ # Copied from expr.py's compile_set_expr.
271+ aliases = {}
272+ columns = select.columns
273+ if not isinstance(columns, (tuple, list)):
274+ columns = [columns]
275+ else:
276+ columns = list(columns)
277+ for i, column in enumerate(columns):
278+ if column not in aliases:
279+ if isinstance(column, Column):
280+ aliases[column] = columns[i] = Alias(
281+ column, name=names.next())
282+ elif isinstance(column, Alias):
283+ aliases[column.expr] = column
284+ select.columns = columns
285+ # Copied from expr.py's compile_set_expr.
286+ statement = SQLRaw("(%s)" % compile_select(compile, select, state))
287+ select = Select(SQLRaw("*"), tables=Alias(statement,
288+ name=names.next()))
289+
290+ if (limit is not Undef) and (offset is not Undef):
291+ rownum_alias = Alias(SQLRaw("ROWNUM"), name=names.next())
292+
293+ # If we have an SQLRaw here that is because we are dealing with a
294+ # subquery.
295+ if isinstance(select.columns, SQLRaw):
296+ select.columns = [SQLRaw('"' + select.tables.name + '".*'),
297+ rownum_alias]
298+ else:
299+ select.columns.append(rownum_alias)
300+
301+ where_expr = Le(SQLRaw("ROWNUM"), limit + offset)
302+ if select.where is Undef:
303+ select.where = where_expr
304+ else:
305+ select.where = And(select.where, where_expr)
306+
307+ statement = SQLRaw("(%s)" % compile_select(compile, select, state))
308+ select = Select(SQLRaw("*"), tables=Alias(statement, names.next()),
309+ where=Gt(rownum_alias, offset))
310+ elif limit is not Undef:
311+ expr = Le(SQLRaw("ROWNUM"), limit)
312+ if select.where is Undef:
313+ select.where = expr
314+ else:
315+ select.where = And(select.where, expr)
316+ elif offset is not Undef:
317+ rownum_alias = Alias(SQLRaw("ROWNUM"), name=names.next())
318+
319+ # If we have an SQLRaw here that is because we are dealing with a
320+ # subquery.
321+ if isinstance(select.columns, SQLRaw):
322+ select.columns = [SQLRaw('"' + select.tables.name + '".*'),
323+ rownum_alias]
324+ else:
325+ select.columns.append(rownum_alias)
326+
327+ statement = SQLRaw("(%s)" % compile_select(compile, select, state))
328+ select = Select(SQLRaw("*"), tables=Alias(statement,
329+ name=names.next()),
330+ where=Gt(rownum_alias, offset))
331+
332+ return compile_select(compile, select, state)
333+
334+
335+@compile.when(Insert)
336+def compile_insert_oracle(compile, insert, state):
337+ # Shamelessly copied from PostgreSQL.
338+ if not insert.map and insert.primary_columns is not Undef:
339+ insert.map.update(dict.fromkeys(insert.primary_columns,
340+ SQLRaw("DEFAULT")))
341+ return compile_insert(compile, insert, state)
342+
343+
344+@compile.when(Sequence)
345+def compile_sequence_oracle(compile, sequence, state):
346+ return "%s.nextval" % sequence.name
347+
348+
349+@compile.when(bool)
350+def compile_bool(compile, expr, state):
351+ return compile_eq(compile, Eq(1, int(expr)), state)
352+
353+
354+class currval(FuncExpr):
355+
356+ name = "currval"
357+
358+ def __init__(self, column):
359+ self.column = column
360+
361+@compile.when(currval)
362+def compile_currval(compile, expr, state):
363+ """Compile a L{currval}."""
364+ state.push("context", COLUMN_PREFIX)
365+ table = compile(expr.column.table, state, token=True)
366+ state.pop()
367+ return "%s_seq.currval" % (table)
368+
369+
370+class Rowid(Expr):
371+
372+ def __init__(self, rowid):
373+ self.rowid = rowid
374+
375+@compile.when(Rowid)
376+def compile_rowid(compile, expr, state):
377+ state.parameters.append(expr.rowid)
378+ return "?"
379+
380+
381+@compile.when(SQLToken)
382+def compile_oracle_sql_token(compile, expr, state):
383+ if "." in expr and state.context in (TABLE, COLUMN_PREFIX):
384+ return ".".join(compile_sql_token(compile, subexpr, state)
385+ for subexpr in expr.split("."))
386+ # A bit of a hack: we're skipping the built-in reserved word list and
387+ # using our own set.
388+ if is_safe_token(expr) and not str(expr) in RESERVED:
389+ return expr
390+ elif (state.context in (COLUMN, COLUMN_NAME)
391+ and str(expr).lower() in ("rowid", "rownum")):
392+ return expr
393+ return '"%s"' % expr.replace('"', "|")
394+
395+
396+class OracleResult(Result):
397+
398+ def __init__(self, connection, raw_cursor, rowid = None):
399+ super(OracleResult, self).__init__(connection, raw_cursor)
400+ self.lastrowid = rowid
401+
402+ def get_insert_identity(self, primary_key, primary_variables):
403+ return Eq(Column("rowid"), Rowid(self.lastrowid))
404+
405+ @staticmethod
406+ def set_variable(variable, value):
407+ if isinstance(value, float):
408+ value = Decimal(str(value))
409+
410+ variable.set(value, from_db=True)
411+
412+ @staticmethod
413+ def from_database(row):
414+ """Convert Oracle-specific datatypes to 'normal' Python types.
415+
416+ If there are anny C{buffer} instances in the row, convert them
417+ to strings.
418+ """
419+ for value in row:
420+ if isinstance(value, oracle.LOB):
421+ yield value.read()
422+ else:
423+ yield value
424+
425+
426+class OracleConnection(Connection):
427+
428+ result_factory = OracleResult
429+ compile = compile
430+ param_mark = "?"
431+
432+ def as_read_committed(self):
433+ return _isolation_context(self)
434+
435+ def is_disconnection_error(self, exc):
436+ if isinstance(exc, (oracle.OperationalError, oracle.DatabaseError)):
437+ error, = exc.args
438+ # Sometimes exceptions get caught here without a code attribute.
439+ if hasattr(error, "code") and error.code in (3135, 3113):
440+ return True
441+ return False
442+
443+ def execute(self, statement, params=None, noresult=False):
444+ """Execute a statement with the given parameters.
445+
446+ This method is completely overidden because the original from the base
447+ class expects to receive only a C{raw_cursor} from C{raw_execute}, and
448+ we need to receive also the C{rowid}, as we cannot set it in the
449+ cursor object.
450+
451+ @type statement: L{Expr} or C{str}.
452+ @param statement: The statement to execute. It will be compiled if
453+ necessary.
454+ @param noresult: If True, no result will be returned.
455+ @raise DisconnectionError: Raised when the connection is lost.
456+ Reconnection happens automatically on rollback.
457+ @return: The result of C{self.result_factory}, or None if C{noresult}
458+ is True.
459+ """
460+ if self._closed:
461+ raise ClosedError("Connection is closed")
462+ self._ensure_connected()
463+ if self._event:
464+ self._event.emit("register-transaction")
465+ if isinstance(statement, Expr):
466+ if params is not None:
467+ raise ValueError("Can't pass parameters with expressions")
468+ state = State()
469+ statement = self.compile(statement, state)
470+ params = state.parameters
471+ statement = convert_to_sequential(statement)
472+ raw_cursor, rowid = self.raw_execute(statement, params)
473+ if noresult:
474+ self._check_disconnect(raw_cursor.close)
475+ return None
476+ return self.result_factory(self, raw_cursor, rowid)
477+
478+ def raw_execute(self, statement, params):
479+ """Execute a raw statement with the given parameters.
480+
481+ This method is completely overidden because the original from the base
482+ class converts params to a tuple, and we need a dictionary! It's
483+ acceptable to override this method in subclasses, but it is not
484+ intended to be called externally. If the global C{DEBUG} flag is
485+ True, the statement will be printed to standard out.
486+
487+ @return: The DBAPI cursor object, as fetched from L{build_raw_cursor}.
488+ """
489+ rowid = None
490+ raw_cursor = self.build_raw_cursor()
491+
492+ statement = str(statement)
493+
494+ if statement.startswith("INSERT INTO"):
495+ statement = statement + " RETURNING ROWID INTO :out_rowid"
496+
497+ # Make sure params is a list as we need to add to it.
498+ if params is None:
499+ params = []
500+ elif not isinstance(params, list):
501+ params = list(params)
502+
503+ rowid = raw_cursor.var(oracle.ROWID)
504+ params.append(rowid)
505+
506+ if not params:
507+ params = ()
508+ else:
509+ params = tuple(self.to_database(params))
510+
511+ trace("connection_raw_execute", self, raw_cursor, statement,
512+ params or ())
513+
514+ try:
515+ self._check_disconnect(raw_cursor.execute, statement, params)
516+ if rowid:
517+ rowid = rowid.getvalue()
518+ except DatabaseError, de:
519+ error, = de.args
520+ if error == 8177:
521+ raise OperationalError("database is locked")
522+ else:
523+ raise
524+ except Exception, error:
525+ trace("connection_raw_execute_error", self, raw_cursor,
526+ statement, params or (), error)
527+ raise
528+ else:
529+ trace("connection_raw_execute_success", self, raw_cursor,
530+ statement, params or ())
531+ return raw_cursor, rowid
532+
533+ @staticmethod
534+ def to_database(params):
535+ for bind_var in params:
536+ if isinstance(bind_var, (RawStrVariable, PickleVariable)):
537+ yield oracle.Binary(bind_var.get(to_db=True))
538+ elif isinstance(bind_var, Variable):
539+ yield bind_var.get(to_db=True)
540+ else:
541+ yield bind_var
542+
543+
544+def convert_to_sequential(statement):
545+ """Convert a query using ? bind variables to a query using
546+ sequential bind variables. For example, SELECT ? FROM DUAL
547+ will be converted to SELECT :1 FROM DUAL"""
548+ param_no = 1
549+ tokens = statement.split("'")
550+ for i in range(0, len(tokens), 2):
551+ while True:
552+ old_tokens = tokens[i]
553+ new_tokens = old_tokens.replace('?', ':%s' % param_no, 1)
554+ if old_tokens == new_tokens:
555+ break
556+ else:
557+ tokens[i] = new_tokens
558+ param_no += 1
559+ return "'".join(tokens)
560+
561+
562+class _isolation_context(object):
563+
564+ def __init__ (self, connection):
565+ self.connection = connection
566+
567+ def __enter__(self):
568+ self.connection.commit()
569+ self.connection.execute(
570+ "ALTER SESSION SET isolation_level = read committed")
571+
572+ def __exit__(self, exc_type, exc_value, traceback):
573+ if exc_value:
574+ self.connection.rollback()
575+ else:
576+ self.connection.commit()
577+ self.connection.execute(
578+ "ALTER SESSION SET isolation_level = serializable")
579+ self.connection.commit()
580+
581+
582+class _type_converter(object):
583+ """This class wraps an input type handler and an output type handler.
584+ The reason being to allow for saving the encoding for the session."""
585+ def __init__(self, encoding):
586+ self.encoding = encoding
587+
588+ def convert_unicode(self, value):
589+ if isinstance(value, Variable):
590+ value = value.get(to_db=True)
591+ if not isinstance(value, unicode):
592+ return unicode(value, "utf8")
593+ else:
594+ return value
595+
596+ def OutputTypeHandler(self, cursor, name, defaultType, size, precision,
597+ scale):
598+ """Prepare cx_Oracle to turn an Oracle datatype into a Python datatype.
599+
600+ This function's purpose is to tell cx_Oracle how to convert an Oracle
601+ datatype into a Python datatype.
602+ """
603+ if defaultType in (oracle.UNICODE, oracle.FIXED_UNICODE):
604+ return cursor.var(unicode, size, cursor.arraysize,
605+ outconverter=self.convert_unicode)
606+ elif defaultType in (oracle.STRING, oracle.FIXED_CHAR):
607+ return cursor.var(str, size, cursor.arraysize)
608+
609+ def InputTypeHandler(self, cursor, value, numElements):
610+ """Prepare cx_Oracle to turn a Python datatype into an Oracle datatype.
611+
612+ This function does the opposite of OutputTypeHandler: it tells
613+ cx_Oracle how to convert a Python datatype into an Oracle datatype.
614+ """
615+ if isinstance(value, unicode):
616+ return cursor.var(unicode, arraysize=numElements,
617+ inconverter=self.convert_unicode)
618+
619+class Oracle(Database):
620+
621+ connection_factory = OracleConnection
622+ converter_factory = _type_converter
623+ raw_connection_factory = oracle.connect
624+
625+ def __init__(self, uri):
626+ if oracle is dummy:
627+ raise DatabaseModuleError("'cx_Oracle' failed to load")
628+
629+ if not uri.port:
630+ uri.port = 1521
631+
632+ isolation = uri.options.get("isolation", "serializable")
633+ isolation_mapping = {
634+ "serializable": "SERIALIZABLE",
635+ "read-committed": "READ COMMITTED",
636+ }
637+ try:
638+ self._isolation = isolation_mapping[isolation]
639+ except KeyError:
640+ raise ValueError(
641+ "Unknown serialization level %r: expected one of "
642+ "'serializable' or 'read-committed'" %
643+ (isolation,))
644+
645+ # Optionally set ORACLE_HOME and TNS_ADMIN environment
646+ # variables for controlling tnsnames.ora lookup
647+ oracle_home = uri.options.get('oracle_home')
648+ if oracle_home:
649+ os.environ["ORACLE_HOME"] = oracle_home
650+
651+ # If tns is specified in the options, treat the host part of the URI as
652+ # the TNS name, and therefore the DSN.
653+ if uri.options.get("tns", False):
654+ self._dsn = uri.host
655+ else:
656+ self._dsn = oracle.makedsn(uri.host, uri.port, uri.database)
657+
658+ self._username = uri.username
659+ self._password = uri.password
660+
661+ def raw_connect(self):
662+ if oracle is dummy:
663+ raise ImportError, "Could not import cx_Oracle"
664+
665+ raw_connection = self.raw_connection_factory(self._username,
666+ self._password, self._dsn)
667+
668+ c = raw_connection.cursor()
669+ c.execute("alter session set isolation_level=%s" % self._isolation)
670+ c.close()
671+
672+ type_handler = self.converter_factory(raw_connection.nencoding)
673+ raw_connection.inputtypehandler = type_handler.InputTypeHandler
674+ raw_connection.outputtypehandler = type_handler.OutputTypeHandler
675+
676+ return raw_connection
677+
678+create_from_uri = Oracle
679
680=== modified file 'storm/expr.py'
681--- storm/expr.py 2009-11-02 11:11:20 +0000
682+++ storm/expr.py 2010-03-15 15:48:27 +0000
683@@ -1000,7 +1000,7 @@
684 return "%s = %s" % (compile(eq.expr1, state), compile(eq.expr2, state))
685
686 @compile_python.when(Eq)
687-def compile_eq(compile, eq, state):
688+def compile_python_eq(compile, eq, state):
689 return "%s == %s" % (compile(eq.expr1, state), compile(eq.expr2, state))
690
691
692
693=== modified file 'storm/variables.py'
694--- storm/variables.py 2009-10-03 16:08:04 +0000
695+++ storm/variables.py 2010-03-15 15:48:27 +0000
696@@ -436,6 +436,8 @@
697 if from_db:
698 if value is None:
699 return None
700+ if isinstance(value, datetime):
701+ return value.date()
702 if isinstance(value, date):
703 return value
704 if not isinstance(value, (str, unicode)):
705
706=== modified file 'tests/databases/base.py'
707--- tests/databases/base.py 2009-09-19 15:56:13 +0000
708+++ tests/databases/base.py 2010-03-15 15:48:27 +0000
709@@ -75,7 +75,7 @@
710 raise NotImplementedError
711
712 def create_sample_data(self):
713- self.connection.execute("INSERT INTO number VALUES (1, 2, 3)")
714+ self.connection.execute("INSERT INTO number_table VALUES (1, 2, 3)")
715 self.connection.execute("INSERT INTO test VALUES (10, 'Title 10')")
716 self.connection.execute("INSERT INTO test VALUES (20, 'Title 20')")
717 self.connection.commit()
718@@ -84,7 +84,7 @@
719 pass
720
721 def drop_tables(self):
722- for table in ["number", "test", "datetime_test", "bin_test"]:
723+ for table in ["number_table", "test", "datetime_test", "bin_test"]:
724 try:
725 self.connection.execute("DROP TABLE " + table)
726 self.connection.commit()
727@@ -131,7 +131,7 @@
728 self.assertTrue(result.get_one())
729
730 def test_execute_result(self):
731- result = self.connection.execute("SELECT 1")
732+ result = self.connection.execute(Select(1))
733 self.assertTrue(isinstance(result, Result))
734 self.assertTrue(result.get_one())
735
736@@ -143,15 +143,15 @@
737 self.assertTrue(isinstance(row[0], unicode))
738
739 def test_execute_params(self):
740- result = self.connection.execute("SELECT one FROM number "
741+ result = self.connection.execute("SELECT one FROM number_table "
742 "WHERE 1=?", (1,))
743 self.assertTrue(result.get_one())
744- result = self.connection.execute("SELECT one FROM number "
745+ result = self.connection.execute("SELECT one FROM number_table "
746 "WHERE 1=?", (2,))
747 self.assertFalse(result.get_one())
748
749 def test_execute_empty_params(self):
750- result = self.connection.execute("SELECT one FROM number", ())
751+ result = self.connection.execute("SELECT one FROM number_table", ())
752 self.assertTrue(result.get_one())
753
754 def test_execute_expression(self):
755@@ -356,7 +356,7 @@
756 event.hook("register-transaction", register_transaction)
757
758 connection = self.database.connect(event)
759- connection.execute("SELECT 1")
760+ connection.execute(Select(1))
761 self.assertEqual(len(calls), 1)
762 self.assertEqual(calls[0], marker)
763
764@@ -364,17 +364,17 @@
765 return [int(item)+1 for item in row]
766
767 def test_wb_result_get_one_goes_through_from_database(self):
768- result = self.connection.execute("SELECT one, two FROM number")
769+ result = self.connection.execute("SELECT one, two FROM number_table")
770 result.from_database = self.from_database
771 self.assertEquals(result.get_one(), (2, 3))
772
773 def test_wb_result_get_all_goes_through_from_database(self):
774- result = self.connection.execute("SELECT one, two FROM number")
775+ result = self.connection.execute("SELECT one, two FROM number_table")
776 result.from_database = self.from_database
777 self.assertEquals(result.get_all(), [(2, 3)])
778
779 def test_wb_result_iter_goes_through_from_database(self):
780- result = self.connection.execute("SELECT one, two FROM number")
781+ result = self.connection.execute("SELECT one, two FROM number_table")
782 result.from_database = self.from_database
783 self.assertEquals(iter(result).next(), (2, 3))
784
785@@ -505,6 +505,7 @@
786 "%s.default_port" % (type(self).__name__,
787 type(self).__name__))
788 uri_str = os.environ.get(self.host_environment_variable)
789+
790 if uri_str:
791 uri = URI(uri_str)
792 if not uri.host:
793@@ -542,24 +543,24 @@
794
795 def test_proxy_works(self):
796 """Ensure that we can talk to the database through the proxy."""
797- result = self.connection.execute("SELECT 1")
798+ result = self.connection.execute(Select(1))
799 self.assertEqual(result.get_one(), (1,))
800
801 def test_catch_disconnect_on_execute(self):
802 """Test that database disconnections get caught on execute()."""
803- result = self.connection.execute("SELECT 1")
804+ result = self.connection.execute(Select(1))
805 self.assertTrue(result.get_one())
806 self.proxy.restart()
807 self.assertRaises(DisconnectionError,
808- self.connection.execute, "SELECT 1")
809+ self.connection.execute, Select(1))
810
811 def test_catch_disconnect_on_commit(self):
812 """Test that database disconnections get caught on commit()."""
813- result = self.connection.execute("SELECT 1")
814+ result = self.connection.execute(Select(1))
815 self.assertTrue(result.get_one())
816 self.proxy.restart()
817 self.assertRaises(DisconnectionError, self.connection.commit)
818-
819+
820 def test_wb_catch_already_disconnected_on_rollback(self):
821 """Connection.rollback() swallows disconnection errors.
822
823@@ -567,13 +568,13 @@
824 then it is possible that Storm won't see the disconnection.
825 It should be able to recover from this situation though.
826 """
827- result = self.connection.execute("SELECT 1")
828+ result = self.connection.execute(Select(1))
829 self.assertTrue(result.get_one())
830 self.proxy.restart()
831 # Perform an action that should result in a disconnection.
832 try:
833 cursor = self.connection._raw_connection.cursor()
834- cursor.execute("SELECT 1")
835+ cursor.execute(Select(1))
836 cursor.fetchone()
837 except Error, exc:
838 self.assertTrue(self.connection.is_disconnection_error(exc))
839@@ -600,59 +601,59 @@
840 then it is possible that Storm won't see the disconnection.
841 It should be able to recover from this situation though.
842 """
843- result = self.connection.execute("SELECT 1")
844+ result = self.connection.execute(Select(1))
845 self.assertTrue(result.get_one())
846 self.proxy.restart()
847 # Perform an action that should result in a disconnection.
848 try:
849 cursor = self.connection._raw_connection.cursor()
850- cursor.execute("SELECT 1")
851+ cursor.execute(Select(1))
852 cursor.fetchone()
853 except DatabaseError, exc:
854 self.assertTrue(self.connection.is_disconnection_error(exc))
855 else:
856 self.fail("Disconnection was not caught.")
857 self.assertRaises(DisconnectionError,
858- self.connection.execute, "SELECT 1")
859+ self.connection.execute, Select(1))
860
861 def test_connection_stays_disconnected_in_transaction(self):
862 """Test that connection does not immediately reconnect."""
863- result = self.connection.execute("SELECT 1")
864+ result = self.connection.execute(Select(1))
865 self.assertTrue(result.get_one())
866 self.proxy.restart()
867 self.assertRaises(DisconnectionError,
868- self.connection.execute, "SELECT 1")
869+ self.connection.execute, Select(1))
870 self.assertRaises(DisconnectionError,
871- self.connection.execute, "SELECT 1")
872+ self.connection.execute, Select(1))
873
874 def test_reconnect_after_rollback(self):
875 """Test that we reconnect after rolling back the connection."""
876- result = self.connection.execute("SELECT 1")
877+ result = self.connection.execute(Select(1))
878 self.assertTrue(result.get_one())
879 self.proxy.restart()
880 self.assertRaises(DisconnectionError,
881- self.connection.execute, "SELECT 1")
882+ self.connection.execute, Select(1))
883 self.connection.rollback()
884- result = self.connection.execute("SELECT 1")
885+ result = self.connection.execute(Select(1))
886 self.assertTrue(result.get_one())
887
888 def test_catch_disconnect_on_reconnect(self):
889 """Test that reconnection failures result in DisconnectionError."""
890- result = self.connection.execute("SELECT 1")
891+ result = self.connection.execute(Select(1))
892 self.assertTrue(result.get_one())
893 self.proxy.stop()
894 self.assertRaises(DisconnectionError,
895- self.connection.execute, "SELECT 1")
896+ self.connection.execute, Select(1))
897 # Rollback the connection, but because the proxy is still
898 # down, we get a DisconnectionError again.
899 self.connection.rollback()
900 self.assertRaises(DisconnectionError,
901- self.connection.execute, "SELECT 1")
902+ self.connection.execute, Select(1))
903
904 def test_close_connection_after_disconnect(self):
905- result = self.connection.execute("SELECT 1")
906+ result = self.connection.execute(Select(1))
907 self.assertTrue(result.get_one())
908 self.proxy.stop()
909 self.assertRaises(DisconnectionError,
910- self.connection.execute, "SELECT 1")
911+ self.connection.execute, Select(1))
912 self.connection.close()
913
914=== added file 'tests/databases/oracle.py'
915--- tests/databases/oracle.py 1970-01-01 00:00:00 +0000
916+++ tests/databases/oracle.py 2010-03-15 15:48:27 +0000
917@@ -0,0 +1,372 @@
918+from __future__ import with_statement
919+
920+import os
921+import pickle
922+from binascii import hexlify
923+from datetime import time, timedelta
924+from unittest import TestSuite, defaultTestLoader as loader, TestCase
925+
926+from cx_Oracle import makedsn, DatabaseError, LOB
927+
928+from storm.database import *
929+from storm.databases.oracle import (
930+ _type_converter, convert_to_sequential, Oracle, _isolation_context,
931+ OracleConnection, OracleResult, compile)
932+from storm.event import EventSystem
933+from storm.exceptions import NotSupportedError
934+from storm.expr import Select, SQLToken, Alias
935+from storm.tracer import debug
936+from storm.uri import URI
937+from storm.variables import PickleVariable, RawStrVariable, TimeDeltaVariable
938+
939+from tests.databases.base import (
940+ DatabaseTest, DatabaseDisconnectionTest, UnsupportedDatabaseTest)
941+from tests.helper import TestHelper
942+from tests.mocker import ANY, ARGS
943+from tests.store.oracle import create_table_helpers, drop_tables
944+
945+
946+DEBUG = True
947+debug(DEBUG)
948+
949+class Marker(object):
950+ pass
951+
952+marker = Marker()
953+
954+class OracleDatabaseTest(DatabaseTest, TestHelper):
955+ _table_names = ['number_table', 'test', 'datetime_test', 'bin_test',
956+ 'like_case_insensitive_test', 'insert_returning_test']
957+ supports_microseconds = False
958+ def is_supported(self):
959+ return bool(os.environ.get('STORM_ORACLE_URI'))
960+
961+ def create_database(self):
962+ self.database = create_database(os.environ['STORM_ORACLE_URI'])
963+
964+ def create_tables(self):
965+ try:
966+ with self.connection.as_read_committed():
967+ self.connection.execute("CREATE TABLE number_table "
968+ "(id INTEGER, one INTEGER, two INTEGER,"
969+ " three INTEGER)")
970+ self.connection.execute("CREATE TABLE test "
971+ "(id INTEGER PRIMARY KEY, title NVARCHAR2(2000))")
972+ self.connection.execute("CREATE TABLE datetime_test "
973+ "(id INTEGER PRIMARY KEY,"
974+ " dt TIMESTAMP(9), d DATE, t DATE, "
975+ " td INTERVAL DAY TO SECOND)")
976+ self.connection.execute("CREATE TABLE bin_test "
977+ "(id INTEGER PRIMARY KEY, b BLOB)")
978+ self.connection.execute("CREATE TABLE like_case_insensitive_test "
979+ "(id INTEGER PRIMARY KEY, description CLOB)")
980+ self.connection.execute("CREATE TABLE insert_returning_test "
981+ "(id INTEGER PRIMARY KEY, "
982+ "id1 INTEGER DEFAULT 123, "
983+ " id2 INTEGER DEFAULT 456)")
984+ create_table_helpers(self.connection, self._table_names)
985+
986+ except Exception, e:
987+ self.connection.rollback()
988+ raise
989+ finally:
990+ debug(DEBUG)
991+
992+ def create_sample_data(self):
993+ self.connection.execute("INSERT INTO number_table (one, two, three) VALUES (1, 2, 3)")
994+ self.connection.execute("INSERT INTO test VALUES (10, 'Title 10')")
995+ self.connection.execute("INSERT INTO test VALUES (20, 'Title 20')")
996+ self.connection.commit()
997+
998+ def drop_tables(self):
999+ try:
1000+ drop_tables(self.connection, self._table_names)
1001+ finally:
1002+ debug(DEBUG)
1003+
1004+ def test_binary(self):
1005+ """Ensure database works with high bits and embedded zeros.
1006+ Overriding this because we need to send the data as a hexlified
1007+ string."""
1008+
1009+ value = "\xff\x00\xff\x00"
1010+ self.connection.execute("INSERT INTO bin_test (b) VALUES (?)",
1011+ (hexlify(value),))
1012+ self.connection.commit()
1013+ result = self.connection.execute("SELECT b FROM bin_test")
1014+ variable = RawStrVariable()
1015+ result.set_variable(variable, result.get_one()[0])
1016+ self.assertEquals(variable.get(), value)
1017+
1018+ def test_binary_ascii(self):
1019+ """Some databases like pysqlite2 may return unicode for strings.
1020+ Overriding this because we need to send the data as a hexlified
1021+ string."""
1022+ bin_text = 'Value'
1023+ self.connection.execute("INSERT INTO bin_test VALUES (10, ?)",
1024+ (hexlify(bin_text),))
1025+ result = self.connection.execute("SELECT b FROM bin_test")
1026+ variable = RawStrVariable()
1027+ # If the following doesn't raise a TypeError we're good.
1028+ result.set_variable(variable, result.get_one()[0])
1029+ self.assertEquals(variable.get(), "Value")
1030+
1031+ def test_execute_result(self):
1032+ result = self.connection.execute("SELECT 1 FROM DUAL")
1033+ self.assertTrue(isinstance(result, Result))
1034+ self.assertTrue(result.get_one())
1035+
1036+ def test_execute_sends_event(self):
1037+ event = EventSystem(marker)
1038+ calls = []
1039+ def register_transaction(owner):
1040+ calls.append(owner)
1041+ event.hook("register-transaction", register_transaction)
1042+
1043+ connection = self.database.connect(event)
1044+ connection.execute("SELECT 1 FROM DUAL")
1045+ self.assertEqual(len(calls), 1)
1046+ self.assertEqual(calls[0], marker)
1047+
1048+ def test_timedelta(self):
1049+ value = timedelta(12, 34)
1050+ self.connection.execute("INSERT INTO datetime_test (td) VALUES (?)",
1051+ (value,))
1052+ result = self.connection.execute("SELECT td FROM datetime_test")
1053+ variable = TimeDeltaVariable()
1054+ result.set_variable(variable, result.get_one()[0])
1055+ self.assertEquals(variable.get(), value)
1056+
1057+ def test_pickle(self):
1058+ value = {"a": 1, "b": 2}
1059+ value_dump = pickle.dumps(value, -1)
1060+ self.connection.execute("INSERT INTO bin_test (b) VALUES (?)",
1061+ (hexlify(value_dump),))
1062+ result = self.connection.execute("SELECT b FROM bin_test")
1063+ variable = PickleVariable()
1064+ result.set_variable(variable, result.get_one()[0])
1065+ self.assertEquals(variable.get(), value)
1066+
1067+ def test_time(self):
1068+ """Oracle doesn't have a time-only datatype, so we need to test
1069+ that using one fails gracefully."""
1070+ self.assertRaises(NotSupportedError, self.connection.execute,
1071+ 'SELECT * FROM ', (time(1, 2, 3),))
1072+
1073+
1074+class OracleUnsupportedDatabaseTest(UnsupportedDatabaseTest, TestHelper):
1075+ dbapi_module_names = ['cx_Oracle']
1076+ db_module_name = 'oracle'
1077+
1078+# FIXME - Oracle doesn't seem to work with these disconnection tests
1079+# class OracleDatabaseDisconnectionTest(DatabaseDisconnectionTest, TestHelper):
1080+# environment_variable = "STORM_ORACLE_URI"
1081+# host_environment_variable = "STORM_ORACLE_HOST_URI"
1082+# default_port=1521
1083+
1084+class MockObject(object):
1085+ def __init__(self, allowed_sets, allowed_gets, *args, **argd):
1086+ self.allowed_sets = allowed_sets
1087+ self.allowed_gets = allowed_gets
1088+ self.args = args
1089+ self.argd = argd
1090+ self.was_set = set()
1091+ self.was_gotten = set()
1092+
1093+ def __getattr__(self, name):
1094+ if self.allowed_gets.get(name, False):
1095+ self.was_gotten.add(name)
1096+ return self.allowed_gets[name]
1097+
1098+ else:
1099+ raise NotImplementedError()
1100+
1101+ def __setattr__(self, name, val):
1102+
1103+ if self.allowed_sets.get(name, False):
1104+ self.was_set.add(name)
1105+ self.__dict__[name] = val
1106+ else:
1107+ raise NotImplementedError()
1108+
1109+ def __delattr__(self, name):
1110+ raise NotImplementedError()
1111+
1112+ def __call__(self, *args, **argd):
1113+ return self
1114+
1115+def dummy_func(*args, **argd):
1116+ pass
1117+
1118+
1119+class UnicodeUnitTests(TestCase):
1120+ unit = True
1121+ def setUp(self):
1122+ self.converter = _type_converter('utf8')
1123+
1124+ def test_identity(self):
1125+ expected = u"Some string"
1126+ actual = self.converter.convert_unicode(expected)
1127+ self.assertEqual(actual, expected)
1128+
1129+ def test_cross_conversion(self):
1130+ initial = unicode('Some string', 'latin-1')
1131+ expected = unicode('Some string', 'utf8')
1132+ actual = self.converter.convert_unicode(initial)
1133+ self.assertEqual(actual, expected)
1134+
1135+class OracleUnitTests(TestHelper):
1136+ unit = True
1137+ def test_tns_uri(self):
1138+ uri = URI('oracle://foo:bar@test?tns=true')
1139+ ora = Oracle(uri)
1140+ self.assertEqual(ora._dsn, 'test')
1141+ self.assertEqual(ora._username, 'foo')
1142+ self.assertEqual(ora._password, 'bar')
1143+
1144+ def test_host_str(self):
1145+ uri = URI('oracle://foo:bar@localhost:1521/test')
1146+ ora = Oracle(uri)
1147+ self.assertEqual(ora._username, 'foo')
1148+ self.assertEqual(ora._password, 'bar')
1149+ expected_dsn = makedsn('localhost', 1521, 'test')
1150+ self.assertEqual(ora._dsn, expected_dsn)
1151+
1152+ def test_type_handlers_set(self):
1153+ mocker = self.mocker
1154+ mock = mocker.mock()
1155+ converter = mocker.mock()
1156+ mock(ARGS)
1157+ mocker.result(mock)
1158+ mock.cursor()
1159+ mocker.result(mock)
1160+ mock.execute(ARGS)
1161+ mocker.result(None)
1162+ mock.nencoding
1163+ mocker.result('utf8')
1164+ converter('utf8')
1165+ mocker.result(converter)
1166+
1167+ converter.InputTypeHandler
1168+ mocker.result(converter)
1169+
1170+ mock.inputtypehandler = converter
1171+ mocker.result(None)
1172+
1173+ converter.OutputTypeHandler
1174+ mocker.result(converter)
1175+ mock.outputtypehandler = converter
1176+ mock.close()
1177+ mocker.result(None)
1178+ mocker.replay()
1179+
1180+ class DummyOracle(Oracle):
1181+ raw_connection_factory = mock
1182+ converter_factory = converter
1183+ def __init__(self):
1184+ self._username = ''
1185+ self._password = ''
1186+ self._dsn = ''
1187+ self._isolation = ''
1188+
1189+ DummyOracle().raw_connect()
1190+
1191+
1192+class OracleConnectionTests(TestHelper):
1193+ unit = True
1194+ def test_isolation_context_pass(self):
1195+ mocker = self.mocker
1196+ mock = mocker.mock()
1197+ mock.commit()
1198+ mocker.count(0, None)
1199+ mock.execute('ALTER SESSION SET isolation_level = read committed')
1200+ mock.execute('ALTER SESSION SET isolation_level = serializable')
1201+ mocker.replay()
1202+
1203+ with _isolation_context(mock):
1204+ pass
1205+
1206+ def test_isolation_context_fail(self):
1207+ mocker = self.mocker
1208+ mock = mocker.mock()
1209+ mock.commit()
1210+ mocker.result(None)
1211+ mock.execute('ALTER SESSION SET isolation_level = read committed')
1212+ mock.rollback()
1213+ mocker.result(None)
1214+ mock.execute('ALTER SESSION SET isolation_level = serializable')
1215+ mock.commit()
1216+ mocker.result(None)
1217+ mocker.replay()
1218+
1219+ try:
1220+ with _isolation_context(mock):
1221+ raise TypeError()
1222+ except TypeError:
1223+ pass
1224+
1225+ def test_is_disconnection_error(self):
1226+ codes = [3135, 3113]
1227+ non_codes = [0, 1000]
1228+
1229+ database = self.mocker.mock()
1230+ database.raw_connect()
1231+ self.mocker.result(database)
1232+ self.mocker.replay()
1233+
1234+ conn = OracleConnection(database)
1235+
1236+ class DummyError(object):
1237+ def __init__(self, code):
1238+ self.code = code
1239+
1240+ for code in codes:
1241+ exc = DatabaseError(DummyError(code))
1242+ assert conn.is_disconnection_error(exc)
1243+
1244+ for code in non_codes:
1245+ exc = DatabaseError(DummyError(code))
1246+ assert not conn.is_disconnection_error(exc)
1247+
1248+ def test_from_database_LOB(self):
1249+ mocker = self.mocker
1250+ mock = mocker.mock(LOB)
1251+
1252+ iter(mock)
1253+ mocker.generate([mock])
1254+ expected = 'result string'
1255+ mock.read()
1256+ mocker.result(expected)
1257+ mocker.replay()
1258+
1259+ row_iter = OracleResult.from_database(mock)
1260+ self.assertEqual(row_iter.next(), expected)
1261+ self.assertRaises(StopIteration, row_iter.next)
1262+
1263+
1264+class QueryUnitTests(TestHelper):
1265+ unit = True
1266+ def test_convert_to_sequential(self):
1267+ initial = 'SELECT ? FROM DUAL'
1268+ expected = 'SELECT :1 FROM DUAL'
1269+ actual = convert_to_sequential(initial)
1270+ self.assertEqual(expected, actual)
1271+
1272+ def test_convert_to_sequential_in_str(self):
1273+ expected = "SELECT '?' FROM DUAL"
1274+ actual = convert_to_sequential(expected)
1275+ self.assertEqual(expected, actual)
1276+
1277+ def test_conver_to_sequential_not_in_str(self):
1278+ initial = "SELECT 'Foo', ? FROM DUAL"
1279+ expected = "SELECT 'Foo', :1 FROM DUAL"
1280+ actual = convert_to_sequential(initial)
1281+ self.assertEqual(expected, actual)
1282+
1283+ def test_double_quotes(self):
1284+ initial = 'word"word'
1285+ expected = '"word|word"'
1286+ actual = compile(SQLToken(initial))
1287+ self.assertEqual(expected, actual)
1288+
1289+
1290
1291=== modified file 'tests/store/base.py'
1292--- tests/store/base.py 2010-02-08 11:30:44 +0000
1293+++ tests/store/base.py 2010-03-15 15:48:27 +0000
1294@@ -280,11 +280,11 @@
1295 return store._cache
1296
1297 def test_execute(self):
1298- result = self.store.execute("SELECT 1")
1299+ result = self.store.execute(Select(1))
1300 self.assertTrue(isinstance(result, Result))
1301 self.assertEquals(result.get_one(), (1,))
1302
1303- result = self.store.execute("SELECT 1", noresult=True)
1304+ result = self.store.execute(Select(1), noresult=True)
1305 self.assertEquals(result, None)
1306
1307 def test_execute_params(self):
1308@@ -302,7 +302,7 @@
1309 def test_close(self):
1310 store = Store(self.database)
1311 store.close()
1312- self.assertRaises(ClosedError, store.execute, "SELECT 1")
1313+ self.assertRaises(ClosedError, store.execute, Select(1))
1314
1315 def test_get(self):
1316 foo = self.store.get(Foo, 10)
1317@@ -1423,7 +1423,7 @@
1318 result.group_by(Foo)
1319 foo1 = self.store.get(Foo, 10)
1320 foo2 = self.store.get(Foo, 20)
1321- self.assertEquals(list(result), [(5, foo1), (16, foo2)])
1322+ self.assertEquals(sorted(list(result)), [(5, foo1), (16, foo2)])
1323
1324 def test_find_group_by_table_contains(self):
1325 result = self.store.find(
1326@@ -5675,7 +5675,7 @@
1327 def register_transaction(owner):
1328 calls.append(owner)
1329 self.store._event.hook("register-transaction", register_transaction)
1330- self.store.execute("SELECT 1")
1331+ self.store.execute(Select(1))
1332 self.assertEqual(len(calls), 1)
1333 self.assertEqual(calls[0], self.store)
1334
1335
1336=== added file 'tests/store/oracle.py'
1337--- tests/store/oracle.py 1970-01-01 00:00:00 +0000
1338+++ tests/store/oracle.py 2010-03-15 15:48:27 +0000
1339@@ -0,0 +1,234 @@
1340+#
1341+# Copyright (c) 2006, 2007 Canonical
1342+#
1343+# Written by Gustavo Niemeyer <gustavo@niemeyer.net>
1344+#
1345+# This file is part of Storm Object Relational Mapper.
1346+#
1347+# Storm is free software; you can redistribute it and/or modify
1348+# it under the terms of the GNU Lesser General Public License as
1349+# published by the Free Software Foundation; either version 2.1 of
1350+# the License, or (at your option) any later version.
1351+#
1352+# Storm is distributed in the hope that it will be useful,
1353+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1354+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1355+# GNU Lesser General Public License for more details.
1356+#
1357+# You should have received a copy of the GNU Lesser General Public License
1358+# along with this program. If not, see <http://www.gnu.org/licenses/>.
1359+#
1360+from __future__ import with_statement
1361+import os, sys
1362+
1363+from storm.database import create_database
1364+from storm.databases.oracle import Oracle
1365+from storm.uri import URI
1366+from cx_Oracle import DatabaseError
1367+
1368+from storm.database import Result
1369+from tests.store.base import StoreTest, EmptyResultSetTest
1370+from tests.store.base import Foo
1371+from storm.properties import Int, Float, RawStr, Unicode, Property, Pickle
1372+from tests.helper import TestHelper, MakePath
1373+
1374+from storm.info import ClassAlias
1375+
1376+from storm.tracer import debug
1377+
1378+DEBUG = False
1379+debug(DEBUG)
1380+
1381+def _safe_drop(connection, query):
1382+ """
1383+ This function will execute a given DROP query
1384+ and ignore any error codes that indicate that
1385+ the object doesn't exist.
1386+ """
1387+ with connection.as_read_committed():
1388+ try:
1389+ connection.execute(query)
1390+ except DatabaseError, exc:
1391+ error, = exc.args
1392+ if error.code not in [942, 4080, 2289, 903]:
1393+ raise
1394+
1395+def create_table_helpers(connection, table_names):
1396+ with connection.as_read_committed():
1397+ for table_name in table_names:
1398+ connection.execute("CREATE SEQUENCE %s_s "
1399+ "START WITH 1 "
1400+ "INCREMENT BY 1 "
1401+ "NOMAXVALUE "
1402+ "ORDER NOCYCLE" % (table_name),
1403+ )
1404+ connection.execute("CREATE OR REPLACE TRIGGER %s_t "
1405+ "BEFORE INSERT ON %s "
1406+ "FOR EACH ROW "
1407+ "WHEN (new.id is null) "
1408+ "BEGIN "
1409+ "SELECT %s_s.nextval INTO :new.id FROM DUAL; "
1410+ "END;" % (table_name, table_name, table_name),
1411+ )
1412+
1413+def drop_tables(connection, tables):
1414+ for table in tables:
1415+ _safe_drop(connection, "DROP TRIGGER %s_t" % table)
1416+ _safe_drop(connection, "DROP SEQUENCE %s_s" % table)
1417+
1418+ _safe_drop(connection, "DROP TABLE %s" % table)
1419+
1420+class OracleStoreTest(TestHelper, StoreTest):
1421+ DUMMY_SELECT = 'SELECT 1 FROM DUAL'
1422+ helpers = [MakePath]
1423+ _tables = ['foo', 'bar', 'bin', 'link', 'money', 'selfref', 'foovalue']
1424+
1425+ def create_tables(self):
1426+ connection = self.connection
1427+ with connection.as_read_committed():
1428+ connection.execute("CREATE TABLE foo (id NUMBER, "
1429+ "title NVARCHAR2(2000) DEFAULT 'Default Title')")
1430+ connection.execute("CREATE TABLE bar "
1431+ "(id NUMBER, foo_id NUMBER, "
1432+ "title NVARCHAR2(2000))")
1433+ connection.execute("CREATE TABLE bin "
1434+ "(id NUMBER, bin BLOB, foo_id INTEGER)")
1435+ connection.execute("CREATE TABLE link "
1436+ "(id NUMBER, foo_id NUMBER, bar_id NUMBER)")
1437+ connection.execute("CREATE TABLE money "
1438+ "(id NUMBER, value NUMBER(6,4))")
1439+ connection.execute("CREATE TABLE selfref "
1440+ "(id NUMBER, title NVARCHAR2(2000),"
1441+ " selfref_id NUMBER)")
1442+ connection.execute("CREATE TABLE foovalue "
1443+ "(id NUMBER, foo_id INTEGER,"
1444+ " value1 INTEGER, value2 INTEGER)")
1445+ create_table_helpers(connection, self._tables)
1446+
1447+ def drop_tables(self):
1448+ drop_tables(self.connection, self._tables)
1449+
1450+ def create_sample_data(self):
1451+ connection = self.connection
1452+ connection.execute("INSERT INTO foo (id, title)"
1453+ " VALUES (10, 'Title 30')")
1454+ connection.execute("INSERT INTO foo (id, title)"
1455+ " VALUES (20, 'Title 20')")
1456+ connection.execute("INSERT INTO foo (id, title)"
1457+ " VALUES (30, 'Title 10')")
1458+ connection.execute("INSERT INTO bar (id, foo_id, title)"
1459+ " VALUES (100, 10, 'Title 300')")
1460+ connection.execute("INSERT INTO bar (id, foo_id, title)"
1461+ " VALUES (200, 20, 'Title 200')")
1462+ connection.execute("INSERT INTO bar (id, foo_id, title)"
1463+ " VALUES (300, 30, 'Title 100')")
1464+ connection.execute("INSERT INTO bin (id, bin) VALUES (10, rawtohex('Blob 30'))")
1465+ connection.execute("INSERT INTO bin (id, bin) VALUES (20, rawtohex('Blob 20'))")
1466+ connection.execute("INSERT INTO bin (id, bin) VALUES (30, rawtohex('Blob 10'))")
1467+ connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (10, 100)")
1468+ connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (10, 200)")
1469+ connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (10, 300)")
1470+ connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (20, 100)")
1471+ connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (20, 200)")
1472+ connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (30, 300)")
1473+ connection.execute("INSERT INTO money (id, value)"
1474+ " VALUES (10, '12.3455')")
1475+ connection.execute("INSERT INTO selfref (id, title, selfref_id)"
1476+ " VALUES (15, 'SelfRef 15', NULL)")
1477+ connection.execute("INSERT INTO selfref (id, title, selfref_id)"
1478+ " VALUES (25, 'SelfRef 25', NULL)")
1479+ connection.execute("INSERT INTO selfref (id, title, selfref_id)"
1480+ " VALUES (35, 'SelfRef 35', 15)")
1481+ connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
1482+ " VALUES (1, 10, 2, 1)")
1483+ connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
1484+ " VALUES (2, 10, 2, 1)")
1485+ connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
1486+ " VALUES (3, 10, 2, 1)")
1487+ connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
1488+ " VALUES (4, 10, 2, 2)")
1489+ connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
1490+ " VALUES (5, 20, 1, 3)")
1491+ connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
1492+ " VALUES (6, 20, 1, 3)")
1493+ connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
1494+ " VALUES (7, 20, 1, 4)")
1495+ connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
1496+ " VALUES (8, 20, 1, 4)")
1497+ connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
1498+ " VALUES (9, 20, 1, 2)")
1499+ connection.commit()
1500+
1501+ #Overriding this because SELECT ? should be SELECT ? FROM DUAL
1502+ def test_execute_params(self):
1503+ result = self.store.execute("SELECT ? FROM DUAL", (1,))
1504+ self.assertTrue(isinstance(result, Result))
1505+ self.assertEquals(result.get_one(), (1,))
1506+
1507+ def test_is_in_empty_list(self):
1508+ result2 = self.store.find(Foo, Foo.id.is_in([]))
1509+ self.assertEquals(result2.count(), 0)
1510+
1511+ def setUp(self):
1512+ TestHelper.setUp(self)
1513+ StoreTest.setUp(self)
1514+ self.connection = self.database.connect()
1515+
1516+ def tearDown(self):
1517+ TestHelper.tearDown(self)
1518+ StoreTest.tearDown(self)
1519+ self.connection.close()
1520+
1521+ def is_supported(self):
1522+ return bool(os.environ.get("STORM_ORACLE_URI"))
1523+
1524+ def create_database(self):
1525+ self.database = create_database(os.environ["STORM_ORACLE_URI"])
1526+
1527+
1528+class testOracleEmptyResultSet(TestHelper, EmptyResultSetTest):
1529+
1530+ helpers = [MakePath]
1531+
1532+ def setUp(self):
1533+ TestHelper.setUp(self)
1534+ EmptyResultSetTest.setUp(self)
1535+ self.connection = self.database.connect()
1536+
1537+ def tearDown(self):
1538+ TestHelper.tearDown(self)
1539+ EmptyResultSetTest.tearDown(self)
1540+ self.connection.close()
1541+
1542+ def is_supported(self):
1543+ return bool(os.environ.get("STORM_ORACLE_URI"))
1544+
1545+ def create_database(self):
1546+ self.database = create_database(os.environ["STORM_ORACLE_URI"])
1547+
1548+ def create_tables(self):
1549+ connection = self.connection
1550+ with connection.as_read_committed():
1551+ connection.execute("CREATE TABLE foo (id NUMBER, "
1552+ "title VARCHAR2(4000) DEFAULT 'Default Title')")
1553+ #connection.execute("CREATE OR REPLACE SEQUENCE foo_seq "
1554+ #"START WITH 1 "
1555+ #"INCREMENT BY 1 "
1556+ #"NOMAXVALUE")
1557+ #connection.execute("CREATE OR REPLACE TRIGGER foo_trigger "
1558+ #"BEFORE INSERT ON foo "
1559+ #"FOR EACH ROW "
1560+ #"BEGIN "
1561+ #"SELECT foo_seq.nextval INTO :new.id FROM DUAL; "
1562+ #"END")
1563+
1564+ def drop_tables(self):
1565+ for table in ["foo"]:
1566+ connection = self.connection
1567+ try:
1568+ #connection.execute("DROP TRIGGER %s_trigger" % table)
1569+ #connection.execute("DROP SEQUENCE %s_seq" % table)
1570+ connection.execute("DROP TABLE %s" % table)
1571+ connection.commit()
1572+ except:
1573+ connection.rollback()

Subscribers

People subscribed via source and target branches

to status/vote changes: