HTD

Merge lp:~ahayzen/htd/3.0-directory-restructure into lp:htd/3.0

Proposed by Andrew Hayzen
Status: Merged
Approved by: Andrew Hayzen
Approved revision: 24
Merged at revision: 24
Proposed branch: lp:~ahayzen/htd/3.0-directory-restructure
Merge into: lp:htd/3.0
Diff against target: 8942 lines (+4811/-3916)
39 files modified
COPYING.LESSER.txt (+165/-0)
COPYING.txt (+674/-0)
MANIFEST.in (+1/-0)
README.txt (+49/-0)
htd30/__init__.py (+266/-0)
htd30/_base/__init__.py (+64/-0)
htd30/_base/file.py (+177/-0)
htd30/_base/trace.py (+213/-0)
htd30/_base/types.py (+273/-0)
htd30/_objects/__init__.py (+26/-0)
htd30/_objects/attribute.py (+179/-0)
htd30/_objects/attributes.py (+212/-0)
htd30/_objects/auth.py (+201/-0)
htd30/_objects/connection.py (+287/-0)
htd30/_objects/cursor.py (+380/-0)
htd30/_objects/record.py (+109/-0)
htd30/_objects/user.py (+81/-0)
htd30/_workers/__init__.py (+333/-0)
htd30/_workers/file_fix.py (+317/-0)
htd30/_workers/file_var.py (+604/-0)
htd30/_workers/header.py (+186/-0)
setup.py (+14/-8)
src/htd30/__init__.py (+0/-266)
src/htd30/_base/__init__.py (+0/-64)
src/htd30/_base/file.py (+0/-177)
src/htd30/_base/trace.py (+0/-213)
src/htd30/_base/types.py (+0/-273)
src/htd30/_objects/__init__.py (+0/-26)
src/htd30/_objects/attribute.py (+0/-179)
src/htd30/_objects/attributes.py (+0/-212)
src/htd30/_objects/auth.py (+0/-201)
src/htd30/_objects/connection.py (+0/-287)
src/htd30/_objects/cursor.py (+0/-380)
src/htd30/_objects/record.py (+0/-109)
src/htd30/_objects/user.py (+0/-81)
src/htd30/_workers/__init__.py (+0/-333)
src/htd30/_workers/file_fix.py (+0/-317)
src/htd30/_workers/file_var.py (+0/-604)
src/htd30/_workers/header.py (+0/-186)
To merge this branch: bzr merge lp:~ahayzen/htd/3.0-directory-restructure
Reviewer Review Type Date Requested Status
Andrew Hayzen Approve
Review via email: mp+115620@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Andrew Hayzen (ahayzen) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'COPYING.LESSER.txt'
2--- COPYING.LESSER.txt 1970-01-01 00:00:00 +0000
3+++ COPYING.LESSER.txt 2012-07-18 20:53:21 +0000
4@@ -0,0 +1,165 @@
5+GNU LESSER GENERAL PUBLIC LICENSE
6+ Version 3, 29 June 2007
7+
8+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
9+ Everyone is permitted to copy and distribute verbatim copies
10+ of this license document, but changing it is not allowed.
11+
12+
13+ This version of the GNU Lesser General Public License incorporates
14+the terms and conditions of version 3 of the GNU General Public
15+License, supplemented by the additional permissions listed below.
16+
17+ 0. Additional Definitions.
18+
19+ As used herein, "this License" refers to version 3 of the GNU Lesser
20+General Public License, and the "GNU GPL" refers to version 3 of the GNU
21+General Public License.
22+
23+ "The Library" refers to a covered work governed by this License,
24+other than an Application or a Combined Work as defined below.
25+
26+ An "Application" is any work that makes use of an interface provided
27+by the Library, but which is not otherwise based on the Library.
28+Defining a subclass of a class defined by the Library is deemed a mode
29+of using an interface provided by the Library.
30+
31+ A "Combined Work" is a work produced by combining or linking an
32+Application with the Library. The particular version of the Library
33+with which the Combined Work was made is also called the "Linked
34+Version".
35+
36+ The "Minimal Corresponding Source" for a Combined Work means the
37+Corresponding Source for the Combined Work, excluding any source code
38+for portions of the Combined Work that, considered in isolation, are
39+based on the Application, and not on the Linked Version.
40+
41+ The "Corresponding Application Code" for a Combined Work means the
42+object code and/or source code for the Application, including any data
43+and utility programs needed for reproducing the Combined Work from the
44+Application, but excluding the System Libraries of the Combined Work.
45+
46+ 1. Exception to Section 3 of the GNU GPL.
47+
48+ You may convey a covered work under sections 3 and 4 of this License
49+without being bound by section 3 of the GNU GPL.
50+
51+ 2. Conveying Modified Versions.
52+
53+ If you modify a copy of the Library, and, in your modifications, a
54+facility refers to a function or data to be supplied by an Application
55+that uses the facility (other than as an argument passed when the
56+facility is invoked), then you may convey a copy of the modified
57+version:
58+
59+ a) under this License, provided that you make a good faith effort to
60+ ensure that, in the event an Application does not supply the
61+ function or data, the facility still operates, and performs
62+ whatever part of its purpose remains meaningful, or
63+
64+ b) under the GNU GPL, with none of the additional permissions of
65+ this License applicable to that copy.
66+
67+ 3. Object Code Incorporating Material from Library Header Files.
68+
69+ The object code form of an Application may incorporate material from
70+a header file that is part of the Library. You may convey such object
71+code under terms of your choice, provided that, if the incorporated
72+material is not limited to numerical parameters, data structure
73+layouts and accessors, or small macros, inline functions and templates
74+(ten or fewer lines in length), you do both of the following:
75+
76+ a) Give prominent notice with each copy of the object code that the
77+ Library is used in it and that the Library and its use are
78+ covered by this License.
79+
80+ b) Accompany the object code with a copy of the GNU GPL and this license
81+ document.
82+
83+ 4. Combined Works.
84+
85+ You may convey a Combined Work under terms of your choice that,
86+taken together, effectively do not restrict modification of the
87+portions of the Library contained in the Combined Work and reverse
88+engineering for debugging such modifications, if you also do each of
89+the following:
90+
91+ a) Give prominent notice with each copy of the Combined Work that
92+ the Library is used in it and that the Library and its use are
93+ covered by this License.
94+
95+ b) Accompany the Combined Work with a copy of the GNU GPL and this license
96+ document.
97+
98+ c) For a Combined Work that displays copyright notices during
99+ execution, include the copyright notice for the Library among
100+ these notices, as well as a reference directing the user to the
101+ copies of the GNU GPL and this license document.
102+
103+ d) Do one of the following:
104+
105+ 0) Convey the Minimal Corresponding Source under the terms of this
106+ License, and the Corresponding Application Code in a form
107+ suitable for, and under terms that permit, the user to
108+ recombine or relink the Application with a modified version of
109+ the Linked Version to produce a modified Combined Work, in the
110+ manner specified by section 6 of the GNU GPL for conveying
111+ Corresponding Source.
112+
113+ 1) Use a suitable shared library mechanism for linking with the
114+ Library. A suitable mechanism is one that (a) uses at run time
115+ a copy of the Library already present on the user's computer
116+ system, and (b) will operate properly with a modified version
117+ of the Library that is interface-compatible with the Linked
118+ Version.
119+
120+ e) Provide Installation Information, but only if you would otherwise
121+ be required to provide such information under section 6 of the
122+ GNU GPL, and only to the extent that such information is
123+ necessary to install and execute a modified version of the
124+ Combined Work produced by recombining or relinking the
125+ Application with a modified version of the Linked Version. (If
126+ you use option 4d0, the Installation Information must accompany
127+ the Minimal Corresponding Source and Corresponding Application
128+ Code. If you use option 4d1, you must provide the Installation
129+ Information in the manner specified by section 6 of the GNU GPL
130+ for conveying Corresponding Source.)
131+
132+ 5. Combined Libraries.
133+
134+ You may place library facilities that are a work based on the
135+Library side by side in a single library together with other library
136+facilities that are not Applications and are not covered by this
137+License, and convey such a combined library under terms of your
138+choice, if you do both of the following:
139+
140+ a) Accompany the combined library with a copy of the same work based
141+ on the Library, uncombined with any other library facilities,
142+ conveyed under the terms of this License.
143+
144+ b) Give prominent notice with the combined library that part of it
145+ is a work based on the Library, and explaining where to find the
146+ accompanying uncombined form of the same work.
147+
148+ 6. Revised Versions of the GNU Lesser General Public License.
149+
150+ The Free Software Foundation may publish revised and/or new versions
151+of the GNU Lesser General Public License from time to time. Such new
152+versions will be similar in spirit to the present version, but may
153+differ in detail to address new problems or concerns.
154+
155+ Each version is given a distinguishing version number. If the
156+Library as you received it specifies that a certain numbered version
157+of the GNU Lesser General Public License "or any later version"
158+applies to it, you have the option of following the terms and
159+conditions either of that published version or of any later version
160+published by the Free Software Foundation. If the Library as you
161+received it does not specify a version number of the GNU Lesser
162+General Public License, you may choose any version of the GNU Lesser
163+General Public License ever published by the Free Software Foundation.
164+
165+ If the Library as you received it specifies that a proxy can decide
166+whether future versions of the GNU Lesser General Public License shall
167+apply, that proxy's public statement of acceptance of any version is
168+permanent authorization for you to choose that version for the
169+Library.
170
171=== added file 'COPYING.txt'
172--- COPYING.txt 1970-01-01 00:00:00 +0000
173+++ COPYING.txt 2012-07-18 20:53:21 +0000
174@@ -0,0 +1,674 @@
175+GNU GENERAL PUBLIC LICENSE
176+ Version 3, 29 June 2007
177+
178+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
179+ Everyone is permitted to copy and distribute verbatim copies
180+ of this license document, but changing it is not allowed.
181+
182+ Preamble
183+
184+ The GNU General Public License is a free, copyleft license for
185+software and other kinds of works.
186+
187+ The licenses for most software and other practical works are designed
188+to take away your freedom to share and change the works. By contrast,
189+the GNU General Public License is intended to guarantee your freedom to
190+share and change all versions of a program--to make sure it remains free
191+software for all its users. We, the Free Software Foundation, use the
192+GNU General Public License for most of our software; it applies also to
193+any other work released this way by its authors. You can apply it to
194+your programs, too.
195+
196+ When we speak of free software, we are referring to freedom, not
197+price. Our General Public Licenses are designed to make sure that you
198+have the freedom to distribute copies of free software (and charge for
199+them if you wish), that you receive source code or can get it if you
200+want it, that you can change the software or use pieces of it in new
201+free programs, and that you know you can do these things.
202+
203+ To protect your rights, we need to prevent others from denying you
204+these rights or asking you to surrender the rights. Therefore, you have
205+certain responsibilities if you distribute copies of the software, or if
206+you modify it: responsibilities to respect the freedom of others.
207+
208+ For example, if you distribute copies of such a program, whether
209+gratis or for a fee, you must pass on to the recipients the same
210+freedoms that you received. You must make sure that they, too, receive
211+or can get the source code. And you must show them these terms so they
212+know their rights.
213+
214+ Developers that use the GNU GPL protect your rights with two steps:
215+(1) assert copyright on the software, and (2) offer you this License
216+giving you legal permission to copy, distribute and/or modify it.
217+
218+ For the developers' and authors' protection, the GPL clearly explains
219+that there is no warranty for this free software. For both users' and
220+authors' sake, the GPL requires that modified versions be marked as
221+changed, so that their problems will not be attributed erroneously to
222+authors of previous versions.
223+
224+ Some devices are designed to deny users access to install or run
225+modified versions of the software inside them, although the manufacturer
226+can do so. This is fundamentally incompatible with the aim of
227+protecting users' freedom to change the software. The systematic
228+pattern of such abuse occurs in the area of products for individuals to
229+use, which is precisely where it is most unacceptable. Therefore, we
230+have designed this version of the GPL to prohibit the practice for those
231+products. If such problems arise substantially in other domains, we
232+stand ready to extend this provision to those domains in future versions
233+of the GPL, as needed to protect the freedom of users.
234+
235+ Finally, every program is threatened constantly by software patents.
236+States should not allow patents to restrict development and use of
237+software on general-purpose computers, but in those that do, we wish to
238+avoid the special danger that patents applied to a free program could
239+make it effectively proprietary. To prevent this, the GPL assures that
240+patents cannot be used to render the program non-free.
241+
242+ The precise terms and conditions for copying, distribution and
243+modification follow.
244+
245+ TERMS AND CONDITIONS
246+
247+ 0. Definitions.
248+
249+ "This License" refers to version 3 of the GNU General Public License.
250+
251+ "Copyright" also means copyright-like laws that apply to other kinds of
252+works, such as semiconductor masks.
253+
254+ "The Program" refers to any copyrightable work licensed under this
255+License. Each licensee is addressed as "you". "Licensees" and
256+"recipients" may be individuals or organizations.
257+
258+ To "modify" a work means to copy from or adapt all or part of the work
259+in a fashion requiring copyright permission, other than the making of an
260+exact copy. The resulting work is called a "modified version" of the
261+earlier work or a work "based on" the earlier work.
262+
263+ A "covered work" means either the unmodified Program or a work based
264+on the Program.
265+
266+ To "propagate" a work means to do anything with it that, without
267+permission, would make you directly or secondarily liable for
268+infringement under applicable copyright law, except executing it on a
269+computer or modifying a private copy. Propagation includes copying,
270+distribution (with or without modification), making available to the
271+public, and in some countries other activities as well.
272+
273+ To "convey" a work means any kind of propagation that enables other
274+parties to make or receive copies. Mere interaction with a user through
275+a computer network, with no transfer of a copy, is not conveying.
276+
277+ An interactive user interface displays "Appropriate Legal Notices"
278+to the extent that it includes a convenient and prominently visible
279+feature that (1) displays an appropriate copyright notice, and (2)
280+tells the user that there is no warranty for the work (except to the
281+extent that warranties are provided), that licensees may convey the
282+work under this License, and how to view a copy of this License. If
283+the interface presents a list of user commands or options, such as a
284+menu, a prominent item in the list meets this criterion.
285+
286+ 1. Source Code.
287+
288+ The "source code" for a work means the preferred form of the work
289+for making modifications to it. "Object code" means any non-source
290+form of a work.
291+
292+ A "Standard Interface" means an interface that either is an official
293+standard defined by a recognized standards body, or, in the case of
294+interfaces specified for a particular programming language, one that
295+is widely used among developers working in that language.
296+
297+ The "System Libraries" of an executable work include anything, other
298+than the work as a whole, that (a) is included in the normal form of
299+packaging a Major Component, but which is not part of that Major
300+Component, and (b) serves only to enable use of the work with that
301+Major Component, or to implement a Standard Interface for which an
302+implementation is available to the public in source code form. A
303+"Major Component", in this context, means a major essential component
304+(kernel, window system, and so on) of the specific operating system
305+(if any) on which the executable work runs, or a compiler used to
306+produce the work, or an object code interpreter used to run it.
307+
308+ The "Corresponding Source" for a work in object code form means all
309+the source code needed to generate, install, and (for an executable
310+work) run the object code and to modify the work, including scripts to
311+control those activities. However, it does not include the work's
312+System Libraries, or general-purpose tools or generally available free
313+programs which are used unmodified in performing those activities but
314+which are not part of the work. For example, Corresponding Source
315+includes interface definition files associated with source files for
316+the work, and the source code for shared libraries and dynamically
317+linked subprograms that the work is specifically designed to require,
318+such as by intimate data communication or control flow between those
319+subprograms and other parts of the work.
320+
321+ The Corresponding Source need not include anything that users
322+can regenerate automatically from other parts of the Corresponding
323+Source.
324+
325+ The Corresponding Source for a work in source code form is that
326+same work.
327+
328+ 2. Basic Permissions.
329+
330+ All rights granted under this License are granted for the term of
331+copyright on the Program, and are irrevocable provided the stated
332+conditions are met. This License explicitly affirms your unlimited
333+permission to run the unmodified Program. The output from running a
334+covered work is covered by this License only if the output, given its
335+content, constitutes a covered work. This License acknowledges your
336+rights of fair use or other equivalent, as provided by copyright law.
337+
338+ You may make, run and propagate covered works that you do not
339+convey, without conditions so long as your license otherwise remains
340+in force. You may convey covered works to others for the sole purpose
341+of having them make modifications exclusively for you, or provide you
342+with facilities for running those works, provided that you comply with
343+the terms of this License in conveying all material for which you do
344+not control copyright. Those thus making or running the covered works
345+for you must do so exclusively on your behalf, under your direction
346+and control, on terms that prohibit them from making any copies of
347+your copyrighted material outside their relationship with you.
348+
349+ Conveying under any other circumstances is permitted solely under
350+the conditions stated below. Sublicensing is not allowed; section 10
351+makes it unnecessary.
352+
353+ 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
354+
355+ No covered work shall be deemed part of an effective technological
356+measure under any applicable law fulfilling obligations under article
357+11 of the WIPO copyright treaty adopted on 20 December 1996, or
358+similar laws prohibiting or restricting circumvention of such
359+measures.
360+
361+ When you convey a covered work, you waive any legal power to forbid
362+circumvention of technological measures to the extent such circumvention
363+is effected by exercising rights under this License with respect to
364+the covered work, and you disclaim any intention to limit operation or
365+modification of the work as a means of enforcing, against the work's
366+users, your or third parties' legal rights to forbid circumvention of
367+technological measures.
368+
369+ 4. Conveying Verbatim Copies.
370+
371+ You may convey verbatim copies of the Program's source code as you
372+receive it, in any medium, provided that you conspicuously and
373+appropriately publish on each copy an appropriate copyright notice;
374+keep intact all notices stating that this License and any
375+non-permissive terms added in accord with section 7 apply to the code;
376+keep intact all notices of the absence of any warranty; and give all
377+recipients a copy of this License along with the Program.
378+
379+ You may charge any price or no price for each copy that you convey,
380+and you may offer support or warranty protection for a fee.
381+
382+ 5. Conveying Modified Source Versions.
383+
384+ You may convey a work based on the Program, or the modifications to
385+produce it from the Program, in the form of source code under the
386+terms of section 4, provided that you also meet all of these conditions:
387+
388+ a) The work must carry prominent notices stating that you modified
389+ it, and giving a relevant date.
390+
391+ b) The work must carry prominent notices stating that it is
392+ released under this License and any conditions added under section
393+ 7. This requirement modifies the requirement in section 4 to
394+ "keep intact all notices".
395+
396+ c) You must license the entire work, as a whole, under this
397+ License to anyone who comes into possession of a copy. This
398+ License will therefore apply, along with any applicable section 7
399+ additional terms, to the whole of the work, and all its parts,
400+ regardless of how they are packaged. This License gives no
401+ permission to license the work in any other way, but it does not
402+ invalidate such permission if you have separately received it.
403+
404+ d) If the work has interactive user interfaces, each must display
405+ Appropriate Legal Notices; however, if the Program has interactive
406+ interfaces that do not display Appropriate Legal Notices, your
407+ work need not make them do so.
408+
409+ A compilation of a covered work with other separate and independent
410+works, which are not by their nature extensions of the covered work,
411+and which are not combined with it such as to form a larger program,
412+in or on a volume of a storage or distribution medium, is called an
413+"aggregate" if the compilation and its resulting copyright are not
414+used to limit the access or legal rights of the compilation's users
415+beyond what the individual works permit. Inclusion of a covered work
416+in an aggregate does not cause this License to apply to the other
417+parts of the aggregate.
418+
419+ 6. Conveying Non-Source Forms.
420+
421+ You may convey a covered work in object code form under the terms
422+of sections 4 and 5, provided that you also convey the
423+machine-readable Corresponding Source under the terms of this License,
424+in one of these ways:
425+
426+ a) Convey the object code in, or embodied in, a physical product
427+ (including a physical distribution medium), accompanied by the
428+ Corresponding Source fixed on a durable physical medium
429+ customarily used for software interchange.
430+
431+ b) Convey the object code in, or embodied in, a physical product
432+ (including a physical distribution medium), accompanied by a
433+ written offer, valid for at least three years and valid for as
434+ long as you offer spare parts or customer support for that product
435+ model, to give anyone who possesses the object code either (1) a
436+ copy of the Corresponding Source for all the software in the
437+ product that is covered by this License, on a durable physical
438+ medium customarily used for software interchange, for a price no
439+ more than your reasonable cost of physically performing this
440+ conveying of source, or (2) access to copy the
441+ Corresponding Source from a network server at no charge.
442+
443+ c) Convey individual copies of the object code with a copy of the
444+ written offer to provide the Corresponding Source. This
445+ alternative is allowed only occasionally and noncommercially, and
446+ only if you received the object code with such an offer, in accord
447+ with subsection 6b.
448+
449+ d) Convey the object code by offering access from a designated
450+ place (gratis or for a charge), and offer equivalent access to the
451+ Corresponding Source in the same way through the same place at no
452+ further charge. You need not require recipients to copy the
453+ Corresponding Source along with the object code. If the place to
454+ copy the object code is a network server, the Corresponding Source
455+ may be on a different server (operated by you or a third party)
456+ that supports equivalent copying facilities, provided you maintain
457+ clear directions next to the object code saying where to find the
458+ Corresponding Source. Regardless of what server hosts the
459+ Corresponding Source, you remain obligated to ensure that it is
460+ available for as long as needed to satisfy these requirements.
461+
462+ e) Convey the object code using peer-to-peer transmission, provided
463+ you inform other peers where the object code and Corresponding
464+ Source of the work are being offered to the general public at no
465+ charge under subsection 6d.
466+
467+ A separable portion of the object code, whose source code is excluded
468+from the Corresponding Source as a System Library, need not be
469+included in conveying the object code work.
470+
471+ A "User Product" is either (1) a "consumer product", which means any
472+tangible personal property which is normally used for personal, family,
473+or household purposes, or (2) anything designed or sold for incorporation
474+into a dwelling. In determining whether a product is a consumer product,
475+doubtful cases shall be resolved in favor of coverage. For a particular
476+product received by a particular user, "normally used" refers to a
477+typical or common use of that class of product, regardless of the status
478+of the particular user or of the way in which the particular user
479+actually uses, or expects or is expected to use, the product. A product
480+is a consumer product regardless of whether the product has substantial
481+commercial, industrial or non-consumer uses, unless such uses represent
482+the only significant mode of use of the product.
483+
484+ "Installation Information" for a User Product means any methods,
485+procedures, authorization keys, or other information required to install
486+and execute modified versions of a covered work in that User Product from
487+a modified version of its Corresponding Source. The information must
488+suffice to ensure that the continued functioning of the modified object
489+code is in no case prevented or interfered with solely because
490+modification has been made.
491+
492+ If you convey an object code work under this section in, or with, or
493+specifically for use in, a User Product, and the conveying occurs as
494+part of a transaction in which the right of possession and use of the
495+User Product is transferred to the recipient in perpetuity or for a
496+fixed term (regardless of how the transaction is characterized), the
497+Corresponding Source conveyed under this section must be accompanied
498+by the Installation Information. But this requirement does not apply
499+if neither you nor any third party retains the ability to install
500+modified object code on the User Product (for example, the work has
501+been installed in ROM).
502+
503+ The requirement to provide Installation Information does not include a
504+requirement to continue to provide support service, warranty, or updates
505+for a work that has been modified or installed by the recipient, or for
506+the User Product in which it has been modified or installed. Access to a
507+network may be denied when the modification itself materially and
508+adversely affects the operation of the network or violates the rules and
509+protocols for communication across the network.
510+
511+ Corresponding Source conveyed, and Installation Information provided,
512+in accord with this section must be in a format that is publicly
513+documented (and with an implementation available to the public in
514+source code form), and must require no special password or key for
515+unpacking, reading or copying.
516+
517+ 7. Additional Terms.
518+
519+ "Additional permissions" are terms that supplement the terms of this
520+License by making exceptions from one or more of its conditions.
521+Additional permissions that are applicable to the entire Program shall
522+be treated as though they were included in this License, to the extent
523+that they are valid under applicable law. If additional permissions
524+apply only to part of the Program, that part may be used separately
525+under those permissions, but the entire Program remains governed by
526+this License without regard to the additional permissions.
527+
528+ When you convey a copy of a covered work, you may at your option
529+remove any additional permissions from that copy, or from any part of
530+it. (Additional permissions may be written to require their own
531+removal in certain cases when you modify the work.) You may place
532+additional permissions on material, added by you to a covered work,
533+for which you have or can give appropriate copyright permission.
534+
535+ Notwithstanding any other provision of this License, for material you
536+add to a covered work, you may (if authorized by the copyright holders of
537+that material) supplement the terms of this License with terms:
538+
539+ a) Disclaiming warranty or limiting liability differently from the
540+ terms of sections 15 and 16 of this License; or
541+
542+ b) Requiring preservation of specified reasonable legal notices or
543+ author attributions in that material or in the Appropriate Legal
544+ Notices displayed by works containing it; or
545+
546+ c) Prohibiting misrepresentation of the origin of that material, or
547+ requiring that modified versions of such material be marked in
548+ reasonable ways as different from the original version; or
549+
550+ d) Limiting the use for publicity purposes of names of licensors or
551+ authors of the material; or
552+
553+ e) Declining to grant rights under trademark law for use of some
554+ trade names, trademarks, or service marks; or
555+
556+ f) Requiring indemnification of licensors and authors of that
557+ material by anyone who conveys the material (or modified versions of
558+ it) with contractual assumptions of liability to the recipient, for
559+ any liability that these contractual assumptions directly impose on
560+ those licensors and authors.
561+
562+ All other non-permissive additional terms are considered "further
563+restrictions" within the meaning of section 10. If the Program as you
564+received it, or any part of it, contains a notice stating that it is
565+governed by this License along with a term that is a further
566+restriction, you may remove that term. If a license document contains
567+a further restriction but permits relicensing or conveying under this
568+License, you may add to a covered work material governed by the terms
569+of that license document, provided that the further restriction does
570+not survive such relicensing or conveying.
571+
572+ If you add terms to a covered work in accord with this section, you
573+must place, in the relevant source files, a statement of the
574+additional terms that apply to those files, or a notice indicating
575+where to find the applicable terms.
576+
577+ Additional terms, permissive or non-permissive, may be stated in the
578+form of a separately written license, or stated as exceptions;
579+the above requirements apply either way.
580+
581+ 8. Termination.
582+
583+ You may not propagate or modify a covered work except as expressly
584+provided under this License. Any attempt otherwise to propagate or
585+modify it is void, and will automatically terminate your rights under
586+this License (including any patent licenses granted under the third
587+paragraph of section 11).
588+
589+ However, if you cease all violation of this License, then your
590+license from a particular copyright holder is reinstated (a)
591+provisionally, unless and until the copyright holder explicitly and
592+finally terminates your license, and (b) permanently, if the copyright
593+holder fails to notify you of the violation by some reasonable means
594+prior to 60 days after the cessation.
595+
596+ Moreover, your license from a particular copyright holder is
597+reinstated permanently if the copyright holder notifies you of the
598+violation by some reasonable means, this is the first time you have
599+received notice of violation of this License (for any work) from that
600+copyright holder, and you cure the violation prior to 30 days after
601+your receipt of the notice.
602+
603+ Termination of your rights under this section does not terminate the
604+licenses of parties who have received copies or rights from you under
605+this License. If your rights have been terminated and not permanently
606+reinstated, you do not qualify to receive new licenses for the same
607+material under section 10.
608+
609+ 9. Acceptance Not Required for Having Copies.
610+
611+ You are not required to accept this License in order to receive or
612+run a copy of the Program. Ancillary propagation of a covered work
613+occurring solely as a consequence of using peer-to-peer transmission
614+to receive a copy likewise does not require acceptance. However,
615+nothing other than this License grants you permission to propagate or
616+modify any covered work. These actions infringe copyright if you do
617+not accept this License. Therefore, by modifying or propagating a
618+covered work, you indicate your acceptance of this License to do so.
619+
620+ 10. Automatic Licensing of Downstream Recipients.
621+
622+ Each time you convey a covered work, the recipient automatically
623+receives a license from the original licensors, to run, modify and
624+propagate that work, subject to this License. You are not responsible
625+for enforcing compliance by third parties with this License.
626+
627+ An "entity transaction" is a transaction transferring control of an
628+organization, or substantially all assets of one, or subdividing an
629+organization, or merging organizations. If propagation of a covered
630+work results from an entity transaction, each party to that
631+transaction who receives a copy of the work also receives whatever
632+licenses to the work the party's predecessor in interest had or could
633+give under the previous paragraph, plus a right to possession of the
634+Corresponding Source of the work from the predecessor in interest, if
635+the predecessor has it or can get it with reasonable efforts.
636+
637+ You may not impose any further restrictions on the exercise of the
638+rights granted or affirmed under this License. For example, you may
639+not impose a license fee, royalty, or other charge for exercise of
640+rights granted under this License, and you may not initiate litigation
641+(including a cross-claim or counterclaim in a lawsuit) alleging that
642+any patent claim is infringed by making, using, selling, offering for
643+sale, or importing the Program or any portion of it.
644+
645+ 11. Patents.
646+
647+ A "contributor" is a copyright holder who authorizes use under this
648+License of the Program or a work on which the Program is based. The
649+work thus licensed is called the contributor's "contributor version".
650+
651+ A contributor's "essential patent claims" are all patent claims
652+owned or controlled by the contributor, whether already acquired or
653+hereafter acquired, that would be infringed by some manner, permitted
654+by this License, of making, using, or selling its contributor version,
655+but do not include claims that would be infringed only as a
656+consequence of further modification of the contributor version. For
657+purposes of this definition, "control" includes the right to grant
658+patent sublicenses in a manner consistent with the requirements of
659+this License.
660+
661+ Each contributor grants you a non-exclusive, worldwide, royalty-free
662+patent license under the contributor's essential patent claims, to
663+make, use, sell, offer for sale, import and otherwise run, modify and
664+propagate the contents of its contributor version.
665+
666+ In the following three paragraphs, a "patent license" is any express
667+agreement or commitment, however denominated, not to enforce a patent
668+(such as an express permission to practice a patent or covenant not to
669+sue for patent infringement). To "grant" such a patent license to a
670+party means to make such an agreement or commitment not to enforce a
671+patent against the party.
672+
673+ If you convey a covered work, knowingly relying on a patent license,
674+and the Corresponding Source of the work is not available for anyone
675+to copy, free of charge and under the terms of this License, through a
676+publicly available network server or other readily accessible means,
677+then you must either (1) cause the Corresponding Source to be so
678+available, or (2) arrange to deprive yourself of the benefit of the
679+patent license for this particular work, or (3) arrange, in a manner
680+consistent with the requirements of this License, to extend the patent
681+license to downstream recipients. "Knowingly relying" means you have
682+actual knowledge that, but for the patent license, your conveying the
683+covered work in a country, or your recipient's use of the covered work
684+in a country, would infringe one or more identifiable patents in that
685+country that you have reason to believe are valid.
686+
687+ If, pursuant to or in connection with a single transaction or
688+arrangement, you convey, or propagate by procuring conveyance of, a
689+covered work, and grant a patent license to some of the parties
690+receiving the covered work authorizing them to use, propagate, modify
691+or convey a specific copy of the covered work, then the patent license
692+you grant is automatically extended to all recipients of the covered
693+work and works based on it.
694+
695+ A patent license is "discriminatory" if it does not include within
696+the scope of its coverage, prohibits the exercise of, or is
697+conditioned on the non-exercise of one or more of the rights that are
698+specifically granted under this License. You may not convey a covered
699+work if you are a party to an arrangement with a third party that is
700+in the business of distributing software, under which you make payment
701+to the third party based on the extent of your activity of conveying
702+the work, and under which the third party grants, to any of the
703+parties who would receive the covered work from you, a discriminatory
704+patent license (a) in connection with copies of the covered work
705+conveyed by you (or copies made from those copies), or (b) primarily
706+for and in connection with specific products or compilations that
707+contain the covered work, unless you entered into that arrangement,
708+or that patent license was granted, prior to 28 March 2007.
709+
710+ Nothing in this License shall be construed as excluding or limiting
711+any implied license or other defenses to infringement that may
712+otherwise be available to you under applicable patent law.
713+
714+ 12. No Surrender of Others' Freedom.
715+
716+ If conditions are imposed on you (whether by court order, agreement or
717+otherwise) that contradict the conditions of this License, they do not
718+excuse you from the conditions of this License. If you cannot convey a
719+covered work so as to satisfy simultaneously your obligations under this
720+License and any other pertinent obligations, then as a consequence you may
721+not convey it at all. For example, if you agree to terms that obligate you
722+to collect a royalty for further conveying from those to whom you convey
723+the Program, the only way you could satisfy both those terms and this
724+License would be to refrain entirely from conveying the Program.
725+
726+ 13. Use with the GNU Affero General Public License.
727+
728+ Notwithstanding any other provision of this License, you have
729+permission to link or combine any covered work with a work licensed
730+under version 3 of the GNU Affero General Public License into a single
731+combined work, and to convey the resulting work. The terms of this
732+License will continue to apply to the part which is the covered work,
733+but the special requirements of the GNU Affero General Public License,
734+section 13, concerning interaction through a network will apply to the
735+combination as such.
736+
737+ 14. Revised Versions of this License.
738+
739+ The Free Software Foundation may publish revised and/or new versions of
740+the GNU General Public License from time to time. Such new versions will
741+be similar in spirit to the present version, but may differ in detail to
742+address new problems or concerns.
743+
744+ Each version is given a distinguishing version number. If the
745+Program specifies that a certain numbered version of the GNU General
746+Public License "or any later version" applies to it, you have the
747+option of following the terms and conditions either of that numbered
748+version or of any later version published by the Free Software
749+Foundation. If the Program does not specify a version number of the
750+GNU General Public License, you may choose any version ever published
751+by the Free Software Foundation.
752+
753+ If the Program specifies that a proxy can decide which future
754+versions of the GNU General Public License can be used, that proxy's
755+public statement of acceptance of a version permanently authorizes you
756+to choose that version for the Program.
757+
758+ Later license versions may give you additional or different
759+permissions. However, no additional obligations are imposed on any
760+author or copyright holder as a result of your choosing to follow a
761+later version.
762+
763+ 15. Disclaimer of Warranty.
764+
765+ THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
766+APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
767+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
768+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
769+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
770+PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
771+IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
772+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
773+
774+ 16. Limitation of Liability.
775+
776+ IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
777+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
778+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
779+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
780+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
781+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
782+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
783+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
784+SUCH DAMAGES.
785+
786+ 17. Interpretation of Sections 15 and 16.
787+
788+ If the disclaimer of warranty and limitation of liability provided
789+above cannot be given local legal effect according to their terms,
790+reviewing courts shall apply local law that most closely approximates
791+an absolute waiver of all civil liability in connection with the
792+Program, unless a warranty or assumption of liability accompanies a
793+copy of the Program in return for a fee.
794+
795+ END OF TERMS AND CONDITIONS
796+
797+ How to Apply These Terms to Your New Programs
798+
799+ If you develop a new program, and you want it to be of the greatest
800+possible use to the public, the best way to achieve this is to make it
801+free software which everyone can redistribute and change under these terms.
802+
803+ To do so, attach the following notices to the program. It is safest
804+to attach them to the start of each source file to most effectively
805+state the exclusion of warranty; and each file should have at least
806+the "copyright" line and a pointer to where the full notice is found.
807+
808+ <one line to give the program's name and a brief idea of what it does.>
809+ Copyright (C) <year> <name of author>
810+
811+ This program is free software: you can redistribute it and/or modify
812+ it under the terms of the GNU General Public License as published by
813+ the Free Software Foundation, either version 3 of the License, or
814+ (at your option) any later version.
815+
816+ This program is distributed in the hope that it will be useful,
817+ but WITHOUT ANY WARRANTY; without even the implied warranty of
818+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
819+ GNU General Public License for more details.
820+
821+ You should have received a copy of the GNU General Public License
822+ along with this program. If not, see <http://www.gnu.org/licenses/>.
823+
824+Also add information on how to contact you by electronic and paper mail.
825+
826+ If the program does terminal interaction, make it output a short
827+notice like this when it starts in an interactive mode:
828+
829+ <program> Copyright (C) <year> <name of author>
830+ This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
831+ This is free software, and you are welcome to redistribute it
832+ under certain conditions; type `show c' for details.
833+
834+The hypothetical commands `show w' and `show c' should show the appropriate
835+parts of the General Public License. Of course, your program's commands
836+might be different; for a GUI interface, you would use an "about box".
837+
838+ You should also get your employer (if you work as a programmer) or school,
839+if any, to sign a "copyright disclaimer" for the program, if necessary.
840+For more information on this, and how to apply and follow the GNU GPL, see
841+<http://www.gnu.org/licenses/>.
842+
843+ The GNU General Public License does not permit incorporating your program
844+into proprietary programs. If your program is a subroutine library, you
845+may consider it more useful to permit linking proprietary applications with
846+the library. If this is what you want to do, use the GNU Lesser General
847+Public License instead of this License. But first, please read
848+<http://www.gnu.org/philosophy/why-not-lgpl.html>.
849
850=== added file 'MANIFEST.in'
851--- MANIFEST.in 1970-01-01 00:00:00 +0000
852+++ MANIFEST.in 2012-07-18 20:53:21 +0000
853@@ -0,0 +1,1 @@
854+include COPYING.txt COPYING.LESSER.txt
855
856=== added file 'README.txt'
857--- README.txt 1970-01-01 00:00:00 +0000
858+++ README.txt 2012-07-18 20:53:21 +0000
859@@ -0,0 +1,49 @@
860+OVERVIEW
861+
862+HTD (HayzenTech Database) is a pure python library which acts as a simple
863+database engine, licenced under LGPLv3.
864+
865+For more information please refer to http://www.hayzentech.co.uk/htd
866+
867+
868+FEATURES
869+
870+- Intuitive API
871+ The API is very easy to understand as has been made consistant through all
872+ the functions.
873+- Variable-length records
874+ When the data is stored in the table it only takes up the space required.
875+- Thread safety
876+ Multiple threads can use the same module at the same time safely.
877+- File locking
878+ So multiple processes can use the same table at the same time.
879+- Add your own file or data types
880+ HTD has the ability for the developer to ‘register’ a file or data type
881+ into HTD.
882+- No Configuration required
883+ If using the core library HTD doesn’t require any configuration, just
884+ import and start using HTD.
885+
886+
887+INSTALLING
888+
889+1. Open up terminal or command prompt and cd to the directory containing
890+ setup.py
891+
892+2. *NIX users run 'python setup.py install', where python is you target python
893+ interpreter. This command may need admin rights to run, for example in
894+ Ubuntu run 'sudo python setup.py install'.
895+
896+ Windows users run 'c:\python\python.exe setup.py install', where
897+ 'c:\python\python.exe' is you target python interpreter.
898+
899+3. Open you interpreter and the following code should run successfully.
900+ >>> import htdXY
901+ Where XY is the version of your HTD engine, for example version 3.0 is 30
902+
903+
904+DOCUMENTATION
905+
906+Documentation is avaliable at http://www.hayzentech.co.uk/htd/documentation.php
907+Refer to the package 'htd-core' and the series which you have downloaded.
908+
909
910=== added directory 'htd30'
911=== added file 'htd30/__init__.py'
912--- htd30/__init__.py 1970-01-01 00:00:00 +0000
913+++ htd30/__init__.py 2012-07-18 20:53:21 +0000
914@@ -0,0 +1,266 @@
915+#!/usr/bin/env python
916+'''
917+This file is part of HTD (HayzenTech Database)
918+
919+Copyright (C) 2011 - 2012 Andrew Hayzen
920+
921+HTD is free software: you can redistribute it and/or modify
922+it under the terms of the GNU Lesser General Public License as published by
923+the Free Software Foundation, either version 3 of the License, or
924+(at your option) any later version.
925+
926+HTD is distributed in the hope that it will be useful,
927+but WITHOUT ANY WARRANTY; without even the implied warranty of
928+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
929+GNU Lesser General Public License for more details.
930+
931+You should have received a copy of the GNU Lesser General Public License
932+along with HTD. If not, see <http://www.gnu.org/licenses/>
933+
934+This document conforms to the PEP 8 Style Guide
935+
936+Version 3.00 PRE-ALPHA-3
937+'''
938+
939+# Import built-in modules
940+from os import path as _path
941+
942+
943+#--------------------------------------------------------------------- Settings
944+__build_id__ = 300039
945+__version__ = 3.00
946+
947+
948+class _Config():
949+ debug = True
950+ errors = True
951+ force_sync = False
952+ tracing = False
953+
954+Config = _Config()
955+
956+
957+#------------------------------------------------------------------- Exceptions
958+class AuthenticationError(Exception):
959+ pass
960+
961+
962+class ConnectionError(Exception):
963+ pass
964+
965+
966+class VersionError(Exception):
967+ pass
968+
969+
970+#-------------------------------- Create containers for datatypes and filetypes
971+class _RegisterContainer():
972+ pass
973+
974+DataTypes = _RegisterContainer()
975+FileTypes = _RegisterContainer()
976+
977+
978+#--------------------------------------------------------------- Register Class
979+class _Register(object):
980+ def __init__(self):
981+ self.dtype_codes = {}
982+ self.dtypes = {}
983+ self._default_dtype = None
984+
985+ self.fhandlers = {}
986+ self._default_fhandler = None
987+
988+ @property
989+ def default_dtype(self):
990+ return self._default_dtype
991+
992+ @default_dtype.setter
993+ def default_dtype(self, dtype):
994+ if dtype.__name__ not in self.dtypes:
995+ raise ValueError("Default dtype must be a registered dtype.")
996+
997+ self._default_dtype = dtype.__name__
998+
999+ @property
1000+ def default_fhandler(self):
1001+ return self._default_fhandler
1002+
1003+ @default_fhandler.setter
1004+ def default_fhandler(self, fhandler):
1005+ if fhandler not in self.fhandlers:
1006+ raise ValueError("Default fhandler must be a registered fhandler.")
1007+
1008+ self._default_fhandler = fhandler
1009+
1010+ def register_dtype(self, dtype):
1011+ setattr(DataTypes, dtype.__name__, dtype())
1012+
1013+ self.dtype_codes[dtype.c] = dtype.__name__
1014+ self.dtypes[dtype.__name__] = dtype.c
1015+
1016+ return True
1017+
1018+ def register_fhandler(self, fhandler, is_location, is_file=None):
1019+ setattr(FileTypes, fhandler.__name__, fhandler)
1020+
1021+ self.fhandlers[fhandler] = {"is_loc": is_location, "is_file": is_file}
1022+
1023+ return True
1024+
1025+ def resolve_fhandler(self, location):
1026+ """ File Handler resolver - determins file type """
1027+
1028+ # Filter fhandlers by location
1029+ fhandlers = [f for f in self.fhandlers if
1030+ self.fhandlers[f]["is_loc"](location)]
1031+
1032+ # Filter fhandlers by file
1033+ output = []
1034+
1035+ for f in fhandlers:
1036+ # if file_format exists then uses File and Header standard
1037+ if f.file_format is not None:
1038+ if not _path.exists(location):
1039+ raise ConnectionError("No HTD file found (%s)" % location)
1040+
1041+ # Open file and get fhandler code
1042+ tmp = open(location, "rb")
1043+ tmp.seek(8)
1044+ if tmp.read(1) == f.file_format:
1045+ output.append(f)
1046+ tmp.close()
1047+
1048+ elif self.fhandlers[f]["is_file"] is not None:
1049+ if self.fhandlers[f]["is_file"](location):
1050+ output.append(f)
1051+
1052+ fhandlers = output
1053+
1054+ # Load fhandler
1055+ if len(fhandlers) == 0:
1056+ raise ConnectionError("Unknown File Format")
1057+ elif len(fhandlers) > 1:
1058+ raise ConnectionError("Could not determine File Format either (%s)"
1059+ % fhandlers)
1060+ else:
1061+ return fhandlers[0]
1062+
1063+Register = _Register()
1064+
1065+#---------------------------------------------------------------------- Imports
1066+
1067+# Load base objects
1068+from ._base import Progress
1069+
1070+from . import _base
1071+from . import _workers
1072+
1073+# Load HTD Objects
1074+from ._objects import Attribute, User, Connection as connect
1075+
1076+
1077+#------------------------------------------------------ Top level HTD functions
1078+@_base._trace()
1079+def create(name, location, user=None, identifier=None, htd_file=None,
1080+ login_user=None):
1081+ """
1082+ Create a new HTD File
1083+ To create a table parse the identifier for the table otherwise leave
1084+ the identifier blank to create a db
1085+ """
1086+
1087+ atts = {}
1088+
1089+ # Check for identifier
1090+ if identifier is not None:
1091+ # Convert single into a list
1092+ if not isinstance(identifier, list):
1093+ identifier = [identifier]
1094+
1095+ # Create ids
1096+ for k, i in enumerate(identifier):
1097+ atts[i.name] = i
1098+ atts[i.name]._id = k
1099+
1100+ # Loop through identifier(s)
1101+ for i in identifier:
1102+ # Error if not an Attribute object
1103+ if not isinstance(i, Attribute):
1104+ raise ValueError("Identifier(s) must be an Attribute object.")
1105+
1106+ table = None # Set TABLE mode
1107+ else:
1108+ # Create db attributes
1109+ name_att = Attribute(DataTypes.STRING, name="name")
1110+ name_att._id = 0
1111+ loc_att = Attribute(DataTypes.STRING)
1112+ loc_att._id = 1
1113+
1114+ # Load attribute and identifier
1115+ atts = {"name": name_att, "location": loc_att}
1116+ identifier = [name_att]
1117+ table = False # Set DB mode master file
1118+
1119+ # Check for user
1120+ if user is not None and user != []:
1121+ # Convert single into a list
1122+ if not isinstance(user, list):
1123+ user = [user]
1124+
1125+ # Store users for resorting
1126+ tmp = user[:]
1127+ user = {}
1128+
1129+ # Check 1 user has full access
1130+ acc = False
1131+
1132+ # Loop through user(s)
1133+ for i in tmp:
1134+ # Error if not a User Object
1135+ if not isinstance(i, User):
1136+ raise ValueError("User(s) must be an User object.")
1137+
1138+ user[i.username] = i
1139+
1140+ # Check for full access
1141+ if i.access == "roadx":
1142+ acc = True
1143+
1144+ # raise error if no access
1145+ if not acc:
1146+ raise ValueError("One User must have full access (roadx)")
1147+
1148+ # Check login user
1149+ if login_user is None:
1150+ # Check for multi users
1151+ if len(user) > 1:
1152+ raise ValueError("A login_user must be specified")
1153+ else:
1154+ # select only user
1155+ login_user = [user[i] for i in user][0]
1156+ else:
1157+ # Check login user is in users
1158+ if login_user not in user:
1159+ raise ValueError("login_user must be in users")
1160+ else:
1161+ login_user = user[login_user]
1162+ else:
1163+ # Load guest account
1164+ user = {"guest": User("guest", None)}
1165+ login_user = User("guest", None)
1166+
1167+ if htd_file is None:
1168+ htd_file = Register.default_fhandler
1169+
1170+ # Build Header
1171+ header = {"TAB": table, "ATT": atts, "USR": user, "KEY": identifier,
1172+ "VER": __version__}
1173+
1174+ # Load and set header
1175+ f = htd_file(_path.sep.join([location, name]), header)
1176+
1177+ # Return con obj
1178+ return connect(_path.sep.join([location, name]), login_user, file_obj=f)
1179+
1180+_base._t.names.extend(dir()) # Load methods/class etc into tracing
1181
1182=== added directory 'htd30/_base'
1183=== added file 'htd30/_base/__init__.py'
1184--- htd30/_base/__init__.py 1970-01-01 00:00:00 +0000
1185+++ htd30/_base/__init__.py 2012-07-18 20:53:21 +0000
1186@@ -0,0 +1,64 @@
1187+'''
1188+This file is part of HTD (HayzenTech Database)
1189+
1190+Copyright (C) 2011 - 2012 Andrew Hayzen
1191+
1192+HTD is free software: you can redistribute it and/or modify
1193+it under the terms of the GNU Lesser General Public License as published by
1194+the Free Software Foundation, either version 3 of the License, or
1195+(at your option) any later version.
1196+
1197+HTD is distributed in the hope that it will be useful,
1198+but WITHOUT ANY WARRANTY; without even the implied warranty of
1199+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1200+GNU Lesser General Public License for more details.
1201+
1202+You should have received a copy of the GNU Lesser General Public License
1203+along with HTD. If not, see <http://www.gnu.org/licenses/>
1204+'''
1205+
1206+from .file import File
1207+from .trace import Progress, ProgressItem, _t, _trace, _access_check
1208+from . import types
1209+
1210+from os import path
1211+import threading
1212+
1213+
1214+def convert_location(location, add):
1215+ folder = location.split(path.sep)
1216+ name = "".join([add, folder[-1]]) # create new filename
1217+
1218+ folder.pop() # remove name
1219+ folder.append(name)
1220+
1221+ return path.sep.join(folder)
1222+
1223+
1224+def location_is_file(location):
1225+ if not path.isabs(location):
1226+ return False
1227+
1228+ directory, name = path.split(location)
1229+
1230+ if not path.isdir(directory):
1231+ return False
1232+
1233+ return True
1234+
1235+
1236+class ThreadLock(object):
1237+ def __init__(self):
1238+ self._threading_lock = threading.Lock()
1239+
1240+ def _acquire(self):
1241+ """ Acquire threading lock """
1242+ return self._threading_lock.acquire()
1243+
1244+ def _release(self):
1245+ """ Release threading lock """
1246+ return self._threading_lock.release()
1247+
1248+ @property
1249+ def _t_locked(self):
1250+ return self._threading_lock.locked()
1251
1252=== added file 'htd30/_base/file.py'
1253--- htd30/_base/file.py 1970-01-01 00:00:00 +0000
1254+++ htd30/_base/file.py 2012-07-18 20:53:21 +0000
1255@@ -0,0 +1,177 @@
1256+'''
1257+This file is part of HTD (HayzenTech Database)
1258+
1259+Copyright (C) 2011 - 2012 Andrew Hayzen
1260+
1261+HTD is free software: you can redistribute it and/or modify
1262+it under the terms of the GNU Lesser General Public License as published by
1263+the Free Software Foundation, either version 3 of the License, or
1264+(at your option) any later version.
1265+
1266+HTD is distributed in the hope that it will be useful,
1267+but WITHOUT ANY WARRANTY; without even the implied warranty of
1268+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1269+GNU Lesser General Public License for more details.
1270+
1271+You should have received a copy of the GNU Lesser General Public License
1272+along with HTD. If not, see <http://www.gnu.org/licenses/>
1273+
1274+Base File for HTD
1275+Provides slicing and file locking support for files
1276+'''
1277+
1278+import os
1279+
1280+if os.name == "posix":
1281+ import fcntl
1282+elif os.name == "nt":
1283+ import win32con
1284+ import win32file
1285+ import pywintypes
1286+ __overlapped = pywintypes.OVERLAPPED()
1287+else:
1288+ print("HTD Warning: Unknown operating system, FileLock unavailable.")
1289+
1290+
1291+class File(object):
1292+ """ HTD File that provides slicing and file locking """
1293+ def __init__(self, location):
1294+ self.__file = open(location, "rb+")
1295+ self.__lkcur = False
1296+
1297+ # Add file methods to self
1298+ for i in [i for i in dir(self.__file) if '_' not in i and
1299+ i != "close"]:
1300+ setattr(self, i, getattr(self.__file, i))
1301+
1302+ def __delitem__(self, key):
1303+ # Delete Slice (MUST be a slice)
1304+ self.seek(key.stop)
1305+ text = self.read()
1306+ self.seek(key.start)
1307+ self.write(text)
1308+ self.truncate()
1309+ self.flush()
1310+
1311+ def __getitem__(self, key):
1312+ # Return slice (MUST be a slice)
1313+ self.seek(key.start, 0)
1314+ return self.read((key.stop - key.start))
1315+
1316+ def __len__(self):
1317+ self.seek(0, 2)
1318+ return self.tell()
1319+
1320+ def __setitem__(self, key, value):
1321+ # Slice (MUST be a slice)
1322+ i = key.start
1323+ e = key.stop
1324+ diff = e - i
1325+ if (diff != len(value)):
1326+ self.seek(e)
1327+ text = self.read()
1328+ self.seek(i)
1329+ self.write(value)
1330+ self.write(text)
1331+ self.truncate()
1332+ else:
1333+ self.seek(i)
1334+ self.write(value)
1335+ self.flush()
1336+
1337+ if os.name == "posix":
1338+ def _lock_reading(self):
1339+ """ Lock file for reading """
1340+
1341+ fcntl.flock(self.__file, fcntl.LOCK_SH | fcntl.LOCK_NB)
1342+ self.__lkcur = None
1343+
1344+ return True
1345+
1346+ elif os.name == "nt":
1347+ def _lock_reading(self):
1348+ """ Lock file for reading """
1349+
1350+ self._unlock() # NT requires unlock first
1351+ f = win32file._get_osfhandle(self.fileno())
1352+ try:
1353+ win32file.LockFileEx(f, 0 | win32con.LOCKFILE_FAIL_IMMEDIATELY,
1354+ 0, -0x10000, pywintypes.OVERLAPPED())
1355+ except pywintypes.error as exc_value:
1356+ if exc_value[0] == 33: # File locked exception
1357+ raise IOError("File is locked")
1358+ else:
1359+ raise IOError("Unknown Error")
1360+
1361+ self.__lkcur = None
1362+
1363+ return True
1364+
1365+ if os.name == "posix":
1366+ def _lock_writing(self):
1367+ """ Lock file for writing """
1368+
1369+ fcntl.flock(self.__file, fcntl.LOCK_EX | fcntl.LOCK_NB)
1370+ self.__lkcur = True
1371+
1372+ return True
1373+
1374+ elif os.name == "nt":
1375+ def _lock_writing(self):
1376+ """ Lock file for writing """
1377+
1378+ self._unlock() # NT requires unlock first
1379+ f = win32file._get_osfhandle(self.fileno())
1380+ try:
1381+ win32file.LockFileEx(f, win32con.LOCKFILE_EXCLUSIVE_LOCK |
1382+ win32con.LOCKFILE_FAIL_IMMEDIATELY, 0,
1383+ - 0x10000, pywintypes.OVERLAPPED())
1384+ except pywintypes.error as exc_value:
1385+ if exc_value[0] == 33: # File locked exception
1386+ raise IOError("File is locked")
1387+ else:
1388+ raise IOError("Unknown Error")
1389+
1390+ self.__lkcur = True
1391+
1392+ return True
1393+
1394+ if os.name == "posix":
1395+ def _unlock(self):
1396+ """ Unlock file """
1397+
1398+ fcntl.flock(self.__file, fcntl.LOCK_UN)
1399+ self.__lkcur = False
1400+
1401+ return True
1402+
1403+ elif os.name == "nt":
1404+ def _unlock(self):
1405+ """ Unlock file """
1406+
1407+ f = win32file._get_osfhandle(self.fileno())
1408+ try:
1409+ win32file.UnlockFileEx(f, 0, -0x10000, pywintypes.OVERLAPPED())
1410+ except pywintypes.error as exc_value:
1411+ if exc_value[0] == 158: # Don't raise already locked
1412+ pass
1413+ else:
1414+ raise
1415+
1416+ self.__lkcur = False
1417+
1418+ return True
1419+
1420+ def close(self):
1421+ self.flush()
1422+ try:
1423+ self._unlock()
1424+ except IOError:
1425+ pass
1426+ self.__file.close()
1427+
1428+ return True
1429+
1430+ @property
1431+ def lock_type(self):
1432+ return self.__lkcur
1433
1434=== added file 'htd30/_base/trace.py'
1435--- htd30/_base/trace.py 1970-01-01 00:00:00 +0000
1436+++ htd30/_base/trace.py 2012-07-18 20:53:21 +0000
1437@@ -0,0 +1,213 @@
1438+'''
1439+This file is part of HTD (HayzenTech Database)
1440+
1441+Copyright (C) 2011 - 2012 Andrew Hayzen
1442+
1443+HTD is free software: you can redistribute it and/or modify
1444+it under the terms of the GNU Lesser General Public License as published by
1445+the Free Software Foundation, either version 3 of the License, or
1446+(at your option) any later version.
1447+
1448+HTD is distributed in the hope that it will be useful,
1449+but WITHOUT ANY WARRANTY; without even the implied warranty of
1450+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1451+GNU Lesser General Public License for more details.
1452+
1453+You should have received a copy of the GNU Lesser General Public License
1454+along with HTD. If not, see <http://www.gnu.org/licenses/>
1455+
1456+Tracing for HTD
1457+Captures progress, logging, triggers, errors and permission levels
1458+'''
1459+
1460+from functools import wraps
1461+from .. import AuthenticationError, Config
1462+from copy import copy
1463+from threading import current_thread
1464+
1465+
1466+#-------------------------------------------------------------- Progress object
1467+class ProgressItem():
1468+ def __init__(self, thread, length, func):
1469+ self.thread = thread
1470+ self.length = length
1471+ self.func = func
1472+ self.counter = 0
1473+
1474+ def count(self, n=1):
1475+ self.counter += n
1476+ return True
1477+
1478+
1479+class Manage():
1480+ def __init__(self):
1481+ self.tasks = []
1482+ self.threads = {}
1483+
1484+ def add(self, progress):
1485+ if not isinstance(progress, ProgressItem):
1486+ raise ValueError("Must be a ProgressItem instance")
1487+
1488+ self.tasks.append(progress)
1489+
1490+ if progress.thread not in self.threads:
1491+ self.threads[progress.thread] = []
1492+
1493+ self.threads[progress.thread].append(progress)
1494+
1495+ return True
1496+
1497+ def progress(self):
1498+ """ Return all running tasks """
1499+ return [{"length":job.length, "count":job.counter, "name":job.func,
1500+ "fraction":job.counter / job.length, "thread":job.thread}
1501+ for job in copy(self.tasks)]
1502+
1503+ def remove(self, progress):
1504+ if not isinstance(progress, ProgressItem):
1505+ raise ValueError("Must be a ProgressItem instance")
1506+
1507+ try:
1508+ self.tasks.remove(progress)
1509+ self.threads[progress.thread].remove(progress)
1510+ except (ValueError, KeyError):
1511+ pass
1512+
1513+ return True
1514+
1515+ def thread_progress(self, thread):
1516+ """ Return the running tasks for a specific thread """
1517+ if thread not in self.threads:
1518+ raise KeyError("Thread does not exist")
1519+
1520+ return [{"length":job.length, "count":job.counter, "name":job.func,
1521+ "fraction":job.counter / job.length} for job in
1522+ copy(self.threads[thread])]
1523+
1524+Progress = Manage()
1525+
1526+
1527+class Tracing():
1528+ """
1529+ Tracing class that manages tracing of function calls
1530+ | Permission | Progress | Logging | Triggers |
1531+ """
1532+ def __init__(self):
1533+ self._perms = {"r": "read", "o": "overwrite", "a": "append",
1534+ "d": "delete", "x": "admin"}
1535+
1536+ self.names = ["Attributes", "Auth", "Connection", "Cursor"]
1537+
1538+ def access_check(self, level, module):
1539+ """ Run access test """
1540+
1541+ if level == "" or not hasattr(module, "_access"):
1542+ return True
1543+
1544+ perm = [i for i in level if i not in module._access]
1545+
1546+ if len(perm) == 0:
1547+ return True
1548+
1549+ err = ", ".join(['(%s - %s)' % (i, self._perms[i]) for i in perm])
1550+
1551+ err = 'User doesn\'t have to permission [%s]' % err
1552+
1553+ raise AuthenticationError(AuthenticationError, err) # Raise exception
1554+
1555+ return False
1556+
1557+ def trace(self, level=""):
1558+ """ Trace function """
1559+
1560+ def wrapper(func):
1561+ """ Wrapper function """
1562+
1563+ @wraps(func)
1564+ def callf(*args, **kwargs):
1565+ if len(args) > 0:
1566+ tmp = args[0]
1567+ else:
1568+ tmp = None
1569+
1570+ # Check that object isn't closed
1571+ if hasattr(tmp, "alive") and not tmp.alive:
1572+ return None
1573+
1574+ # Acquire lock
1575+ locked = False
1576+ if hasattr(tmp, "_t_locked") and not tmp._t_locked:
1577+ tmp._acquire()
1578+ locked = True
1579+
1580+ # Trace before
1581+ func_name = func.__name__
1582+
1583+ if len(args) > 0 and type(args[0]).__name__ in self.names:
1584+ func_name = ".".join([args[0].__class__.__name__,
1585+ func_name])
1586+
1587+ func_name = ".".join(["htd30", func_name])
1588+ calling = "Calling %s" % (func_name)
1589+
1590+ if Config.tracing:
1591+ # Build parameters
1592+ tmp = [str(i) for i in args]
1593+ tmp.extend(["%s=%s" % (i, str(kwargs[i])) for i in kwargs])
1594+ param = ", ".join(tmp)
1595+
1596+ print("HTD TRACE: %s(%s)" % (calling, param))
1597+
1598+ # Add to progress
1599+ prog = ProgressItem(current_thread(), 1, calling)
1600+
1601+ Progress.add(prog)
1602+
1603+ try:
1604+ self.access_check(level, tmp) # Run access check
1605+ result = func(*args, **kwargs) # Call the function
1606+ except Exception as e: # Catch errors
1607+ result = False
1608+
1609+ # run unlocked
1610+ if hasattr(tmp, "_file"):
1611+ if (hasattr(args[0]._file, "_t_locked") and
1612+ tmp._file._t_locked):
1613+ tmp._file.unlock()
1614+
1615+ if hasattr(tmp, "_t_locked") and tmp._t_locked:
1616+ tmp._release()
1617+
1618+ txt = "%s, %s (%s)" % (type(e).__name__, e, calling)
1619+
1620+ if Config.tracing:
1621+ print("HTD TRACE: %s" % txt)
1622+
1623+ if Config.debug:
1624+ print("HTD DEBUG: %s" % txt)
1625+
1626+ if Config.errors:
1627+ raise
1628+ finally:
1629+ prog.count()
1630+
1631+ if Config.tracing:
1632+ print("HTD TRACE: %s returned %s" % (func_name,
1633+ result))
1634+
1635+ Progress.remove(prog)
1636+
1637+ # Release lock
1638+ if hasattr(tmp, "_t_locked") and tmp._t_locked and locked:
1639+ tmp._release()
1640+
1641+ return result
1642+
1643+ return callf
1644+
1645+ return wrapper
1646+
1647+# Start tracing
1648+_t = Tracing()
1649+_trace = _t.trace
1650+_access_check = _t.access_check
1651
1652=== added file 'htd30/_base/types.py'
1653--- htd30/_base/types.py 1970-01-01 00:00:00 +0000
1654+++ htd30/_base/types.py 2012-07-18 20:53:21 +0000
1655@@ -0,0 +1,273 @@
1656+'''
1657+This file is part of HTD (HayzenTech Database)
1658+
1659+Copyright (C) 2011 - 2012 Andrew Hayzen
1660+
1661+HTD is free software: you can redistribute it and/or modify
1662+it under the terms of the GNU Lesser General Public License as published by
1663+the Free Software Foundation, either version 3 of the License, or
1664+(at your option) any later version.
1665+
1666+HTD is distributed in the hope that it will be useful,
1667+but WITHOUT ANY WARRANTY; without even the implied warranty of
1668+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1669+GNU Lesser General Public License for more details.
1670+
1671+You should have received a copy of the GNU Lesser General Public License
1672+along with HTD. If not, see <http://www.gnu.org/licenses/>
1673+
1674+HTD Data Types
1675+Data types and (un)sanisite functions are managed here
1676+'''
1677+
1678+from datetime import date, datetime, time, timedelta
1679+from .. import __version__, DataTypes, Register
1680+import sys
1681+
1682+
1683+#-------------------------------------------------------------------- HTD Types
1684+# C = identifier F = Type check, R = range, S = sanitise, U = unsanitise
1685+class Dtype(object):
1686+ def __repr__(self):
1687+ return ("<HTD%s Data Type (%s)>" %
1688+ (__version__, self.__class__.__name__))
1689+
1690+reg_dtype = Register.register_dtype # Register dtypes
1691+
1692+
1693+class BOOL(Dtype):
1694+ c = "\x01"
1695+
1696+ def f(self, d):
1697+ return type(d) is bool
1698+ r = False
1699+
1700+ def s(self, d):
1701+ if d:
1702+ return b"1"
1703+ else:
1704+ return b"0"
1705+
1706+ def u(self, d):
1707+ if d == b"1":
1708+ return True
1709+ else:
1710+ return False
1711+
1712+reg_dtype(BOOL)
1713+
1714+
1715+class BYTES(Dtype):
1716+ c = "\x02"
1717+
1718+ def f(self, d):
1719+ return type(d) is bytes
1720+ r = False
1721+ s = bytes
1722+ u = bytes
1723+
1724+reg_dtype(BYTES)
1725+
1726+
1727+class INTEGER(Dtype):
1728+ c = "\x04"
1729+
1730+ def f(self, d):
1731+ return type(d) is int
1732+ r = True
1733+
1734+ def s(self, d):
1735+ return str(d).encode("ISO-8859-1")
1736+ u = int
1737+
1738+reg_dtype(INTEGER)
1739+
1740+if sys.version_info < (3, 0):
1741+ class LONG(Dtype):
1742+ c = "\x05"
1743+
1744+ def f(self, d):
1745+ return type(d) is long
1746+ r = True
1747+
1748+ def s(self, d):
1749+ return str(d).encode("ISO-8859-1")
1750+ u = long
1751+
1752+ reg_dtype(LONG)
1753+
1754+
1755+class FLOAT(Dtype):
1756+ c = "\x08"
1757+
1758+ def f(self, d):
1759+ return type(d) is float
1760+ r = True
1761+
1762+ def s(self, d):
1763+ return str(d).encode("ISO-8859-1")
1764+ u = float
1765+
1766+reg_dtype(FLOAT)
1767+
1768+if sys.version_info < (3, 0):
1769+ class STRING(Dtype):
1770+ c = "\x11"
1771+
1772+ def f(self, d):
1773+ return type(d) is str
1774+ r = False
1775+ s = str
1776+ u = str
1777+
1778+ reg_dtype(STRING)
1779+
1780+ class UTF8(Dtype):
1781+ c = "\x14"
1782+
1783+ def f(self, d):
1784+ return type(d) is unicode
1785+ r = False
1786+
1787+ def s(self, raw):
1788+ return raw.encode("utf8")
1789+
1790+ def u(self, rawstr):
1791+ return rawstr.decode("utf8")
1792+
1793+ reg_dtype(UTF8)
1794+
1795+ class UTF16(Dtype):
1796+ c = "\x15"
1797+
1798+ def f(self, d):
1799+ return type(d) is unicode
1800+ r = False
1801+
1802+ def s(self, raw):
1803+ return raw.encode("utf16")
1804+
1805+ def u(self, rawstr):
1806+ return rawstr.decode("utf16")
1807+
1808+ reg_dtype(UTF16)
1809+
1810+ class UTF32(Dtype):
1811+ c = "\x16"
1812+
1813+ def f(self, d):
1814+ return type(d) is unicode
1815+ r = False
1816+
1817+ def s(self, raw):
1818+ return raw.encode("utf32")
1819+
1820+ def u(self, rawstr):
1821+ return rawstr.decode("utf32")
1822+
1823+ reg_dtype(UTF32)
1824+else:
1825+ class STRING(Dtype):
1826+ c = "\x12"
1827+
1828+ def f(self, d):
1829+ return type(d) is str
1830+ r = False
1831+
1832+ def s(self, raw):
1833+ return raw.encode("ISO-8859-1")
1834+
1835+ def u(self, rawstr):
1836+ return rawstr.decode("ISO-8859-1")
1837+
1838+ reg_dtype(STRING)
1839+
1840+
1841+class DATE(Dtype):
1842+ c = "\x21"
1843+
1844+ def f(self, d):
1845+ return type(d) is date
1846+ r = False
1847+
1848+ def s(self, raw):
1849+ return raw.isoformat().encode("ISO-8859-1")
1850+
1851+ def u(self, rawstr):
1852+ data = rawstr.decode("ISO-8859-1").split("-")
1853+ return date(int(data[0]), int(data[1]), int(data[2]))
1854+
1855+reg_dtype(DATE)
1856+
1857+
1858+class TIME(Dtype):
1859+ c = "\x24"
1860+
1861+ def f(self, d):
1862+ return type(d) is time
1863+ r = False
1864+
1865+ def s(self, raw):
1866+ return raw.isoformat().encode("ISO-8859-1")
1867+
1868+ def u(self, rawstr):
1869+ data = rawstr.decode("ISO-8859-1").split(":")
1870+ data[2] = data[2].split(".")
1871+ data[2].append(0)
1872+ return time(int(data[0]), int(data[1]), int(data[2][0]),
1873+ int(data[2][1]))
1874+
1875+reg_dtype(TIME)
1876+
1877+
1878+class DATETIME(Dtype):
1879+ c = "\x28"
1880+
1881+ def f(self, d):
1882+ return type(d) is datetime
1883+ r = False
1884+
1885+ def s(self, raw):
1886+ return raw.isoformat().encode("ISO-8859-1")
1887+
1888+ def u(self, rawstr):
1889+ data = rawstr.decode("ISO-8859-1").split(".")
1890+ dt = datetime.strptime(data[0], "%Y-%m-%dT%H:%M:%S")
1891+ if len(data) > 1:
1892+ us = int(data[1].rstrip("Z"), 10)
1893+ else:
1894+ us = 0
1895+ return dt + timedelta(microseconds=us)
1896+
1897+reg_dtype(DATETIME)
1898+
1899+
1900+class PRESERVE(Dtype):
1901+ c = "\x31"
1902+
1903+ def f(self, *args):
1904+ return True
1905+
1906+ r = False
1907+
1908+ def s(self, raw):
1909+ for ty in Register.dtype_codes:
1910+ x = getattr(DataTypes, Register.dtype_codes[ty])
1911+ if x.f(raw) and isinstance(x, PRESERVE) is False:
1912+ return b"".join([x.s(raw), ty.encode("ISO-8859-1")])
1913+
1914+ raise ValueError("Unknown type to preserve (%s)" % type(raw))
1915+
1916+ def u(self, rawstr):
1917+ ty = rawstr.decode("ISO-8859-1")[-1]
1918+ rawstr = rawstr[:-1]
1919+
1920+ if ty in Register.dtype_codes:
1921+ x = getattr(DataTypes, Register.dtype_codes[ty])
1922+ else:
1923+ raise TypeError("Could not restore type.")
1924+
1925+ return x.u(rawstr)
1926+
1927+reg_dtype(PRESERVE)
1928+Register.default_dtype = PRESERVE
1929
1930=== added directory 'htd30/_objects'
1931=== added file 'htd30/_objects/__init__.py'
1932--- htd30/_objects/__init__.py 1970-01-01 00:00:00 +0000
1933+++ htd30/_objects/__init__.py 2012-07-18 20:53:21 +0000
1934@@ -0,0 +1,26 @@
1935+'''
1936+This file is part of HTD (HayzenTech Database)
1937+
1938+Copyright (C) 2011 - 2012 Andrew Hayzen
1939+
1940+HTD is free software: you can redistribute it and/or modify
1941+it under the terms of the GNU Lesser General Public License as published by
1942+the Free Software Foundation, either version 3 of the License, or
1943+(at your option) any later version.
1944+
1945+HTD is distributed in the hope that it will be useful,
1946+but WITHOUT ANY WARRANTY; without even the implied warranty of
1947+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1948+GNU Lesser General Public License for more details.
1949+
1950+You should have received a copy of the GNU Lesser General Public License
1951+along with HTD. If not, see <http://www.gnu.org/licenses/>
1952+'''
1953+
1954+from .attribute import Attribute
1955+from .attributes import Attributes
1956+from .auth import Auth
1957+from .connection import Connection
1958+from .cursor import Cursor
1959+from .record import Record
1960+from .user import User
1961
1962=== added file 'htd30/_objects/attribute.py'
1963--- htd30/_objects/attribute.py 1970-01-01 00:00:00 +0000
1964+++ htd30/_objects/attribute.py 2012-07-18 20:53:21 +0000
1965@@ -0,0 +1,179 @@
1966+'''
1967+This file is part of HTD (HayzenTech Database)
1968+
1969+Copyright (C) 2011 - 2012 Andrew Hayzen
1970+
1971+HTD is free software: you can redistribute it and/or modify
1972+it under the terms of the GNU Lesser General Public License as published by
1973+the Free Software Foundation, either version 3 of the License, or
1974+(at your option) any later version.
1975+
1976+HTD is distributed in the hope that it will be useful,
1977+but WITHOUT ANY WARRANTY; without even the implied warranty of
1978+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1979+GNU Lesser General Public License for more details.
1980+
1981+You should have received a copy of the GNU Lesser General Public License
1982+along with HTD. If not, see <http://www.gnu.org/licenses/>
1983+
1984+HTD Attribute
1985+'''
1986+
1987+from .. import __version__, Register, DataTypes
1988+from .._workers import sanitise, unsanitise
1989+from struct import pack
1990+from sys import version_info
1991+
1992+
1993+class Attribute(object):
1994+ """ Class that manages a specific attribute """
1995+ def __init__(self, dtype=None, size=None, name=None):
1996+ # Save variables
1997+ self.dtype = dtype
1998+ self.name = name
1999+ self.size = size
2000+
2001+ # Set extra options
2002+ self.range = False
2003+ self.reg_expr = None
2004+ self.limit_list = [False, []]
2005+ self.required = False
2006+ self._id = None
2007+ self._default = None
2008+
2009+ def __repr__(self):
2010+ return ("<HTD%s Attribute (<name '%s'> <type '%s'> <len %s>)>" %
2011+ (__version__, self.name, self.dtype.__class__.__name__,
2012+ self.size))
2013+
2014+ @property
2015+ def default(self):
2016+ return unsanitise({"val": self._default}, {"val": self})["val"]
2017+
2018+ @default.setter
2019+ def default(self, value):
2020+ value = sanitise({"val": value}, {"val": self})
2021+ if not value:
2022+ self._default = None
2023+ else:
2024+ self._default = value["val"]
2025+
2026+ @property
2027+ def dtype(self):
2028+ return self._dtype
2029+
2030+ @dtype.setter
2031+ def dtype(self, dtype):
2032+ # Check type
2033+ if dtype is None:
2034+ dtype = Register.default_dtype
2035+ else:
2036+ dtype = dtype.__class__.__name__
2037+
2038+ try:
2039+ dtype = getattr(DataTypes, dtype)
2040+ except AttributeError:
2041+ raise ValueError("Attribute type must be a valid HTD data type")
2042+
2043+ self._dtype = dtype
2044+
2045+ @property
2046+ def limit_list(self):
2047+ return self._limit_list
2048+
2049+ @limit_list.setter
2050+ def limit_list(self, limits):
2051+ if limits is False or limits == [False, []]:
2052+ self._limit_list = [False, []]
2053+ elif isinstance(limits, (list, tuple)) is False:
2054+ raise ValueError("Limits must be list type")
2055+ else:
2056+ self._limit_list = [True, limits]
2057+
2058+ @property
2059+ def name(self):
2060+ return self._name
2061+
2062+ @name.setter
2063+ def name(self, name):
2064+ # Check name
2065+ if name is None:
2066+ self._name = None
2067+ return
2068+
2069+ if "." in name:
2070+ raise ValueError("Attribute name must not contain '.'")
2071+
2072+ if isinstance(name, str) is False:
2073+ raise ValueError("Attribute name must be str type.")
2074+
2075+ self._name = name
2076+
2077+ @property
2078+ def range(self):
2079+ return self._range
2080+
2081+ @range.setter
2082+ def range(self, ran):
2083+ if ran is False or ran == [False, False]:
2084+ self._range = ran
2085+ return
2086+
2087+ if isinstance(ran, (list, tuple)) is False:
2088+ raise ValueError("Range must be list type")
2089+
2090+ lower, upper = ran
2091+
2092+ if lower >= upper:
2093+ raise ValueError("The lower value must be less than the upper.")
2094+
2095+ self._range = [lower, upper]
2096+
2097+ @property
2098+ def reg_expr(self):
2099+ return self._re
2100+
2101+ @reg_expr.setter
2102+ def reg_expr(self, expr):
2103+ if type(expr) is not bytes and not not expr:
2104+ expr = str(expr).encode("ISO-8859-1")
2105+ self._re = expr
2106+
2107+ @property
2108+ def required(self):
2109+ return self._required
2110+
2111+ @required.setter
2112+ def required(self, required):
2113+ if required in [True, False]:
2114+ self._required = required
2115+ else:
2116+ raise ValueError("Required must be either True or False")
2117+
2118+ @property
2119+ def size(self):
2120+ return self._size
2121+
2122+ @size.setter
2123+ def size(self, size):
2124+ # Check size
2125+ if size is None:
2126+ self._size = 255
2127+ return
2128+
2129+ if isinstance(size, int) is False:
2130+ raise ValueError("Attribute size must be an Integer")
2131+ else:
2132+ if not 1 <= size <= 4294967295:
2133+ raise ValueError("Attribute size must be greater than or " \
2134+ "equal to 1")
2135+
2136+ self._size = size
2137+
2138+ def to_dict(self):
2139+ """ Convert to string """
2140+
2141+ return {"dtype": self.dtype.c, "id": self._id,
2142+ "limit_list": self.limit_list, "default": self._default,
2143+ "range": self.range, "reg_expr": self.reg_expr,
2144+ "required": self.required, "size": pack("I", self.size)}
2145
2146=== added file 'htd30/_objects/attributes.py'
2147--- htd30/_objects/attributes.py 1970-01-01 00:00:00 +0000
2148+++ htd30/_objects/attributes.py 2012-07-18 20:53:21 +0000
2149@@ -0,0 +1,212 @@
2150+'''
2151+This file is part of HTD (HayzenTech Database)
2152+
2153+Copyright (C) 2011 - 2012 Andrew Hayzen
2154+
2155+HTD is free software: you can redistribute it and/or modify
2156+it under the terms of the GNU Lesser General Public License as published by
2157+the Free Software Foundation, either version 3 of the License, or
2158+(at your option) any later version.
2159+
2160+HTD is distributed in the hope that it will be useful,
2161+but WITHOUT ANY WARRANTY; without even the implied warranty of
2162+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2163+GNU Lesser General Public License for more details.
2164+
2165+You should have received a copy of the GNU Lesser General Public License
2166+along with HTD. If not, see <http://www.gnu.org/licenses/>
2167+
2168+HTD Attributes
2169+'''
2170+
2171+from .. import __version__
2172+from .._base import _access_check, _trace, ThreadLock
2173+from .._objects.attribute import Attribute
2174+from .._workers import sanitise, unsanitise
2175+from weakref import ref
2176+
2177+
2178+class Attributes(ThreadLock):
2179+ """ Class that manages attribute settings in a table """
2180+ def __init__(self, parent):
2181+ # Set vars
2182+ self.alive = True
2183+ self._parent = ref(parent)
2184+ self._parent()._children.append(self)
2185+ self._file = self._parent()._file
2186+
2187+ self._access = self._parent()._access # Load user access
2188+
2189+ self._index = 0 # var for loop
2190+
2191+ ThreadLock.__init__(self)
2192+
2193+ @_trace("r")
2194+ def __contains__(self, key):
2195+ """ Return True if attribute exists in db """
2196+
2197+ self._file.lock_reading()
2198+
2199+ out = False
2200+ if key in self._file.header["ATT"]:
2201+ out = True
2202+
2203+ self._file.unlock()
2204+
2205+ return out
2206+
2207+ @_trace("d")
2208+ def __delitem__(self, key):
2209+ """ Delete attribute and data from db """
2210+
2211+ self._file.lock_writing()
2212+
2213+ # Confirm att exists
2214+ if key not in self._file.header["ATT"]:
2215+ raise KeyError('Could not find attribute to delete. (%s)' % key)
2216+
2217+ if key in self._file.header["KEY"]:
2218+ raise ValueError('Cannot remove attribute that is an identifier')
2219+
2220+ self._file.delete_attribute(key)
2221+
2222+ # Delete attribute, rebuild header
2223+ header = self._file.header
2224+
2225+ i = header["ATT"][key]._id
2226+
2227+ del header["ATT"][key]
2228+
2229+ # resort IDs
2230+ for a in header["ATT"]:
2231+ att = header["ATT"][a]
2232+ if att._id > i:
2233+ att._id -= 1
2234+
2235+ self._file.header_update(header)
2236+
2237+ self._file.unlock()
2238+
2239+ @_trace("r")
2240+ def __getitem__(self, key):
2241+ """ Return attributes settings """
2242+
2243+ self._file.lock_reading()
2244+
2245+ attributes = self._file.header["ATT"]
2246+
2247+ if key not in attributes:
2248+ raise KeyError('Failed to find sanitise settings for attribute ' \
2249+ '(%s)' % (key))
2250+
2251+ self._file.unlock()
2252+
2253+ return attributes[key]
2254+
2255+ def __iter__(self):
2256+ self._index = 0
2257+
2258+ return self
2259+
2260+ @_trace("r")
2261+ def __len__(self):
2262+ """ The number of attributes in the table """
2263+
2264+ self._file.lock_reading()
2265+
2266+ leng = len(self._file.header["ATT"])
2267+
2268+ self._file.unlock()
2269+
2270+ return leng
2271+
2272+ def __next__(self):
2273+ return self.next()
2274+
2275+ def next(self):
2276+ tmp = None
2277+
2278+ self._file.lock_reading()
2279+ try:
2280+ tmp = self._file.header["ATTBYID"][self._index][1]
2281+ except KeyError:
2282+ raise StopIteration
2283+ else:
2284+ out = self._file.header["ATT"][tmp]
2285+ self._index += 1
2286+
2287+ return out
2288+ finally:
2289+ self._file.unlock()
2290+
2291+ def __repr__(self):
2292+ return "<HTD%s Attributes (%s)>" % (__version__,
2293+ self._parent()._location)
2294+
2295+ @_trace()
2296+ def __setitem__(self, key, v):
2297+ if isinstance(v, Attribute) is False:
2298+ raise ValueError("Attribute must be a Attribute instance.")
2299+
2300+ self._file.lock_writing()
2301+
2302+ header = self._file.header
2303+ v.name = key
2304+
2305+ if v.required and v.default is None and self._file.get_length() > 0:
2306+ raise ValueError("Table must be empty to set an attribute if it " \
2307+ "is required or have a default value (%s)" % key)
2308+
2309+ new = False
2310+ if key in header["ATT"]:
2311+ _access_check("o", self)
2312+ v._id = header["ATT"][key]._id
2313+ old = header["ATT"][key]
2314+
2315+ # Confirm old data
2316+ for i in self._file.select(key):
2317+ sanitise(unsanitise(i, {key: v}), {key: v})
2318+ else:
2319+ _access_check("a", self)
2320+ v._id = len(header["ATT"])
2321+ new = True
2322+
2323+ header["ATT"][key] = v
2324+
2325+ self._file.header_update(header)
2326+
2327+ if new:
2328+ self._file.add_attribute(key)
2329+ else:
2330+ # Iter through elements
2331+ gen = self._file.modify_attribute(key, old, v)
2332+
2333+ run = True
2334+
2335+ try:
2336+ nxt = next(gen)
2337+ except StopIteration:
2338+ run = False
2339+
2340+ while run:
2341+ try:
2342+ nxt = gen.send(sanitise(unsanitise(nxt, {key: v}), {key:
2343+ v}))
2344+ except StopIteration:
2345+ break
2346+
2347+ gen.close()
2348+
2349+ self._file.unlock()
2350+
2351+ @_trace()
2352+ def close(self):
2353+ self.alive = False
2354+
2355+ self._file = None
2356+
2357+ # Remove from parent
2358+ if self._parent() is not None and self in self._parent()._children:
2359+ self._parent()._children.remove(self)
2360+
2361+ return True
2362
2363=== added file 'htd30/_objects/auth.py'
2364--- htd30/_objects/auth.py 1970-01-01 00:00:00 +0000
2365+++ htd30/_objects/auth.py 2012-07-18 20:53:21 +0000
2366@@ -0,0 +1,201 @@
2367+'''
2368+This file is part of HTD (HayzenTech Database)
2369+
2370+Copyright (C) 2011 - 2012 Andrew Hayzen
2371+
2372+HTD is free software: you can redistribute it and/or modify
2373+it under the terms of the GNU Lesser General Public License as published by
2374+the Free Software Foundation, either version 3 of the License, or
2375+(at your option) any later version.
2376+
2377+HTD is distributed in the hope that it will be useful,
2378+but WITHOUT ANY WARRANTY; without even the implied warranty of
2379+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2380+GNU Lesser General Public License for more details.
2381+
2382+You should have received a copy of the GNU Lesser General Public License
2383+along with HTD. If not, see <http://www.gnu.org/licenses/>
2384+'''
2385+
2386+from .. import Register
2387+from .._base import _access_check, _trace, ThreadLock
2388+from .._objects.user import User
2389+from weakref import ref
2390+
2391+
2392+class Auth(ThreadLock):
2393+ """ Authentication object manages users and access for a table """
2394+ def __init__(self, parent, location, file_obj=None):
2395+ # Store variables
2396+ self.alive = True
2397+ self._parent = ref(parent)
2398+ self._parent()._children.append(self)
2399+ self._location = location
2400+
2401+ self._access = parent._access # Load access
2402+
2403+ self._tlist = []
2404+ self._index = 0
2405+
2406+ ThreadLock.__init__(self) # Start inherit
2407+
2408+ if file_obj is None:
2409+ htdfile = Register.resolve_fhandler(self.location)
2410+ self._file = htdfile(location) # Open file
2411+ self._opened = True
2412+ else:
2413+ self._file = file_obj
2414+ self._opened = False
2415+
2416+ @_trace("rx")
2417+ def __contains__(self, key):
2418+ self._file.lock_reading()
2419+
2420+ out = key in self._file.header["USR"]
2421+
2422+ self._file.unlock()
2423+
2424+ return out
2425+
2426+ @_trace("dx")
2427+ def __delitem__(self, key):
2428+ self._file.lock_writing()
2429+
2430+ if key not in self._file.header["USR"]:
2431+ raise KeyError('No User, "%s", found to delete' % key)
2432+
2433+ if len(self._file.header["USR"]) == 1:
2434+ raise ValueError("Must be at least 2 users to delete a user")
2435+
2436+ if self._parent().user.username == key:
2437+ raise ValueError("Cannot delete your own user.")
2438+
2439+ users = self._file.header["USR"]
2440+ del users[key]
2441+
2442+ x = sum([1 for i in users if i.access == "roadx"])
2443+ if x < 1:
2444+ raise ValueError("At least 1 remaining user must have full access")
2445+
2446+ # Update header
2447+ header = self._file.header
2448+ header["USR"] = users
2449+ self._file.header_update(header)
2450+
2451+ self._file.unlock()
2452+
2453+ @_trace("rx")
2454+ def __getitem__(self, key):
2455+ self._file.lock_reading()
2456+
2457+ try:
2458+ out = self._file.header["USR"][key]
2459+ except KeyError:
2460+ out = False
2461+ raise KeyError('No User, "%s", found' % key)
2462+
2463+ self._file.unlock()
2464+
2465+ return out
2466+
2467+ @_trace("r")
2468+ def __iter__(self):
2469+ self._file.lock_reading()
2470+
2471+ self._tlist = [u for u in self._file.header["USR"]]
2472+ self._index = 0
2473+
2474+ self._file.unlock()
2475+
2476+ return self
2477+
2478+ @_trace("r")
2479+ def __len__(self):
2480+ self._file.lock_reading()
2481+
2482+ out = len(self._file.header["USR"])
2483+
2484+ self._file.unlock()
2485+
2486+ return out
2487+
2488+ def __next__(self):
2489+ return self.next()
2490+
2491+ def next(self):
2492+ try:
2493+ u = self._tlist[self._index]
2494+ except IndexError:
2495+ self._tlist = []
2496+ self._index = 0
2497+ raise StopIteration
2498+ else:
2499+ self._index += 1
2500+
2501+ self._file.lock_reading()
2502+
2503+ out = self._file.header["USR"][u]
2504+
2505+ self._file.unlock()
2506+
2507+ return out
2508+
2509+ def __repr__(self):
2510+ if self.alive is False:
2511+ return "<HTD%s Closed Auth>" % self.version
2512+
2513+ return "<HTD%s Auth (%s)>" % (self.version, self.location)
2514+
2515+ @_trace("x")
2516+ def __setitem__(self, key, v):
2517+ if isinstance(v, User) is False:
2518+ raise ValueError("User must be an User instance.")
2519+
2520+ self._file.lock_writing()
2521+
2522+ v.username = key
2523+
2524+ if key in self._file.header["USR"]:
2525+ _access_check("o", self)
2526+
2527+ if len(self._file.header["USR"]) == 1 and v.access != "roadx":
2528+ raise ValueError("At least one User must have full access")
2529+ else:
2530+ _access_check("a", self)
2531+
2532+ header = self._file.header
2533+ header["USR"][key] = v
2534+
2535+ self._file.header_update(header)
2536+
2537+ self._file.unlock()
2538+
2539+ @_trace()
2540+ def close(self):
2541+ """ Close the cursor """
2542+ self.alive = False # Set closed
2543+
2544+ if self._opened: # Close file if I opened it
2545+ self._file.close()
2546+
2547+ # Remove from parent
2548+ if self._parent() is not None and self in self._parent()._children:
2549+ self._parent()._children.remove(self)
2550+
2551+ return True
2552+
2553+ @property
2554+ def location(self):
2555+ """ The location of the table """
2556+ return "%s.htd" % self._location
2557+
2558+ @property
2559+ def version(self):
2560+ """ Version of the open table """
2561+ self._file.lock_reading()
2562+
2563+ out = self._file.header["VER"]
2564+
2565+ self._file.unlock()
2566+
2567+ return out
2568
2569=== added file 'htd30/_objects/connection.py'
2570--- htd30/_objects/connection.py 1970-01-01 00:00:00 +0000
2571+++ htd30/_objects/connection.py 2012-07-18 20:53:21 +0000
2572@@ -0,0 +1,287 @@
2573+'''
2574+This file is part of HTD (HayzenTech Database)
2575+
2576+Copyright (C) 2011 - 2012 Andrew Hayzen
2577+
2578+HTD is free software: you can redistribute it and/or modify
2579+it under the terms of the GNU Lesser General Public License as published by
2580+the Free Software Foundation, either version 3 of the License, or
2581+(at your option) any later version.
2582+
2583+HTD is distributed in the hope that it will be useful,
2584+but WITHOUT ANY WARRANTY; without even the implied warranty of
2585+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2586+GNU Lesser General Public License for more details.
2587+
2588+You should have received a copy of the GNU Lesser General Public License
2589+along with HTD. If not, see <http://www.gnu.org/licenses/>
2590+
2591+HTD Connection
2592+'''
2593+
2594+from .._objects.auth import Auth
2595+from .._objects.cursor import Cursor
2596+from .._objects.user import User
2597+from .._base import _trace, ThreadLock
2598+from .. import __version__, ConnectionError, AuthenticationError, Register
2599+from copy import copy
2600+
2601+
2602+@_trace()
2603+class Connection(ThreadLock):
2604+ """ Class that provides a connection to a table or db """
2605+ def __init__(self, location, user=None, file_obj=None):
2606+ ThreadLock.__init__(self) # Load subclasses
2607+
2608+ # Set vars
2609+ self.alive = True
2610+ self._children = []
2611+ self._location = location
2612+
2613+ # Load user
2614+ if user is None:
2615+ user = User("guest", None)
2616+ self.user = user
2617+
2618+ # Generator for __iter__
2619+ self._gen = None
2620+ self._gen_cur = None
2621+
2622+ if file_obj is None: # Open file if not passed
2623+ self.htdfile = Register.resolve_fhandler(self.location)
2624+ file_obj = self.htdfile(location)
2625+
2626+ self._file = file_obj
2627+
2628+ # Load db mode
2629+ self._isdb = False
2630+
2631+ if self._file.header["TAB"] is False:
2632+ self._isdb = True
2633+ elif self._file.header["TAB"]:
2634+ raise ConnectionError("This table is part of a db, please " \
2635+ "connect to the db")
2636+
2637+ # Load and check auth
2638+ if user.username not in self._file.header["USR"]:
2639+ raise AuthenticationError()
2640+
2641+ u = self._file.header["USR"][user.username]
2642+
2643+ if user.password != u.password:
2644+ raise AuthenticationError()
2645+
2646+ # Load access
2647+ self._access = u.access
2648+
2649+ @_trace('xd')
2650+ def __delitem__(self, name):
2651+ """ Split table with parent - returns connection to child """
2652+
2653+ if not self.is_db:
2654+ raise ValueError("This feature requires a connection to a " \
2655+ "database")
2656+
2657+ cur = self[None]
2658+
2659+ if name in cur:
2660+ # Split table from parent
2661+ data = cur[name] # get loc
2662+ del cur[name]
2663+ cur.close()
2664+
2665+ con = load_connection(data['location']) # Connect
2666+ cur = con[None]
2667+
2668+ # Update header
2669+ cur._file.lock_reading()
2670+
2671+ header = cur._file.header
2672+ header["TAB"] = None
2673+
2674+ cur._file.header_update(header)
2675+
2676+ cur._file.unlock()
2677+
2678+ cur.close()
2679+ else:
2680+ cur.close()
2681+ raise KeyError("No table to split named '%s'." % name)
2682+
2683+ @_trace()
2684+ def __getitem__(self, key):
2685+ """ Open a cursor on the table """
2686+
2687+ if key is None:
2688+ return Cursor(self, self._location, None, file_obj=self._file)
2689+
2690+ # Get table data from db
2691+ cur = Cursor(self, self._location, None, file_obj=self._file)
2692+ tbd = cur[key]
2693+ cur.close()
2694+
2695+ return Cursor(self, tbd['location'], key)
2696+
2697+ @_trace('r')
2698+ def __iter__(self):
2699+ """ If a db - return the tables """
2700+ # Check connections is DB
2701+ if not self._isdb:
2702+ self._gen = None
2703+ return self
2704+
2705+ # Open cursor, list table, close
2706+ self._gen_cur = self[None]
2707+ self._gen = self._gen_cur.select(["name", "location"])
2708+
2709+ return self
2710+
2711+ def __len__(self):
2712+ # Check for db mode
2713+ if self._isdb:
2714+ cur = self[None]
2715+ leng = len(cur)
2716+ cur.close()
2717+ return leng
2718+ else:
2719+ return 1
2720+
2721+ def __next__(self):
2722+ return self.next()
2723+
2724+ def next(self):
2725+ if not self._gen:
2726+ raise StopIteration
2727+
2728+ try:
2729+ tmp = next(self._gen)
2730+ except StopIteration:
2731+ self._gen_cur.close() # Close cursor
2732+ raise StopIteration
2733+ else:
2734+ return tmp
2735+
2736+ def __repr__(self):
2737+ if self.alive is False:
2738+ return "<HTD%s Closed Connection>" % __version__
2739+
2740+ return ("<HTD%s Connection (%s@%s), DB Mode %s>" %
2741+ (__version__, self.user.username, self.location,
2742+ self.is_db))
2743+
2744+ @_trace('xa')
2745+ def __setitem__(self, name, child):
2746+ """ Join table (connection) with parent """
2747+
2748+ if not self.is_db:
2749+ return False
2750+
2751+ if isinstance(child, self.__class__) is False:
2752+ raise ValueError("Child must be a Connection instance")
2753+
2754+ cur = child[None]
2755+
2756+ # Check that child is a table
2757+ cur._file.lock_reading()
2758+ header = cur._file.header
2759+
2760+ if header["TAB"] is not None:
2761+ cur._file.unlock()
2762+ cur.close()
2763+ raise ValueError("Child must be a table")
2764+
2765+ # Add location to parent
2766+ cur = self[None]
2767+ cur[name] = {"location": child._location}
2768+ cur.close()
2769+
2770+ child.close()
2771+
2772+ cur = self[name]
2773+
2774+ # Update child header
2775+ cur._file.lock_writing()
2776+
2777+ header = cur._file.header
2778+ header["TAB"] = True
2779+
2780+ # Clear users
2781+ header["USR"] = {"guest": User("guest", None, "roadx")}
2782+
2783+ cur._file.header_update(header)
2784+
2785+ cur._file.unlock()
2786+
2787+ cur.close()
2788+
2789+ return self
2790+
2791+ @property
2792+ @_trace('x')
2793+ def authentication(self):
2794+ """ Create a new authentication object """
2795+
2796+ return Auth(self, self._location, file_obj=self._file)
2797+
2798+ @property
2799+ @_trace()
2800+ def children(self):
2801+ """ Open children """
2802+
2803+ for cur in copy(self._children):
2804+ if cur is None:
2805+ self._children.remove(cur)
2806+ else:
2807+ if not cur.alive:
2808+ del self._children._parent
2809+ self._children.remove(cur)
2810+
2811+ return self._children
2812+
2813+ @_trace()
2814+ def close(self):
2815+ """ Close the connection """
2816+
2817+ for cur in copy(self.children):
2818+ cur.close() # Close children
2819+
2820+ self._file.close()
2821+
2822+ self.alive = False
2823+
2824+ return True
2825+
2826+ @_trace('d')
2827+ def delete(self, children=False):
2828+ """ Delete connection db (and tables) and disconnect """
2829+
2830+ for cur in copy(self.children):
2831+ cur.close() # Close cursors
2832+
2833+ if self.is_db:
2834+ for i in list(self):
2835+ # Delete children
2836+ htdfile = Register.resolve_fhandler("%s.htd" % i["location"])
2837+ f = htdfile(i["location"])
2838+ f.delete_files()
2839+
2840+ # Delete main file
2841+ self._file.delete_files()
2842+
2843+ self.alive = False
2844+
2845+ return True
2846+
2847+ @property
2848+ def is_db(self):
2849+ """ Is the connection to a DB or table """
2850+ return self._isdb
2851+
2852+ @property
2853+ def location(self):
2854+ """ The location of the table """
2855+ return "%s.htd" % self._location
2856+
2857+
2858+def load_connection(*args, **kwargs):
2859+ return Connection(*args, **kwargs)
2860
2861=== added file 'htd30/_objects/cursor.py'
2862--- htd30/_objects/cursor.py 1970-01-01 00:00:00 +0000
2863+++ htd30/_objects/cursor.py 2012-07-18 20:53:21 +0000
2864@@ -0,0 +1,380 @@
2865+'''
2866+This file is part of HTD (HayzenTech Database)
2867+
2868+Copyright (C) 2011 - 2012 Andrew Hayzen
2869+
2870+HTD is free software: you can redistribute it and/or modify
2871+it under the terms of the GNU Lesser General Public License as published by
2872+the Free Software Foundation, either version 3 of the License, or
2873+(at your option) any later version.
2874+
2875+HTD is distributed in the hope that it will be useful,
2876+but WITHOUT ANY WARRANTY; without even the implied warranty of
2877+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2878+GNU Lesser General Public License for more details.
2879+
2880+You should have received a copy of the GNU Lesser General Public License
2881+along with HTD. If not, see <http://www.gnu.org/licenses/>
2882+
2883+HTD Cursor
2884+'''
2885+
2886+from .. import Register, __version__
2887+from .._base import _access_check, Progress, ProgressItem, _trace, ThreadLock
2888+from .._objects.attributes import Attributes
2889+from .._objects.record import Record
2890+from .._workers import sanitise, unsanitise
2891+from weakref import ref
2892+from threading import current_thread
2893+
2894+
2895+@_trace()
2896+class Cursor(ThreadLock):
2897+ """ Class that manages a cursor on a table """
2898+ def __init__(self, parent, location, table, header=None, file_obj=None):
2899+ # Store variables
2900+ self.alive = True
2901+ self.table = table
2902+ self._location = location
2903+ self._children = []
2904+ self._gen = None
2905+ self._parent = ref(parent)
2906+ self._parent()._children.append(self)
2907+
2908+ self._access = parent._access
2909+
2910+ ThreadLock.__init__(self) # Start inherit
2911+
2912+ # Open file
2913+ if file_obj is None:
2914+ htdfile = Register.resolve_fhandler(self.location)
2915+ self._file = htdfile(location, header)
2916+ self._opened = True
2917+ else:
2918+ self._file = file_obj
2919+ self._opened = False
2920+
2921+ @_trace("r")
2922+ def __contains__(self, key):
2923+ self._file.lock_reading() # Lock file
2924+
2925+ # Filter primarykey and get line
2926+ if len(self._file.header["KEY"]) == 1:
2927+ key = [key]
2928+
2929+ key = sanitise(dict(zip(self._file.header["KEY"], key)),
2930+ self._file.header["ATT"])
2931+
2932+ line = self._file.get_line(key)
2933+
2934+ self._file.unlock() # Unlock
2935+
2936+ return line[0] is not False # Return True if key exists
2937+
2938+ @_trace("d")
2939+ def __delitem__(self, key):
2940+ self._file.lock_writing()
2941+
2942+ if isinstance(key, slice): # delete a slice
2943+ start = key.start
2944+ if start is None:
2945+ start = 0
2946+
2947+ stop = key.stop
2948+ if stop is None:
2949+ stop = len(self._file)
2950+
2951+ if start >= stop:
2952+ raise ValueError("Stop index must be greater than start.")
2953+
2954+ self._file.delete_region(start=start, stop=stop)
2955+
2956+ self._file.unlock()
2957+ else:
2958+ # San data
2959+ if isinstance(key, (list, tuple)) is False:
2960+ key = [key]
2961+
2962+ key = sanitise(dict(zip(self._file.header["KEY"], key)),
2963+ self._file.header["ATT"])
2964+
2965+ # Check if record exists and get line
2966+ pos, data, nline = self._file.get_line(key)
2967+
2968+ if pos is False:
2969+ raise KeyError('No record, %s, was found to delete.' % (key))
2970+
2971+ del self._file[pos] # Delete i from file
2972+
2973+ self._file.unlock()
2974+
2975+ @_trace("r")
2976+ def __getitem__(self, key):
2977+ if isinstance(key, slice):
2978+ return self.select(start=key.start or 0, stop=key.stop or -1)
2979+
2980+ self._file.lock_reading() # Lock file
2981+
2982+ # Convert key
2983+ if isinstance(key, (list, tuple)) is False:
2984+ key = [key]
2985+
2986+ key = sanitise(dict(zip(self._file.header["KEY"], key)),
2987+ self._file.header["ATT"])
2988+
2989+ # Check key exists
2990+ pos, data, nline = self._file.get_line(key)
2991+
2992+ if pos is False:
2993+ raise KeyError("".join(['No key, ', str(key), ', found']))
2994+
2995+ data = unsanitise(data, self._file.header["ATT"])
2996+
2997+ self._file.unlock() # Unlock
2998+
2999+ return Record(key, data, self._parent(), self.table,
3000+ self._file.header["KEY"])
3001+
3002+ @_trace("r")
3003+ def __iter__(self):
3004+ """ If a db - return the tables """
3005+ # Open cursor, list table, close
3006+ self._gen = self.select(attribute=self._file.header["KEY"])
3007+
3008+ return self
3009+
3010+ def __next__(self):
3011+ return self.next()
3012+
3013+ def next(self):
3014+ try:
3015+ tmp = next(self._gen)
3016+ except StopIteration:
3017+ self._gen.close() # Close cursor
3018+ raise StopIteration
3019+ else:
3020+ return tmp
3021+
3022+ @_trace()
3023+ def __len__(self):
3024+ """ The number of records in the table """
3025+
3026+ self._file.lock_reading()
3027+
3028+ length = len(self._file) # Get length
3029+
3030+ self._file.unlock()
3031+
3032+ return length
3033+
3034+ def __repr__(self):
3035+ if self.alive is False:
3036+ return "<HTD%s Closed Cursor>" % __version__
3037+
3038+ return "<HTD%s Cursor (%s)>" % (__version__, self.location)
3039+
3040+ @_trace()
3041+ def __setitem__(self, key, data):
3042+ self._file.lock_writing()
3043+
3044+ # Filter primarykey
3045+ if isinstance(key, (list, tuple)) is False:
3046+ key = [key]
3047+
3048+ key = sanitise(dict(zip(self._file.header["KEY"], key)),
3049+ self._file.header["ATT"])
3050+
3051+ # Check if record exists and get current att
3052+ pos, datatmp, nline = self._file.get_line(key)
3053+
3054+ # If record exists get current record
3055+ if pos is False:
3056+ datatmp = {}
3057+
3058+ data = sanitise(data, self._file.header["ATT"]) # Run filters
3059+
3060+ # Confirm data
3061+ for att in self._file.header["ATT"]:
3062+ if att not in data and att in datatmp:
3063+ data[att] = datatmp[att]
3064+ elif att not in data and att not in datatmp:
3065+ data[att] = None
3066+
3067+ # Add identifier to data
3068+ for att in self._file.header["KEY"]:
3069+ data[att] = key[att]
3070+
3071+ # Write to file
3072+ if pos is not False:
3073+ _access_check("o", self)
3074+ self._file[pos] = data
3075+ else:
3076+ _access_check("a", self)
3077+ self._file.insert_line(nline, data)
3078+
3079+ self._file.unlock()
3080+
3081+ @_trace()
3082+ def assign(self, generator):
3083+ """ Assign the contents of the generator to the table """
3084+
3085+ self._file.lock_writing()
3086+
3087+ gen = self._file.assign(app="a" in self._access, ovr="o" in
3088+ self._access)
3089+
3090+ next(gen)
3091+
3092+ for data in generator:
3093+ key = [data[a] for a in self._file.header["KEY"]]
3094+ key = sanitise(dict(zip(self._file.header["KEY"], key)),
3095+ self._file.header["ATT"])
3096+
3097+ gen.send([key, sanitise(data, self._file.header["ATT"])])
3098+
3099+ gen.close()
3100+
3101+ self._file.unlock()
3102+
3103+ @property
3104+ def attributes(self):
3105+ """ Manage the attribute settings """
3106+ return Attributes(self)
3107+
3108+ @_trace()
3109+ def close(self):
3110+ """ Close the cursor """
3111+ self.alive = False # Set closed
3112+
3113+ # Close children
3114+ for child in self._children:
3115+ child.close()
3116+
3117+ if self._opened: # Close file if I opened it
3118+ self._file.close()
3119+
3120+ # Remove from parent
3121+ if self._parent() is not None and self in self._parent()._children:
3122+ self._parent()._children.remove(self)
3123+
3124+ return True
3125+
3126+ @property
3127+ def composite(self):
3128+ """ Is the table using a composite key """
3129+
3130+ return isinstance(self.identifier, (list, tuple))
3131+
3132+ @property
3133+ @_trace("r")
3134+ def identifier(self):
3135+ """ The identifier for the table """
3136+
3137+ self._file.lock_reading()
3138+
3139+ key = self._file.header["KEY"]
3140+ output = [self._file.header["ATT"][att] for att in key]
3141+
3142+ if len(output) == 1:
3143+ output = output[0]
3144+
3145+ self._file.unlock()
3146+
3147+ return output
3148+
3149+ @identifier.setter
3150+ @_trace("o")
3151+ def identifier(self, key):
3152+ """ Set the identifier for the table """
3153+
3154+ self._file.lock_writing()
3155+
3156+ # Confirm data
3157+ if isinstance(key, str):
3158+ key = [key]
3159+
3160+ if isinstance(key, (list, tuple)) is False:
3161+ raise ValueError("Identifier must be a list or list of strings")
3162+
3163+ if len(self._file) > 0:
3164+ raise ValueError("The table must be empty before changing the" \
3165+ " identifier.")
3166+
3167+ header = self._file.header
3168+ header["KEY"] = key
3169+
3170+ self._file.header_update(header)
3171+
3172+ self._file.unlock()
3173+
3174+ @property
3175+ def location(self):
3176+ """ The location of the table """
3177+ return "%s.htd" % self._location
3178+
3179+ @_trace("r")
3180+ def select(self, attribute=None, start=0, stop=-1):
3181+ """ List selected attribute(s) """
3182+
3183+ self._file.lock_reading()
3184+
3185+ if attribute is None: # Select all atts
3186+ attbyid = self._file.header["ATTBYID"]
3187+ attribute = [attbyid[i][1] for i in attbyid]
3188+
3189+ if isinstance(attribute, str):
3190+ attribute = [attribute]
3191+
3192+ if not isinstance(attribute, (list, tuple)):
3193+ raise ValueError("Attribute must be a list, tuple or str.")
3194+
3195+ numatts = len(attribute) # Count atts
3196+
3197+ # Check atts exist
3198+ if sum([1 for att in attribute if att not in
3199+ self._file.header["ATT"]]) > 0:
3200+ raise KeyError('No find attribute, %s, to list.' % attribute)
3201+
3202+ if stop == -1: # load stop
3203+ stop = len(self._file)
3204+
3205+ if start > stop:
3206+ raise ValueError("Stop index must be greater than start.")
3207+
3208+ gen = self._file.select(attribute, start, stop) # load generator
3209+
3210+ # Load progress
3211+ calling = "Calling htd.Cursor.select(%s, %s, %s)" % (attribute, start,
3212+ stop)
3213+ prog = ProgressItem(current_thread(), stop - start, calling)
3214+ Progress.add(prog)
3215+
3216+ for i in gen:
3217+ tmp = unsanitise(i, self._file.header["ATT"])
3218+
3219+ self._file.unlock()
3220+
3221+ prog.count()
3222+
3223+ if numatts == 1:
3224+ yield tmp[attribute[0]]
3225+ else:
3226+ yield tmp
3227+
3228+ self._file.lock_reading()
3229+
3230+ Progress.remove(prog)
3231+
3232+ self._file.unlock()
3233+
3234+ @property
3235+ @_trace("r")
3236+ def version(self):
3237+ """ Version of the open table """
3238+ self._file.lock_reading()
3239+
3240+ out = self._file.header["VER"]
3241+
3242+ self._file.unlock()
3243+
3244+ return out
3245
3246=== added file 'htd30/_objects/record.py'
3247--- htd30/_objects/record.py 1970-01-01 00:00:00 +0000
3248+++ htd30/_objects/record.py 2012-07-18 20:53:21 +0000
3249@@ -0,0 +1,109 @@
3250+'''
3251+This file is part of HTD (HayzenTech Database)
3252+
3253+Copyright (C) 2011 - 2012 Andrew Hayzen
3254+
3255+HTD is free software: you can redistribute it and/or modify
3256+it under the terms of the GNU Lesser General Public License as published by
3257+the Free Software Foundation, either version 3 of the License, or
3258+(at your option) any later version.
3259+
3260+HTD is distributed in the hope that it will be useful,
3261+but WITHOUT ANY WARRANTY; without even the implied warranty of
3262+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3263+GNU Lesser General Public License for more details.
3264+
3265+You should have received a copy of the GNU Lesser General Public License
3266+along with HTD. If not, see <http://www.gnu.org/licenses/>
3267+
3268+HTD Record
3269+'''
3270+
3271+from .. import ConnectionError
3272+from weakref import ref
3273+
3274+
3275+class Record(object):
3276+ """ Class that provides read-only record and foreign key support """
3277+ def __init__(self, key, data={}, con=None, table=None, identifier=[]):
3278+ # Set data
3279+ self.key = key
3280+ self.data = data
3281+
3282+ # Set references
3283+ if con is not None:
3284+ self._con = ref(con)
3285+ else:
3286+ self._con = None
3287+
3288+ self.table = table
3289+
3290+ # Load identifier
3291+ if identifier == [] and con is not None:
3292+ cur = self._con()[table]
3293+ identifier = cur.identifier
3294+ cur.close()
3295+
3296+ self.identifier = identifier
3297+
3298+ # Load composite
3299+ self.composite = False
3300+
3301+ leng = len(self.identifier)
3302+
3303+ # Copy identifer into data
3304+ if leng > 1:
3305+ self.composite = True
3306+
3307+ # Copy identifier into data
3308+ for i, k in enumerate(self.identifier):
3309+ if k not in self.data:
3310+ self.data[k] = self.key[i]
3311+ elif leng == 1:
3312+ if self.identifier[0] not in self.data:
3313+ self.data[self.identifier[0]] = self.key
3314+
3315+ if isinstance(self.key, (list, tuple)):
3316+ self.key = self.key[0]
3317+
3318+ def __getitem__(self, key):
3319+ # Check if foreign key lookup needed
3320+ if "." in key:
3321+ table, key = key.split(".", 1) # Split table and attribute
3322+
3323+ # Check the object exists
3324+ if self._con() is None:
3325+ raise ConnectionError("This connection has been closed")
3326+
3327+ cur = self._con()[table] # Open cursor
3328+
3329+ # Generate foreign key from attributes
3330+ if cur.composite:
3331+ fkey = []
3332+ for a in cur.identifier:
3333+ if a.name not in self.data:
3334+ raise ValueError("%s could not be found in relation" %
3335+ a.name)
3336+
3337+ fkey.append(self.data[a.name])
3338+
3339+ else:
3340+ if cur.identifier.name not in self.data:
3341+ raise ValueError("%s could not be found in relation" %
3342+ cur.identifier.name)
3343+
3344+ fkey = self.data[cur.identifier.name]
3345+
3346+ data = cur[fkey][key] # Get the required data
3347+ cur.close() # Close cursor
3348+
3349+ return data # Return data
3350+ else:
3351+ return self.data[key] # Return key from data
3352+
3353+ def __len__(self):
3354+ """ Return number of non-NULL fields """
3355+ return len([i for i in self.data if i is not None])
3356+
3357+ def __repr__(self):
3358+ return str(self.data)
3359
3360=== added file 'htd30/_objects/user.py'
3361--- htd30/_objects/user.py 1970-01-01 00:00:00 +0000
3362+++ htd30/_objects/user.py 2012-07-18 20:53:21 +0000
3363@@ -0,0 +1,81 @@
3364+'''
3365+This file is part of HTD (HayzenTech Database)
3366+
3367+Copyright (C) 2011 - 2012 Andrew Hayzen
3368+
3369+HTD is free software: you can redistribute it and/or modify
3370+it under the terms of the GNU Lesser General Public License as published by
3371+the Free Software Foundation, either version 3 of the License, or
3372+(at your option) any later version.
3373+
3374+HTD is distributed in the hope that it will be useful,
3375+but WITHOUT ANY WARRANTY; without even the implied warranty of
3376+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3377+GNU Lesser General Public License for more details.
3378+
3379+You should have received a copy of the GNU Lesser General Public License
3380+along with HTD. If not, see <http://www.gnu.org/licenses/>
3381+
3382+HTD User
3383+'''
3384+
3385+from .. import __version__
3386+from hashlib import sha256
3387+
3388+
3389+class User(object):
3390+ """ User object """
3391+ def __init__(self, username=None, password=None, access="roadx"):
3392+ self.username = username
3393+ self.password = password
3394+ self.access = access
3395+
3396+ def __repr__(self):
3397+ return ("<HTD%s User (%s with '%s' permissions)>" %
3398+ (__version__, self.username, self.access))
3399+
3400+ @property
3401+ def access(self):
3402+ return self.__access
3403+
3404+ @access.setter
3405+ def access(self, access):
3406+ """
3407+ Set the access level for the user
3408+ NOTE: Only used for creating new users, not authenticating connections
3409+ """
3410+ for i in access:
3411+ if i not in "roadx":
3412+ raise ValueError("Invalid permission given, '%s'." % i)
3413+
3414+ self.__access = access
3415+
3416+ @property
3417+ def password(self):
3418+ return self.__password
3419+
3420+ @password.setter
3421+ def password(self, password):
3422+ """ Set the password for the user """
3423+ if password is None:
3424+ self.__password = None
3425+ else:
3426+ self.__password = sha256(password.encode("ISO-8859-1")).hexdigest()
3427+
3428+ def to_dict(self):
3429+ return {"pass": self.password, "acc": self.access}
3430+
3431+ @property
3432+ def username(self):
3433+ return self.__username
3434+
3435+ @username.setter
3436+ def username(self, username):
3437+ if username is None:
3438+ self.__username = None
3439+ return
3440+
3441+ if isinstance(username, str) is False:
3442+ raise ValueError("Username must a string")
3443+
3444+ self.__username = username
3445
3446=== added directory 'htd30/_workers'
3447=== added file 'htd30/_workers/__init__.py'
3448--- htd30/_workers/__init__.py 1970-01-01 00:00:00 +0000
3449+++ htd30/_workers/__init__.py 2012-07-18 20:53:21 +0000
3450@@ -0,0 +1,333 @@
3451+'''
3452+This file is part of HTD (HayzenTech Database)
3453+
3454+Copyright (C) 2011 - 2012 Andrew Hayzen
3455+
3456+HTD is free software: you can redistribute it and/or modify
3457+it under the terms of the GNU Lesser General Public License as published by
3458+the Free Software Foundation, either version 3 of the License, or
3459+(at your option) any later version.
3460+
3461+HTD is distributed in the hope that it will be useful,
3462+but WITHOUT ANY WARRANTY; without even the implied warranty of
3463+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3464+GNU Lesser General Public License for more details.
3465+
3466+You should have received a copy of the GNU Lesser General Public License
3467+along with HTD. If not, see <http://www.gnu.org/licenses/>
3468+
3469+Data Filter to import/export raw data
3470+Also loads default file handlers
3471+get_line alog and locking classes
3472+'''
3473+
3474+from .. import AuthenticationError
3475+from .._base import File, convert_location
3476+import os
3477+from random import choice
3478+from re import sub
3479+from struct import unpack, pack
3480+from sys import platform
3481+from time import time
3482+
3483+# Load fsync
3484+if platform == 'linux2':
3485+ fsync = os.fdatasync
3486+else:
3487+ fsync = os.fsync
3488+
3489+
3490+def sanitise(data, attributes):
3491+ """ Sanitise data so that it can be stored in the db """
3492+
3493+ if data is None:
3494+ return None
3495+
3496+ # Check data attributes exist
3497+ atts = [a for a in data if a not in attributes]
3498+
3499+ if len(atts) > 0:
3500+ raise AttributeError('Failed to find sanitise settings for ' \
3501+ 'attribute %s' % atts)
3502+
3503+ for att in attributes:
3504+ if att in data: # Load data
3505+ tmp = data[att]
3506+ else:
3507+ tmp = None
3508+
3509+ attr = attributes[att] # load attribute
3510+
3511+ if tmp is None:
3512+ if attr._default is not None:
3513+ tmp = attr.default # Load default value
3514+ elif attr.required: # Check required
3515+ raise ValueError("Required attribute (%s) was NULL" % att)
3516+ else:
3517+ continue
3518+
3519+ typed = attr.dtype # Load dtype
3520+
3521+ if typed.f(tmp) is False: # Type check
3522+ name = ('%s, failed to match preset pattern for %s. (%s)' %
3523+ (att, typed, tmp))
3524+
3525+ raise ValueError(name)
3526+
3527+ if typed.r and attr.range is not False: # Range check
3528+ if not (attr.range[1] < tmp < attr.range[0]):
3529+ raise ValueError('"%s" failed to match range for "%s". (%s)' %
3530+ (att, typed, tmp))
3531+
3532+ if attr.limit_list[0] and tmp not in attr.limit_list[1]: # List check
3533+ raise KeyError('"%s" failed to match an item in list.' % att)
3534+
3535+ tmp = typed.s(tmp) # Convert data
3536+
3537+ if len(tmp) > attr.size: # Length Check
3538+ raise ValueError('String length check failed for "%s". (%s)' %
3539+ (att, tmp))
3540+
3541+ data[att] = tmp # Update data
3542+
3543+ return data
3544+
3545+
3546+def unsanitise(data, attributes):
3547+ """ Unsantise data so that it can be used by the client """
3548+ for att in data:
3549+ dtype = attributes[att].dtype
3550+ patt = attributes[att].reg_expr
3551+ if data[att] is not None:
3552+ if not not patt: # Run reg expr
3553+ data[att] = sub(patt, b"", data[att])
3554+ data[att] = dtype.u(data[att])
3555+
3556+ return data
3557+
3558+
3559+class GetLineAlgorithm(object):
3560+ def assign(self, app=False, ovr=False):
3561+ leng = len(self)
3562+ pre = b""
3563+ end = leng == 0
3564+ err_str = 'User doesn\'t have to permission '
3565+ app_err = ''.join([err_str, '[a - append]'])
3566+ ovr_err = ''.join([err_str, '[o - overwrite]'])
3567+
3568+ while 1:
3569+ key, data = (yield)
3570+
3571+ skey = b"".join([key[att] for att in self.header["KEY"]])
3572+
3573+ if end and (skey > pre) - (skey < pre) == 1:
3574+ if app:
3575+ self.insert_line(leng, data)
3576+ pre = skey
3577+ else:
3578+ raise AuthenticationError(app_err)
3579+ else:
3580+ pos, datal, nline = self.get_line(key, new=True)
3581+
3582+ if pos is False:
3583+ if app:
3584+ self.insert_line(nline, data)
3585+ else:
3586+ raise AuthenticationError(app_err)
3587+ else:
3588+ if ovr:
3589+ self[pos] = data
3590+ else:
3591+ raise AuthenticationError(ovr_err)
3592+
3593+ end = False
3594+
3595+ leng += 1
3596+
3597+ def get_line(self, key, new=False):
3598+ """ Find line for key """
3599+ # Set up att widths, order
3600+ atts = []
3601+ attwidth = {}
3602+ primarykey = []
3603+
3604+ for att in self.header["KEY"]:
3605+ atts.append(att)
3606+ attwidth[att] = self.header["ATT"][att].size
3607+ primarykey.append(key[att])
3608+
3609+ primarykey = b"".join(primarykey) # Generate primarykey
3610+
3611+ # Calculate upper and lower Will take log(2, numrecords)
3612+ size = len(self)
3613+
3614+ upper = size
3615+ lower = 0
3616+
3617+ # Find line
3618+ datal = None
3619+ datap = []
3620+ cline = upper // 2
3621+ nline = cline
3622+
3623+ if size == 0:
3624+ cline = False
3625+ nline = 0
3626+
3627+ while size > 0:
3628+ # Check if cline in index
3629+ datal = self[cline]
3630+
3631+ if datal is False: # Fix for end of file
3632+ nline = cline
3633+ cline = False
3634+ break
3635+
3636+ # Build data
3637+ datap = []
3638+ for att in atts:
3639+ v = datal[att]
3640+ if type(v) is not bytes:
3641+ v = str(v).encode("ISO-8859-1")
3642+ datap.append(v)
3643+
3644+ # Join, add to cache
3645+ datap = b"".join(datap)
3646+
3647+ # Check data not primarykey
3648+ if datap == primarykey:
3649+ nline = False
3650+ break
3651+ elif datap == b'':
3652+ nline = cline
3653+ break
3654+
3655+ # Compare
3656+ comp = (datap > primarykey) - (datap < primarykey)
3657+
3658+ # Calc movement
3659+ # upper 10 lower 0 cline 5
3660+ if comp > 0:
3661+ upper = cline
3662+ # upper 1 lower 0 cline 1 => 0
3663+ if (upper - lower) == 1:
3664+ cline = lower
3665+ continue
3666+ else:
3667+ # upper 5 lower 0 cline 5 => 2.5
3668+ cline = lower + int((upper - lower) // 2)
3669+ elif comp < 0:
3670+ # upper 1 lower 0 cline 0 => 1
3671+ if (upper - lower) == 1:
3672+ cline = upper
3673+ lower = upper
3674+ continue
3675+ else:
3676+ # upper 10 lower 0 cline 5 => 7.5
3677+ lower = cline
3678+ cline = lower + int((upper - lower) // 2)
3679+ else:
3680+ nline = False
3681+ break
3682+
3683+ if upper == lower:
3684+ nline = cline
3685+ cline = False
3686+ break
3687+
3688+ # Return result
3689+ return [cline, datal, nline]
3690+
3691+
3692+class LockFile():
3693+ def __init__(self, location):
3694+ self.__location = "%s.htd" % (convert_location(location, ".~lock_"))
3695+
3696+ if not os.path.exists(self.__location):
3697+ f = open(self.__location, "wb")
3698+ f.close()
3699+
3700+ self.__file = File(self.__location)
3701+
3702+ while 1:
3703+ try:
3704+ self.__file._lock_reading()
3705+ break
3706+ except IOError:
3707+ pass
3708+
3709+ self._files = [self.__file]
3710+
3711+ self._confirm_funcs = []
3712+
3713+ @property
3714+ def locked(self):
3715+ """ Return the locks on the cursor """
3716+ locked = True
3717+
3718+ try:
3719+ self.__file._lock_writing()
3720+ locked = False
3721+ except IOError:
3722+ pass
3723+ finally:
3724+ self.__file._lock_reading()
3725+
3726+ return locked
3727+
3728+ def lock_reading(self):
3729+ self._acquire()
3730+
3731+ while 1:
3732+ try:
3733+ self._file._lock_reading()
3734+ break
3735+ except IOError:
3736+ pass
3737+
3738+ for i in self._files:
3739+ i.flush() # Flush files
3740+
3741+ for i in self._confirm_funcs:
3742+ i()
3743+
3744+ return
3745+
3746+ def lock_writing(self):
3747+ self._acquire()
3748+
3749+ while 1:
3750+ try:
3751+ self._file._lock_writing()
3752+ break
3753+ except IOError:
3754+ pass
3755+
3756+ for i in self._files:
3757+ i.flush() # Flush files
3758+
3759+ for i in self._confirm_funcs:
3760+ i()
3761+
3762+ return
3763+
3764+ def unlock(self):
3765+ for i in self._files:
3766+ i.flush() # Flush files
3767+
3768+ if self.force_sync and self._file.lock_type:
3769+ for i in self._files:
3770+ fsync(i) # sync file if lock_writing
3771+
3772+ try:
3773+ self._file._unlock()
3774+ except IOError:
3775+ pass
3776+
3777+ self._release()
3778+
3779+ return
3780+
3781+# Import default file handlers
3782+from . import file_var
3783+from . import file_fix
3784
3785=== added file 'htd30/_workers/file_fix.py'
3786--- htd30/_workers/file_fix.py 1970-01-01 00:00:00 +0000
3787+++ htd30/_workers/file_fix.py 2012-07-18 20:53:21 +0000
3788@@ -0,0 +1,317 @@
3789+'''
3790+This file is part of HTD (HayzenTech Database)
3791+
3792+Copyright (C) 2011 - 2012 Andrew Hayzen
3793+
3794+HTD is free software: you can redistribute it and/or modify
3795+it under the terms of the GNU Lesser General Public License as published by
3796+the Free Software Foundation, either version 3 of the License, or
3797+(at your option) any later version.
3798+
3799+HTD is distributed in the hope that it will be useful,
3800+but WITHOUT ANY WARRANTY; without even the implied warranty of
3801+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3802+GNU Lesser General Public License for more details.
3803+
3804+You should have received a copy of the GNU Lesser General Public License
3805+along with HTD. If not, see <http://www.gnu.org/licenses/>
3806+
3807+HTD_FILE_FIXED_WIDTH
3808+'''
3809+
3810+from .. import Config, Register
3811+from .._base import File, ThreadLock, convert_location, location_is_file
3812+from .._workers import GetLineAlgorithm, LockFile
3813+from .._workers.header import Header
3814+import os
3815+
3816+
3817+class FIXED_WIDTH_FILE(ThreadLock, Header, GetLineAlgorithm, LockFile):
3818+ file_format = b"\x05"
3819+
3820+ def __init__(self, location, header=None):
3821+ # Load settings
3822+ self.force_sync = Config.force_sync
3823+ self._rec_width = 0
3824+
3825+ # Load classes
3826+ ThreadLock.__init__(self)
3827+ Header.__init__(self)
3828+ GetLineAlgorithm.__init__(self)
3829+
3830+ # Check for file
3831+ if not os.path.exists("%s.htd" % location):
3832+ if header is None:
3833+ raise IOError("No HTD file found (%s.htd)" % location)
3834+
3835+ f = open("%s.htd" % location, "wb")
3836+ f.close()
3837+
3838+ self._file = File("%s.htd" % location)
3839+ self._Header__update(header, True)
3840+ else:
3841+ if header is not None:
3842+ raise IOError('File already exists (%s.htd)' % location)
3843+
3844+ self._file = File("%s.htd" % location)
3845+
3846+ self.header_update() # Load header
3847+
3848+ # Set variables
3849+ self.location = location
3850+ self.escape_char = b"\x00"
3851+
3852+ LockFile.__init__(self, location)
3853+ self._files.append(self._file)
3854+ self._confirm_funcs.append(self.header_confirm)
3855+
3856+ def __delitem__(self, key):
3857+ # Delete record
3858+ pos = (self._rec_width * key) + self.header["SIZ"]
3859+
3860+ del self._file[pos:pos + self._rec_width] # Delete from file
3861+
3862+ def __getitem__(self, key):
3863+ pos = (self._rec_width * key) + self.header["SIZ"] # Find pos
3864+
3865+ # Loop through attributes
3866+ output = {}
3867+ atts = self.header["ATTBYID"]
3868+
3869+ self._file.seek(pos)
3870+
3871+ r = self._file.read
3872+
3873+ # Read atts
3874+ for att in atts:
3875+ data = r(atts[att][0] + 1)
3876+
3877+ if data == b"":
3878+ return False
3879+
3880+ data = data.rsplit(b".", 1)
3881+
3882+ if len(data[0]) == 0:
3883+ output[atts[att][1]] = None
3884+ else:
3885+ output[atts[att][1]] = data[0]
3886+
3887+ return output
3888+
3889+ def __len__(self):
3890+ pos = len(self._file) - self.header["SIZ"]
3891+ return pos // self._rec_width
3892+
3893+ def __setitem__(self, key, value):
3894+ # Write record
3895+ # Record MUST exist
3896+ pos = (self._rec_width * key) + self.header["SIZ"]
3897+
3898+ # Loop through attributes
3899+ string = []
3900+ atts = self.header["ATTBYID"]
3901+
3902+ for att in atts:
3903+ string.append(b"".join([value[atts[att][1]] or
3904+ b"", b"."]).ljust(atts[att][0] + 1,
3905+ self.escape_char))
3906+
3907+ string = b"".join(string)
3908+
3909+ self._file[pos:pos + self._rec_width] = string
3910+
3911+ def _modify_attribute(self, attribute, mode):
3912+ if mode == 1: # Calc old length
3913+ pos = len(self._file) - self.header["SIZ"]
3914+ leng = pos // (self._rec_width -
3915+ self.header["ATT"][attribute].size)
3916+ elif mode == -1:
3917+ leng = len(self)
3918+
3919+ if leng == 0:
3920+ return True
3921+
3922+ atts = self.header["ATTBYID"]
3923+ self._file.seek(self.header["SIZ"])
3924+
3925+ x = 0
3926+ pos = self.header["SIZ"]
3927+ while x < leng:
3928+ for a in atts:
3929+ size = atts[a][0] + 1
3930+ if atts[a][1] == attribute:
3931+ if mode == 1:
3932+ self._file[pos:pos] = self.escape_char * size
3933+ pos += size
3934+ elif mode == -1:
3935+ del self._file[pos:pos + size]
3936+ else:
3937+ pos += atts[a][0] + 1
3938+ x += 1
3939+
3940+ return True
3941+
3942+ def add_attribute(self, attribute):
3943+ return self._modify_attribute(attribute, 1)
3944+
3945+ def close(self):
3946+ """ Close file and sub classes """
3947+ # Remove self from cache file
3948+ self._file._lock_writing()
3949+
3950+ locked = self.locked
3951+
3952+ # Close lock file
3953+ self._LockFile__file.close()
3954+
3955+ # Check for other locks
3956+ if not locked:
3957+ try:
3958+ os.remove(self._LockFile__location)
3959+ except IOError:
3960+ pass
3961+
3962+ # Close file
3963+ self._file._unlock()
3964+ self._file.close()
3965+
3966+ return True
3967+
3968+ def delete_attribute(self, attribute):
3969+ return self._modify_attribute(attribute, -1)
3970+
3971+ def delete_region(self, start=None, stop=None):
3972+ spos = (self._rec_width * start) + self.header["SIZ"]
3973+ epos = (self._rec_width * stop) + self.header["SIZ"]
3974+
3975+ del self._file[spos:epos]
3976+
3977+ return True
3978+
3979+ def delete_files(self):
3980+ # Close files
3981+ for f in self._files:
3982+ f.close()
3983+
3984+ # Delete table
3985+ os.unlink("%s.htd" % self.location)
3986+
3987+ lock_loc = "%s.htd" % (convert_location(self.location, ".~lock_"))
3988+ if os.path.exists(lock_loc):
3989+ os.unlink(lock_loc)
3990+
3991+ return True
3992+
3993+ def header_update(self, header=None):
3994+ if header is not None:
3995+ self._Header__update(header)
3996+ else:
3997+ self._Header__load()
3998+
3999+ self._rec_width = sum([self.header["ATT"][i].size + 1 for i in
4000+ self.header["ATT"]])
4001+ return
4002+
4003+ def insert_line(self, key, data):
4004+ # Write record
4005+ # Record MUST NOT exist
4006+ pos = (self._rec_width * key) + self.header["SIZ"]
4007+
4008+ # Loop through attributes
4009+ string = []
4010+ atts = self.header["ATTBYID"]
4011+
4012+ for att in atts:
4013+ string.append(b"".join([data[atts[att][1]] or
4014+ b"", b"."]).ljust(atts[att][0] + 1,
4015+ self.escape_char))
4016+
4017+ string = b"".join(string)
4018+
4019+ self._file[pos:pos] = string
4020+
4021+ return
4022+
4023+ def modify_attribute(self, attribute, old, new):
4024+ nsize = new.size + 1
4025+ osize = old.size + 1
4026+
4027+ # Calc old length
4028+ pos = len(self._file) - self.header["SIZ"]
4029+ leng = pos // (self._rec_width - (nsize - osize))
4030+
4031+ if leng == 0:
4032+ raise StopIteration
4033+
4034+ atts = self.header["ATTBYID"]
4035+ self._file.seek(self.header["SIZ"])
4036+
4037+ x = 0
4038+ pos = self.header["SIZ"]
4039+
4040+ while x < leng:
4041+ for a in atts:
4042+ if atts[a][1] == attribute:
4043+ raw = self._file[pos:pos + osize]
4044+
4045+ if raw == b"": # break if end of file
4046+ raise StopIteration
4047+
4048+ raw = raw.rsplit(b".", 1)
4049+
4050+ if len(raw[0]) == 0:
4051+ raw = None
4052+ else:
4053+ raw = raw[0]
4054+
4055+ tmp = (yield {attribute: raw})
4056+
4057+ output = b"".join([tmp[attribute] or
4058+ b"", b"."]).ljust(nsize,
4059+ self.escape_char)
4060+
4061+ self._file[pos:pos + osize] = output
4062+
4063+ pos += nsize
4064+ else:
4065+ pos += atts[a][0] + 1
4066+ x += 1
4067+
4068+ raise StopIteration
4069+
4070+ def select(self, attribute, start=0, stop=-1):
4071+ pos = start
4072+ atts = self.header["ATTBYID"]
4073+ s = self._file.seek
4074+ r = self._file.read
4075+
4076+ while pos < stop:
4077+ output = {}
4078+ s((self._rec_width * pos) + self.header["SIZ"])
4079+
4080+ for att in atts: # Read atts
4081+ if atts[att][1] not in attribute:
4082+ s(atts[att][0] + 1, 1)
4083+ continue
4084+
4085+ d = r(atts[att][0] + 1)
4086+
4087+ if d == b"":
4088+ raise StopIteration
4089+
4090+ d = d.rsplit(b".", 1)
4091+
4092+ if len(d[0]) == 0:
4093+ d = None
4094+ else:
4095+ d = d[0]
4096+
4097+ output[atts[att][1]] = d
4098+
4099+ pos += 1
4100+ yield output
4101+
4102+ raise StopIteration
4103+
4104+
4105+Register.register_fhandler(FIXED_WIDTH_FILE, location_is_file)
4106
4107=== added file 'htd30/_workers/file_var.py'
4108--- htd30/_workers/file_var.py 1970-01-01 00:00:00 +0000
4109+++ htd30/_workers/file_var.py 2012-07-18 20:53:21 +0000
4110@@ -0,0 +1,604 @@
4111+'''
4112+This file is part of HTD (HayzenTech Database)
4113+
4114+Copyright (C) 2011 - 2012 Andrew Hayzen
4115+
4116+HTD is free software: you can redistribute it and/or modify
4117+it under the terms of the GNU Lesser General Public License as published by
4118+the Free Software Foundation, either version 3 of the License, or
4119+(at your option) any later version.
4120+
4121+HTD is distributed in the hope that it will be useful,
4122+but WITHOUT ANY WARRANTY; without even the implied warranty of
4123+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4124+GNU Lesser General Public License for more details.
4125+
4126+You should have received a copy of the GNU Lesser General Public License
4127+along with HTD. If not, see <http://www.gnu.org/licenses/>
4128+
4129+HTD_FILE_VARIABLE_WIDTH
4130+'''
4131+
4132+from .. import Config, Register
4133+from .._base import File, ThreadLock, convert_location, location_is_file
4134+from .._workers import GetLineAlgorithm, LockFile
4135+from .._workers.header import Header
4136+
4137+import os
4138+from struct import error, unpack, pack
4139+from time import time, sleep
4140+
4141+cache_size = 50
4142+
4143+
4144+class Index():
4145+ def __init__(self, location):
4146+ # Set vars
4147+ self.__cache = []
4148+ self.__cache_size = 53
4149+ self.__location = "%s.htd" % convert_location(location, ".~index_")
4150+ self.__file = None
4151+ self.__tms = (0, 0)
4152+
4153+ # Create and connect to index
4154+ if not os.path.exists(self.__location):
4155+ self.__create_file(self._file, self.header["SIZ"],
4156+ len(self.header["ATT"]))
4157+ else:
4158+ self.__file = File(self.__location)
4159+ self.__create_cache()
4160+
4161+ def __close(self):
4162+ """ Disconnect from index """
4163+ if self.__file is not None:
4164+ self.__file.close()
4165+ self.__file = None
4166+
4167+ self.__cache = []
4168+ self.__cache_size = 50
4169+
4170+ return
4171+
4172+ def __create_cache(self, change=True):
4173+ """ Create cache from connected file """
4174+ # Calculate Cache Size
4175+ length = self.__get_length()
4176+ self.__cache_size = int((length * 0.01033) + 52.77)
4177+
4178+ # Work out how many items
4179+ length = length // self.__cache_size
4180+
4181+ # Set vars
4182+ x = 0
4183+ output = []
4184+ txt = "Q" * self.__cache_size
4185+ siz = self.__cache_size * 8
4186+
4187+ # Seek
4188+ self.__file.seek(9)
4189+
4190+ # Read file
4191+ while x < length:
4192+ tmp = self.__file.read(siz)
4193+ output.append(sum(unpack(txt, tmp)))
4194+ x += 1
4195+
4196+ # Set cache
4197+ self.__cache = output
4198+
4199+ if change:
4200+ self.__create_tms()
4201+
4202+ return True
4203+
4204+ def __create_file(self, fileobj, pos, numatts):
4205+ """ Create index file from fileobj """
4206+ # Create file
4207+ if not os.path.exists(self.__location):
4208+ f = open(self.__location, "wb+")
4209+ f.close()
4210+
4211+ if self.__file is None:
4212+ self.__file = File(self.__location)
4213+
4214+ self.__clear() # Clear cache
4215+
4216+ # Get length of data and seek to start
4217+ fileobj.seek(0, 2)
4218+ end = fileobj.tell()
4219+ fileobj.seek(pos)
4220+
4221+ self.__file.seek(0)
4222+ self.__file.write(b"\x00" * 9)
4223+
4224+ # Loop through data
4225+ while pos < end:
4226+ length = 0
4227+ x = 0
4228+ while x < numatts:
4229+ d = fileobj.read(4)
4230+ wid = unpack("I", d)[0]
4231+ fileobj.seek(wid, 1)
4232+ wid += 4
4233+ pos += wid
4234+ length += wid
4235+ x += 1
4236+ self.__file.write(pack("Q", length))
4237+
4238+ # Create cache
4239+ self.__create_cache()
4240+ return True
4241+
4242+ def __create_tms(self):
4243+ # Load timestamp
4244+ while 1:
4245+ timestamp = int(time() * 10 ** 6)
4246+ if timestamp != self.__tms[0]:
4247+ time_id = 1
4248+ break
4249+ elif self.__tms[1] == 255:
4250+ sleep(0.1)
4251+ continue
4252+ else:
4253+ time_id = self.__tms[1]
4254+ break
4255+
4256+ self.__tms = (timestamp, time_id)
4257+ self.__file[0:9] = pack("QB", *self.__tms)
4258+
4259+ return
4260+
4261+ def index_confirm(self):
4262+ """ Check for changes in the header """
4263+ self._file.seek(0)
4264+
4265+ timestamp, time_id = unpack("QB", self._file.read(9))
4266+ otimestamp, otime_id = self.__tms # Load old times
4267+
4268+ if otime_id != time_id or otimestamp != timestamp:
4269+ self.__create_cache()
4270+
4271+ return
4272+
4273+ def __clear(self):
4274+ """ Clear index """
4275+ if self.__file is not None:
4276+ self.__file.seek(9)
4277+ self.__file.truncate()
4278+
4279+ # Create cache
4280+ self.__create_cache()
4281+ return True
4282+
4283+ def __clear_slice(self, start, stop):
4284+ """ Clear index """
4285+ if self.__file is not None:
4286+ # Convert -1 to max
4287+ if stop == -1:
4288+ stop = self.__get_length()
4289+
4290+ # Clear slice
4291+ del self.__file[start * 8 + 9:stop * 8 + 9]
4292+
4293+ # Create cache
4294+ self.__create_cache()
4295+ return True
4296+
4297+ def __delete(self):
4298+ """ Delete index file """
4299+ if os.path.exists(self.__location) is False:
4300+ return False
4301+
4302+ self.__cache = []
4303+ os.unlink(self.__location)
4304+ return True
4305+
4306+ def __delete_key(self, key):
4307+ """ Delete a record from index """
4308+ key *= 8
4309+ del self.__file[key + 9:key + 17]
4310+
4311+ self.__create_cache()
4312+ return True
4313+
4314+ def __get_length(self):
4315+ """ Get length (number of records) of index """
4316+ return (len(self.__file) - 9) // 8
4317+
4318+ def __get_position(self, key):
4319+ """ Get position in db of key """
4320+ # Convert -1 to max
4321+ if key == -1:
4322+ key = self.__get_length()
4323+
4324+ # Load cache
4325+ cache = key // self.__cache_size
4326+ if cache == 0:
4327+ length = 0
4328+ else:
4329+ length = sum(self.__cache[0:cache])
4330+
4331+ key = key % self.__cache_size
4332+
4333+ self.__file.seek(cache * (self.__cache_size * 8) + 9)
4334+
4335+ # Read width
4336+ data = self.__file.read(key * 8)
4337+
4338+ if key == 1:
4339+ length += unpack("Q", data)[0]
4340+ elif key == 0:
4341+ pass
4342+ else:
4343+ try:
4344+ length += sum(unpack("Q" * key, data))
4345+ except error:
4346+ return False
4347+
4348+ return length + self.header["SIZ"]
4349+
4350+ def __get_width(self, key):
4351+ """ Get length of record """
4352+ self.__file.seek(8 * key + 9)
4353+ data = self.__file.read(8)
4354+
4355+ if len(data) == 0:
4356+ return 0
4357+ else:
4358+ return unpack("Q", data)[0]
4359+
4360+ def __insert(self, pos, value):
4361+ """ Insert record into position """
4362+ data = pack("Q", value)
4363+ pos *= 8
4364+ self.__file[pos + 9:pos + 9] = data
4365+
4366+ self.__create_cache()
4367+ return True
4368+
4369+ def __set(self, key, value):
4370+ """ Change record width to value """
4371+ key *= 8
4372+ data = pack("Q", value)
4373+ self.__file[key + 9:key + 17] = data
4374+
4375+ self.__create_cache()
4376+ return True
4377+
4378+
4379+class VARIABLE_WIDTH_FILE(ThreadLock, Header, Index, GetLineAlgorithm,
4380+ LockFile):
4381+ file_format = b"\x01"
4382+
4383+ def __init__(self, location, header=None):
4384+ # Load settings
4385+ self.force_sync = Config.force_sync
4386+
4387+ # Load classes
4388+ ThreadLock.__init__(self)
4389+ Header.__init__(self)
4390+ GetLineAlgorithm.__init__(self)
4391+
4392+ # Check for file
4393+ if not os.path.exists("%s.htd" % location):
4394+ if header is None:
4395+ raise IOError("No HTD file found (%s.htd)" % location)
4396+
4397+ f = open("%s.htd" % location, "wb")
4398+ f.close()
4399+
4400+ self._file = File("%s.htd" % location)
4401+ self._Header__update(header, True)
4402+ new = True
4403+ else:
4404+ if header is not None:
4405+ raise IOError('File already exists (%s.htd)' % location)
4406+
4407+ self._file = File("%s.htd" % location)
4408+ new = False
4409+
4410+ self.header_update() # Load header
4411+
4412+ # Set variables
4413+ self.location = location
4414+
4415+ LockFile.__init__(self, location)
4416+ self._files.append(self._file)
4417+ self._confirm_funcs.append(self.header_confirm)
4418+
4419+ Index.__init__(self, location)
4420+ self._files.append(self._Index__file)
4421+ self._confirm_funcs.append(self.index_confirm)
4422+
4423+ if new:
4424+ self._Index__clear()
4425+
4426+ def __delitem__(self, key):
4427+ # Delete record
4428+
4429+ # Find pos
4430+ pos = self._Index__get_position(key)
4431+ if pos is False:
4432+ raise IndexError("No record found to delete (%s)" % key)
4433+
4434+ wid = self._Index__get_width(key) # Get width
4435+
4436+ del self._file[pos:pos + wid] # Delete from file
4437+
4438+ self._Index__delete_key(key) # Update index and cache
4439+
4440+ def __len__(self):
4441+ return self._Index__get_length()
4442+
4443+ def __getitem__(self, key):
4444+ # Find pos
4445+ pos = self._Index__get_position(key)
4446+
4447+ if pos is False:
4448+ return False
4449+
4450+ # Loop through attributes
4451+ output = {}
4452+ atts = self.header["ATTBYID"]
4453+
4454+ self._file.seek(pos)
4455+
4456+ r = self._file.read
4457+
4458+ # Read atts
4459+ for att in atts:
4460+ try:
4461+ wid = unpack("I", r(4))[0]
4462+ except error:
4463+ output = False
4464+ break
4465+
4466+ if wid == 0:
4467+ output[atts[att][1]] = None
4468+ else:
4469+ output[atts[att][1]] = r(wid)
4470+
4471+ return output
4472+
4473+ def __setitem__(self, key, value):
4474+ # Write record
4475+ # Record MUST exist
4476+ pos = self._Index__get_position(key)
4477+
4478+ if pos is False:
4479+ raise IndexError("No record found to set (%s)" % key)
4480+
4481+ # Get width
4482+ owid = self._Index__get_width(key)
4483+
4484+ # Loop through attributes
4485+ string = []
4486+ atts = self.header["ATTBYID"]
4487+
4488+ for att in atts:
4489+ val = value[atts[att][1]]
4490+ if val is None:
4491+ string.append(b"\x00" * 4)
4492+ else:
4493+ wid = pack("I", len(val))
4494+ string.extend([wid, val])
4495+
4496+ string = b"".join(string)
4497+
4498+ self._file[pos:pos + owid] = string
4499+
4500+ # Update index
4501+ wid = len(string)
4502+ if owid != wid:
4503+ self._Index__set(key, wid)
4504+
4505+ def _modify_attribute(self, attribute, mode):
4506+ leng = len(self)
4507+ if leng == 0:
4508+ return True
4509+
4510+ attributes = self.header["ATTBYID"]
4511+ self._file.seek(self.header["SIZ"])
4512+
4513+ x = 0
4514+ pos = self.header["SIZ"]
4515+
4516+ while x < leng:
4517+ for a in attributes:
4518+ if attributes[a][1] == attribute:
4519+ if mode == 1:
4520+ self._file[pos:pos] = b"\x00\x00\x00\x00"
4521+ pos += 4
4522+ elif mode == -1:
4523+ wid = unpack("I", self._file.read(4))[0]
4524+ del self._file[pos:pos + wid + 4]
4525+ self._file.seek(pos)
4526+ else:
4527+ wid = unpack("I", self._file.read(4))[0]
4528+ self._file.seek(wid, 1)
4529+ pos += wid + 4
4530+ x += 1
4531+
4532+ num_atts = len(self.header["ATT"])
4533+ if mode == -1:
4534+ num_atts -= 1
4535+
4536+ self._Index__create_file(self._file, self.header["SIZ"], num_atts)
4537+
4538+ return True
4539+
4540+ def add_attribute(self, attribute):
4541+ return self._modify_attribute(attribute, 1)
4542+
4543+ def close(self):
4544+ """ Close file and sub classes """
4545+ # Remove self from cache file
4546+ self._file._lock_writing()
4547+
4548+ self._Index__close() # Close index
4549+
4550+ locked = self.locked
4551+
4552+ # Close lock file
4553+ self._LockFile__file.close()
4554+
4555+ # Check for other locks
4556+ if not locked:
4557+ try:
4558+ os.remove(self._LockFile__location)
4559+ except IOError:
4560+ pass
4561+
4562+ # Close file
4563+ self._file._unlock()
4564+ self._file.close()
4565+
4566+ return True
4567+
4568+ def delete_attribute(self, attribute):
4569+ return self._modify_attribute(attribute, -1)
4570+
4571+ def delete_files(self):
4572+ # Close files
4573+ for f in self._files:
4574+ f.close()
4575+
4576+ # Delete table
4577+ os.unlink("%s.htd" % self.location)
4578+
4579+ for i in [".~lock_", ".~index_"]:
4580+ loc = "%s.htd" % (convert_location(self.location, i))
4581+ if os.path.exists(loc):
4582+ os.unlink(loc)
4583+
4584+ return True
4585+
4586+ def delete_region(self, start=None, stop=None):
4587+ spos = self._Index__get_position(start)
4588+ epos = self._Index__get_position(stop)
4589+
4590+ del self._file[spos:epos]
4591+
4592+ self._Index__clear_slice(start, stop)
4593+
4594+ return True
4595+
4596+ def header_update(self, header=None):
4597+ if header is not None:
4598+ self._Header__update(header)
4599+ else:
4600+ self._Header__load()
4601+
4602+ return
4603+
4604+ def insert_line(self, key, data):
4605+ # Write record (Record MUST NOT exist)
4606+ pos = self._Index__get_position(key)
4607+
4608+ if pos is False:
4609+ return False
4610+
4611+ # Loop through attributes
4612+ string = []
4613+ atts = self.header["ATTBYID"]
4614+
4615+ for att in atts:
4616+ val = data[atts[att][1]]
4617+ if val is None:
4618+ string.append(b"\x00" * 4)
4619+ else:
4620+ wid = pack("I", len(val))
4621+ string.extend([wid, val])
4622+
4623+ string = b"".join(string)
4624+
4625+ wid = len(string)
4626+ self._file[pos:pos] = string
4627+
4628+ self._Index__insert(key, wid) # Update index
4629+
4630+ return
4631+
4632+ def modify_attribute(self, attribute, old, new):
4633+ leng = len(self)
4634+
4635+ if leng == 0:
4636+ raise StopIteration
4637+
4638+ atts = self.header["ATTBYID"]
4639+ self._file.seek(self.header["SIZ"])
4640+ num_atts = len(self.header["ATT"])
4641+
4642+ x = 0
4643+ pos = self.header["SIZ"]
4644+
4645+ while x < leng:
4646+ for a in atts:
4647+ self._file.seek(pos)
4648+ if atts[a][1] == attribute:
4649+ wid = unpack("I", self._file.read(4))[0] # read old
4650+ raw = self._file[pos + 4:pos + wid + 4]
4651+
4652+ if raw == b"": # break if end of file
4653+ self._Index__create_file(self._file,
4654+ self.header["SIZ"], num_atts)
4655+ raise StopIteration
4656+
4657+ output = (yield {attribute: raw})[attribute]
4658+ nleng = len(output)
4659+ nwid = pack("I", nleng) # write new
4660+ self._file[pos:pos + wid + 4] = b"".join([nwid, output])
4661+
4662+ pos += nleng + 4
4663+ else:
4664+ wid = unpack("I", self._file.read(4))[0]
4665+ self._file.seek(wid, 1)
4666+ pos += wid + 4
4667+ x += 1
4668+
4669+ self._Index__create_file(self._file, self.header["SIZ"], num_atts)
4670+
4671+ raise StopIteration
4672+
4673+ def select(self, attribute, start=0, stop=-1):
4674+ key = start
4675+ atts = self.header["ATTBYID"]
4676+ s = self._file.seek
4677+ r = self._file.read
4678+
4679+ pos = self._Index__get_position(key) # Find pos
4680+
4681+ if pos is False:
4682+ raise IndexError("No key found for specified range")
4683+
4684+ while key < stop:
4685+ output = {}
4686+ s(pos)
4687+
4688+ # Read atts
4689+ for att in atts:
4690+ try:
4691+ wid = unpack("I", r(4))[0]
4692+ pos += 4
4693+ except error:
4694+ raise StopIteration
4695+ break
4696+
4697+ pos += wid
4698+
4699+ if atts[att][1] not in attribute:
4700+ s(wid, 1)
4701+ continue
4702+
4703+ if wid == 0:
4704+ output[atts[att][1]] = None
4705+ else:
4706+ output[atts[att][1]] = r(wid)
4707+
4708+ key += 1
4709+ yield output
4710+
4711+ raise StopIteration
4712+
4713+Register.register_fhandler(VARIABLE_WIDTH_FILE, location_is_file)
4714+Register.default_fhandler = VARIABLE_WIDTH_FILE
4715
4716=== added file 'htd30/_workers/header.py'
4717--- htd30/_workers/header.py 1970-01-01 00:00:00 +0000
4718+++ htd30/_workers/header.py 2012-07-18 20:53:21 +0000
4719@@ -0,0 +1,186 @@
4720+'''
4721+This file is part of HTD (HayzenTech Database)
4722+
4723+Copyright (C) 2011 - 2012 Andrew Hayzen
4724+
4725+HTD is free software: you can redistribute it and/or modify
4726+it under the terms of the GNU Lesser General Public License as published by
4727+the Free Software Foundation, either version 3 of the License, or
4728+(at your option) any later version.
4729+
4730+HTD is distributed in the hope that it will be useful,
4731+but WITHOUT ANY WARRANTY; without even the implied warranty of
4732+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4733+GNU Lesser General Public License for more details.
4734+
4735+You should have received a copy of the GNU Lesser General Public License
4736+along with HTD. If not, see <http://www.gnu.org/licenses/>
4737+
4738+Header object - Manages the I/O for the header
4739+'''
4740+
4741+from .. import DataTypes, Register
4742+from .._objects import Attribute, User
4743+from copy import deepcopy
4744+from struct import pack, unpack
4745+from time import time, sleep
4746+
4747+
4748+class Header():
4749+ def __init__(self):
4750+ self.header = {"TMS": (0, 0)}
4751+
4752+ def __load(self):
4753+ """ Load header into cache """
4754+
4755+ # Check data exists
4756+ self._file.seek(0)
4757+ data = self._file.read(8)
4758+
4759+ if data == b"":
4760+ return False
4761+
4762+ # Read data
4763+ length = unpack("Q", data)[0]
4764+ file_format = self._file.read(1)
4765+ timestamp, time_id = unpack("QB", self._file.read(9))
4766+ header = {"FOR": file_format, "TMS": (timestamp, time_id)}
4767+ pos = self._file.tell()
4768+
4769+ while pos < length:
4770+ wid = unpack("Q", self._file.read(8))[0]
4771+ data = self._file.read(wid)
4772+ header[str(data[0:3].decode("ISO-8859-1"))] = data[3:]
4773+ pos += 8 + wid
4774+
4775+ # Convert data
4776+
4777+ # Convert Version
4778+ header["VER"] = float(header["HTD"])
4779+ del header["HTD"]
4780+
4781+ # Convert each
4782+ for att in header:
4783+ if att not in ["VER", "FOR", "TMS"]:
4784+ header[att] = eval(header[att])
4785+
4786+ if att == "ATT":
4787+ for a in deepcopy(header[att]):
4788+ val = header[att][a]
4789+
4790+ dtype = getattr(DataTypes,
4791+ Register.dtype_codes[val["dtype"]])
4792+
4793+ tmp = Attribute(name=a, dtype=dtype)
4794+ tmp._id = val["id"]
4795+ size = val["size"]
4796+
4797+ if not isinstance(size, bytes):
4798+ size = size.encode("ISO-8859-1")
4799+
4800+ tmp.size = unpack("I", size)[0]
4801+ tmp.limit_list = val["limit_list"]
4802+ tmp.range = val["range"]
4803+ tmp.reg_expr = val["reg_expr"]
4804+ tmp.required = val["required"]
4805+ tmp._default = val["default"]
4806+
4807+ header[att][a] = tmp
4808+
4809+ if att == "USR":
4810+ for a in deepcopy(header[att]):
4811+ val = header[att][a]
4812+ header[att][a] = User(username=a, password=val["pass"],
4813+ access=val["acc"])
4814+ header[att][a]._User__password = val["pass"] # sha256
4815+
4816+ # Add calculated
4817+ header["SIZ"] = length
4818+ header["ATTBYID"] = self._get_atts_by_id(header["ATT"])
4819+
4820+ # Update var
4821+ self.header = header
4822+
4823+ return self.header
4824+
4825+ def __update(self, header, blank=False): # Convert from new objects to raw
4826+ """ Set new header """
4827+ # Get old header
4828+ if not blank:
4829+ oldheader = deepcopy(self.header)
4830+
4831+ # Build inital str
4832+ leng = pack("Q", len(str(header["VER"])) + 3)
4833+
4834+ write = [self.file_format]
4835+
4836+ # Load timestamp
4837+ while 1:
4838+ timestamp = pack("Q", int(time() * 10 ** 6))
4839+ if timestamp != self.header["TMS"][0]:
4840+ time_id = b"\x01"
4841+ break
4842+ elif self.header["TMS"][1] == 255:
4843+ sleep(0.1)
4844+ continue
4845+ else:
4846+ time_id = pack("B", self.header["TMS"][1] + 1)
4847+ break
4848+
4849+ write.extend([timestamp, time_id, leng, b"HTD",
4850+ str(header["VER"]).encode("ISO-8859-1")])
4851+
4852+ # Add header atts
4853+ for att in ["KEY", "USR", "TAB", "ATT"]:
4854+ if att in ["ATT", "USR"]:
4855+ tmp = {}
4856+ for i in header[att]:
4857+ tmp[i] = header[att][i].to_dict()
4858+ elif att == "KEY":
4859+ tmp = []
4860+ for i in header[att]:
4861+ if isinstance(i, Attribute):
4862+ i = i.name
4863+ tmp.append(i)
4864+ header[att] = tmp
4865+ else:
4866+ tmp = header[att]
4867+
4868+ length = len(str(tmp)) + 3
4869+ write.extend([pack("Q", length), att.encode("ISO-8859-1"),
4870+ str(tmp).encode("ISO-8859-1")])
4871+
4872+ write = b"".join(write)
4873+ write = b"".join([pack("Q", len(write) + 8), write]) # Add length
4874+
4875+ # Write to file
4876+ if blank:
4877+ self._file[0:1] = write
4878+ else:
4879+ self._file[0:oldheader["SIZ"]] = write
4880+
4881+ self.__load()
4882+
4883+ return True
4884+
4885+ def _get_atts_by_id(self, atts):
4886+ """ Sort attributes by id """
4887+ attributes = {}
4888+
4889+ for att in atts:
4890+ attr = atts[att]
4891+ attributes[attr._id] = [attr.size, att]
4892+
4893+ return attributes
4894+
4895+ def header_confirm(self):
4896+ """ Check for changes in the header """
4897+ self._file.seek(9)
4898+
4899+ timestamp, time_id = unpack("QB", self._file.read(9))
4900+ otimestamp, otime_id = self.header["TMS"] # Load old times
4901+
4902+ if otime_id != time_id or otimestamp != timestamp:
4903+ self.header_update()
4904+
4905+ return
4906
4907=== modified file 'setup.py'
4908--- setup.py 2012-07-13 19:52:47 +0000
4909+++ setup.py 2012-07-18 20:53:21 +0000
4910@@ -2,20 +2,26 @@
4911 from distutils.core import setup
4912
4913 setup(name="htd30",
4914+ description="HTD (HayzenTech Database) is a pure python library which " \
4915+ "acts as a simple database engine.",
4916 version="3.0",
4917 author="Andrew Hayzen",
4918+ author_email="htd@hayzentech.co.uk",
4919 url="http://www.hayzentech.co.uk/htd",
4920 classifiers=["Development Status :: 4 - Beta",
4921 "Intended Audience :: Developers",
4922- "License :: OSI Approved :: GNU Library or Lesser General" \
4923- " Public License (LGPL)",
4924+ "License :: OSI Approved :: GNU Lesser General Public " \
4925+ "License v3 (LGPLv3)",
4926 "Programming Language :: Python",
4927+ "Programming Language :: Python :: 3",
4928 "Topic :: Database :: Database Engines/Servers",
4929- 'Operating System :: POSIX',
4930- 'Operating System :: Microsoft :: Windows'],
4931+ "Topic :: Software Development :: Libraries :: Python " \
4932+ "Modules",
4933+ "Operating System :: POSIX",
4934+ "Operating System :: Microsoft :: Windows"],
4935 packages=['htd30', 'htd30/_base', 'htd30/_objects', 'htd30/_workers'],
4936- package_dir={'htd30': 'src/htd30',
4937- 'htd30/_base': 'src/htd30/_base',
4938- 'htd30/_objects': 'src/htd30/_objects',
4939- 'htd30/_workers': 'src/htd30/_workers'}
4940+ package_dir={'htd30': 'htd30',
4941+ 'htd30/_base': 'htd30/_base',
4942+ 'htd30/_objects': 'htd30/_objects',
4943+ 'htd30/_workers': 'htd30/_workers'}
4944 )
4945
4946=== removed directory 'src'
4947=== removed directory 'src/htd30'
4948=== removed file 'src/htd30/__init__.py'
4949--- src/htd30/__init__.py 2012-07-17 21:42:28 +0000
4950+++ src/htd30/__init__.py 1970-01-01 00:00:00 +0000
4951@@ -1,266 +0,0 @@
4952-#!/usr/bin/env python
4953-'''
4954-This file is part of HTD (HayzenTech Database)
4955-
4956-Copyright (C) 2011 - 2012 Andrew Hayzen
4957-
4958-HTD is free software: you can redistribute it and/or modify
4959-it under the terms of the GNU Lesser General Public License as published by
4960-the Free Software Foundation, either version 3 of the License, or
4961-(at your option) any later version.
4962-
4963-HTD is distributed in the hope that it will be useful,
4964-but WITHOUT ANY WARRANTY; without even the implied warranty of
4965-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
4966-GNU Lesser General Public License for more details.
4967-
4968-You should have received a copy of the GNU Lesser General Public License
4969-along with HTD. If not, see <http://www.gnu.org/licenses/>
4970-
4971-This document conforms to the PEP 8 Style Guide
4972-
4973-Version 3.00 PRE-ALPHA-3
4974-'''
4975-
4976-# Import built-in modules
4977-from os import path as _path
4978-
4979-
4980-#--------------------------------------------------------------------- Settings
4981-__build_id__ = 300039
4982-__version__ = 3.00
4983-
4984-
4985-class _Config():
4986- debug = True
4987- errors = True
4988- force_sync = False
4989- tracing = False
4990-
4991-Config = _Config()
4992-
4993-
4994-#------------------------------------------------------------------- Exceptions
4995-class AuthenticationError(Exception):
4996- pass
4997-
4998-
4999-class ConnectionError(Exception):
5000- pass
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches

to all changes: