Merge lp:~syuu1228/eucalyptus/rados4eucalyptus-devel into lp:eucalyptus

Proposed by Takuya ASADA
Status: Needs review
Proposed branch: lp:~syuu1228/eucalyptus/rados4eucalyptus-devel
Merge into: lp:eucalyptus
Diff against target: 16619 lines (has conflicts)
Text conflict in clc/build.xml
Contents conflict in clc/modules/database/conf/scripts/caches.groovy
To merge this branch: bzr merge lp:~syuu1228/eucalyptus/rados4eucalyptus-devel
Reviewer Review Type Date Requested Status
Neil Soman Pending
Review via email: mp+40838@code.launchpad.net

Description of the change

Hi,

I'm trying to integrate a distributed storage system named "RADOS" for both S3 service and EBS service on Eucalyptus, to make it scalable.

RADOS is a part of Ceph filesystem, which provides distributed object store.
It provides API for C/C++ and S3 compatible server, EBS compatible block device for qemu/kvm.
Details are described on this page:
http://ceph.newdream.net/2009/05/the-rados-distributed-object-store/

This patch is minimum implement of the work, it's implements SystemStorageManager, FileIO, ChunkedDataFile for RADOS.
This provides RADOS based Walrus backend, when it's enabled Walrus stores objects to RADOS cluster instead of local file system.

Existing local file system module is moved from storage-common to storage-fs, and new rados module added as storage-rados.
It can switch via configure option --with-rados=[rados home], if it specified storage-rados will install. Otherwise storage-fs will install.

And this patch doesn't include following features, it going to be another patches:
- Zerocopy on JNI
  Currently the implementation copy buffer between librados and Java code, we need to prevent it to get better performance.
  To make it zerocopy, we need to modify Ceph implementation, not only my code.
  I'm discussing it on Ceph ML now, but it takes few more time.
- Multiple Walrus support for CLC
  We need to support multiple Walrus to make it scalable.
  I just implemented "quick hack" support Multiple Walrus, but I need to make it better before posting the patch.
- RADOS based EBS(rbd) support
  I only worked on Walrus now, but I also would like to work on rbd support for Storage Controller.

*Performance test1: chunk size
On storage-rados, default chunk size(8KB) in StorageManager.sendObject() is too small.
Here's throughput on default chunk size:
- storage-fs: 33.34MB/s
- storage-rados: 4.81MB/s

And here's read throughput when extending chunk size from 8KB to 80MB on storage-rados:
8K 4.81MB/s
80K 13.37MB/s
800K 29.52MB/s
1M 31.58MB/s
2M 33.54MB/s
3M 35.19MB/s
4M 35.67MB/s
5M 37.49MB/s
6M 35MB/s
8M 33.65MB/s
80M 23.85MB/s

So we need to change the default chunk size to 5MB on storage-rados.
I also measured 5MB on rados-fs for comparison, it's 65.60MB/s.
That means changing the default chunk size also makes rados-fs faster anyway.

Testing environment as follows:
Throughput are measured by s3cmd from Gigabit ethernet, same segment with Walrus.
RADOS cluster constructed with 9 nodes, 1 node for monitor, the others are storage(OSD).
Test file size is 1GB, single file.

[node assignment]
node0: CC/CLC, s3cmd
node1: Walrus, RADOS Monitor
node2: RADOS Storage
node3: RADOS Storage
node4: RADOS Storage
node5: RADOS Storage
node6: RADOS Storage
node7: RADOS Storage
node8: RADOS Storage
node9: RADOS Storage

[node spec]
CPU: Athlon II X4 605e
Memory: 16GB
HDD: SATA 250GB via Areca SATA Host Adapter RAID Controller
OS: Ubuntu Server 10.04

*Performance test2: scalability
I also measured the throughput when requesting multiple read request concurrently.
(This actually requires multiple Walrus implementation which doesn't include the patch, as I described earlier)

Test condition as follows:
Compared the performance when number of Walrus node is 1, 2, 4, 8 using storage-rados, and also storage-fs.
Storage node for RADOS is always 8 node, sharing Walrus node.
Read requests are sending from 2 nodes, each node sends 1, 2, 4, 8, 16, 32, 64 requests concurrently.
So overall requests are 2, 4, 8, 16, 32, 64, 128.

The graph is on following URL:
http://cid-35288454e2692e6b.photos.live.com/self.aspx/public/graph.png
Y-axis is average throughput(per request), unit is MB/s.
X-axis is number of request.

From the graph we can see the system scaling when adding Walrus nodes.
Even on 1 Walrus node, it's faster than storage-fs. This probably means 8 nodes of storage cluster is faster than local filesystem when multiple requests occurred.

Testing environment as follows:
Throughput are measured by modified s3cmd from Gigabit ethernet, same segment with Walrus, 2 nodes.
RADOS cluster constructed 1 node for monitor, 8 nodes for storage(OSD).
Test file size is 10MB, 64 files.

[node assignment]
node0: CC/CLC, s3cmd
node1: Walrus, RADOS Monitor, RADOS Storage
node2: Walrus, RADOS Storage
node3: Walrus, RADOS Storage
node4: Walrus, RADOS Storage
node5: Walrus, RADOS Storage
node6: Walrus, RADOS Storage
node7: Walrus, RADOS Storage
node8: Walrus, RADOS Storage
node9: s3cmd

[node spec]
same as test1

*Documents
Here's a document for install procedure:
http://r4eucalyptus.wikia.com/wiki/Installing_RADOS4Eucalyptus_2.0

To post a comment you must log in.

Unmerged revisions

1239. By tasada <tasada@7ae02a3f>

storage-rados module implemented. local filesystem classes moved to storage-rados module.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Makedefs.in'
2--- Makedefs.in 2010-10-22 17:50:41 +0000
3+++ Makedefs.in 2010-11-15 08:38:14 +0000
4@@ -11,6 +11,8 @@
5 export AXIS2C_SERVICES = @AXIS2C_SERVICES@
6 export AXIS2_HOME = @AXIS2_HOME@
7 export LIBVIRT_HOME = @LIBVIRT_HOME@
8+export RADOS_HOME = @RADOS_HOME@
9+export STORAGE_MODULE = @STORAGE_MODULE@
10
11 # where eucalyptus will install and other standard stuff: eventually we'll
12 # comply to the GNU standard
13
14=== modified file 'clc/Makefile'
15--- clc/Makefile 2010-06-02 20:29:20 +0000
16+++ clc/Makefile 2010-11-15 08:38:14 +0000
17@@ -50,27 +50,42 @@
18 # echo "Cannot find java deps, please run 'make deps' in the 'clc/' subdirectory."; exit 1; fi
19 @ANT_OPTS="-Xmx512m" $(ANT) build
20 @make -C modules/storage-controller/native
21+ @if test "${STORAGE_MODULE}" = "storage-rados"; then \
22+ make -C modules/storage-rados/native RADOS_HOME="${RADOS_HOME}"; \
23+ fi
24 @make -C modules/bootstrap
25
26 clean:
27 $(ANT) clean
28 make -C modules/bootstrap clean
29+ @if test "${STORAGE_MODULE}" = "storage-rados"; then \
30+ make -C modules/storage-rados/native clean; \
31+ fi
32 make -C modules/storage-controller/native clean
33
34 distclean: clean
35 $(ANT) distclean
36 make -C modules/bootstrap distclean
37+ @if test "${STORAGE_MODULE}" = "storage-rados"; then \
38+ make -C modules/storage-rados/native distclean; \
39+ fi
40 make -C modules/storage-controller/native distclean
41
42 install: build
43 @$(ANT) install
44 @make -C modules/storage-controller/native install
45+ @if test "${STORAGE_MODULE}" = "storage-rados"; then \
46+ make -C modules/storage-rados/native install; \
47+ fi
48 @make -C modules/bootstrap install
49
50 deploy: install
51
52 uninstall:
53 make -C modules/storage-controller/native uninstall
54+ @if test "${STORAGE_MODULE}" = "storage-rados"; then \
55+ make -C modules/storage-rados/native uninstall; \
56+ fi
57 make -C modules/bootstrap uninstall
58 $(RM) -rf $(DESTDIR)$(etcdir)/eucalyptus/cloud.d
59 $(RM) -rf $(DESTDIR)$(datarootdir)/eucalyptus/*jar
60
61=== modified file 'clc/build.xml'
62--- clc/build.xml 2010-11-09 19:31:25 +0000
63+++ clc/build.xml 2010-11-15 08:38:14 +0000
64@@ -98,6 +98,7 @@
65 <ant dir="modules/cloud" inheritall="false" target="builder" />
66 <ant dir="modules/cluster-manager" inheritall="false" target="builder" />
67 <ant dir="modules/storage-common" inheritall="false" target="builder" />
68+ <ant dir="modules/storage-rados" inheritall="false" target="builder" />
69 <ant dir="modules/walrus" inheritall="false" target="builder" />
70 <ant dir="modules/storage-controller" inheritall="false" target="builder" />
71 <ant dir="modules/dns" inheritall="false" target="builder" />
72@@ -122,6 +123,7 @@
73 <ant dir="modules/wsstack" inheritall="false" target="jar" />
74 <ant dir="modules/cluster-manager" inheritall="false" target="jar" />
75 <ant dir="modules/storage-common" inheritall="false" target="jar" />
76+ <ant dir="modules/storage-rados" inheritall="false" target="jar" />
77 <ant dir="modules/walrus" inheritall="false" target="jar" />
78 <ant dir="modules/storage-controller" inheritall="false" target="jar" />
79 <ant dir="modules/dns" inheritall="false" target="jar" />
80@@ -177,6 +179,7 @@
81 <ant dir="modules/wsstack" inheritall="false" target="clean" />
82 <ant dir="modules/cluster-manager" inheritall="false" target="clean" />
83 <ant dir="modules/storage-common" inheritall="false" target="clean" />
84+ <ant dir="modules/storage-rados" inheritall="false" target="clean" />
85 <ant dir="modules/walrus" inheritall="false" target="clean" />
86 <ant dir="modules/storage-controller" inheritall="false" target="clean" />
87 <ant dir="modules/dns" inheritall="false" target="clean" />
88@@ -244,6 +247,11 @@
89 <ant dir="modules/core" inheritall="false" target="install" />
90 <ant dir="modules/wsstack" inheritall="false" target="install" />
91 <ant dir="modules/storage-common" inheritall="false" target="install" />
92+<<<<<<< TREE
93+=======
94+ <ant dir="modules/storage-rados" inheritall="false" target="install" />
95+ <ant dir="modules/interface" inheritall="false" target="install" />
96+>>>>>>> MERGE-SOURCE
97 <ant dir="modules/cluster-manager" inheritall="false" target="install" />
98 <ant dir="modules/walrus" inheritall="false" target="install" />
99 <ant dir="modules/storage-controller" inheritall="false" target="install" />
100
101=== added file 'clc/build.xml.in'
102--- clc/build.xml.in 1970-01-01 00:00:00 +0000
103+++ clc/build.xml.in 2010-11-15 08:38:14 +0000
104@@ -0,0 +1,292 @@
105+<!--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
106+ ~ Copyright (c) 2009 Eucalyptus Systems, Inc.
107+ ~
108+ ~ This program is free software: you can redistribute it and/or modify
109+ ~ it under the terms of the GNU General Public License as published by
110+ ~ the Free Software Foundation, only version 3 of the License.
111+ ~
112+ ~
113+ ~ This file is distributed in the hope that it will be useful, but WITHOUT
114+ ~ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
115+ ~ FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
116+ ~ for more details.
117+ ~
118+ ~ You should have received a copy of the GNU General Public License along
119+ ~ with this program. If not, see http://www.gnu.org/licenses/.
120+ ~
121+ ~ Please contact Eucalyptus Systems, Inc., 130 Castilian
122+ ~ Dr., Goleta, CA 93101 USA or visit http://www.eucalyptus.com/licenses/
123+ ~ if you need additional information or have any questions.
124+ ~
125+ ~ This file may incorporate work covered under the following copyright and
126+ ~ permission notice:
127+ ~
128+ ~ Software License Agreement (BSD License)
129+ ~
130+ ~ Copyright (c) 2008, Regents of the University of California
131+ ~ All rights reserved.
132+ ~
133+ ~ Redistribution and use of this software in source and binary forms, with
134+ ~ or without modification, are permitted provided that the following
135+ ~ conditions are met:
136+ ~
137+ ~ Redistributions of source code must retain the above copyright notice,
138+ ~ this list of conditions and the following disclaimer.
139+ ~
140+ ~ Redistributions in binary form must reproduce the above copyright
141+ ~ notice, this list of conditions and the following disclaimer in the
142+ ~ documentation and/or other materials provided with the distribution.
143+ ~
144+ ~ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
145+ ~ IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
146+ ~ TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
147+ ~ PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
148+ ~ OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
149+ ~ EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
150+ ~ PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
151+ ~ PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
152+ ~ LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
153+ ~ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
154+ ~ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
155+ ~ THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
156+ ~ LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
157+ ~ SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
158+ ~ IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
159+ ~ BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
160+ ~ THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
161+ ~ OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
162+ ~ WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
163+ ~ ANY SUCH LICENSES OR RIGHTS.
164+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
165+ ~ Author: chris grzegorczyk grze@eucalyptus.com
166+ -->
167+
168+<project name="eucalyptus" basedir=".">
169+ <property environment="localenv" />
170+ <property name="euca.home.0" value="${localenv.EUCALYPTUS}" />
171+ <condition property="euca.home" value="/opt/eucalyptus" else="${localenv.EUCALYPTUS}">
172+ <contains string="${euca.home.0}" substring="localenv" />
173+ </condition>
174+ <property name="DESTDIR.0" value="${localenv.DESTDIR}" />
175+ <condition property="DESTDIR" value="/" else="${localenv.DESTDIR}">
176+ <contains string="${DESTDIR.0}" substring="localenv" />
177+ </condition>
178+ <property name="project.basedir" value="${user.dir}" />
179+ <loadfile srcfile="${project.basedir}/../VERSION" property="euca.version">
180+ <filterchain>
181+ <filterreader classname="org.apache.tools.ant.filters.StripLineBreaks" />
182+ </filterchain>
183+ </loadfile>
184+ <!--================================== other vars ==================================-->
185+ <property name="jvm.heap" value="256m" />
186+ <property name="deps.url" value="http://eucalyptussoftware.com/downloads/dependencies/${euca.version}" />
187+ <property name="deps.dir" value="deps" />
188+ <property name="deps.lib.dir" value="lib" />
189+ <property name="tools.dir" value="tools/src" />
190+ <property name="deps.lib" value="cloud-lib" />
191+ <property name="bind.timestamp" value="target/.bind-timestamp" />
192+ <!--================================== build targets ==================================-->
193+ <target name="build">
194+ <ant dir="modules/msgs" inheritall="false" target="builder" />
195+ <ant dir="modules/database" inheritall="false" target="builder" />
196+ <ant dir="modules/hsqldb" inheritall="false" target="builder" />
197+ <ant dir="modules/authentication" inheritall="false" target="builder" />
198+ <ant dir="modules/cloud" inheritall="false" target="builder" />
199+ <ant dir="modules/configuration" inheritall="false" target="builder" />
200+ <ant dir="modules/core" inheritall="false" target="builder" />
201+ <ant dir="modules/wsstack" inheritall="false" target="builder" />
202+ <ant dir="modules/component" inheritall="false" target="builder" />
203+ <ant dir="modules/interface" inheritall="false" target="builder" />
204+ <ant dir="modules/group-manager" inheritall="false" target="builder" />
205+ <ant dir="modules/cluster-manager" inheritall="false" target="builder" />
206+ <ant dir="modules/image-manager" inheritall="false" target="builder" />
207+ <ant dir="modules/key-manager" inheritall="false" target="builder" />
208+ <ant dir="modules/storage-common" inheritall="false" target="builder" />
209+ <ant dir="modules/@STORAGE_MODULE@" inheritall="false" target="builder" />
210+ <ant dir="modules/walrus" inheritall="false" target="builder" />
211+ <ant dir="modules/storage-controller" inheritall="false" target="builder" />
212+ <ant dir="modules/dns" inheritall="false" target="builder" />
213+ <ant dir="modules/www" inheritall="false" target="builder" />
214+ <ant dir="." inheritall="false" target="jar" />
215+ </target>
216+ <target name="jar" depends="bind">
217+ <ant dir="modules/msgs" inheritall="false" target="jar" />
218+ <ant dir="modules/database" inheritall="false" target="jar" />
219+ <ant dir="modules/hsqldb" inheritall="false" target="jar" />
220+ <ant dir="modules/authentication" inheritall="false" target="jar" />
221+ <ant dir="modules/cloud" inheritall="false" target="jar" />
222+ <ant dir="modules/configuration" inheritall="false" target="jar" />
223+ <ant dir="modules/component" inheritall="false" target="jar" />
224+ <ant dir="modules/core" inheritall="false" target="jar" />
225+ <ant dir="modules/wsstack" inheritall="false" target="jar" />
226+ <ant dir="modules/interface" inheritall="false" target="jar" />
227+ <ant dir="modules/group-manager" inheritall="false" target="jar" />
228+ <ant dir="modules/cluster-manager" inheritall="false" target="jar" />
229+ <ant dir="modules/image-manager" inheritall="false" target="jar" />
230+ <ant dir="modules/key-manager" inheritall="false" target="jar" />
231+ <ant dir="modules/storage-common" inheritall="false" target="jar" />
232+ <ant dir="modules/@STORAGE_MODULE@" inheritall="false" target="jar" />
233+ <ant dir="modules/walrus" inheritall="false" target="jar" />
234+ <ant dir="modules/storage-controller" inheritall="false" target="jar" />
235+ <ant dir="modules/dns" inheritall="false" target="jar" />
236+ <ant dir="modules/www" inheritall="false" target="jar" />
237+ </target>
238+ <target name="shouldBind">
239+ <uptodate property="bind.notRequired" targetfile="${bind.timestamp}">
240+ <srcfiles dir="modules">
241+ <include name="**/*.java"/>
242+ </srcfiles>
243+ </uptodate>
244+ <echo message="[BINDING] Up-to-date check shows bind.notRequred=${bind.notRequired}." />
245+ </target>
246+ <target name="bind" depends="shouldBind" unless="bind.notRequired">
247+ <echo message="[BINDING] Running binding compiler." />
248+ <path id="bindingclasspath">
249+ <dirset dir="modules">
250+ <include name="**/build" />
251+ </dirset>
252+ <fileset dir="lib">
253+ <include name="**/*.jar" />
254+ </fileset>
255+ </path>
256+ <delete>
257+ <fileset dir="modules" includes="**/*JiBX*" />
258+ </delete>
259+ <delete file="modules/msgs/src/main/resources/msgs-binding.xml" />
260+ <taskdef name="bindings" classname="com.eucalyptus.binding.BuildBindings" classpathref="bindingclasspath" />
261+ <bindings>
262+ <classfileset dir="${project.basedir}/modules" includes="*/build/**" />
263+ <bindingfileset dir="${project.basedir}/modules" includes="**/*-binding.xml" />
264+ </bindings>
265+ <touch file="${bind.timestamp}"/>
266+ </target>
267+
268+ <!--================================== clean targets ==================================-->
269+ <target name="clean">
270+ <delete file="${bind.timestamp}"/>
271+ <ant dir="modules/msgs" inheritall="false" target="clean" />
272+ <ant dir="modules/database" inheritall="false" target="clean" />
273+ <ant dir="modules/hsqldb" inheritall="false" target="clean" />
274+ <ant dir="modules/cloud" inheritall="false" target="clean" />
275+ <ant dir="modules/authentication" inheritall="false" target="clean" />
276+ <ant dir="modules/configuration" inheritall="false" target="clean" />
277+ <ant dir="modules/component" inheritall="false" target="clean" />
278+ <ant dir="modules/core" inheritall="false" target="clean" />
279+ <ant dir="modules/wsstack" inheritall="false" target="clean" />
280+ <ant dir="modules/interface" inheritall="false" target="clean" />
281+ <ant dir="modules/cluster-manager" inheritall="false" target="clean" />
282+ <ant dir="modules/image-manager" inheritall="false" target="clean" />
283+ <ant dir="modules/group-manager" inheritall="false" target="clean" />
284+ <ant dir="modules/key-manager" inheritall="false" target="clean" />
285+ <ant dir="modules/storage-common" inheritall="false" target="clean" />
286+ <ant dir="modules/@STORAGE_MODULE@" inheritall="false" target="clean" />
287+ <ant dir="modules/walrus" inheritall="false" target="clean" />
288+ <ant dir="modules/storage-controller" inheritall="false" target="clean" />
289+ <ant dir="modules/dns" inheritall="false" target="clean" />
290+ <ant dir="modules/www" inheritall="false" target="clean" />
291+ <delete dir="${target.dir}" />
292+ </target>
293+ <target name="distclean" depends="clean">
294+ <delete dir="${deps.dir}" />
295+ <delete dir="${deps.lib.dir}" />
296+ <delete dir="target" />
297+ </target>
298+ <!--================================== install target ==================================-->
299+ <property name="euca.conf.dir" value="${euca.home}/etc/eucalyptus/cloud.d" />
300+ <property name="euca.lib.dir" value="${euca.home}/usr/share/eucalyptus" />
301+ <property name="euca.var.dir" value="${euca.home}/var/lib/eucalyptus" />
302+ <property name="euca.run.dir" value="${euca.home}/var/run/eucalyptus" />
303+ <property name="euca.log.dir" value="${euca.home}/var/log/eucalyptus" />
304+ <property name="euca.bin.dir" value="${euca.home}/usr/sbin" />
305+ <target name="install">
306+ <!-- copy libraries over -->
307+ <mkdir dir="${DESTDIR}${euca.lib.dir}" />
308+ <copy todir="${DESTDIR}${euca.lib.dir}" overwrite="true">
309+ <fileset dir="${deps.lib.dir}">
310+ <include name="**/*" />
311+ <exclude name="**/gwt-api-checker-*.jar" />
312+ <exclude name="**/gwt-dev-*.jar" />
313+ <exclude name="**/gwt-soyc-vis-*.jar" />
314+ <exclude name="**/gwt-user-*.jar" />
315+ </fileset>
316+ </copy>
317+ <!--copy admin tools -->
318+ <mkdir dir="${DESTDIR}${euca.bin.dir}" />
319+ <copy todir="${DESTDIR}${euca.bin.dir}" overwrite="true">
320+ <fileset dir="${tools.dir}">
321+ <include name="**/*" />
322+ </fileset>
323+ </copy>
324+ <chmod dir="${DESTDIR}${euca.bin.dir}" perm="0755">
325+ <include name="**/euca-*" />
326+ </chmod>
327+ <mkdir dir="${DESTDIR}${euca.conf.dir}" />
328+ <chmod dir="${DESTDIR}${euca.conf.dir}" perm="0755" />
329+ <mkdir dir="${DESTDIR}${euca.var.dir}/" />
330+ <mkdir dir="${DESTDIR}${euca.var.dir}/db" />
331+ <mkdir dir="${DESTDIR}${euca.var.dir}/keys" />
332+ <mkdir dir="${DESTDIR}${euca.log.dir}" />
333+ <mkdir dir="${DESTDIR}${euca.var.dir}/webapps" />
334+ <mkdir dir="${DESTDIR}${euca.var.dir}/modules" />
335+ <mkdir dir="${DESTDIR}${euca.run.dir}" />
336+ <ant dir="modules/msgs" inheritall="false" target="install" />
337+ <ant dir="modules/database" inheritall="false" target="install" />
338+ <ant dir="modules/hsqldb" inheritall="false" target="install" />
339+ <ant dir="modules/cloud" inheritall="false" target="install" />
340+ <ant dir="modules/authentication" inheritall="false" target="install" />
341+ <ant dir="modules/configuration" inheritall="false" target="install" />
342+ <ant dir="modules/component" inheritall="false" target="install" />
343+ <ant dir="modules/core" inheritall="false" target="install" />
344+ <ant dir="modules/wsstack" inheritall="false" target="install" />
345+ <ant dir="modules/storage-common" inheritall="false" target="install" />
346+ <ant dir="modules/@STORAGE_MODULE@" inheritall="false" target="install" />
347+ <ant dir="modules/interface" inheritall="false" target="install" />
348+ <ant dir="modules/cluster-manager" inheritall="false" target="install" />
349+ <ant dir="modules/image-manager" inheritall="false" target="install" />
350+ <ant dir="modules/group-manager" inheritall="false" target="install" />
351+ <ant dir="modules/key-manager" inheritall="false" target="install" />
352+ <ant dir="modules/walrus" inheritall="false" target="install" />
353+ <ant dir="modules/storage-controller" inheritall="false" target="install" />
354+ <ant dir="modules/dns" inheritall="false" target="install" />
355+ <ant dir="modules/www" inheritall="false" target="install" />
356+ </target>
357+ <!--================================== unzipping target ==================================-->
358+ <target name="deps">
359+ <condition property="needDownload">
360+ <and>
361+ <not>
362+ <available file="${deps.lib.dir}/bcprov.jar" />
363+ </not>
364+ <and>
365+ <not>
366+ <available file="${deps.dir}/${deps.lib}.tar.gz" />
367+ </not>
368+ </and>
369+ </and>
370+ </condition>
371+ <ant target="download-deps" />
372+ <condition property="needUnzipping">
373+ <and>
374+ <not>
375+ <available file="${deps.lib.dir}/bcprov.jar" />
376+ </not>
377+ <and>
378+ <available file="${deps.dir}/${deps.lib}.tar.gz" />
379+ </and>
380+ </and>
381+ </condition>
382+ <ant target="untar" />
383+ </target>
384+ <target name="dist-deps" depends="deps">
385+ <delete dir="${deps.dir}" />
386+ </target>
387+ <target name="untar" if="needUnzipping">
388+ <untar compression="gzip" src="${deps.dir}/${deps.lib}.tar.gz" dest="${deps.lib.dir}" />
389+ </target>
390+ <target name="download-deps" if="needDownload">
391+ <mkdir dir="${deps.dir}" />
392+ <mkdir dir="${deps.lib.dir}" />
393+ <!--=========== download the dependency libraries ===========-->
394+ <get src="${deps.url}/${deps.lib}.tar.gz" dest="${deps.dir}/${deps.lib}.tar.gz" />
395+ </target>
396+</project>
397
398=== added file 'clc/modules/database/conf/scripts/caches.groovy.OTHER'
399--- clc/modules/database/conf/scripts/caches.groovy.OTHER 1970-01-01 00:00:00 +0000
400+++ clc/modules/database/conf/scripts/caches.groovy.OTHER 2010-11-15 08:38:14 +0000
401@@ -0,0 +1,7 @@
402+cacheProps = [
403+ 'hibernate.cache.provider_class': 'net.sf.ehcache.hibernate.SingletonEhCacheProvider',
404+ 'hibernate.cache.region_prefix': "eucalyptus_${context_name}_cache",
405+ 'hibernate.cache.use_second_level_cache': 'false',
406+ 'hibernate.cache.use_query_cache': 'false',
407+ 'hibernate.cache.use_structured_entries': 'false',
408+]
409
410=== removed directory 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud'
411=== removed directory 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws'
412=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java'
413--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java 2009-08-27 18:38:18 +0000
414+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java 1970-01-01 00:00:00 +0000
415@@ -1,77 +0,0 @@
416-/*******************************************************************************
417-*Copyright (c) 2009 Eucalyptus Systems, Inc.
418-*
419-* This program is free software: you can redistribute it and/or modify
420-* it under the terms of the GNU General Public License as published by
421-* the Free Software Foundation, only version 3 of the License.
422-*
423-*
424-* This file is distributed in the hope that it will be useful, but WITHOUT
425-* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
426-* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
427-* for more details.
428-*
429-* You should have received a copy of the GNU General Public License along
430-* with this program. If not, see <http://www.gnu.org/licenses/>.
431-*
432-* Please contact Eucalyptus Systems, Inc., 130 Castilian
433-* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
434-* if you need additional information or have any questions.
435-*
436-* This file may incorporate work covered under the following copyright and
437-* permission notice:
438-*
439-* Software License Agreement (BSD License)
440-*
441-* Copyright (c) 2008, Regents of the University of California
442-* All rights reserved.
443-*
444-* Redistribution and use of this software in source and binary forms, with
445-* or without modification, are permitted provided that the following
446-* conditions are met:
447-*
448-* Redistributions of source code must retain the above copyright notice,
449-* this list of conditions and the following disclaimer.
450-*
451-* Redistributions in binary form must reproduce the above copyright
452-* notice, this list of conditions and the following disclaimer in the
453-* documentation and/or other materials provided with the distribution.
454-*
455-* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
456-* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
457-* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
458-* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
459-* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
460-* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
461-* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
462-* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
463-* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
464-* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
465-* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
466-* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
467-* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
468-* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
469-* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
470-* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
471-* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
472-* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
473-* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
474-* ANY SUCH LICENSES OR RIGHTS.
475-*******************************************************************************/
476-package edu.ucsb.eucalyptus.cloud.ws;
477-
478-import java.io.IOException;
479-import java.io.RandomAccessFile;
480-
481-import org.jboss.netty.handler.stream.ChunkedFile;
482-
483-import edu.ucsb.eucalyptus.constants.IsData;
484-
485-public class ChunkedDataFile extends ChunkedFile implements IsData {
486-
487- public ChunkedDataFile(RandomAccessFile file, long offset, long length,
488- int chunkSize) throws IOException {
489- super(file, offset, length, chunkSize);
490- }
491-}
492-
493
494=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java'
495--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java 2009-08-27 18:38:18 +0000
496+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java 1970-01-01 00:00:00 +0000
497@@ -1,120 +0,0 @@
498-/*******************************************************************************
499-*Copyright (c) 2009 Eucalyptus Systems, Inc.
500-*
501-* This program is free software: you can redistribute it and/or modify
502-* it under the terms of the GNU General Public License as published by
503-* the Free Software Foundation, only version 3 of the License.
504-*
505-*
506-* This file is distributed in the hope that it will be useful, but WITHOUT
507-* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
508-* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
509-* for more details.
510-*
511-* You should have received a copy of the GNU General Public License along
512-* with this program. If not, see <http://www.gnu.org/licenses/>.
513-*
514-* Please contact Eucalyptus Systems, Inc., 130 Castilian
515-* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
516-* if you need additional information or have any questions.
517-*
518-* This file may incorporate work covered under the following copyright and
519-* permission notice:
520-*
521-* Software License Agreement (BSD License)
522-*
523-* Copyright (c) 2008, Regents of the University of California
524-* All rights reserved.
525-*
526-* Redistribution and use of this software in source and binary forms, with
527-* or without modification, are permitted provided that the following
528-* conditions are met:
529-*
530-* Redistributions of source code must retain the above copyright notice,
531-* this list of conditions and the following disclaimer.
532-*
533-* Redistributions in binary form must reproduce the above copyright
534-* notice, this list of conditions and the following disclaimer in the
535-* documentation and/or other materials provided with the distribution.
536-*
537-* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
538-* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
539-* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
540-* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
541-* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
542-* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
543-* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
544-* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
545-* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
546-* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
547-* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
548-* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
549-* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
550-* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
551-* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
552-* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
553-* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
554-* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
555-* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
556-* ANY SUCH LICENSES OR RIGHTS.
557-*******************************************************************************/
558-package edu.ucsb.eucalyptus.cloud.ws;
559-
560-import java.io.ByteArrayOutputStream;
561-import java.io.RandomAccessFile;
562-import java.util.zip.GZIPOutputStream;
563-
564-import org.apache.log4j.Logger;
565-import org.jboss.netty.buffer.ChannelBuffers;
566-import org.jboss.netty.handler.stream.ChunkedInput;
567-
568-import edu.ucsb.eucalyptus.constants.IsData;
569-
570-public class CompressedChunkedFile implements ChunkedInput, IsData {
571- private Logger LOG = Logger.getLogger( CompressedChunkedFile.class );
572- private RandomAccessFile file;
573- private long offset;
574- private int CHUNK_SIZE = 8192;
575- private long fileLength;
576-
577- public CompressedChunkedFile(RandomAccessFile file, long fileLength) {
578- this.file = file;
579- this.offset = 0;
580- this.fileLength = fileLength;
581- }
582-
583- public CompressedChunkedFile(RandomAccessFile file, long start, long end, int chunkSize) {
584- this.file = file;
585- this.offset = start;
586- this.fileLength = end;
587- CHUNK_SIZE = chunkSize;
588- }
589-
590- @Override
591- public void close() throws Exception {
592- file.close();
593- }
594-
595- @Override
596- public boolean hasNextChunk() throws Exception {
597- return offset < fileLength && file.getChannel().isOpen();
598- }
599-
600- @Override
601- public Object nextChunk() throws Exception {
602- long offset = this.offset;
603- if (offset >= fileLength) {
604- return null;
605- }
606- int chunkSize = (int) Math.min(CHUNK_SIZE, fileLength - offset);
607- byte[] chunk = new byte[chunkSize];
608- file.readFully(chunk);
609- this.offset = offset + chunkSize;
610- ByteArrayOutputStream out = new ByteArrayOutputStream();
611- GZIPOutputStream zip = new GZIPOutputStream(out);
612- zip.write(chunk);
613- zip.close();
614- return ChannelBuffers.wrappedBuffer(out.toByteArray());
615- }
616-}
617-
618
619=== added file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageIO.java'
620--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageIO.java 1970-01-01 00:00:00 +0000
621+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageIO.java 2010-11-15 08:38:14 +0000
622@@ -0,0 +1,80 @@
623+/*******************************************************************************
624+*Copyright (c) 2009 Eucalyptus Systems, Inc.
625+*
626+* This program is free software: you can redistribute it and/or modify
627+* it under the terms of the GNU General Public License as published by
628+* the Free Software Foundation, only version 3 of the License.
629+*
630+*
631+* This file is distributed in the hope that it will be useful, but WITHOUT
632+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
633+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
634+* for more details.
635+*
636+* You should have received a copy of the GNU General Public License along
637+* with this program. If not, see <http://www.gnu.org/licenses/>.
638+*
639+* Please contact Eucalyptus Systems, Inc., 130 Castilian
640+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
641+* if you need additional information or have any questions.
642+*
643+* This file may incorporate work covered under the following copyright and
644+* permission notice:
645+*
646+* Software License Agreement (BSD License)
647+*
648+* Copyright (c) 2008, Regents of the University of California
649+* All rights reserved.
650+*
651+* Redistribution and use of this software in source and binary forms, with
652+* or without modification, are permitted provided that the following
653+* conditions are met:
654+*
655+* Redistributions of source code must retain the above copyright notice,
656+* this list of conditions and the following disclaimer.
657+*
658+* Redistributions in binary form must reproduce the above copyright
659+* notice, this list of conditions and the following disclaimer in the
660+* documentation and/or other materials provided with the distribution.
661+*
662+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
663+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
664+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
665+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
666+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
667+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
668+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
669+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
670+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
671+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
672+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
673+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
674+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
675+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
676+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
677+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
678+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
679+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
680+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
681+* ANY SUCH LICENSES OR RIGHTS.
682+*******************************************************************************/
683+/*
684+ *
685+ * Author: Sunil Soman sunils@cs.ucsb.edu
686+ */
687+
688+package edu.ucsb.eucalyptus.storage;
689+
690+import java.io.IOException;
691+import java.nio.ByteBuffer;
692+
693+public abstract class StorageIO {
694+
695+ public abstract int read(long offset) throws IOException;
696+
697+ public abstract void write(byte[] bytes) throws IOException;
698+
699+ public abstract ByteBuffer getBuffer();
700+
701+ public abstract void finish();
702+}
703
704=== modified file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageManager.java'
705--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageManager.java 2010-04-26 20:50:11 +0000
706+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageManager.java 2010-11-15 08:38:14 +0000
707@@ -75,7 +75,6 @@
708
709 import edu.ucsb.eucalyptus.cloud.BucketLogData;
710 import edu.ucsb.eucalyptus.msgs.WalrusDataGetRequestType;
711-import edu.ucsb.eucalyptus.storage.fs.FileIO;
712
713 public interface StorageManager {
714
715@@ -95,9 +94,9 @@
716
717 public void putObject(String bucket, String object, byte[] base64Data, boolean append) throws IOException;
718
719- public FileIO prepareForRead(String bucket, String object) throws Exception;
720+ public StorageIO prepareForRead(String bucket, String object) throws Exception;
721
722- public FileIO prepareForWrite(String bucket, String object) throws Exception;
723+ public StorageIO prepareForWrite(String bucket, String object) throws Exception;
724
725 public int readObject(String bucket, String object, byte[] bytes, long offset) throws IOException;
726
727
728=== removed directory 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs'
729=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileIO.java'
730--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileIO.java 2009-08-27 18:38:18 +0000
731+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileIO.java 1970-01-01 00:00:00 +0000
732@@ -1,83 +0,0 @@
733-/*******************************************************************************
734-*Copyright (c) 2009 Eucalyptus Systems, Inc.
735-*
736-* This program is free software: you can redistribute it and/or modify
737-* it under the terms of the GNU General Public License as published by
738-* the Free Software Foundation, only version 3 of the License.
739-*
740-*
741-* This file is distributed in the hope that it will be useful, but WITHOUT
742-* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
743-* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
744-* for more details.
745-*
746-* You should have received a copy of the GNU General Public License along
747-* with this program. If not, see <http://www.gnu.org/licenses/>.
748-*
749-* Please contact Eucalyptus Systems, Inc., 130 Castilian
750-* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
751-* if you need additional information or have any questions.
752-*
753-* This file may incorporate work covered under the following copyright and
754-* permission notice:
755-*
756-* Software License Agreement (BSD License)
757-*
758-* Copyright (c) 2008, Regents of the University of California
759-* All rights reserved.
760-*
761-* Redistribution and use of this software in source and binary forms, with
762-* or without modification, are permitted provided that the following
763-* conditions are met:
764-*
765-* Redistributions of source code must retain the above copyright notice,
766-* this list of conditions and the following disclaimer.
767-*
768-* Redistributions in binary form must reproduce the above copyright
769-* notice, this list of conditions and the following disclaimer in the
770-* documentation and/or other materials provided with the distribution.
771-*
772-* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
773-* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
774-* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
775-* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
776-* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
777-* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
778-* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
779-* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
780-* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
781-* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
782-* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
783-* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
784-* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
785-* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
786-* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
787-* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
788-* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
789-* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
790-* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
791-* ANY SUCH LICENSES OR RIGHTS.
792-*******************************************************************************/
793-/*
794- *
795- * Author: Sunil Soman sunils@cs.ucsb.edu
796- */
797-
798-package edu.ucsb.eucalyptus.storage.fs;
799-
800-import java.io.IOException;
801-import java.nio.ByteBuffer;
802-import java.nio.channels.FileChannel;
803-
804-public abstract class FileIO {
805-
806- protected FileChannel channel;
807-
808- public abstract int read(long offset) throws IOException;
809-
810- public abstract void write(byte[] bytes) throws IOException;
811-
812- public abstract ByteBuffer getBuffer();
813-
814- public abstract void finish();
815-}
816
817=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java'
818--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java 2009-10-02 17:47:47 +0000
819+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java 1970-01-01 00:00:00 +0000
820@@ -1,110 +0,0 @@
821-/*******************************************************************************
822-*Copyright (c) 2009 Eucalyptus Systems, Inc.
823-*
824-* This program is free software: you can redistribute it and/or modify
825-* it under the terms of the GNU General Public License as published by
826-* the Free Software Foundation, only version 3 of the License.
827-*
828-*
829-* This file is distributed in the hope that it will be useful, but WITHOUT
830-* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
831-* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
832-* for more details.
833-*
834-* You should have received a copy of the GNU General Public License along
835-* with this program. If not, see <http://www.gnu.org/licenses/>.
836-*
837-* Please contact Eucalyptus Systems, Inc., 130 Castilian
838-* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
839-* if you need additional information or have any questions.
840-*
841-* This file may incorporate work covered under the following copyright and
842-* permission notice:
843-*
844-* Software License Agreement (BSD License)
845-*
846-* Copyright (c) 2008, Regents of the University of California
847-* All rights reserved.
848-*
849-* Redistribution and use of this software in source and binary forms, with
850-* or without modification, are permitted provided that the following
851-* conditions are met:
852-*
853-* Redistributions of source code must retain the above copyright notice,
854-* this list of conditions and the following disclaimer.
855-*
856-* Redistributions in binary form must reproduce the above copyright
857-* notice, this list of conditions and the following disclaimer in the
858-* documentation and/or other materials provided with the distribution.
859-*
860-* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
861-* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
862-* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
863-* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
864-* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
865-* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
866-* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
867-* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
868-* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
869-* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
870-* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
871-* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
872-* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
873-* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
874-* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
875-* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
876-* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
877-* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
878-* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
879-* ANY SUCH LICENSES OR RIGHTS.
880-*******************************************************************************/
881-/*
882- *
883- * Author: Sunil Soman sunils@cs.ucsb.edu
884- */
885-
886-package edu.ucsb.eucalyptus.storage.fs;
887-
888-import org.apache.log4j.Logger;
889-
890-import java.io.FileInputStream;
891-import java.io.FileNotFoundException;
892-import java.io.IOException;
893-import java.nio.ByteBuffer;
894-
895-public class FileReader extends FileIO {
896-
897- private static Logger LOG = Logger.getLogger(FileReader.class);
898- private ByteBuffer buffer;
899-
900- public FileReader(String filename) throws Exception {
901- try {
902- channel = new FileInputStream(filename).getChannel();
903- buffer = ByteBuffer.allocate(102400/*TODO: NEIL WalrusQueryDispatcher.DATA_MESSAGE_SIZE */);
904- } catch( FileNotFoundException ex) {
905- LOG.error(ex);
906- throw ex;
907- }
908- }
909-
910- public int read(long offset) throws IOException {
911- buffer.clear();
912- int bytesRead = channel.read(buffer, offset);
913- buffer.flip();
914- return bytesRead;
915- }
916-
917- public void write(byte[] bytes) throws IOException {}
918-
919- public ByteBuffer getBuffer() {
920- return buffer;
921- }
922-
923- public void finish() {
924- try {
925- channel.close();
926- } catch(IOException ex) {
927- LOG.error(ex);
928- }
929- }
930-}
931
932=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java'
933--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java 2010-11-03 19:16:45 +0000
934+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java 1970-01-01 00:00:00 +0000
935@@ -1,465 +0,0 @@
936-/*******************************************************************************
937- *Copyright (c) 2009 Eucalyptus Systems, Inc.
938- *
939- * This program is free software: you can redistribute it and/or modify
940- * it under the terms of the GNU General Public License as published by
941- * the Free Software Foundation, only version 3 of the License.
942- *
943- *
944- * This file is distributed in the hope that it will be useful, but WITHOUT
945- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
946- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
947- * for more details.
948- *
949- * You should have received a copy of the GNU General Public License along
950- * with this program. If not, see <http://www.gnu.org/licenses/>.
951- *
952- * Please contact Eucalyptus Systems, Inc., 130 Castilian
953- * Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
954- * if you need additional information or have any questions.
955- *
956- * This file may incorporate work covered under the following copyright and
957- * permission notice:
958- *
959- * Software License Agreement (BSD License)
960- *
961- * Copyright (c) 2008, Regents of the University of California
962- * All rights reserved.
963- *
964- * Redistribution and use of this software in source and binary forms, with
965- * or without modification, are permitted provided that the following
966- * conditions are met:
967- *
968- * Redistributions of source code must retain the above copyright notice,
969- * this list of conditions and the following disclaimer.
970- *
971- * Redistributions in binary form must reproduce the above copyright
972- * notice, this list of conditions and the following disclaimer in the
973- * documentation and/or other materials provided with the distribution.
974- *
975- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
976- * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
977- * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
978- * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
979- * OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
980- * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
981- * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
982- * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
983- * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
984- * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
985- * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
986- * THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
987- * LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
988- * SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
989- * IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
990- * BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
991- * THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
992- * OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
993- * WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
994- * ANY SUCH LICENSES OR RIGHTS.
995- *******************************************************************************/
996-/*
997- *
998- * Author: Sunil Soman sunils@cs.ucsb.edu
999- */
1000-
1001-package edu.ucsb.eucalyptus.storage.fs;
1002-
1003-import com.eucalyptus.auth.util.Hashes;
1004-import com.eucalyptus.context.Contexts;
1005-import com.eucalyptus.http.MappingHttpResponse;
1006-import com.eucalyptus.util.EucalyptusCloudException;
1007-import com.eucalyptus.util.ExecutionException;
1008-import com.eucalyptus.util.WalrusProperties;
1009-import com.eucalyptus.ws.util.ChannelUtil;
1010-import com.eucalyptus.ws.util.WalrusBucketLogger;
1011-
1012-import edu.ucsb.eucalyptus.cloud.BucketLogData;
1013-import edu.ucsb.eucalyptus.cloud.ws.CompressedChunkedFile;
1014-import edu.ucsb.eucalyptus.cloud.ws.ChunkedDataFile;
1015-import edu.ucsb.eucalyptus.msgs.WalrusDataGetRequestType;
1016-import edu.ucsb.eucalyptus.storage.StorageManager;
1017-import edu.ucsb.eucalyptus.util.StreamConsumer;
1018-import edu.ucsb.eucalyptus.util.SystemUtil;
1019-
1020-import org.apache.log4j.Logger;
1021-import org.jboss.netty.channel.Channel;
1022-import org.jboss.netty.channel.ChannelFuture;
1023-import org.jboss.netty.channel.ChannelFutureListener;
1024-import org.jboss.netty.handler.codec.http.DefaultHttpResponse;
1025-import org.jboss.netty.handler.codec.http.HttpHeaders;
1026-import org.jboss.netty.handler.stream.ChunkedInput;
1027-
1028-import java.io.*;
1029-import java.nio.channels.FileChannel;
1030-import java.util.ArrayList;
1031-import java.util.List;
1032-
1033-public class FileSystemStorageManager implements StorageManager {
1034-
1035- public static final String FILE_SEPARATOR = "/";
1036- public static final String lvmRootDirectory = "/dev";
1037- private static boolean initialized = false;
1038- private static String eucaHome = "/opt/eucalyptus";
1039- public static final String EUCA_ROOT_WRAPPER = "/usr/lib/eucalyptus/euca_rootwrap";
1040- public static final int MAX_LOOP_DEVICES = 256;
1041- private static Logger LOG = Logger.getLogger(FileSystemStorageManager.class);
1042-
1043- private String rootDirectory;
1044- public FileSystemStorageManager(String rootDirectory) {
1045- this.rootDirectory = rootDirectory;
1046- }
1047-
1048- public void checkPreconditions() throws EucalyptusCloudException {
1049- try {
1050- String eucaHomeDir = System.getProperty("euca.home");
1051- if(eucaHomeDir == null) {
1052- throw new EucalyptusCloudException("euca.home not set");
1053- }
1054- eucaHome = eucaHomeDir;
1055- if(!new File(eucaHome + EUCA_ROOT_WRAPPER).exists()) {
1056- throw new EucalyptusCloudException("root wrapper (euca_rootwrap) does not exist");
1057- }
1058- String returnValue = getLvmVersion();
1059- if(returnValue.length() == 0) {
1060- throw new EucalyptusCloudException("Is lvm installed?");
1061- } else {
1062- LOG.info(returnValue);
1063- }
1064- } catch(ExecutionException ex) {
1065- String error = "Unable to run command: " + ex.getMessage();
1066- LOG.error(error);
1067- throw new EucalyptusCloudException(error);
1068- }
1069- }
1070-
1071- public void setRootDirectory(String rootDirectory) {
1072- this.rootDirectory = rootDirectory;
1073- }
1074-
1075- public boolean bucketExists(String bucket) {
1076- return new File (rootDirectory + FILE_SEPARATOR + bucket).exists();
1077- }
1078-
1079- public boolean objectExists(String bucket, String object) {
1080- return new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object).exists();
1081- }
1082-
1083- public void createBucket(String bucket) throws IOException {
1084- File bukkit = new File (rootDirectory + FILE_SEPARATOR + bucket);
1085- if(!bukkit.exists()) {
1086- if(!bukkit.mkdirs()) {
1087- throw new IOException("Unable to create bucket: " + bucket);
1088- }
1089- }
1090- }
1091-
1092- public long getSize(String bucket, String object) {
1093- File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
1094- if(objectFile.exists())
1095- return objectFile.length();
1096- return -1;
1097- }
1098-
1099- public void deleteBucket(String bucket) throws IOException {
1100- File bukkit = new File (rootDirectory + FILE_SEPARATOR + bucket);
1101- if(!bukkit.delete()) {
1102- throw new IOException("Unable to delete bucket: " + bucket);
1103- }
1104- }
1105-
1106- public void createObject(String bucket, String object) throws IOException {
1107- File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
1108- if (!objectFile.exists()) {
1109- if (!objectFile.createNewFile()) {
1110- throw new IOException("Unable to create: " + objectFile.getAbsolutePath());
1111- }
1112- }
1113- }
1114-
1115- public FileIO prepareForRead(String bucket, String object) throws Exception {
1116- return new FileReader(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
1117- }
1118-
1119- public FileIO prepareForWrite(String bucket, String object) throws Exception {
1120- return new FileWriter(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
1121- }
1122-
1123- public int readObject(String bucket, String object, byte[] bytes, long offset) throws IOException {
1124- return readObject(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object, bytes, offset);
1125- }
1126-
1127- public int readObject(String path, byte[] bytes, long offset) throws IOException {
1128- File objectFile = new File (path);
1129- if (!objectFile.exists()) {
1130- throw new IOException("Unable to read: " + path);
1131- }
1132- BufferedInputStream inputStream = new BufferedInputStream(new FileInputStream(objectFile));
1133- if (offset > 0) {
1134- inputStream.skip(offset);
1135- }
1136- int bytesRead = inputStream.read(bytes);
1137- inputStream.close();
1138- return bytesRead;
1139- }
1140-
1141- public void deleteObject(String bucket, String object) throws IOException {
1142- File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
1143- if (objectFile.exists()) {
1144- if(!objectFile.delete()) {
1145- throw new IOException("Unable to delete: " + objectFile.getAbsolutePath());
1146- }
1147- }
1148- }
1149-
1150- public void deleteAbsoluteObject(String object) throws IOException {
1151- File objectFile = new File (object);
1152- if (objectFile.exists()) {
1153- if(!objectFile.delete()) {
1154- throw new IOException("Unable to delete: " + object);
1155- }
1156- }
1157- }
1158-
1159- public void putObject(String bucket, String object, byte[] base64Data, boolean append) throws IOException {
1160- File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
1161- if (!objectFile.exists()) {
1162- objectFile.createNewFile();
1163- }
1164- BufferedOutputStream outputStream = new BufferedOutputStream(new FileOutputStream(objectFile, append));
1165- outputStream.write(base64Data);
1166- outputStream.close();
1167- }
1168-
1169- public void renameObject(String bucket, String oldName, String newName) throws IOException {
1170- File oldObjectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + oldName);
1171- File newObjectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + newName);
1172- if(oldObjectFile.exists()) {
1173- if (!oldObjectFile.renameTo(newObjectFile)) {
1174- throw new IOException("Unable to rename " + oldObjectFile.getAbsolutePath() + " to " + newObjectFile.getAbsolutePath());
1175- }
1176- }
1177- }
1178-
1179- public void copyObject(String sourceBucket, String sourceObject, String destinationBucket, String destinationObject) throws IOException {
1180- File oldObjectFile = new File (rootDirectory + FILE_SEPARATOR + sourceBucket + FILE_SEPARATOR + sourceObject);
1181- File newObjectFile = new File (rootDirectory + FILE_SEPARATOR + destinationBucket + FILE_SEPARATOR + destinationObject);
1182- if(!oldObjectFile.equals(newObjectFile)) {
1183- FileInputStream fileInputStream = new FileInputStream(oldObjectFile);
1184- FileChannel fileIn = fileInputStream.getChannel();
1185- FileOutputStream fileOutputStream = new FileOutputStream(newObjectFile);
1186- FileChannel fileOut = fileOutputStream.getChannel();
1187- fileIn.transferTo(0, fileIn.size(), fileOut);
1188- fileIn.close();
1189- fileInputStream.close();
1190- fileOut.close();
1191- fileOutputStream.close();
1192- }
1193- }
1194-
1195- public String getObjectPath(String bucket, String object) {
1196- return rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object;
1197- }
1198-
1199- public long getObjectSize(String bucket, String object) {
1200- String absoluteObjectPath = rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object;
1201-
1202- File objectFile = new File(absoluteObjectPath);
1203- if(objectFile.exists())
1204- return objectFile.length();
1205- return -1;
1206- }
1207-
1208- public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
1209- try {
1210- Channel channel = request.getChannel();
1211- RandomAccessFile raf = new RandomAccessFile(new File(getObjectPath(bucketName, objectName)), "r");
1212- httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
1213- if(etag != null)
1214- httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
1215- httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
1216- if(contentDisposition != null)
1217- httpResponse.addHeader("Content-Disposition", contentDisposition);
1218- final ChunkedInput file;
1219- isCompressed = isCompressed == null ? false : isCompressed;
1220- if(isCompressed) {
1221- file = new CompressedChunkedFile(raf, size);
1222- } else {
1223- file = new ChunkedDataFile(raf, 0, size, 8192);
1224- httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
1225- }
1226- if(logData != null) {
1227- logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
1228- logData.setBytesSent(size);
1229- }
1230- if(versionId != null) {
1231- httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
1232- }
1233- channel.write(httpResponse);
1234- channel.write(file).addListener(new ChannelFutureListener( ) {
1235- @Override public void operationComplete( ChannelFuture future ) throws Exception {
1236- Contexts.clear(request.getCorrelationId());
1237- file.close();
1238- if(logData != null) {
1239- logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
1240- WalrusBucketLogger.getInstance().addLogEntry(logData);
1241- }
1242- }
1243- });
1244- } catch(Exception ex) {
1245- LOG.error(ex, ex);
1246- }
1247- }
1248-
1249- public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long start, long end, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
1250- try {
1251- Channel channel = request.getChannel();
1252- RandomAccessFile raf = new RandomAccessFile(new File(getObjectPath(bucketName, objectName)), "r");
1253- httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
1254- if(etag != null)
1255- httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
1256- httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
1257- if(contentDisposition != null)
1258- httpResponse.addHeader("Content-Disposition", contentDisposition);
1259- final ChunkedInput file;
1260- isCompressed = isCompressed == null ? false : isCompressed;
1261- if(isCompressed) {
1262- file = new CompressedChunkedFile(raf, start, end, (int)Math.min((end - start), 8192));
1263- } else {
1264- file = new ChunkedDataFile(raf, start, end, (int)Math.min((end - start), 8192));
1265- httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf((end - start)));
1266- }
1267- httpResponse.addHeader("Content-Range", start + "-" + end + "/" + size);
1268- if(logData != null) {
1269- logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
1270- logData.setBytesSent(size);
1271- }
1272- if(versionId != null) {
1273- httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
1274- }
1275- channel.write(httpResponse);
1276- channel.write(file).addListener(new ChannelFutureListener( ) {
1277- @Override public void operationComplete( ChannelFuture future ) throws Exception {
1278- Contexts.clear(request.getCorrelationId());
1279- file.close();
1280- if(logData != null) {
1281- logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
1282- WalrusBucketLogger.getInstance().addLogEntry(logData);
1283- }
1284- }
1285- });
1286- } catch(Exception ex) {
1287- LOG.error(ex, ex);
1288- }
1289- }
1290-
1291- public void sendHeaders(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, Long size, String etag,
1292- String lastModified, String contentType, String contentDisposition, String versionId, final BucketLogData logData) {
1293- Channel channel = request.getChannel();
1294- httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
1295- httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
1296- if(etag != null)
1297- httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
1298- httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
1299- if(contentDisposition != null)
1300- httpResponse.addHeader("Content-Disposition", contentDisposition);
1301- if(versionId != null) {
1302- httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
1303- }
1304- if(logData != null) {
1305- logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
1306- }
1307- channel.write(httpResponse).addListener(new ChannelFutureListener( ) {
1308- @Override public void operationComplete( ChannelFuture future ) throws Exception {
1309- Contexts.clear(request.getCorrelationId());
1310- if(logData != null) {
1311- logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
1312- WalrusBucketLogger.getInstance().addLogEntry(logData);
1313- }
1314- }
1315- });
1316- }
1317-
1318- private String removeLoopback(String loDevName) throws ExecutionException {
1319- return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-d", loDevName});
1320- }
1321-
1322- private int losetup(String absoluteFileName, String loDevName) {
1323- try
1324- {
1325- Runtime rt = Runtime.getRuntime();
1326- Process proc = rt.exec(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", loDevName, absoluteFileName});
1327- StreamConsumer error = new StreamConsumer(proc.getErrorStream());
1328- StreamConsumer output = new StreamConsumer(proc.getInputStream());
1329- error.start();
1330- output.start();
1331- int errorCode = proc.waitFor();
1332- output.join();
1333- LOG.info("losetup " + loDevName + " " + absoluteFileName);
1334- LOG.info(output.getReturnValue());
1335- LOG.info(error.getReturnValue());
1336- return errorCode;
1337- } catch (Throwable t) {
1338- LOG.error(t);
1339- }
1340- return -1;
1341- }
1342-
1343- private String findFreeLoopback() throws ExecutionException {
1344- return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-f"}).replaceAll("\n", "");
1345- }
1346-
1347- private String removeLogicalVolume(String lvName) throws ExecutionException {
1348- return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvremove", "-f", lvName});
1349- }
1350-
1351- private String reduceVolumeGroup(String vgName, String pvName) throws ExecutionException {
1352- return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "vgreduce", vgName, pvName});
1353- }
1354-
1355- private String removePhysicalVolume(String loDevName) throws ExecutionException {
1356- return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "pvremove", loDevName});
1357- }
1358-
1359- private String createVolumeFromLv(String lvName, String volumeKey) throws ExecutionException {
1360- return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "dd", "if=" + lvName, "of=" + volumeKey, "bs=1M"});
1361- }
1362-
1363- private String enableLogicalVolume(String lvName) throws ExecutionException {
1364- return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvchange", "-ay", lvName});
1365- }
1366-
1367- private String disableLogicalVolume(String lvName) throws ExecutionException {
1368- return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvchange", "-an", lvName});
1369- }
1370-
1371- private String removeVolumeGroup(String vgName) throws ExecutionException {
1372- return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "vgremove", vgName});
1373- }
1374-
1375- private String getLvmVersion() throws ExecutionException {
1376- return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvm", "version"});
1377- }
1378-
1379-
1380-
1381- public String createLoopback(String fileName) throws EucalyptusCloudException, ExecutionException {
1382- int number_of_retries = 0;
1383- int status = -1;
1384- String loDevName;
1385- do {
1386- loDevName = findFreeLoopback();
1387- if(loDevName.length() > 0) {
1388- status = losetup(fileName, loDevName);
1389- }
1390- if(number_of_retries++ >= MAX_LOOP_DEVICES)
1391- break;
1392- } while(status != 0);
1393-
1394- if(status != 0) {
1395- throw new EucalyptusCloudException("Could not create loopback device for " + fileName +
1396- ". Please check the max loop value and permissions");
1397- }
1398- return loDevName;
1399- }
1400-}
1401
1402=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java'
1403--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java 2009-10-02 17:47:47 +0000
1404+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java 1970-01-01 00:00:00 +0000
1405@@ -1,108 +0,0 @@
1406-/*******************************************************************************
1407-*Copyright (c) 2009 Eucalyptus Systems, Inc.
1408-*
1409-* This program is free software: you can redistribute it and/or modify
1410-* it under the terms of the GNU General Public License as published by
1411-* the Free Software Foundation, only version 3 of the License.
1412-*
1413-*
1414-* This file is distributed in the hope that it will be useful, but WITHOUT
1415-* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
1416-* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
1417-* for more details.
1418-*
1419-* You should have received a copy of the GNU General Public License along
1420-* with this program. If not, see <http://www.gnu.org/licenses/>.
1421-*
1422-* Please contact Eucalyptus Systems, Inc., 130 Castilian
1423-* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
1424-* if you need additional information or have any questions.
1425-*
1426-* This file may incorporate work covered under the following copyright and
1427-* permission notice:
1428-*
1429-* Software License Agreement (BSD License)
1430-*
1431-* Copyright (c) 2008, Regents of the University of California
1432-* All rights reserved.
1433-*
1434-* Redistribution and use of this software in source and binary forms, with
1435-* or without modification, are permitted provided that the following
1436-* conditions are met:
1437-*
1438-* Redistributions of source code must retain the above copyright notice,
1439-* this list of conditions and the following disclaimer.
1440-*
1441-* Redistributions in binary form must reproduce the above copyright
1442-* notice, this list of conditions and the following disclaimer in the
1443-* documentation and/or other materials provided with the distribution.
1444-*
1445-* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
1446-* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
1447-* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
1448-* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
1449-* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
1450-* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
1451-* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
1452-* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
1453-* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
1454-* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
1455-* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
1456-* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
1457-* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
1458-* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
1459-* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
1460-* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
1461-* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
1462-* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
1463-* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
1464-* ANY SUCH LICENSES OR RIGHTS.
1465-*******************************************************************************/
1466-/*
1467- *
1468- * Author: Sunil Soman sunils@cs.ucsb.edu
1469- */
1470-
1471-package edu.ucsb.eucalyptus.storage.fs;
1472-
1473-import org.apache.log4j.Logger;
1474-
1475-import java.io.FileNotFoundException;
1476-import java.io.FileOutputStream;
1477-import java.io.IOException;
1478-import java.nio.ByteBuffer;
1479-
1480-public class FileWriter extends FileIO {
1481-
1482- private static Logger LOG = Logger.getLogger(FileWriter.class);
1483-
1484- public FileWriter(String filename) throws Exception {
1485- try {
1486- channel = new FileOutputStream(filename).getChannel();
1487- } catch( FileNotFoundException ex) {
1488- LOG.error(ex);
1489- throw ex;
1490- }
1491- }
1492-
1493- public int read(long offset) throws IOException {
1494- return -1;
1495- }
1496-
1497- public void write(byte[] bytes) throws IOException {
1498- ByteBuffer buffer = ByteBuffer.wrap(bytes);
1499- channel.write(buffer);
1500- }
1501-
1502- public ByteBuffer getBuffer() {
1503- return null;
1504- }
1505-
1506- public void finish() {
1507- try {
1508- channel.close();
1509- } catch(IOException ex) {
1510- LOG.error(ex);
1511- }
1512- }
1513-}
1514
1515=== added directory 'clc/modules/storage-fs'
1516=== added file 'clc/modules/storage-fs/build.xml'
1517--- clc/modules/storage-fs/build.xml 1970-01-01 00:00:00 +0000
1518+++ clc/modules/storage-fs/build.xml 2010-11-15 08:38:14 +0000
1519@@ -0,0 +1,68 @@
1520+<!--
1521+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1522+ ~ Copyright (c) 2009 Eucalyptus Systems, Inc.
1523+ ~
1524+ ~ This program is free software: you can redistribute it and/or modify
1525+ ~ it under the terms of the GNU General Public License as published by
1526+ ~ the Free Software Foundation, only version 3 of the License.
1527+ ~
1528+ ~
1529+ ~ This file is distributed in the hope that it will be useful, but WITHOUT
1530+ ~ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
1531+ ~ FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
1532+ ~ for more details.
1533+ ~
1534+ ~ You should have received a copy of the GNU General Public License along
1535+ ~ with this program. If not, see http://www.gnu.org/licenses/.
1536+ ~
1537+ ~ Please contact Eucalyptus Systems, Inc., 130 Castilian
1538+ ~ Dr., Goleta, CA 93101 USA or visit http://www.eucalyptus.com/licenses/
1539+ ~ if you need additional information or have any questions.
1540+ ~
1541+ ~ This file may incorporate work covered under the following copyright and
1542+ ~ permission notice:
1543+ ~
1544+ ~ Software License Agreement (BSD License)
1545+ ~
1546+ ~ Copyright (c) 2008, Regents of the University of California
1547+ ~ All rights reserved.
1548+ ~
1549+ ~ Redistribution and use of this software in source and binary forms, with
1550+ ~ or without modification, are permitted provided that the following
1551+ ~ conditions are met:
1552+ ~
1553+ ~ Redistributions of source code must retain the above copyright notice,
1554+ ~ this list of conditions and the following disclaimer.
1555+ ~
1556+ ~ Redistributions in binary form must reproduce the above copyright
1557+ ~ notice, this list of conditions and the following disclaimer in the
1558+ ~ documentation and/or other materials provided with the distribution.
1559+ ~
1560+ ~ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
1561+ ~ IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
1562+ ~ TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
1563+ ~ PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
1564+ ~ OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
1565+ ~ EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
1566+ ~ PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
1567+ ~ PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
1568+ ~ LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
1569+ ~ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
1570+ ~ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
1571+ ~ THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
1572+ ~ LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
1573+ ~ SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
1574+ ~ IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
1575+ ~ BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
1576+ ~ THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
1577+ ~ OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
1578+ ~ WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
1579+ ~ ANY SUCH LICENSES OR RIGHTS.
1580+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1581+ ~ Author: Sunil Soman sunils@cs.ucsb.edu
1582+ -->
1583+
1584+<project name="eucalyptus-storage-fs" basedir=".">
1585+ <import file="../module-inc.xml"/>
1586+</project>
1587+
1588
1589=== added directory 'clc/modules/storage-fs/conf'
1590=== added directory 'clc/modules/storage-fs/src'
1591=== added directory 'clc/modules/storage-fs/src/main'
1592=== added directory 'clc/modules/storage-fs/src/main/java'
1593=== added directory 'clc/modules/storage-fs/src/main/java/edu'
1594=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb'
1595=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus'
1596=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud'
1597=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws'
1598=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java'
1599--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java 1970-01-01 00:00:00 +0000
1600+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java 2010-11-15 08:38:14 +0000
1601@@ -0,0 +1,77 @@
1602+/*******************************************************************************
1603+*Copyright (c) 2009 Eucalyptus Systems, Inc.
1604+*
1605+* This program is free software: you can redistribute it and/or modify
1606+* it under the terms of the GNU General Public License as published by
1607+* the Free Software Foundation, only version 3 of the License.
1608+*
1609+*
1610+* This file is distributed in the hope that it will be useful, but WITHOUT
1611+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
1612+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
1613+* for more details.
1614+*
1615+* You should have received a copy of the GNU General Public License along
1616+* with this program. If not, see <http://www.gnu.org/licenses/>.
1617+*
1618+* Please contact Eucalyptus Systems, Inc., 130 Castilian
1619+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
1620+* if you need additional information or have any questions.
1621+*
1622+* This file may incorporate work covered under the following copyright and
1623+* permission notice:
1624+*
1625+* Software License Agreement (BSD License)
1626+*
1627+* Copyright (c) 2008, Regents of the University of California
1628+* All rights reserved.
1629+*
1630+* Redistribution and use of this software in source and binary forms, with
1631+* or without modification, are permitted provided that the following
1632+* conditions are met:
1633+*
1634+* Redistributions of source code must retain the above copyright notice,
1635+* this list of conditions and the following disclaimer.
1636+*
1637+* Redistributions in binary form must reproduce the above copyright
1638+* notice, this list of conditions and the following disclaimer in the
1639+* documentation and/or other materials provided with the distribution.
1640+*
1641+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
1642+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
1643+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
1644+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
1645+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
1646+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
1647+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
1648+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
1649+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
1650+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
1651+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
1652+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
1653+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
1654+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
1655+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
1656+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
1657+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
1658+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
1659+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
1660+* ANY SUCH LICENSES OR RIGHTS.
1661+*******************************************************************************/
1662+package edu.ucsb.eucalyptus.cloud.ws;
1663+
1664+import java.io.IOException;
1665+import java.io.RandomAccessFile;
1666+
1667+import org.jboss.netty.handler.stream.ChunkedFile;
1668+
1669+import edu.ucsb.eucalyptus.constants.IsData;
1670+
1671+public class ChunkedDataFile extends ChunkedFile implements IsData {
1672+
1673+ public ChunkedDataFile(RandomAccessFile file, long offset, long length,
1674+ int chunkSize) throws IOException {
1675+ super(file, offset, length, chunkSize);
1676+ }
1677+}
1678+
1679
1680=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java'
1681--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java 1970-01-01 00:00:00 +0000
1682+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java 2010-11-15 08:38:14 +0000
1683@@ -0,0 +1,120 @@
1684+/*******************************************************************************
1685+*Copyright (c) 2009 Eucalyptus Systems, Inc.
1686+*
1687+* This program is free software: you can redistribute it and/or modify
1688+* it under the terms of the GNU General Public License as published by
1689+* the Free Software Foundation, only version 3 of the License.
1690+*
1691+*
1692+* This file is distributed in the hope that it will be useful, but WITHOUT
1693+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
1694+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
1695+* for more details.
1696+*
1697+* You should have received a copy of the GNU General Public License along
1698+* with this program. If not, see <http://www.gnu.org/licenses/>.
1699+*
1700+* Please contact Eucalyptus Systems, Inc., 130 Castilian
1701+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
1702+* if you need additional information or have any questions.
1703+*
1704+* This file may incorporate work covered under the following copyright and
1705+* permission notice:
1706+*
1707+* Software License Agreement (BSD License)
1708+*
1709+* Copyright (c) 2008, Regents of the University of California
1710+* All rights reserved.
1711+*
1712+* Redistribution and use of this software in source and binary forms, with
1713+* or without modification, are permitted provided that the following
1714+* conditions are met:
1715+*
1716+* Redistributions of source code must retain the above copyright notice,
1717+* this list of conditions and the following disclaimer.
1718+*
1719+* Redistributions in binary form must reproduce the above copyright
1720+* notice, this list of conditions and the following disclaimer in the
1721+* documentation and/or other materials provided with the distribution.
1722+*
1723+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
1724+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
1725+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
1726+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
1727+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
1728+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
1729+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
1730+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
1731+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
1732+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
1733+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
1734+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
1735+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
1736+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
1737+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
1738+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
1739+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
1740+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
1741+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
1742+* ANY SUCH LICENSES OR RIGHTS.
1743+*******************************************************************************/
1744+package edu.ucsb.eucalyptus.cloud.ws;
1745+
1746+import java.io.ByteArrayOutputStream;
1747+import java.io.RandomAccessFile;
1748+import java.util.zip.GZIPOutputStream;
1749+
1750+import org.apache.log4j.Logger;
1751+import org.jboss.netty.buffer.ChannelBuffers;
1752+import org.jboss.netty.handler.stream.ChunkedInput;
1753+
1754+import edu.ucsb.eucalyptus.constants.IsData;
1755+
1756+public class CompressedChunkedFile implements ChunkedInput, IsData {
1757+ private Logger LOG = Logger.getLogger( CompressedChunkedFile.class );
1758+ private RandomAccessFile file;
1759+ private long offset;
1760+ private int CHUNK_SIZE = 8192;
1761+ private long fileLength;
1762+
1763+ public CompressedChunkedFile(RandomAccessFile file, long fileLength) {
1764+ this.file = file;
1765+ this.offset = 0;
1766+ this.fileLength = fileLength;
1767+ }
1768+
1769+ public CompressedChunkedFile(RandomAccessFile file, long start, long end, int chunkSize) {
1770+ this.file = file;
1771+ this.offset = start;
1772+ this.fileLength = end;
1773+ CHUNK_SIZE = chunkSize;
1774+ }
1775+
1776+ @Override
1777+ public void close() throws Exception {
1778+ file.close();
1779+ }
1780+
1781+ @Override
1782+ public boolean hasNextChunk() throws Exception {
1783+ return offset < fileLength && file.getChannel().isOpen();
1784+ }
1785+
1786+ @Override
1787+ public Object nextChunk() throws Exception {
1788+ long offset = this.offset;
1789+ if (offset >= fileLength) {
1790+ return null;
1791+ }
1792+ int chunkSize = (int) Math.min(CHUNK_SIZE, fileLength - offset);
1793+ byte[] chunk = new byte[chunkSize];
1794+ file.readFully(chunk);
1795+ this.offset = offset + chunkSize;
1796+ ByteArrayOutputStream out = new ByteArrayOutputStream();
1797+ GZIPOutputStream zip = new GZIPOutputStream(out);
1798+ zip.write(chunk);
1799+ zip.close();
1800+ return ChannelBuffers.wrappedBuffer(out.toByteArray());
1801+ }
1802+}
1803+
1804
1805=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage'
1806=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java'
1807--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java 1970-01-01 00:00:00 +0000
1808+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java 2010-11-15 08:38:14 +0000
1809@@ -0,0 +1,71 @@
1810+/*******************************************************************************
1811+*Copyright (c) 2010 Takuya ASADA
1812+*
1813+* This program is free software: you can redistribute it and/or modify
1814+* it under the terms of the GNU General Public License as published by
1815+* the Free Software Foundation, only version 3 of the License.
1816+*
1817+*
1818+* This file is distributed in the hope that it will be useful, but WITHOUT
1819+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
1820+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
1821+* for more details.
1822+*
1823+* You should have received a copy of the GNU General Public License along
1824+* with this program. If not, see <http://www.gnu.org/licenses/>.
1825+*
1826+* Please contact Eucalyptus Systems, Inc., 130 Castilian
1827+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
1828+* if you need additional information or have any questions.
1829+*
1830+* This file may incorporate work covered under the following copyright and
1831+* permission notice:
1832+*
1833+* Software License Agreement (BSD License)
1834+*
1835+* Copyright (c) 2008, Regents of the University of California
1836+* All rights reserved.
1837+*
1838+* Redistribution and use of this software in source and binary forms, with
1839+* or without modification, are permitted provided that the following
1840+* conditions are met:
1841+*
1842+* Redistributions of source code must retain the above copyright notice,
1843+* this list of conditions and the following disclaimer.
1844+*
1845+* Redistributions in binary form must reproduce the above copyright
1846+* notice, this list of conditions and the following disclaimer in the
1847+* documentation and/or other materials provided with the distribution.
1848+*
1849+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
1850+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
1851+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
1852+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
1853+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
1854+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
1855+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
1856+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
1857+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
1858+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
1859+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
1860+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
1861+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
1862+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
1863+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
1864+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
1865+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
1866+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
1867+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
1868+* ANY SUCH LICENSES OR RIGHTS.
1869+*******************************************************************************/
1870+
1871+package edu.ucsb.eucalyptus.storage;
1872+
1873+import edu.ucsb.eucalyptus.cloud.entities.WalrusInfo;
1874+import edu.ucsb.eucalyptus.storage.fs.FileSystemStorageManager;
1875+
1876+public class StorageManagerFactory {
1877+ public static StorageManager getStorageManager(WalrusInfo walrusInfo) {
1878+ return new FileSystemStorageManager(walrusInfo);
1879+ }
1880+}
1881
1882=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs'
1883=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java'
1884--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java 1970-01-01 00:00:00 +0000
1885+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java 2010-11-15 08:38:14 +0000
1886@@ -0,0 +1,114 @@
1887+/*******************************************************************************
1888+*Copyright (c) 2009 Eucalyptus Systems, Inc.
1889+*
1890+* This program is free software: you can redistribute it and/or modify
1891+* it under the terms of the GNU General Public License as published by
1892+* the Free Software Foundation, only version 3 of the License.
1893+*
1894+*
1895+* This file is distributed in the hope that it will be useful, but WITHOUT
1896+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
1897+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
1898+* for more details.
1899+*
1900+* You should have received a copy of the GNU General Public License along
1901+* with this program. If not, see <http://www.gnu.org/licenses/>.
1902+*
1903+* Please contact Eucalyptus Systems, Inc., 130 Castilian
1904+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
1905+* if you need additional information or have any questions.
1906+*
1907+* This file may incorporate work covered under the following copyright and
1908+* permission notice:
1909+*
1910+* Software License Agreement (BSD License)
1911+*
1912+* Copyright (c) 2008, Regents of the University of California
1913+* All rights reserved.
1914+*
1915+* Redistribution and use of this software in source and binary forms, with
1916+* or without modification, are permitted provided that the following
1917+* conditions are met:
1918+*
1919+* Redistributions of source code must retain the above copyright notice,
1920+* this list of conditions and the following disclaimer.
1921+*
1922+* Redistributions in binary form must reproduce the above copyright
1923+* notice, this list of conditions and the following disclaimer in the
1924+* documentation and/or other materials provided with the distribution.
1925+*
1926+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
1927+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
1928+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
1929+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
1930+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
1931+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
1932+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
1933+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
1934+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
1935+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
1936+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
1937+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
1938+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
1939+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
1940+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
1941+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
1942+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
1943+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
1944+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
1945+* ANY SUCH LICENSES OR RIGHTS.
1946+*******************************************************************************/
1947+/*
1948+ *
1949+ * Author: Sunil Soman sunils@cs.ucsb.edu
1950+ */
1951+
1952+package edu.ucsb.eucalyptus.storage.fs;
1953+
1954+import edu.ucsb.eucalyptus.storage.StorageIO;
1955+
1956+import org.apache.log4j.Logger;
1957+
1958+import java.io.FileInputStream;
1959+import java.io.FileNotFoundException;
1960+import java.io.IOException;
1961+import java.nio.ByteBuffer;
1962+import java.nio.channels.FileChannel;
1963+
1964+public class FileReader extends StorageIO {
1965+
1966+ private static Logger LOG = Logger.getLogger(FileReader.class);
1967+ private ByteBuffer buffer;
1968+ protected FileChannel channel;
1969+
1970+ public FileReader(String filename) throws Exception {
1971+ try {
1972+ channel = new FileInputStream(filename).getChannel();
1973+ buffer = ByteBuffer.allocate(102400/*TODO: NEIL WalrusQueryDispatcher.DATA_MESSAGE_SIZE */);
1974+ } catch( FileNotFoundException ex) {
1975+ LOG.error(ex);
1976+ throw ex;
1977+ }
1978+ }
1979+
1980+ public int read(long offset) throws IOException {
1981+ buffer.clear();
1982+ int bytesRead = channel.read(buffer, offset);
1983+ buffer.flip();
1984+ return bytesRead;
1985+ }
1986+
1987+ public void write(byte[] bytes) throws IOException {}
1988+
1989+ public ByteBuffer getBuffer() {
1990+ return buffer;
1991+ }
1992+
1993+ public void finish() {
1994+ try {
1995+ channel.close();
1996+ } catch(IOException ex) {
1997+ LOG.error(ex);
1998+ }
1999+ }
2000+}
2001
2002=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java'
2003--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java 1970-01-01 00:00:00 +0000
2004+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java 2010-11-15 08:38:14 +0000
2005@@ -0,0 +1,468 @@
2006+/*******************************************************************************
2007+ *Copyright (c) 2009 Eucalyptus Systems, Inc.
2008+ *
2009+ * This program is free software: you can redistribute it and/or modify
2010+ * it under the terms of the GNU General Public License as published by
2011+ * the Free Software Foundation, only version 3 of the License.
2012+ *
2013+ *
2014+ * This file is distributed in the hope that it will be useful, but WITHOUT
2015+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
2016+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
2017+ * for more details.
2018+ *
2019+ * You should have received a copy of the GNU General Public License along
2020+ * with this program. If not, see <http://www.gnu.org/licenses/>.
2021+ *
2022+ * Please contact Eucalyptus Systems, Inc., 130 Castilian
2023+ * Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
2024+ * if you need additional information or have any questions.
2025+ *
2026+ * This file may incorporate work covered under the following copyright and
2027+ * permission notice:
2028+ *
2029+ * Software License Agreement (BSD License)
2030+ *
2031+ * Copyright (c) 2008, Regents of the University of California
2032+ * All rights reserved.
2033+ *
2034+ * Redistribution and use of this software in source and binary forms, with
2035+ * or without modification, are permitted provided that the following
2036+ * conditions are met:
2037+ *
2038+ * Redistributions of source code must retain the above copyright notice,
2039+ * this list of conditions and the following disclaimer.
2040+ *
2041+ * Redistributions in binary form must reproduce the above copyright
2042+ * notice, this list of conditions and the following disclaimer in the
2043+ * documentation and/or other materials provided with the distribution.
2044+ *
2045+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
2046+ * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
2047+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
2048+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
2049+ * OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
2050+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
2051+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
2052+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
2053+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
2054+ * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
2055+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
2056+ * THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
2057+ * LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
2058+ * SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
2059+ * IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
2060+ * BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
2061+ * THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
2062+ * OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
2063+ * WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
2064+ * ANY SUCH LICENSES OR RIGHTS.
2065+ *******************************************************************************/
2066+/*
2067+ *
2068+ * Author: Sunil Soman sunils@cs.ucsb.edu
2069+ */
2070+
2071+package edu.ucsb.eucalyptus.storage.fs;
2072+
2073+import com.eucalyptus.auth.util.Hashes;
2074+import com.eucalyptus.context.Contexts;
2075+import com.eucalyptus.http.MappingHttpResponse;
2076+import com.eucalyptus.util.EucalyptusCloudException;
2077+import com.eucalyptus.util.ExecutionException;
2078+import com.eucalyptus.util.WalrusProperties;
2079+import com.eucalyptus.ws.util.ChannelUtil;
2080+import com.eucalyptus.ws.util.WalrusBucketLogger;
2081+
2082+import edu.ucsb.eucalyptus.cloud.BucketLogData;
2083+import edu.ucsb.eucalyptus.cloud.ws.CompressedChunkedFile;
2084+import edu.ucsb.eucalyptus.cloud.ws.ChunkedDataFile;
2085+import edu.ucsb.eucalyptus.msgs.WalrusDataGetRequestType;
2086+import edu.ucsb.eucalyptus.storage.StorageManager;
2087+import edu.ucsb.eucalyptus.storage.StorageIO;
2088+import edu.ucsb.eucalyptus.util.StreamConsumer;
2089+import edu.ucsb.eucalyptus.util.SystemUtil;
2090+
2091+import edu.ucsb.eucalyptus.cloud.entities.WalrusInfo;
2092+
2093+import org.apache.log4j.Logger;
2094+import org.jboss.netty.channel.Channel;
2095+import org.jboss.netty.channel.ChannelFuture;
2096+import org.jboss.netty.channel.ChannelFutureListener;
2097+import org.jboss.netty.handler.codec.http.DefaultHttpResponse;
2098+import org.jboss.netty.handler.codec.http.HttpHeaders;
2099+import org.jboss.netty.handler.stream.ChunkedInput;
2100+
2101+import java.io.*;
2102+import java.nio.channels.FileChannel;
2103+import java.util.ArrayList;
2104+import java.util.List;
2105+
2106+public class FileSystemStorageManager implements StorageManager {
2107+
2108+ public static final String FILE_SEPARATOR = "/";
2109+ public static final String lvmRootDirectory = "/dev";
2110+ private static boolean initialized = false;
2111+ private static String eucaHome = "/opt/eucalyptus";
2112+ public static final String EUCA_ROOT_WRAPPER = "/usr/lib/eucalyptus/euca_rootwrap";
2113+ public static final int MAX_LOOP_DEVICES = 256;
2114+ private static Logger LOG = Logger.getLogger(FileSystemStorageManager.class);
2115+
2116+ private String rootDirectory;
2117+ public FileSystemStorageManager(WalrusInfo walrusInfo) {
2118+ this.rootDirectory = walrusInfo.getStorageDir();
2119+ }
2120+
2121+ public void checkPreconditions() throws EucalyptusCloudException {
2122+ try {
2123+ String eucaHomeDir = System.getProperty("euca.home");
2124+ if(eucaHomeDir == null) {
2125+ throw new EucalyptusCloudException("euca.home not set");
2126+ }
2127+ eucaHome = eucaHomeDir;
2128+ if(!new File(eucaHome + EUCA_ROOT_WRAPPER).exists()) {
2129+ throw new EucalyptusCloudException("root wrapper (euca_rootwrap) does not exist");
2130+ }
2131+ String returnValue = getLvmVersion();
2132+ if(returnValue.length() == 0) {
2133+ throw new EucalyptusCloudException("Is lvm installed?");
2134+ } else {
2135+ LOG.info(returnValue);
2136+ }
2137+ } catch(ExecutionException ex) {
2138+ String error = "Unable to run command: " + ex.getMessage();
2139+ LOG.error(error);
2140+ throw new EucalyptusCloudException(error);
2141+ }
2142+ }
2143+
2144+ public void setRootDirectory(String rootDirectory) {
2145+ this.rootDirectory = rootDirectory;
2146+ }
2147+
2148+ public boolean bucketExists(String bucket) {
2149+ return new File (rootDirectory + FILE_SEPARATOR + bucket).exists();
2150+ }
2151+
2152+ public boolean objectExists(String bucket, String object) {
2153+ return new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object).exists();
2154+ }
2155+
2156+ public void createBucket(String bucket) throws IOException {
2157+ File bukkit = new File (rootDirectory + FILE_SEPARATOR + bucket);
2158+ if(!bukkit.exists()) {
2159+ if(!bukkit.mkdirs()) {
2160+ throw new IOException("Unable to create bucket: " + bucket);
2161+ }
2162+ }
2163+ }
2164+
2165+ public long getSize(String bucket, String object) {
2166+ File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
2167+ if(objectFile.exists())
2168+ return objectFile.length();
2169+ return -1;
2170+ }
2171+
2172+ public void deleteBucket(String bucket) throws IOException {
2173+ File bukkit = new File (rootDirectory + FILE_SEPARATOR + bucket);
2174+ if(!bukkit.delete()) {
2175+ throw new IOException("Unable to delete bucket: " + bucket);
2176+ }
2177+ }
2178+
2179+ public void createObject(String bucket, String object) throws IOException {
2180+ File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
2181+ if (!objectFile.exists()) {
2182+ if (!objectFile.createNewFile()) {
2183+ throw new IOException("Unable to create: " + objectFile.getAbsolutePath());
2184+ }
2185+ }
2186+ }
2187+
2188+ public StorageIO prepareForRead(String bucket, String object) throws Exception {
2189+ return new FileReader(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
2190+ }
2191+
2192+ public StorageIO prepareForWrite(String bucket, String object) throws Exception {
2193+ return new FileWriter(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
2194+ }
2195+
2196+ public int readObject(String bucket, String object, byte[] bytes, long offset) throws IOException {
2197+ return readObject(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object, bytes, offset);
2198+ }
2199+
2200+ public int readObject(String path, byte[] bytes, long offset) throws IOException {
2201+ File objectFile = new File (path);
2202+ if (!objectFile.exists()) {
2203+ throw new IOException("Unable to read: " + path);
2204+ }
2205+ BufferedInputStream inputStream = new BufferedInputStream(new FileInputStream(objectFile));
2206+ if (offset > 0) {
2207+ inputStream.skip(offset);
2208+ }
2209+ int bytesRead = inputStream.read(bytes);
2210+ inputStream.close();
2211+ return bytesRead;
2212+ }
2213+
2214+ public void deleteObject(String bucket, String object) throws IOException {
2215+ File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
2216+ if (objectFile.exists()) {
2217+ if(!objectFile.delete()) {
2218+ throw new IOException("Unable to delete: " + objectFile.getAbsolutePath());
2219+ }
2220+ }
2221+ }
2222+
2223+ public void deleteAbsoluteObject(String object) throws IOException {
2224+ File objectFile = new File (object);
2225+ if (objectFile.exists()) {
2226+ if(!objectFile.delete()) {
2227+ throw new IOException("Unable to delete: " + object);
2228+ }
2229+ }
2230+ }
2231+
2232+ public void putObject(String bucket, String object, byte[] base64Data, boolean append) throws IOException {
2233+ File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
2234+ if (!objectFile.exists()) {
2235+ objectFile.createNewFile();
2236+ }
2237+ BufferedOutputStream outputStream = new BufferedOutputStream(new FileOutputStream(objectFile, append));
2238+ outputStream.write(base64Data);
2239+ outputStream.close();
2240+ }
2241+
2242+ public void renameObject(String bucket, String oldName, String newName) throws IOException {
2243+ File oldObjectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + oldName);
2244+ File newObjectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + newName);
2245+ if(oldObjectFile.exists()) {
2246+ if (!oldObjectFile.renameTo(newObjectFile)) {
2247+ throw new IOException("Unable to rename " + oldObjectFile.getAbsolutePath() + " to " + newObjectFile.getAbsolutePath());
2248+ }
2249+ }
2250+ }
2251+
2252+ public void copyObject(String sourceBucket, String sourceObject, String destinationBucket, String destinationObject) throws IOException {
2253+ File oldObjectFile = new File (rootDirectory + FILE_SEPARATOR + sourceBucket + FILE_SEPARATOR + sourceObject);
2254+ File newObjectFile = new File (rootDirectory + FILE_SEPARATOR + destinationBucket + FILE_SEPARATOR + destinationObject);
2255+ if(!oldObjectFile.equals(newObjectFile)) {
2256+ FileInputStream fileInputStream = new FileInputStream(oldObjectFile);
2257+ FileChannel fileIn = fileInputStream.getChannel();
2258+ FileOutputStream fileOutputStream = new FileOutputStream(newObjectFile);
2259+ FileChannel fileOut = fileOutputStream.getChannel();
2260+ fileIn.transferTo(0, fileIn.size(), fileOut);
2261+ fileIn.close();
2262+ fileInputStream.close();
2263+ fileOut.close();
2264+ fileOutputStream.close();
2265+ }
2266+ }
2267+
2268+ public String getObjectPath(String bucket, String object) {
2269+ return rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object;
2270+ }
2271+
2272+ public long getObjectSize(String bucket, String object) {
2273+ String absoluteObjectPath = rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object;
2274+
2275+ File objectFile = new File(absoluteObjectPath);
2276+ if(objectFile.exists())
2277+ return objectFile.length();
2278+ return -1;
2279+ }
2280+
2281+ public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
2282+ try {
2283+ Channel channel = request.getChannel();
2284+ RandomAccessFile raf = new RandomAccessFile(new File(getObjectPath(bucketName, objectName)), "r");
2285+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
2286+ if(etag != null)
2287+ httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
2288+ httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
2289+ if(contentDisposition != null)
2290+ httpResponse.addHeader("Content-Disposition", contentDisposition);
2291+ final ChunkedInput file;
2292+ isCompressed = isCompressed == null ? false : isCompressed;
2293+ if(isCompressed) {
2294+ file = new CompressedChunkedFile(raf, size);
2295+ } else {
2296+ file = new ChunkedDataFile(raf, 0, size, 8192);
2297+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
2298+ }
2299+ if(logData != null) {
2300+ logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
2301+ logData.setBytesSent(size);
2302+ }
2303+ if(versionId != null) {
2304+ httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
2305+ }
2306+ channel.write(httpResponse);
2307+ channel.write(file).addListener(new ChannelFutureListener( ) {
2308+ @Override public void operationComplete( ChannelFuture future ) throws Exception {
2309+ Contexts.clear(request.getCorrelationId());
2310+ file.close();
2311+ if(logData != null) {
2312+ logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
2313+ WalrusBucketLogger.getInstance().addLogEntry(logData);
2314+ }
2315+ }
2316+ });
2317+ } catch(Exception ex) {
2318+ LOG.error(ex, ex);
2319+ }
2320+ }
2321+
2322+ public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long start, long end, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
2323+ try {
2324+ Channel channel = request.getChannel();
2325+ RandomAccessFile raf = new RandomAccessFile(new File(getObjectPath(bucketName, objectName)), "r");
2326+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
2327+ if(etag != null)
2328+ httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
2329+ httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
2330+ if(contentDisposition != null)
2331+ httpResponse.addHeader("Content-Disposition", contentDisposition);
2332+ final ChunkedInput file;
2333+ isCompressed = isCompressed == null ? false : isCompressed;
2334+ if(isCompressed) {
2335+ file = new CompressedChunkedFile(raf, start, end, (int)Math.min((end - start), 8192));
2336+ } else {
2337+ file = new ChunkedDataFile(raf, start, end, (int)Math.min((end - start), 8192));
2338+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf((end - start)));
2339+ }
2340+ httpResponse.addHeader("Content-Range", start + "-" + end + "/" + size);
2341+ if(logData != null) {
2342+ logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
2343+ logData.setBytesSent(size);
2344+ }
2345+ if(versionId != null) {
2346+ httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
2347+ }
2348+ channel.write(httpResponse);
2349+ channel.write(file).addListener(new ChannelFutureListener( ) {
2350+ @Override public void operationComplete( ChannelFuture future ) throws Exception {
2351+ Contexts.clear(request.getCorrelationId());
2352+ file.close();
2353+ if(logData != null) {
2354+ logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
2355+ WalrusBucketLogger.getInstance().addLogEntry(logData);
2356+ }
2357+ }
2358+ });
2359+ } catch(Exception ex) {
2360+ LOG.error(ex, ex);
2361+ }
2362+ }
2363+
2364+ public void sendHeaders(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, Long size, String etag,
2365+ String lastModified, String contentType, String contentDisposition, String versionId, final BucketLogData logData) {
2366+ Channel channel = request.getChannel();
2367+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
2368+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
2369+ if(etag != null)
2370+ httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
2371+ httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
2372+ if(contentDisposition != null)
2373+ httpResponse.addHeader("Content-Disposition", contentDisposition);
2374+ if(versionId != null) {
2375+ httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
2376+ }
2377+ if(logData != null) {
2378+ logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
2379+ }
2380+ channel.write(httpResponse).addListener(new ChannelFutureListener( ) {
2381+ @Override public void operationComplete( ChannelFuture future ) throws Exception {
2382+ Contexts.clear(request.getCorrelationId());
2383+ if(logData != null) {
2384+ logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
2385+ WalrusBucketLogger.getInstance().addLogEntry(logData);
2386+ }
2387+ }
2388+ });
2389+ }
2390+
2391+ private String removeLoopback(String loDevName) throws ExecutionException {
2392+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-d", loDevName});
2393+ }
2394+
2395+ private int losetup(String absoluteFileName, String loDevName) {
2396+ try
2397+ {
2398+ Runtime rt = Runtime.getRuntime();
2399+ Process proc = rt.exec(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", loDevName, absoluteFileName});
2400+ StreamConsumer error = new StreamConsumer(proc.getErrorStream());
2401+ StreamConsumer output = new StreamConsumer(proc.getInputStream());
2402+ error.start();
2403+ output.start();
2404+ int errorCode = proc.waitFor();
2405+ output.join();
2406+ LOG.info("losetup " + loDevName + " " + absoluteFileName);
2407+ LOG.info(output.getReturnValue());
2408+ LOG.info(error.getReturnValue());
2409+ return errorCode;
2410+ } catch (Throwable t) {
2411+ LOG.error(t);
2412+ }
2413+ return -1;
2414+ }
2415+
2416+ private String findFreeLoopback() throws ExecutionException {
2417+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-f"}).replaceAll("\n", "");
2418+ }
2419+
2420+ private String removeLogicalVolume(String lvName) throws ExecutionException {
2421+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvremove", "-f", lvName});
2422+ }
2423+
2424+ private String reduceVolumeGroup(String vgName, String pvName) throws ExecutionException {
2425+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "vgreduce", vgName, pvName});
2426+ }
2427+
2428+ private String removePhysicalVolume(String loDevName) throws ExecutionException {
2429+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "pvremove", loDevName});
2430+ }
2431+
2432+ private String createVolumeFromLv(String lvName, String volumeKey) throws ExecutionException {
2433+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "dd", "if=" + lvName, "of=" + volumeKey, "bs=1M"});
2434+ }
2435+
2436+ private String enableLogicalVolume(String lvName) throws ExecutionException {
2437+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvchange", "-ay", lvName});
2438+ }
2439+
2440+ private String disableLogicalVolume(String lvName) throws ExecutionException {
2441+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvchange", "-an", lvName});
2442+ }
2443+
2444+ private String removeVolumeGroup(String vgName) throws ExecutionException {
2445+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "vgremove", vgName});
2446+ }
2447+
2448+ private String getLvmVersion() throws ExecutionException {
2449+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvm", "version"});
2450+ }
2451+
2452+
2453+
2454+ public String createLoopback(String fileName) throws EucalyptusCloudException, ExecutionException {
2455+ int number_of_retries = 0;
2456+ int status = -1;
2457+ String loDevName;
2458+ do {
2459+ loDevName = findFreeLoopback();
2460+ if(loDevName.length() > 0) {
2461+ status = losetup(fileName, loDevName);
2462+ }
2463+ if(number_of_retries++ >= MAX_LOOP_DEVICES)
2464+ break;
2465+ } while(status != 0);
2466+
2467+ if(status != 0) {
2468+ throw new EucalyptusCloudException("Could not create loopback device for " + fileName +
2469+ ". Please check the max loop value and permissions");
2470+ }
2471+ return loDevName;
2472+ }
2473+}
2474
2475=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java'
2476--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java 1970-01-01 00:00:00 +0000
2477+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java 2010-11-15 08:38:14 +0000
2478@@ -0,0 +1,112 @@
2479+/*******************************************************************************
2480+*Copyright (c) 2009 Eucalyptus Systems, Inc.
2481+*
2482+* This program is free software: you can redistribute it and/or modify
2483+* it under the terms of the GNU General Public License as published by
2484+* the Free Software Foundation, only version 3 of the License.
2485+*
2486+*
2487+* This file is distributed in the hope that it will be useful, but WITHOUT
2488+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
2489+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
2490+* for more details.
2491+*
2492+* You should have received a copy of the GNU General Public License along
2493+* with this program. If not, see <http://www.gnu.org/licenses/>.
2494+*
2495+* Please contact Eucalyptus Systems, Inc., 130 Castilian
2496+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
2497+* if you need additional information or have any questions.
2498+*
2499+* This file may incorporate work covered under the following copyright and
2500+* permission notice:
2501+*
2502+* Software License Agreement (BSD License)
2503+*
2504+* Copyright (c) 2008, Regents of the University of California
2505+* All rights reserved.
2506+*
2507+* Redistribution and use of this software in source and binary forms, with
2508+* or without modification, are permitted provided that the following
2509+* conditions are met:
2510+*
2511+* Redistributions of source code must retain the above copyright notice,
2512+* this list of conditions and the following disclaimer.
2513+*
2514+* Redistributions in binary form must reproduce the above copyright
2515+* notice, this list of conditions and the following disclaimer in the
2516+* documentation and/or other materials provided with the distribution.
2517+*
2518+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
2519+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
2520+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
2521+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
2522+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
2523+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
2524+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
2525+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
2526+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
2527+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
2528+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
2529+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
2530+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
2531+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
2532+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
2533+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
2534+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
2535+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
2536+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
2537+* ANY SUCH LICENSES OR RIGHTS.
2538+*******************************************************************************/
2539+/*
2540+ *
2541+ * Author: Sunil Soman sunils@cs.ucsb.edu
2542+ */
2543+
2544+package edu.ucsb.eucalyptus.storage.fs;
2545+
2546+import edu.ucsb.eucalyptus.storage.StorageIO;
2547+
2548+import org.apache.log4j.Logger;
2549+
2550+import java.io.FileNotFoundException;
2551+import java.io.FileOutputStream;
2552+import java.io.IOException;
2553+import java.nio.ByteBuffer;
2554+import java.nio.channels.FileChannel;
2555+
2556+public class FileWriter extends StorageIO {
2557+
2558+ private static Logger LOG = Logger.getLogger(FileWriter.class);
2559+ protected FileChannel channel;
2560+
2561+ public FileWriter(String filename) throws Exception {
2562+ try {
2563+ channel = new FileOutputStream(filename).getChannel();
2564+ } catch( FileNotFoundException ex) {
2565+ LOG.error(ex);
2566+ throw ex;
2567+ }
2568+ }
2569+
2570+ public int read(long offset) throws IOException {
2571+ return -1;
2572+ }
2573+
2574+ public void write(byte[] bytes) throws IOException {
2575+ ByteBuffer buffer = ByteBuffer.wrap(bytes);
2576+ channel.write(buffer);
2577+ }
2578+
2579+ public ByteBuffer getBuffer() {
2580+ return null;
2581+ }
2582+
2583+ public void finish() {
2584+ try {
2585+ channel.close();
2586+ } catch(IOException ex) {
2587+ LOG.error(ex);
2588+ }
2589+ }
2590+}
2591
2592=== added directory 'clc/modules/storage-fs/src/main/resources'
2593=== added directory 'clc/modules/storage-rados'
2594=== added file 'clc/modules/storage-rados/build.xml'
2595--- clc/modules/storage-rados/build.xml 1970-01-01 00:00:00 +0000
2596+++ clc/modules/storage-rados/build.xml 2010-11-15 08:38:14 +0000
2597@@ -0,0 +1,68 @@
2598+<!--
2599+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2600+ ~ Copyright (c) 2009 Eucalyptus Systems, Inc.
2601+ ~
2602+ ~ This program is free software: you can redistribute it and/or modify
2603+ ~ it under the terms of the GNU General Public License as published by
2604+ ~ the Free Software Foundation, only version 3 of the License.
2605+ ~
2606+ ~
2607+ ~ This file is distributed in the hope that it will be useful, but WITHOUT
2608+ ~ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
2609+ ~ FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
2610+ ~ for more details.
2611+ ~
2612+ ~ You should have received a copy of the GNU General Public License along
2613+ ~ with this program. If not, see http://www.gnu.org/licenses/.
2614+ ~
2615+ ~ Please contact Eucalyptus Systems, Inc., 130 Castilian
2616+ ~ Dr., Goleta, CA 93101 USA or visit http://www.eucalyptus.com/licenses/
2617+ ~ if you need additional information or have any questions.
2618+ ~
2619+ ~ This file may incorporate work covered under the following copyright and
2620+ ~ permission notice:
2621+ ~
2622+ ~ Software License Agreement (BSD License)
2623+ ~
2624+ ~ Copyright (c) 2008, Regents of the University of California
2625+ ~ All rights reserved.
2626+ ~
2627+ ~ Redistribution and use of this software in source and binary forms, with
2628+ ~ or without modification, are permitted provided that the following
2629+ ~ conditions are met:
2630+ ~
2631+ ~ Redistributions of source code must retain the above copyright notice,
2632+ ~ this list of conditions and the following disclaimer.
2633+ ~
2634+ ~ Redistributions in binary form must reproduce the above copyright
2635+ ~ notice, this list of conditions and the following disclaimer in the
2636+ ~ documentation and/or other materials provided with the distribution.
2637+ ~
2638+ ~ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
2639+ ~ IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
2640+ ~ TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
2641+ ~ PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
2642+ ~ OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
2643+ ~ EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
2644+ ~ PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
2645+ ~ PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
2646+ ~ LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
2647+ ~ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
2648+ ~ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
2649+ ~ THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
2650+ ~ LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
2651+ ~ SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
2652+ ~ IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
2653+ ~ BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
2654+ ~ THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
2655+ ~ OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
2656+ ~ WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
2657+ ~ ANY SUCH LICENSES OR RIGHTS.
2658+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2659+ ~ Author: Sunil Soman sunils@cs.ucsb.edu
2660+ -->
2661+
2662+<project name="eucalyptus-storage-rados" basedir=".">
2663+ <import file="../module-inc.xml"/>
2664+</project>
2665+
2666
2667=== added directory 'clc/modules/storage-rados/conf'
2668=== added directory 'clc/modules/storage-rados/include'
2669=== added file 'clc/modules/storage-rados/include/com_dokukino_rados4j_ListCtx.h'
2670--- clc/modules/storage-rados/include/com_dokukino_rados4j_ListCtx.h 1970-01-01 00:00:00 +0000
2671+++ clc/modules/storage-rados/include/com_dokukino_rados4j_ListCtx.h 2010-11-15 08:38:14 +0000
2672@@ -0,0 +1,29 @@
2673+/* DO NOT EDIT THIS FILE - it is machine generated */
2674+#include <jni.h>
2675+/* Header for class com_dokukino_rados4j_ListCtx */
2676+
2677+#ifndef _Included_com_dokukino_rados4j_ListCtx
2678+#define _Included_com_dokukino_rados4j_ListCtx
2679+#ifdef __cplusplus
2680+extern "C" {
2681+#endif
2682+/*
2683+ * Class: com_dokukino_rados4j_ListCtx
2684+ * Method: more
2685+ * Signature: (I[Ljava/lang/String;)I
2686+ */
2687+JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_ListCtx_more
2688+ (JNIEnv *, jobject, jint, jobjectArray);
2689+
2690+/*
2691+ * Class: com_dokukino_rados4j_ListCtx
2692+ * Method: close
2693+ * Signature: ()V
2694+ */
2695+JNIEXPORT void JNICALL Java_com_dokukino_rados4j_ListCtx_close
2696+ (JNIEnv *, jobject);
2697+
2698+#ifdef __cplusplus
2699+}
2700+#endif
2701+#endif
2702
2703=== added file 'clc/modules/storage-rados/include/com_dokukino_rados4j_Pool.h'
2704--- clc/modules/storage-rados/include/com_dokukino_rados4j_Pool.h 1970-01-01 00:00:00 +0000
2705+++ clc/modules/storage-rados/include/com_dokukino_rados4j_Pool.h 2010-11-15 08:38:14 +0000
2706@@ -0,0 +1,93 @@
2707+/* DO NOT EDIT THIS FILE - it is machine generated */
2708+#include <jni.h>
2709+/* Header for class com_dokukino_rados4j_Pool */
2710+
2711+#ifndef _Included_com_dokukino_rados4j_Pool
2712+#define _Included_com_dokukino_rados4j_Pool
2713+#ifdef __cplusplus
2714+extern "C" {
2715+#endif
2716+/*
2717+ * Class: com_dokukino_rados4j_Pool
2718+ * Method: delete
2719+ * Signature: ()I
2720+ */
2721+JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_delete
2722+ (JNIEnv *, jobject);
2723+
2724+/*
2725+ * Class: com_dokukino_rados4j_Pool
2726+ * Method: close
2727+ * Signature: ()V
2728+ */
2729+JNIEXPORT void JNICALL Java_com_dokukino_rados4j_Pool_close
2730+ (JNIEnv *, jobject);
2731+
2732+/*
2733+ * Class: com_dokukino_rados4j_Pool
2734+ * Method: createObj
2735+ * Signature: (Ljava/lang/String;Z)I
2736+ */
2737+JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_createObj
2738+ (JNIEnv *, jobject, jstring, jboolean);
2739+
2740+/*
2741+ * Class: com_dokukino_rados4j_Pool
2742+ * Method: writeObj
2743+ * Signature: (Ljava/lang/String;J[BJ)I
2744+ */
2745+JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_writeObj
2746+ (JNIEnv *, jobject, jstring, jlong, jbyteArray, jlong);
2747+
2748+/*
2749+ * Class: com_dokukino_rados4j_Pool
2750+ * Method: readObj
2751+ * Signature: (Ljava/lang/String;J[BJ)I
2752+ */
2753+JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_readObj
2754+ (JNIEnv *, jobject, jstring, jlong, jbyteArray, jlong);
2755+
2756+/*
2757+ * Class: com_dokukino_rados4j_Pool
2758+ * Method: renameObj
2759+ * Signature: (Ljava/lang/String;Ljava/lang/String;)I
2760+ */
2761+JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_renameObj
2762+ (JNIEnv *, jobject, jstring, jstring);
2763+
2764+/*
2765+ * Class: com_dokukino_rados4j_Pool
2766+ * Method: copyObj
2767+ * Signature: (Ljava/lang/String;Lcom/dokukino/rados4j/Pool;Ljava/lang/String;)I
2768+ */
2769+JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_copyObj
2770+ (JNIEnv *, jobject, jstring, jobject, jstring);
2771+
2772+/*
2773+ * Class: com_dokukino_rados4j_Pool
2774+ * Method: removeObj
2775+ * Signature: (Ljava/lang/String;)I
2776+ */
2777+JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_removeObj
2778+ (JNIEnv *, jobject, jstring);
2779+
2780+/*
2781+ * Class: com_dokukino_rados4j_Pool
2782+ * Method: statObj
2783+ * Signature: (Ljava/lang/String;)Lcom/dokukino/rados4j/Stat;
2784+ */
2785+JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Pool_statObj
2786+ (JNIEnv *, jobject, jstring);
2787+
2788+/*
2789+ * Class: com_dokukino_rados4j_Pool
2790+ * Method: openList
2791+ * Signature: ()Lcom/dokukino/rados4j/ListCtx;
2792+ */
2793+JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Pool_openList
2794+ (JNIEnv *, jobject);
2795+
2796+#ifdef __cplusplus
2797+}
2798+#endif
2799+#endif
2800
2801=== added file 'clc/modules/storage-rados/include/com_dokukino_rados4j_Rados.h'
2802--- clc/modules/storage-rados/include/com_dokukino_rados4j_Rados.h 1970-01-01 00:00:00 +0000
2803+++ clc/modules/storage-rados/include/com_dokukino_rados4j_Rados.h 2010-11-15 08:38:14 +0000
2804@@ -0,0 +1,61 @@
2805+/* DO NOT EDIT THIS FILE - it is machine generated */
2806+#include <jni.h>
2807+/* Header for class com_dokukino_rados4j_Rados */
2808+
2809+#ifndef _Included_com_dokukino_rados4j_Rados
2810+#define _Included_com_dokukino_rados4j_Rados
2811+#ifdef __cplusplus
2812+extern "C" {
2813+#endif
2814+/*
2815+ * Class: com_dokukino_rados4j_Rados
2816+ * Method: initialize
2817+ * Signature: ([Ljava/lang/String;)I
2818+ */
2819+JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_initialize
2820+ (JNIEnv *, jobject, jobjectArray);
2821+
2822+/*
2823+ * Class: com_dokukino_rados4j_Rados
2824+ * Method: shutdown
2825+ * Signature: ()V
2826+ */
2827+JNIEXPORT void JNICALL Java_com_dokukino_rados4j_Rados_shutdown
2828+ (JNIEnv *, jobject);
2829+
2830+/*
2831+ * Class: com_dokukino_rados4j_Rados
2832+ * Method: openPool
2833+ * Signature: (Ljava/lang/String;)Lcom/dokukino/rados4j/Pool;
2834+ */
2835+JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Rados_openPool
2836+ (JNIEnv *, jobject, jstring);
2837+
2838+/*
2839+ * Class: com_dokukino_rados4j_Rados
2840+ * Method: lookupPool
2841+ * Signature: (Ljava/lang/String;)Z
2842+ */
2843+JNIEXPORT jboolean JNICALL Java_com_dokukino_rados4j_Rados_lookupPool
2844+ (JNIEnv *, jobject, jstring);
2845+
2846+/*
2847+ * Class: com_dokukino_rados4j_Rados
2848+ * Method: listPools
2849+ * Signature: ([Ljava/lang/String;)I
2850+ */
2851+JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_listPools
2852+ (JNIEnv *, jobject, jobjectArray);
2853+
2854+/*
2855+ * Class: com_dokukino_rados4j_Rados
2856+ * Method: createPool
2857+ * Signature: (Ljava/lang/String;J)I
2858+ */
2859+JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_createPool
2860+ (JNIEnv *, jobject, jstring, jlong);
2861+
2862+#ifdef __cplusplus
2863+}
2864+#endif
2865+#endif
2866
2867=== added directory 'clc/modules/storage-rados/lib'
2868=== added directory 'clc/modules/storage-rados/native'
2869=== added file 'clc/modules/storage-rados/native/Makefile'
2870--- clc/modules/storage-rados/native/Makefile 1970-01-01 00:00:00 +0000
2871+++ clc/modules/storage-rados/native/Makefile 2010-11-15 08:38:14 +0000
2872@@ -0,0 +1,28 @@
2873+LIBDIR=../lib
2874+RADOS4J_SO=librados4j.so
2875+PLATFORM=linux
2876+CXXFLAGS=-g -fpic -Wall -D_FILE_OFFSET_BITS=64 -O0 -I $(JAVA_HOME)/include -I $(JAVA_HOME)/include/$(PLATFORM) -I. -I../include -I$(RADOS_HOME)/include -L$(RADOS_HOME)/lib -lcrypto -lrados -lcrush
2877+WARN=-Wall -Wno-unused-variable
2878+
2879+.PHONY: all clean install
2880+
2881+all: $(LIBDIR)/$(RADOS4J_SO)
2882+
2883+.cpp.o:
2884+ g++ -c $(WARN) $(CXXFLAGS) $<
2885+
2886+$(LIBDIR)/$(RADOS4J_SO):
2887+ mkdir -p $(LIBDIR)
2888+ gcc -shared $(CXXFLAGS) -o $(LIBDIR)/$(RADOS4J_SO) *.cpp
2889+
2890+clean:
2891+ rm -f $(LIBDIR)/$(RADOS4J_SO) *.o *~* *#*
2892+
2893+distclean: clean
2894+
2895+install:
2896+ mkdir -p $(DESTDIR)$(EUCALYPTUS)/usr/lib/eucalyptus/
2897+ cp $(LIBDIR)/$(RADOS4J_SO) $(DESTDIR)$(EUCALYPTUS)/usr/lib/eucalyptus/
2898+
2899+uninstall:
2900+ rm -r $(DESTDIR)$(EUCALYPTUS)/usr/lib/eucalyptus/$(RADOS4J_SO)
2901
2902=== added file 'clc/modules/storage-rados/native/com_dokukino_rados4j_ListCtx.cpp'
2903--- clc/modules/storage-rados/native/com_dokukino_rados4j_ListCtx.cpp 1970-01-01 00:00:00 +0000
2904+++ clc/modules/storage-rados/native/com_dokukino_rados4j_ListCtx.cpp 2010-11-15 08:38:14 +0000
2905@@ -0,0 +1,109 @@
2906+#include "com_dokukino_rados4j_ListCtx.h"
2907+#include <rados/librados.hpp>
2908+#include <list>
2909+
2910+jclass list_ctx_class = NULL;
2911+jmethodID list_ctx_constructor = NULL;
2912+jfieldID list_ctx_instance_ptr = NULL, list_ctx_rados_ptr = NULL;
2913+
2914+int
2915+list_ctx_field_initialize(JNIEnv *env)
2916+{
2917+ jclass clsj = env->FindClass("Lcom/dokukino/rados4j/ListCtx;");
2918+ if (!clsj)
2919+ return -1;
2920+ list_ctx_class = reinterpret_cast<jclass>(env->NewGlobalRef(clsj));
2921+ env->DeleteLocalRef(clsj);
2922+ list_ctx_constructor = env->GetMethodID(list_ctx_class, "<init>", "()V");
2923+ if (!list_ctx_constructor)
2924+ return -1;
2925+ list_ctx_instance_ptr = env->GetFieldID(list_ctx_class, "instancePtr", "J");
2926+ if (!list_ctx_instance_ptr)
2927+ return -1;
2928+ list_ctx_rados_ptr = env->GetFieldID(list_ctx_class, "radosPtr", "J");
2929+ if (!list_ctx_rados_ptr)
2930+ return -1;
2931+ return 0;
2932+}
2933+
2934+static inline void
2935+list_ctx_set_instance_ptr(JNIEnv *env, jobject obj, librados::Rados::ListCtx *ptr)
2936+{
2937+ env->SetLongField(obj, list_ctx_instance_ptr, reinterpret_cast<long>(ptr));
2938+}
2939+
2940+static inline librados::Rados::ListCtx *
2941+list_ctx_get_instance_ptr(JNIEnv *env, jobject obj)
2942+{
2943+ jlong ptr = env->GetLongField(obj, list_ctx_instance_ptr);
2944+ return reinterpret_cast<librados::Rados::ListCtx *>(ptr);
2945+}
2946+
2947+static inline librados::Rados *
2948+list_ctx_get_rados_ptr(JNIEnv *env, jobject obj)
2949+{
2950+ jlong ptr = env->GetLongField(obj, list_ctx_rados_ptr);
2951+ return reinterpret_cast<librados::Rados *>(ptr);
2952+}
2953+
2954+extern "C"
2955+{
2956+ /*
2957+ * Class: com_dokukino_rados4j_ListCtx
2958+ * Method: more
2959+ * Signature: (I[Ljava/lang/String;)I
2960+ */
2961+ JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_ListCtx_more
2962+ (JNIEnv *env, jobject obj, jint max, jobjectArray entries)
2963+ {
2964+ librados::Rados::ListCtx *ctx;
2965+ librados::Rados *rados;
2966+ std::list<std::string> entries_list;
2967+ jint r;
2968+
2969+ if (!entries)
2970+ return -1;
2971+ ctx = list_ctx_get_instance_ptr(env, obj);
2972+ if (!ctx)
2973+ return -1;
2974+ rados = list_ctx_get_rados_ptr(env, obj);
2975+ if (!rados)
2976+ return -1;
2977+ r = rados->list_objects_more(*ctx, max, entries_list);
2978+ if (r < 0)
2979+ return r;
2980+ int i = 0;
2981+ const int len = env->GetArrayLength(entries);
2982+ for (std::list<std::string>::iterator iter = entries_list.begin();
2983+ i < len && iter != entries_list.end(); ++iter, ++i)
2984+ {
2985+ jstring entry = env->NewStringUTF((*iter).c_str());
2986+ env->SetObjectArrayElement(entries, i, entry);
2987+ }
2988+
2989+ return r;
2990+ }
2991+
2992+ /*
2993+ * Class: com_dokukino_rados4j_ListCtx
2994+ * Method: close
2995+ * Signature: ()V
2996+ */
2997+ JNIEXPORT void JNICALL Java_com_dokukino_rados4j_ListCtx_close
2998+ (JNIEnv *env, jobject obj)
2999+ {
3000+ librados::Rados::ListCtx *ctx;
3001+ librados::Rados *rados;
3002+
3003+ ctx = list_ctx_get_instance_ptr(env, obj);
3004+ if (!ctx)
3005+ return;
3006+ rados = list_ctx_get_rados_ptr(env, obj);
3007+ if (!rados)
3008+ return;
3009+ rados->list_objects_close(*ctx);
3010+ delete ctx;
3011+ list_ctx_set_instance_ptr(env, obj, NULL);
3012+ }
3013+
3014+}
3015
3016=== added file 'clc/modules/storage-rados/native/com_dokukino_rados4j_Pool.cpp'
3017--- clc/modules/storage-rados/native/com_dokukino_rados4j_Pool.cpp 1970-01-01 00:00:00 +0000
3018+++ clc/modules/storage-rados/native/com_dokukino_rados4j_Pool.cpp 2010-11-15 08:38:14 +0000
3019@@ -0,0 +1,484 @@
3020+#include "com_dokukino_rados4j_Pool.h"
3021+#include <rados/librados.hpp>
3022+#include <iostream>
3023+
3024+jclass pool_class = NULL, stat_class = NULL;
3025+jfieldID pool_instance_ptr = NULL, pool_rados_ptr = NULL, stat_psize = NULL, stat_pmtime = NULL;
3026+jmethodID pool_constructor = NULL, stat_constructor = NULL;
3027+
3028+extern jclass list_ctx_class;
3029+extern jmethodID list_ctx_constructor;
3030+extern jfieldID list_ctx_instance_ptr, list_ctx_rados_ptr;
3031+
3032+int
3033+pool_field_initialize(JNIEnv *env)
3034+{
3035+ jclass clsj = env->FindClass("Lcom/dokukino/rados4j/Pool;");
3036+ if (!clsj)
3037+ return -1;
3038+ pool_class = reinterpret_cast<jclass>(env->NewGlobalRef(clsj));
3039+ env->DeleteLocalRef(clsj);
3040+
3041+ pool_constructor = env->GetMethodID(pool_class, "<init>", "()V");
3042+ if (!pool_constructor)
3043+ return -1;
3044+ pool_instance_ptr = env->GetFieldID(pool_class, "instancePtr", "J");
3045+ if (!pool_instance_ptr)
3046+ return -1;
3047+ pool_rados_ptr = env->GetFieldID(pool_class, "radosPtr", "J");
3048+ if (!pool_rados_ptr)
3049+ return -1;
3050+ return 0;
3051+}
3052+
3053+int
3054+stat_field_initialize(JNIEnv *env)
3055+{
3056+ jclass clsj = env->FindClass("Lcom/dokukino/rados4j/Stat;");
3057+ if (!clsj)
3058+ return -1;
3059+ stat_class = reinterpret_cast<jclass>(env->NewGlobalRef(clsj));
3060+ env->DeleteLocalRef(clsj);
3061+
3062+ stat_constructor = env->GetMethodID(stat_class, "<init>", "()V");
3063+ if (!stat_constructor)
3064+ return -1;
3065+ stat_psize = env->GetFieldID(stat_class, "psize", "J");
3066+ if (!stat_psize)
3067+ return -1;
3068+ stat_pmtime = env->GetFieldID(stat_class, "pmtime", "J");
3069+ if (!stat_pmtime)
3070+ return -1;
3071+ return 0;
3072+}
3073+
3074+static inline void
3075+pool_set_instance_ptr(JNIEnv *env, jobject obj, librados::pool_t pool)
3076+{
3077+ env->SetLongField(obj, pool_instance_ptr, reinterpret_cast<jlong>(pool));
3078+}
3079+
3080+static inline void
3081+pool_set_rados_ptr(JNIEnv *env, jobject obj, librados::Rados *rados)
3082+{
3083+ env->SetLongField(obj, pool_rados_ptr, reinterpret_cast<jlong>(rados));
3084+}
3085+
3086+static inline librados::pool_t
3087+pool_get_instance_ptr(JNIEnv *env, jobject obj)
3088+{
3089+ jlong ptr = env->GetLongField(obj, pool_instance_ptr);
3090+ return reinterpret_cast<librados::pool_t>(ptr);
3091+}
3092+
3093+static inline librados::Rados *
3094+pool_get_rados_ptr(JNIEnv *env, jobject obj)
3095+{
3096+ jlong ptr = env->GetLongField(obj, pool_rados_ptr);
3097+ return reinterpret_cast<librados::Rados *>(ptr);
3098+}
3099+
3100+static inline jobject
3101+pool_new_instance(JNIEnv *env)
3102+{
3103+ return env->NewObject(pool_class, pool_constructor);
3104+}
3105+
3106+static inline jobject
3107+stat_new_instance(JNIEnv *env)
3108+{
3109+ return env->NewObject(stat_class, stat_constructor);
3110+}
3111+
3112+static inline void
3113+stat_set_psize(JNIEnv *env, jobject obj, uint64_t psize)
3114+{
3115+ env->SetLongField(obj, stat_psize, static_cast<jlong>(psize));
3116+}
3117+
3118+static inline void
3119+stat_set_pmtime(JNIEnv *env, jobject obj, time_t pmtime)
3120+{
3121+ env->SetLongField(obj, stat_pmtime, static_cast<jlong>(pmtime));
3122+}
3123+
3124+static inline jobject
3125+list_ctx_new_instance(JNIEnv *env)
3126+{
3127+ return env->NewObject(list_ctx_class, list_ctx_constructor);
3128+}
3129+
3130+static inline void
3131+list_ctx_set_instance_ptr(JNIEnv *env, jobject obj, librados::Rados::ListCtx *ctx)
3132+{
3133+ env->SetLongField(obj, list_ctx_instance_ptr, reinterpret_cast<jlong>(ctx));
3134+}
3135+
3136+static inline void
3137+list_ctx_set_rados_ptr(JNIEnv *env, jobject obj, librados::Rados *rados)
3138+{
3139+ env->SetLongField(obj, list_ctx_rados_ptr, reinterpret_cast<jlong>(rados));
3140+}
3141+
3142+extern "C"
3143+{
3144+
3145+ /*
3146+ * Class: com_dokukino_rados4j_Pool
3147+ * Method: delete
3148+ * Signature: ()I
3149+ */
3150+ JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_delete
3151+ (JNIEnv *env, jobject obj)
3152+ {
3153+ librados::Rados *rados;
3154+ librados::pool_t pool_ptr;
3155+
3156+ rados = pool_get_rados_ptr(env, obj);
3157+ if (!rados)
3158+ return -1;
3159+ pool_ptr = pool_get_instance_ptr(env, obj);
3160+ if (!pool_ptr)
3161+ return -1;
3162+ return rados->delete_pool(pool_ptr);
3163+ }
3164+
3165+ /*
3166+ * Class: com_dokukino_rados4j_Pool
3167+ * Method: close
3168+ * Signature: ()V
3169+ */
3170+ JNIEXPORT void JNICALL Java_com_dokukino_rados4j_Pool_close
3171+ (JNIEnv *env, jobject obj)
3172+ {
3173+ librados::Rados *rados;
3174+ librados::pool_t pool_ptr;
3175+
3176+ rados = pool_get_rados_ptr(env, obj);
3177+ if (!rados)
3178+ return;
3179+ pool_ptr = pool_get_instance_ptr(env, obj);
3180+ if (!pool_ptr)
3181+ return;
3182+
3183+ rados->close_pool(pool_ptr);
3184+ pool_set_instance_ptr(env, obj, NULL);
3185+ }
3186+
3187+ /*
3188+ * Class: com_dokukino_rados4j_Pool
3189+ * Method: createObj
3190+ * Signature: (Ljava/lang/String;Z)I
3191+ */
3192+ JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_createObj
3193+ (JNIEnv *env, jobject obj, jstring oid, jboolean exclusive)
3194+ {
3195+ librados::Rados *rados;
3196+ librados::pool_t pool_ptr;
3197+ const char *oid_char;
3198+ jint ret;
3199+
3200+ if (!oid)
3201+ return -1;
3202+ rados = pool_get_rados_ptr(env, obj);
3203+ if (!rados)
3204+ return -1;
3205+ pool_ptr = pool_get_instance_ptr(env, obj);
3206+ if (!pool_ptr)
3207+ return -1;
3208+ oid_char = env->GetStringUTFChars(oid, NULL);
3209+ if (!oid_char)
3210+ return -1;
3211+ ret = rados->create(pool_ptr, oid_char, exclusive);
3212+ env->ReleaseStringUTFChars(oid, oid_char);
3213+ return ret;
3214+ }
3215+
3216+ /*
3217+ * Class: com_dokukino_rados4j_Pool
3218+ * Method: writeObj
3219+ * Signature: (Ljava/lang/String;J[BJ)I
3220+ */
3221+ JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_writeObj
3222+ (JNIEnv *env, jobject obj, jstring oid, jlong off, jbyteArray buf, jlong len)
3223+ {
3224+ librados::Rados *rados;
3225+ librados::pool_t pool_ptr;
3226+ librados::bufferlist bl;
3227+ const char *oid_char;
3228+ jbyte* src;
3229+ jint ret;
3230+
3231+ if (!oid || !buf)
3232+ return -1;
3233+
3234+ rados = pool_get_rados_ptr(env, obj);
3235+ if (!rados)
3236+ return -1;
3237+
3238+ pool_ptr = pool_get_instance_ptr(env, obj);
3239+ if (!pool_ptr)
3240+ return -1;
3241+
3242+ oid_char = env->GetStringUTFChars(oid, NULL);
3243+ if (!oid_char)
3244+ return -1;
3245+
3246+ src = env->GetByteArrayElements(buf, NULL);
3247+ bl.append(reinterpret_cast<char *>(src), len);
3248+ ret = rados->write(pool_ptr, oid_char, off, bl, len);
3249+ env->ReleaseByteArrayElements(buf, src, 0);
3250+ env->ReleaseStringUTFChars(oid, oid_char);
3251+
3252+ return ret;
3253+ }
3254+
3255+ /*
3256+ * Class: com_dokukino_rados4j_Pool
3257+ * Method: readObj
3258+ * Signature: (Ljava/lang/String;J[BJ)I
3259+ */
3260+ JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_readObj
3261+ (JNIEnv *env, jobject obj, jstring oid, jlong off, jbyteArray buf, jlong len)
3262+ {
3263+ librados::Rados *rados;
3264+ librados::pool_t pool_ptr;
3265+ librados::bufferlist bl;
3266+ const char *oid_char;
3267+ jint ret;
3268+
3269+ if (!oid || !buf)
3270+ return -1;
3271+ rados = pool_get_rados_ptr(env, obj);
3272+ if (!rados)
3273+ return -1;
3274+ pool_ptr = pool_get_instance_ptr(env, obj);
3275+ if (!pool_ptr)
3276+ return -1;
3277+ oid_char = env->GetStringUTFChars(oid, NULL);
3278+ if (!oid_char)
3279+ return -1;
3280+
3281+ ret = rados->read(pool_ptr, oid_char, off, bl, len);
3282+
3283+ if (ret > 0)
3284+ env->SetByteArrayRegion(buf, 0, ret, reinterpret_cast<jbyte*>(bl.c_str()));
3285+ env->ReleaseStringUTFChars(oid, oid_char);
3286+
3287+ return ret;
3288+ }
3289+
3290+
3291+ /*
3292+ * Class: com_dokukino_rados4j_Pool
3293+ * Method: renameObj
3294+ * Signature: (Ljava/lang/String;Ljava/lang/String;)I
3295+ */
3296+ JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_renameObj
3297+ (JNIEnv *env, jobject obj, jstring old_oid, jstring new_oid)
3298+ {
3299+ librados::Rados *rados;
3300+ librados::pool_t pool_ptr;
3301+ librados::bufferlist bl;
3302+ const char *old_oid_char, *new_oid_char;
3303+ jint ret;
3304+
3305+ if (!old_oid || !new_oid)
3306+ return -1;
3307+ rados = pool_get_rados_ptr(env, obj);
3308+ if (!rados)
3309+ return -1;
3310+ pool_ptr = pool_get_instance_ptr(env, obj);
3311+ if (!pool_ptr)
3312+ return -1;
3313+ old_oid_char = env->GetStringUTFChars(old_oid, NULL);
3314+ if (!old_oid_char)
3315+ return -1;
3316+ new_oid_char = env->GetStringUTFChars(new_oid, NULL);
3317+ if (!new_oid_char)
3318+ {
3319+ env->ReleaseStringUTFChars(old_oid, old_oid_char);
3320+ return -1;
3321+ }
3322+
3323+ ret = rados->read(pool_ptr, old_oid_char, 0, bl, 0);
3324+ if (ret < 0)
3325+ goto err;
3326+
3327+ ret = rados->write_full(pool_ptr, new_oid_char, bl);
3328+ if (ret < 0)
3329+ goto err;
3330+
3331+ rados->remove(pool_ptr, old_oid_char);
3332+err:
3333+ env->ReleaseStringUTFChars(old_oid, old_oid_char);
3334+ env->ReleaseStringUTFChars(new_oid, new_oid_char);
3335+ return ret;
3336+ }
3337+
3338+ /*
3339+ * Class: com_dokukino_rados4j_Pool
3340+ * Method: copyObj
3341+ * Signature: (Ljava/lang/String;Lcom/dokukino/rados4j/Pool;Ljava/lang/String;)I
3342+ */
3343+ JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_copyObj
3344+ (JNIEnv *env, jobject obj, jstring src_oid, jobject dest_pool, jstring dest_oid)
3345+ {
3346+ librados::Rados *rados;
3347+ librados::pool_t src_pool_ptr, dest_pool_ptr;
3348+ librados::bufferlist bl;
3349+ const char *src_oid_char, *dest_oid_char;
3350+ jint ret;
3351+
3352+ if (!src_oid || !dest_pool || !dest_oid)
3353+ return -1;
3354+ rados = pool_get_rados_ptr(env, obj);
3355+ if (!rados)
3356+ return -1;
3357+ src_pool_ptr = pool_get_instance_ptr(env, obj);
3358+ if (!src_pool_ptr)
3359+ return -1;
3360+ dest_pool_ptr = pool_get_instance_ptr(env, dest_pool);
3361+ if (!dest_pool_ptr)
3362+ return -1;
3363+ src_oid_char = env->GetStringUTFChars(src_oid, NULL);
3364+ if (!src_oid_char)
3365+ return -1;
3366+ dest_oid_char = env->GetStringUTFChars(dest_oid, NULL);
3367+ if (!dest_oid_char)
3368+ {
3369+ env->ReleaseStringUTFChars(src_oid, src_oid_char);
3370+ return -1;
3371+ }
3372+
3373+ ret = rados->read(src_pool_ptr, src_oid_char, 0, bl, 0);
3374+ if (ret < 0)
3375+ goto err;
3376+
3377+ ret = rados->write_full(dest_pool_ptr, dest_oid_char, bl);
3378+ if (ret < 0)
3379+ goto err;
3380+
3381+err:
3382+ env->ReleaseStringUTFChars(src_oid, src_oid_char);
3383+ env->ReleaseStringUTFChars(dest_oid, dest_oid_char);
3384+ return ret;
3385+ }
3386+
3387+
3388+ /*
3389+ * Class: com_dokukino_rados4j_Pool
3390+ * Method: removeObj
3391+ * Signature: (Ljava/lang/String;)I
3392+ */
3393+ JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_removeObj
3394+ (JNIEnv *env, jobject obj, jstring oid)
3395+ {
3396+ librados::Rados *rados;
3397+ librados::pool_t pool_ptr;
3398+ const char *oid_char;
3399+ jint ret;
3400+
3401+ if (!oid)
3402+ return -1;
3403+ rados = pool_get_rados_ptr(env, obj);
3404+ if (!rados)
3405+ return -1;
3406+ pool_ptr = pool_get_instance_ptr(env, obj);
3407+ if (!pool_ptr)
3408+ return -1;
3409+ oid_char = env->GetStringUTFChars(oid, NULL);
3410+ if (!oid_char)
3411+ return -1;
3412+ ret = rados->remove(pool_ptr, oid_char);
3413+ env->ReleaseStringUTFChars(oid, oid_char);
3414+
3415+ return ret;
3416+ }
3417+
3418+ /*
3419+ * Class: com_dokukino_rados4j_Pool
3420+ * Method: statObj
3421+ * Signature: (Ljava/lang/String;)Lcom/dokukino/rados4j/Stat;
3422+ */
3423+ JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Pool_statObj
3424+ (JNIEnv *env, jobject obj, jstring oid)
3425+ {
3426+ librados::Rados *rados;
3427+ librados::pool_t pool_ptr;
3428+ const char *oid_char;
3429+ uint64_t psize;
3430+ time_t pmtime;
3431+ jint ret;
3432+
3433+ if (!oid)
3434+ return NULL;
3435+
3436+ rados = pool_get_rados_ptr(env, obj);
3437+ if (!rados)
3438+ return NULL;
3439+
3440+ pool_ptr = pool_get_instance_ptr(env, obj);
3441+ if (!pool_ptr)
3442+ return NULL;
3443+
3444+ oid_char = env->GetStringUTFChars(oid, NULL);
3445+ if (!oid_char)
3446+ return NULL;
3447+
3448+ ret = rados->stat(pool_ptr, oid_char, &psize, &pmtime);
3449+
3450+ env->ReleaseStringUTFChars(oid, oid_char);
3451+ if (ret < 0)
3452+ return NULL;
3453+ else
3454+ {
3455+ jobject obj = stat_new_instance(env);
3456+ if (!obj)
3457+ return NULL;
3458+ stat_set_psize(env, obj, psize);
3459+ stat_set_pmtime(env, obj, pmtime);
3460+ return obj;
3461+ }
3462+ }
3463+
3464+ /*
3465+ * Class: com_dokukino_rados4j_Pool
3466+ * Method: openList
3467+ * Signature: ()Lcom/dokukino/rados4j/ListCtx;
3468+ */
3469+ JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Pool_openList
3470+ (JNIEnv *env, jobject obj)
3471+ {
3472+ librados::Rados *rados;
3473+ librados::pool_t pool_ptr;
3474+ librados::Rados::ListCtx *ctx = NULL;
3475+ jint ret;
3476+
3477+ rados = pool_get_rados_ptr(env, obj);
3478+ if (!rados)
3479+ return NULL;
3480+
3481+ pool_ptr = pool_get_instance_ptr(env, obj);
3482+ if (!pool_ptr)
3483+ return NULL;
3484+
3485+ ctx = new librados::Rados::ListCtx();
3486+ ret = rados->list_objects_open(pool_ptr, ctx);
3487+
3488+ if (ret < 0)
3489+ {
3490+ delete ctx;
3491+ return NULL;
3492+ }
3493+ else
3494+ {
3495+ jobject obj = list_ctx_new_instance(env);
3496+ if (!obj)
3497+ return NULL;
3498+ list_ctx_set_instance_ptr(env, obj, ctx);
3499+ list_ctx_set_rados_ptr(env, obj, rados);
3500+ return obj;
3501+ }
3502+ }
3503+}
3504
3505=== added file 'clc/modules/storage-rados/native/com_dokukino_rados4j_Rados.cpp'
3506--- clc/modules/storage-rados/native/com_dokukino_rados4j_Rados.cpp 1970-01-01 00:00:00 +0000
3507+++ clc/modules/storage-rados/native/com_dokukino_rados4j_Rados.cpp 2010-11-15 08:38:14 +0000
3508@@ -0,0 +1,245 @@
3509+#include "com_dokukino_rados4j_Rados.h"
3510+#include <rados/librados.hpp>
3511+#include <iostream>
3512+
3513+static jfieldID rados_instance_ptr = NULL;
3514+
3515+extern jclass pool_class;
3516+extern jfieldID pool_instance_ptr, pool_rados_ptr;
3517+extern jmethodID pool_constructor;
3518+
3519+extern int pool_field_initialize(JNIEnv *);
3520+extern int stat_field_initialize(JNIEnv *);
3521+extern int list_ctx_field_initialize(JNIEnv *);
3522+
3523+static int
3524+rados_field_initialize(JNIEnv *env)
3525+{
3526+ jclass clsj = env->FindClass("Lcom/dokukino/rados4j/Rados;");
3527+ if (!clsj)
3528+ {
3529+ printf("findclass failed\n");
3530+ return -1;
3531+ }
3532+ rados_instance_ptr = env->GetFieldID(clsj, "instancePtr", "J");
3533+ if (!rados_instance_ptr)
3534+ {
3535+ printf("getfieldid failed\n");
3536+ return -1;
3537+ }
3538+ return 0;
3539+}
3540+
3541+static inline void
3542+rados_set_instance_ptr(JNIEnv *env, jobject obj, librados::Rados *rados)
3543+{
3544+ env->SetLongField(obj, rados_instance_ptr, reinterpret_cast<jlong>(rados));
3545+}
3546+
3547+static inline librados::Rados *
3548+rados_get_instance_ptr(JNIEnv *env, jobject obj)
3549+{
3550+ jlong ptr = env->GetLongField(obj, rados_instance_ptr);
3551+ return reinterpret_cast<librados::Rados *>(ptr);
3552+}
3553+
3554+static inline jobject
3555+pool_new_instance(JNIEnv *env)
3556+{
3557+ return env->NewObject(pool_class, pool_constructor);
3558+}
3559+
3560+static inline void
3561+pool_set_instance_ptr(JNIEnv *env, jobject obj, librados::pool_t pool)
3562+{
3563+ env->SetLongField(obj, pool_instance_ptr, reinterpret_cast<jlong>(pool));
3564+}
3565+
3566+static inline void
3567+pool_set_rados_ptr(JNIEnv *env, jobject obj, librados::Rados *rados)
3568+{
3569+ env->SetLongField(obj, pool_rados_ptr, reinterpret_cast<jlong>(rados));
3570+}
3571+
3572+extern "C"
3573+{
3574+ /*
3575+ * Class: com_dokukino_rados4j_Rados
3576+ * Method: initialize
3577+ * Signature: ([Ljava/lang/String;)I
3578+ */
3579+ JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_initialize
3580+ (JNIEnv *env, jobject obj, jobjectArray args)
3581+ {
3582+ jint ret;
3583+ librados::Rados *rados;
3584+ ret = rados_field_initialize(env);
3585+ if (ret)
3586+ return -1;
3587+
3588+ ret = pool_field_initialize(env);
3589+ if (ret)
3590+ return -1;
3591+
3592+ ret = stat_field_initialize(env);
3593+ if (ret)
3594+ return -1;
3595+
3596+ ret = list_ctx_field_initialize(env);
3597+ if (ret)
3598+ return -1;
3599+
3600+ rados = new librados::Rados();
3601+ if (!rados)
3602+ return -1;
3603+
3604+ ret = rados->initialize(0, NULL);
3605+ rados_set_instance_ptr(env, obj, rados);
3606+ return ret;
3607+ }
3608+
3609+ /*
3610+ * Class: com_dokukino_rados4j_Rados
3611+ * Method: shutdown
3612+ * Signature: ()V
3613+ */
3614+ JNIEXPORT void JNICALL Java_com_dokukino_rados4j_Rados_shutdown
3615+ (JNIEnv *env, jobject obj)
3616+ {
3617+ librados::Rados *rados;
3618+
3619+ rados = rados_get_instance_ptr(env, obj);
3620+ if (!rados)
3621+ return;
3622+ rados->shutdown();
3623+ delete rados;
3624+ rados_set_instance_ptr(env, obj, NULL);
3625+ }
3626+
3627+ /*
3628+ * Class: com_dokukino_rados4j_Rados
3629+ * Method: openPool
3630+ * Signature: (Ljava/lang/String;Lcom/dokukino/rados4j/Pool;)I
3631+ */
3632+ JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Rados_openPool
3633+ (JNIEnv *env, jobject obj, jstring name)
3634+ {
3635+ librados::Rados *rados;
3636+ const char *name_char;
3637+ librados::pool_t pool;
3638+ jint ret;
3639+
3640+ if (!name)
3641+ return NULL;
3642+ rados = rados_get_instance_ptr(env, obj);
3643+ if (!rados)
3644+ return NULL;
3645+ name_char = env->GetStringUTFChars(name, NULL);
3646+ if (!name_char)
3647+ return NULL;
3648+ ret = rados->open_pool(name_char, &pool);
3649+
3650+ env->ReleaseStringUTFChars(name, name_char);
3651+ if (ret < 0)
3652+ return NULL;
3653+ else
3654+ {
3655+ jobject obj = pool_new_instance(env);
3656+ if (!obj)
3657+ {
3658+ rados->close_pool(pool);
3659+ return NULL;
3660+ }
3661+ pool_set_instance_ptr(env, obj, pool);
3662+ pool_set_rados_ptr(env, obj, rados);
3663+ return obj;
3664+ }
3665+ }
3666+
3667+ /*
3668+ * Class: com_dokukino_rados4j_Rados
3669+ * Method: lookupPool
3670+ * Signature: (Ljava/lang/String;)Z
3671+ */
3672+ JNIEXPORT jboolean JNICALL Java_com_dokukino_rados4j_Rados_lookupPool
3673+ (JNIEnv *env, jobject obj, jstring name)
3674+ {
3675+ librados::Rados *rados;
3676+ const char *name_char;
3677+ int ret;
3678+
3679+ if (!name)
3680+ return -1;
3681+
3682+ rados = rados_get_instance_ptr(env, obj);
3683+ if (!rados)
3684+ return -1;
3685+
3686+ name_char = env->GetStringUTFChars(name, NULL);
3687+ if (!name_char)
3688+ return -1;
3689+
3690+ ret = rados->lookup_pool(name_char);
3691+ env->ReleaseStringUTFChars(name, name_char);
3692+
3693+ return (ret > 0);
3694+ }
3695+
3696+ /*
3697+ * Class: com_dokukino_rados4j_Rados
3698+ * Method: listPools
3699+ * Signature: ([Ljava/lang/String;)I
3700+ */
3701+ JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_listPools
3702+ (JNIEnv *env, jobject obj, jobjectArray entries)
3703+ {
3704+ librados::Rados *rados;
3705+ std::list<std::string> entries_list;
3706+ int r;
3707+
3708+ if (!entries)
3709+ return -1;
3710+
3711+ rados = rados_get_instance_ptr(env, obj);
3712+ if (!rados)
3713+ return -1;
3714+
3715+ r = rados->list_pools(entries_list);
3716+ if (r < 0)
3717+ return r;
3718+
3719+ int i = 0;
3720+ const int len = env->GetArrayLength(entries);
3721+ for (std::list<std::string>::iterator iter = entries_list.begin();
3722+ i < len && iter != entries_list.end(); ++iter, ++i)
3723+ {
3724+ jstring entry = env->NewStringUTF((*iter).c_str());
3725+ env->SetObjectArrayElement(entries, i, entry);
3726+ }
3727+
3728+ return r;
3729+ }
3730+
3731+ /*
3732+ * Class: com_dokukino_rados4j_Rados
3733+ * Method: createPool
3734+ * Signature: (Ljava/lang/String;J)I
3735+ */
3736+ JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_createPool
3737+ (JNIEnv *env, jobject obj, jstring name, jlong auid)
3738+ {
3739+ librados::Rados *rados;
3740+ const char *name_char;
3741+
3742+ if (!name)
3743+ return -1;
3744+ rados = rados_get_instance_ptr(env, obj);
3745+ if (!rados)
3746+ return -1;
3747+ name_char = env->GetStringUTFChars(name, NULL);
3748+ if (!name_char)
3749+ return -1;
3750+ return rados->create_pool(name_char, auid);
3751+ }
3752+
3753+}
3754
3755=== added directory 'clc/modules/storage-rados/src'
3756=== added directory 'clc/modules/storage-rados/src/main'
3757=== added directory 'clc/modules/storage-rados/src/main/java'
3758=== added directory 'clc/modules/storage-rados/src/main/java/com'
3759=== added directory 'clc/modules/storage-rados/src/main/java/com/dokukino'
3760=== added directory 'clc/modules/storage-rados/src/main/java/com/dokukino/rados4j'
3761=== added file 'clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/ListCtx.java'
3762--- clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/ListCtx.java 1970-01-01 00:00:00 +0000
3763+++ clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/ListCtx.java 2010-11-15 08:38:14 +0000
3764@@ -0,0 +1,68 @@
3765+/*******************************************************************************
3766+*Copyright (c) 2010 Takuya ASADA
3767+*
3768+* This program is free software: you can redistribute it and/or modify
3769+* it under the terms of the GNU General Public License as published by
3770+* the Free Software Foundation, only version 3 of the License.
3771+*
3772+*
3773+* This file is distributed in the hope that it will be useful, but WITHOUT
3774+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
3775+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
3776+* for more details.
3777+*
3778+* You should have received a copy of the GNU General Public License along
3779+* with this program. If not, see <http://www.gnu.org/licenses/>.
3780+*
3781+* Please contact Eucalyptus Systems, Inc., 130 Castilian
3782+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
3783+* if you need additional information or have any questions.
3784+*
3785+* This file may incorporate work covered under the following copyright and
3786+* permission notice:
3787+*
3788+* Software License Agreement (BSD License)
3789+*
3790+* Copyright (c) 2008, Regents of the University of California
3791+* All rights reserved.
3792+*
3793+* Redistribution and use of this software in source and binary forms, with
3794+* or without modification, are permitted provided that the following
3795+* conditions are met:
3796+*
3797+* Redistributions of source code must retain the above copyright notice,
3798+* this list of conditions and the following disclaimer.
3799+*
3800+* Redistributions in binary form must reproduce the above copyright
3801+* notice, this list of conditions and the following disclaimer in the
3802+* documentation and/or other materials provided with the distribution.
3803+*
3804+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
3805+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
3806+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
3807+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
3808+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
3809+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
3810+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
3811+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
3812+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
3813+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
3814+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
3815+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
3816+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
3817+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
3818+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
3819+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
3820+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
3821+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
3822+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
3823+* ANY SUCH LICENSES OR RIGHTS.
3824+*******************************************************************************/
3825+package com.dokukino.rados4j;
3826+
3827+public class ListCtx {
3828+ private long radosPtr, instancePtr;
3829+
3830+ public native int more(int max, String[] entries);
3831+ public native void close();
3832+}
3833
3834=== added file 'clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Pool.java'
3835--- clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Pool.java 1970-01-01 00:00:00 +0000
3836+++ clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Pool.java 2010-11-15 08:38:14 +0000
3837@@ -0,0 +1,76 @@
3838+/*******************************************************************************
3839+*Copyright (c) 2010 Takuya ASADA
3840+*
3841+* This program is free software: you can redistribute it and/or modify
3842+* it under the terms of the GNU General Public License as published by
3843+* the Free Software Foundation, only version 3 of the License.
3844+*
3845+*
3846+* This file is distributed in the hope that it will be useful, but WITHOUT
3847+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
3848+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
3849+* for more details.
3850+*
3851+* You should have received a copy of the GNU General Public License along
3852+* with this program. If not, see <http://www.gnu.org/licenses/>.
3853+*
3854+* Please contact Eucalyptus Systems, Inc., 130 Castilian
3855+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
3856+* if you need additional information or have any questions.
3857+*
3858+* This file may incorporate work covered under the following copyright and
3859+* permission notice:
3860+*
3861+* Software License Agreement (BSD License)
3862+*
3863+* Copyright (c) 2008, Regents of the University of California
3864+* All rights reserved.
3865+*
3866+* Redistribution and use of this software in source and binary forms, with
3867+* or without modification, are permitted provided that the following
3868+* conditions are met:
3869+*
3870+* Redistributions of source code must retain the above copyright notice,
3871+* this list of conditions and the following disclaimer.
3872+*
3873+* Redistributions in binary form must reproduce the above copyright
3874+* notice, this list of conditions and the following disclaimer in the
3875+* documentation and/or other materials provided with the distribution.
3876+*
3877+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
3878+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
3879+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
3880+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
3881+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
3882+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
3883+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
3884+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
3885+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
3886+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
3887+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
3888+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
3889+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
3890+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
3891+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
3892+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
3893+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
3894+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
3895+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
3896+* ANY SUCH LICENSES OR RIGHTS.
3897+*******************************************************************************/
3898+package com.dokukino.rados4j;
3899+
3900+public class Pool {
3901+ private long radosPtr, instancePtr;
3902+
3903+ public native int delete();
3904+ public native void close();
3905+ public native int createObj(String oid, boolean exclusive);
3906+ public native int writeObj(String oid, long off, byte[] buf, long len);
3907+ public native int readObj(String oid, long off, byte[] buf, long len);
3908+ public native int renameObj(String oldOid, String newOid);
3909+ public native int copyObj(String srcOid, Pool destPool, String destOid);
3910+ public native int removeObj(String oid);
3911+ public native Stat statObj(String oid);
3912+ public native ListCtx openList();
3913+}
3914
3915=== added file 'clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Rados.java'
3916--- clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Rados.java 1970-01-01 00:00:00 +0000
3917+++ clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Rados.java 2010-11-15 08:38:14 +0000
3918@@ -0,0 +1,77 @@
3919+/*******************************************************************************
3920+*Copyright (c) 2010 Takuya ASADA
3921+*
3922+* This program is free software: you can redistribute it and/or modify
3923+* it under the terms of the GNU General Public License as published by
3924+* the Free Software Foundation, only version 3 of the License.
3925+*
3926+*
3927+* This file is distributed in the hope that it will be useful, but WITHOUT
3928+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
3929+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
3930+* for more details.
3931+*
3932+* You should have received a copy of the GNU General Public License along
3933+* with this program. If not, see <http://www.gnu.org/licenses/>.
3934+*
3935+* Please contact Eucalyptus Systems, Inc., 130 Castilian
3936+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
3937+* if you need additional information or have any questions.
3938+*
3939+* This file may incorporate work covered under the following copyright and
3940+* permission notice:
3941+*
3942+* Software License Agreement (BSD License)
3943+*
3944+* Copyright (c) 2008, Regents of the University of California
3945+* All rights reserved.
3946+*
3947+* Redistribution and use of this software in source and binary forms, with
3948+* or without modification, are permitted provided that the following
3949+* conditions are met:
3950+*
3951+* Redistributions of source code must retain the above copyright notice,
3952+* this list of conditions and the following disclaimer.
3953+*
3954+* Redistributions in binary form must reproduce the above copyright
3955+* notice, this list of conditions and the following disclaimer in the
3956+* documentation and/or other materials provided with the distribution.
3957+*
3958+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
3959+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
3960+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
3961+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
3962+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
3963+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
3964+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
3965+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
3966+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
3967+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
3968+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
3969+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
3970+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
3971+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
3972+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
3973+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
3974+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
3975+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
3976+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
3977+* ANY SUCH LICENSES OR RIGHTS.
3978+*******************************************************************************/
3979+package com.dokukino.rados4j;
3980+
3981+public class Rados {
3982+ private long instancePtr;
3983+
3984+ static {
3985+ System.loadLibrary("rados4j");
3986+ }
3987+
3988+ public native int initialize(String[] args);
3989+ public native void shutdown();
3990+ public native Pool openPool(String name);
3991+ public native boolean lookupPool(String name);
3992+ public native int listPools(String[] v);
3993+ public native int createPool(String name, long auid);
3994+ public int createPool(String name) { return createPool(name, 0); }
3995+}
3996
3997=== added file 'clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Stat.java'
3998--- clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Stat.java 1970-01-01 00:00:00 +0000
3999+++ clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Stat.java 2010-11-15 08:38:14 +0000
4000@@ -0,0 +1,76 @@
4001+/*******************************************************************************
4002+*Copyright (c) 2010 Takuya ASADA
4003+*
4004+* This program is free software: you can redistribute it and/or modify
4005+* it under the terms of the GNU General Public License as published by
4006+* the Free Software Foundation, only version 3 of the License.
4007+*
4008+*
4009+* This file is distributed in the hope that it will be useful, but WITHOUT
4010+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
4011+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
4012+* for more details.
4013+*
4014+* You should have received a copy of the GNU General Public License along
4015+* with this program. If not, see <http://www.gnu.org/licenses/>.
4016+*
4017+* Please contact Eucalyptus Systems, Inc., 130 Castilian
4018+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
4019+* if you need additional information or have any questions.
4020+*
4021+* This file may incorporate work covered under the following copyright and
4022+* permission notice:
4023+*
4024+* Software License Agreement (BSD License)
4025+*
4026+* Copyright (c) 2008, Regents of the University of California
4027+* All rights reserved.
4028+*
4029+* Redistribution and use of this software in source and binary forms, with
4030+* or without modification, are permitted provided that the following
4031+* conditions are met:
4032+*
4033+* Redistributions of source code must retain the above copyright notice,
4034+* this list of conditions and the following disclaimer.
4035+*
4036+* Redistributions in binary form must reproduce the above copyright
4037+* notice, this list of conditions and the following disclaimer in the
4038+* documentation and/or other materials provided with the distribution.
4039+*
4040+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
4041+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
4042+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
4043+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
4044+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
4045+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
4046+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
4047+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
4048+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
4049+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
4050+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
4051+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
4052+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
4053+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
4054+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
4055+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
4056+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
4057+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
4058+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
4059+* ANY SUCH LICENSES OR RIGHTS.
4060+*******************************************************************************/
4061+package com.dokukino.rados4j;
4062+
4063+import java.util.Date;
4064+
4065+public class Stat {
4066+ private long psize;
4067+ private long pmtime;
4068+
4069+ public long getSize() {
4070+ return psize;
4071+ }
4072+
4073+ public long getMtime() {
4074+ return pmtime;
4075+ }
4076+}
4077
4078=== added directory 'clc/modules/storage-rados/src/main/java/edu'
4079=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb'
4080=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus'
4081=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud'
4082=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws'
4083=== added file 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedRadosObject.java'
4084--- clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedRadosObject.java 1970-01-01 00:00:00 +0000
4085+++ clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedRadosObject.java 2010-11-15 08:38:14 +0000
4086@@ -0,0 +1,125 @@
4087+/*******************************************************************************
4088+*Copyright (c) 2009 Eucalyptus Systems, Inc.
4089+*
4090+* This program is free software: you can redistribute it and/or modify
4091+* it under the terms of the GNU General Public License as published by
4092+* the Free Software Foundation, only version 3 of the License.
4093+*
4094+*
4095+* This file is distributed in the hope that it will be useful, but WITHOUT
4096+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
4097+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
4098+* for more details.
4099+*
4100+* You should have received a copy of the GNU General Public License along
4101+* with this program. If not, see <http://www.gnu.org/licenses/>.
4102+*
4103+* Please contact Eucalyptus Systems, Inc., 130 Castilian
4104+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
4105+* if you need additional information or have any questions.
4106+*
4107+* This file may incorporate work covered under the following copyright and
4108+* permission notice:
4109+*
4110+* Software License Agreement (BSD License)
4111+*
4112+* Copyright (c) 2008, Regents of the University of California
4113+* All rights reserved.
4114+*
4115+* Redistribution and use of this software in source and binary forms, with
4116+* or without modification, are permitted provided that the following
4117+* conditions are met:
4118+*
4119+* Redistributions of source code must retain the above copyright notice,
4120+* this list of conditions and the following disclaimer.
4121+*
4122+* Redistributions in binary form must reproduce the above copyright
4123+* notice, this list of conditions and the following disclaimer in the
4124+* documentation and/or other materials provided with the distribution.
4125+*
4126+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
4127+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
4128+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
4129+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
4130+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
4131+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
4132+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
4133+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
4134+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
4135+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
4136+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
4137+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
4138+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
4139+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
4140+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
4141+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
4142+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
4143+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
4144+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
4145+* ANY SUCH LICENSES OR RIGHTS.
4146+*******************************************************************************/
4147+package edu.ucsb.eucalyptus.cloud.ws;
4148+
4149+import java.io.ByteArrayOutputStream;
4150+import java.io.IOException;
4151+
4152+import org.apache.log4j.Logger;
4153+import org.jboss.netty.buffer.ChannelBuffers;
4154+import org.jboss.netty.handler.stream.ChunkedInput;
4155+
4156+import edu.ucsb.eucalyptus.constants.IsData;
4157+
4158+import com.dokukino.rados4j.*;
4159+
4160+public class ChunkedRadosObject implements ChunkedInput, IsData {
4161+ private Logger LOG = Logger.getLogger( ChunkedRadosObject.class );
4162+ private Pool pool;
4163+ private String object;
4164+ private long offset;
4165+ private int CHUNK_SIZE = 8192;
4166+ private long length;
4167+
4168+ public ChunkedRadosObject(Pool pool, String object, long length) {
4169+ this.pool = pool;
4170+ this.object = object;
4171+ this.offset = 0;
4172+ this.length = length;
4173+ }
4174+
4175+ public ChunkedRadosObject(Pool pool, String object, long start, long end, int chunkSize) {
4176+ this.pool = pool;
4177+ this.object = object;
4178+ this.offset = start;
4179+ this.length = end;
4180+ CHUNK_SIZE = chunkSize;
4181+ }
4182+
4183+ @Override
4184+ public void close() throws Exception {
4185+ pool.close();
4186+ }
4187+
4188+ @Override
4189+ public boolean hasNextChunk() throws Exception {
4190+ return offset < length;
4191+ }
4192+
4193+ @Override
4194+ public Object nextChunk() throws Exception {
4195+ long offset = this.offset;
4196+ if (offset >= length) {
4197+ return null;
4198+ }
4199+ int chunkSize = (int) Math.min(CHUNK_SIZE, length - offset);
4200+ byte[] chunk = new byte[chunkSize];
4201+ int ret = pool.readObj(object, offset, chunk, chunkSize);
4202+ if (ret < 0)
4203+ throw new IOException("Unable to read object: " + object);
4204+ this.offset = offset + chunkSize;
4205+ ByteArrayOutputStream out = new ByteArrayOutputStream();
4206+ out.write(chunk);
4207+ out.close();
4208+ return ChannelBuffers.wrappedBuffer(out.toByteArray());
4209+ }
4210+}
4211+
4212
4213=== added file 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedRadosObject.java'
4214--- clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedRadosObject.java 1970-01-01 00:00:00 +0000
4215+++ clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedRadosObject.java 2010-11-15 08:38:14 +0000
4216@@ -0,0 +1,127 @@
4217+/*******************************************************************************
4218+*Copyright (c) 2009 Eucalyptus Systems, Inc.
4219+*
4220+* This program is free software: you can redistribute it and/or modify
4221+* it under the terms of the GNU General Public License as published by
4222+* the Free Software Foundation, only version 3 of the License.
4223+*
4224+*
4225+* This file is distributed in the hope that it will be useful, but WITHOUT
4226+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
4227+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
4228+* for more details.
4229+*
4230+* You should have received a copy of the GNU General Public License along
4231+* with this program. If not, see <http://www.gnu.org/licenses/>.
4232+*
4233+* Please contact Eucalyptus Systems, Inc., 130 Castilian
4234+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
4235+* if you need additional information or have any questions.
4236+*
4237+* This file may incorporate work covered under the following copyright and
4238+* permission notice:
4239+*
4240+* Software License Agreement (BSD License)
4241+*
4242+* Copyright (c) 2008, Regents of the University of California
4243+* All rights reserved.
4244+*
4245+* Redistribution and use of this software in source and binary forms, with
4246+* or without modification, are permitted provided that the following
4247+* conditions are met:
4248+*
4249+* Redistributions of source code must retain the above copyright notice,
4250+* this list of conditions and the following disclaimer.
4251+*
4252+* Redistributions in binary form must reproduce the above copyright
4253+* notice, this list of conditions and the following disclaimer in the
4254+* documentation and/or other materials provided with the distribution.
4255+*
4256+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
4257+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
4258+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
4259+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
4260+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
4261+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
4262+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
4263+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
4264+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
4265+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
4266+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
4267+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
4268+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
4269+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
4270+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
4271+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
4272+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
4273+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
4274+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
4275+* ANY SUCH LICENSES OR RIGHTS.
4276+*******************************************************************************/
4277+package edu.ucsb.eucalyptus.cloud.ws;
4278+
4279+import java.io.ByteArrayOutputStream;
4280+import java.io.IOException;
4281+import java.util.zip.GZIPOutputStream;
4282+
4283+import org.apache.log4j.Logger;
4284+import org.jboss.netty.buffer.ChannelBuffers;
4285+import org.jboss.netty.handler.stream.ChunkedInput;
4286+
4287+import edu.ucsb.eucalyptus.constants.IsData;
4288+
4289+import com.dokukino.rados4j.*;
4290+
4291+public class CompressedChunkedRadosObject implements ChunkedInput, IsData {
4292+ private Logger LOG = Logger.getLogger( CompressedChunkedRadosObject.class );
4293+ private Pool pool;
4294+ private String object;
4295+ private long offset;
4296+ private int CHUNK_SIZE = 8192;
4297+ private long length;
4298+
4299+ public CompressedChunkedRadosObject(Pool pool, String object, long length) {
4300+ this.pool = pool;
4301+ this.object = object;
4302+ this.offset = 0;
4303+ this.length = length;
4304+ }
4305+
4306+ public CompressedChunkedRadosObject(Pool pool, String object, long start, long end, int chunkSize) {
4307+ this.pool = pool;
4308+ this.object = object;
4309+ this.offset = start;
4310+ this.length = end;
4311+ CHUNK_SIZE = chunkSize;
4312+ }
4313+
4314+ @Override
4315+ public void close() throws Exception {
4316+ pool.close();
4317+ }
4318+
4319+ @Override
4320+ public boolean hasNextChunk() throws Exception {
4321+ return offset < length;
4322+ }
4323+
4324+ @Override
4325+ public Object nextChunk() throws Exception {
4326+ long offset = this.offset;
4327+ if (offset >= length) {
4328+ return null;
4329+ }
4330+ int chunkSize = (int) Math.min(CHUNK_SIZE, length - offset);
4331+ byte[] chunk = new byte[chunkSize];
4332+ int ret = pool.readObj(object, offset, chunk, chunkSize);
4333+ if (ret < 0)
4334+ throw new IOException("Unable to read object: " + object);
4335+ this.offset = offset + chunkSize;
4336+ ByteArrayOutputStream out = new ByteArrayOutputStream();
4337+ GZIPOutputStream zip = new GZIPOutputStream(out);
4338+ zip.write(chunk);
4339+ zip.close();
4340+ return ChannelBuffers.wrappedBuffer(out.toByteArray());
4341+ }
4342+}
4343+
4344
4345=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage'
4346=== added file 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java'
4347--- clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java 1970-01-01 00:00:00 +0000
4348+++ clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java 2010-11-15 08:38:14 +0000
4349@@ -0,0 +1,71 @@
4350+/*******************************************************************************
4351+*Copyright (c) 2010 Takuya ASADA
4352+*
4353+* This program is free software: you can redistribute it and/or modify
4354+* it under the terms of the GNU General Public License as published by
4355+* the Free Software Foundation, only version 3 of the License.
4356+*
4357+*
4358+* This file is distributed in the hope that it will be useful, but WITHOUT
4359+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
4360+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
4361+* for more details.
4362+*
4363+* You should have received a copy of the GNU General Public License along
4364+* with this program. If not, see <http://www.gnu.org/licenses/>.
4365+*
4366+* Please contact Eucalyptus Systems, Inc., 130 Castilian
4367+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
4368+* if you need additional information or have any questions.
4369+*
4370+* This file may incorporate work covered under the following copyright and
4371+* permission notice:
4372+*
4373+* Software License Agreement (BSD License)
4374+*
4375+* Copyright (c) 2008, Regents of the University of California
4376+* All rights reserved.
4377+*
4378+* Redistribution and use of this software in source and binary forms, with
4379+* or without modification, are permitted provided that the following
4380+* conditions are met:
4381+*
4382+* Redistributions of source code must retain the above copyright notice,
4383+* this list of conditions and the following disclaimer.
4384+*
4385+* Redistributions in binary form must reproduce the above copyright
4386+* notice, this list of conditions and the following disclaimer in the
4387+* documentation and/or other materials provided with the distribution.
4388+*
4389+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
4390+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
4391+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
4392+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
4393+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
4394+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
4395+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
4396+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
4397+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
4398+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
4399+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
4400+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
4401+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
4402+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
4403+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
4404+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
4405+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
4406+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
4407+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
4408+* ANY SUCH LICENSES OR RIGHTS.
4409+*******************************************************************************/
4410+
4411+package edu.ucsb.eucalyptus.storage;
4412+
4413+import edu.ucsb.eucalyptus.cloud.entities.WalrusInfo;
4414+import edu.ucsb.eucalyptus.storage.rados.RadosStorageManager;
4415+
4416+public class StorageManagerFactory {
4417+ public static StorageManager getStorageManager(WalrusInfo walrusInfo) {
4418+ return new RadosStorageManager(walrusInfo);
4419+ }
4420+}
4421
4422=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados'
4423=== added file 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosReader.java'
4424--- clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosReader.java 1970-01-01 00:00:00 +0000
4425+++ clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosReader.java 2010-11-15 08:38:14 +0000
4426@@ -0,0 +1,118 @@
4427+/*******************************************************************************
4428+*Copyright (c) 2009 Eucalyptus Systems, Inc.
4429+*
4430+* This program is free software: you can redistribute it and/or modify
4431+* it under the terms of the GNU General Public License as published by
4432+* the Free Software Foundation, only version 3 of the License.
4433+*
4434+*
4435+* This file is distributed in the hope that it will be useful, but WITHOUT
4436+* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
4437+* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
4438+* for more details.
4439+*
4440+* You should have received a copy of the GNU General Public License along
4441+* with this program. If not, see <http://www.gnu.org/licenses/>.
4442+*
4443+* Please contact Eucalyptus Systems, Inc., 130 Castilian
4444+* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
4445+* if you need additional information or have any questions.
4446+*
4447+* This file may incorporate work covered under the following copyright and
4448+* permission notice:
4449+*
4450+* Software License Agreement (BSD License)
4451+*
4452+* Copyright (c) 2008, Regents of the University of California
4453+* All rights reserved.
4454+*
4455+* Redistribution and use of this software in source and binary forms, with
4456+* or without modification, are permitted provided that the following
4457+* conditions are met:
4458+*
4459+* Redistributions of source code must retain the above copyright notice,
4460+* this list of conditions and the following disclaimer.
4461+*
4462+* Redistributions in binary form must reproduce the above copyright
4463+* notice, this list of conditions and the following disclaimer in the
4464+* documentation and/or other materials provided with the distribution.
4465+*
4466+* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
4467+* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
4468+* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
4469+* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
4470+* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
4471+* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
4472+* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
4473+* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
4474+* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
4475+* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
4476+* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
4477+* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
4478+* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
4479+* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
4480+* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
4481+* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
4482+* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
4483+* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
4484+* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
4485+* ANY SUCH LICENSES OR RIGHTS.
4486+*******************************************************************************/
4487+
4488+package edu.ucsb.eucalyptus.storage.rados;
4489+
4490+import edu.ucsb.eucalyptus.storage.StorageIO;
4491+
4492+import org.apache.log4j.Logger;
4493+
4494+import java.io.FileNotFoundException;
4495+import java.io.IOException;
4496+import java.nio.ByteBuffer;
4497+
4498+import com.dokukino.rados4j.*;
4499+
4500+public class RadosReader extends StorageIO {
4501+
4502+ private static Logger LOG = Logger.getLogger(RadosReader.class);
4503+ private ByteBuffer buffer;
4504+ private Pool pool;
4505+ private String object;
4506+ private long size;
4507+
4508+ public RadosReader(Pool pool, String object) throws Exception {
4509+ Stat stat;
4510+
4511+ this.pool = pool;
4512+ this.object = object;
4513+ buffer = ByteBuffer.allocate(102400/*TODO: NEIL WalrusQueryDispatcher.DATA_MESSAGE_SIZE */);
4514+ stat = pool.statObj(object);
4515+ if (stat == null) {
4516+ IOException ex = new IOException("Unable to stat object:" + object);
4517+ LOG.error(ex);
4518+ throw ex;
4519+ }
4520+ size = stat.getSize();
4521+ }
4522+
4523+ public int read(long offset) throws IOException {
4524+ byte[] buf;
4525+
4526+ buf = new byte[(int)(size - offset)];
4527+ buffer.clear();
4528+ int bytesRead = pool.readObj(object, offset, buf, buf.length);
4529+ buffer.put(buf, 0, bytesRead);
4530+ buffer.flip();
4531+ return bytesRead;
4532+ }
4533+
4534+ public void write(byte[] bytes) throws IOException {}
4535+
4536+ public ByteBuffer getBuffer() {
4537+ return buffer;
4538+ }
4539+
4540+ public void finish() {
4541+ pool.close();
4542+ }
4543+}
4544+
4545
4546=== added file 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosStorageManager.java'
4547--- clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosStorageManager.java 1970-01-01 00:00:00 +0000
4548+++ clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosStorageManager.java 2010-11-15 08:38:14 +0000
4549@@ -0,0 +1,497 @@
4550+/*******************************************************************************
4551+ *Copyright (c) 2009 Eucalyptus Systems, Inc.
4552+ *
4553+ * This program is free software: you can redistribute it and/or modify
4554+ * it under the terms of the GNU General Public License as published by
4555+ * the Free Software Foundation, only version 3 of the License.
4556+ *
4557+ *
4558+ * This file is distributed in the hope that it will be useful, but WITHOUT
4559+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
4560+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
4561+ * for more details.
4562+ *
4563+ * You should have received a copy of the GNU General Public License along
4564+ * with this program. If not, see <http://www.gnu.org/licenses/>.
4565+ *
4566+ * Please contact Eucalyptus Systems, Inc., 130 Castilian
4567+ * Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
4568+ * if you need additional information or have any questions.
4569+ *
4570+ * This file may incorporate work covered under the following copyright and
4571+ * permission notice:
4572+ *
4573+ * Software License Agreement (BSD License)
4574+ *
4575+ * Copyright (c) 2008, Regents of the University of California
4576+ * All rights reserved.
4577+ *
4578+ * Redistribution and use of this software in source and binary forms, with
4579+ * or without modification, are permitted provided that the following
4580+ * conditions are met:
4581+ *
4582+ * Redistributions of source code must retain the above copyright notice,
4583+ * this list of conditions and the following disclaimer.
4584+ *
4585+ * Redistributions in binary form must reproduce the above copyright
4586+ * notice, this list of conditions and the following disclaimer in the
4587+ * documentation and/or other materials provided with the distribution.
4588+ *
4589+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
4590+ * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
4591+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
4592+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
4593+ * OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
4594+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
4595+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
4596+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
4597+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
4598+ * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
4599+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
4600+ * THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
4601+ * LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
4602+ * SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
4603+ * IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
4604+ * BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
4605+ * THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
4606+ * OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
4607+ * WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
4608+ * ANY SUCH LICENSES OR RIGHTS.
4609+ *******************************************************************************/
4610+
4611+package edu.ucsb.eucalyptus.storage.rados;
4612+
4613+import com.eucalyptus.auth.util.Hashes;
4614+import com.eucalyptus.context.Contexts;
4615+import com.eucalyptus.http.MappingHttpResponse;
4616+import com.eucalyptus.util.EucalyptusCloudException;
4617+import com.eucalyptus.util.ExecutionException;
4618+import com.eucalyptus.util.WalrusProperties;
4619+import com.eucalyptus.ws.util.ChannelUtil;
4620+import com.eucalyptus.ws.util.WalrusBucketLogger;
4621+
4622+import edu.ucsb.eucalyptus.cloud.BucketLogData;
4623+import edu.ucsb.eucalyptus.cloud.ws.CompressedChunkedRadosObject;
4624+import edu.ucsb.eucalyptus.cloud.ws.ChunkedRadosObject;
4625+import edu.ucsb.eucalyptus.msgs.WalrusDataGetRequestType;
4626+import edu.ucsb.eucalyptus.storage.StorageManager;
4627+import edu.ucsb.eucalyptus.storage.StorageIO;
4628+import edu.ucsb.eucalyptus.util.StreamConsumer;
4629+import edu.ucsb.eucalyptus.util.SystemUtil;
4630+
4631+import edu.ucsb.eucalyptus.cloud.entities.WalrusInfo;
4632+
4633+import org.apache.log4j.Logger;
4634+import org.jboss.netty.channel.Channel;
4635+import org.jboss.netty.channel.ChannelFuture;
4636+import org.jboss.netty.channel.ChannelFutureListener;
4637+import org.jboss.netty.handler.codec.http.DefaultHttpResponse;
4638+import org.jboss.netty.handler.codec.http.HttpHeaders;
4639+import org.jboss.netty.handler.stream.ChunkedInput;
4640+
4641+import java.io.*;
4642+import java.util.ArrayList;
4643+import java.util.List;
4644+
4645+import com.dokukino.rados4j.*;
4646+
4647+public class RadosStorageManager implements StorageManager {
4648+
4649+ public static final String lvmRootDirectory = "/dev";
4650+ private static boolean initialized = false;
4651+ private static String eucaHome = "/opt/eucalyptus";
4652+ public static final String EUCA_ROOT_WRAPPER = "/usr/lib/eucalyptus/euca_rootwrap";
4653+ public static final int MAX_LOOP_DEVICES = 256;
4654+ private static Logger LOG = Logger.getLogger(RadosStorageManager.class);
4655+ private Rados rados = new Rados();
4656+
4657+ public RadosStorageManager(WalrusInfo walrusInfo) {
4658+ rados.initialize(new String[0]);
4659+ }
4660+
4661+ public void setRootDirectory(String directory) {
4662+ }
4663+
4664+ public void checkPreconditions() throws EucalyptusCloudException {
4665+ try {
4666+ String eucaHomeDir = System.getProperty("euca.home");
4667+ if(eucaHomeDir == null) {
4668+ throw new EucalyptusCloudException("euca.home not set");
4669+ }
4670+ eucaHome = eucaHomeDir;
4671+ if(!new File(eucaHome + EUCA_ROOT_WRAPPER).exists()) {
4672+ throw new EucalyptusCloudException("root wrapper (euca_rootwrap) does not exist");
4673+ }
4674+ String returnValue = getLvmVersion();
4675+ if(returnValue.length() == 0) {
4676+ throw new EucalyptusCloudException("Is lvm installed?");
4677+ } else {
4678+ LOG.info(returnValue);
4679+ }
4680+ } catch(ExecutionException ex) {
4681+ String error = "Unable to run command: " + ex.getMessage();
4682+ LOG.error(error);
4683+ throw new EucalyptusCloudException(error);
4684+ }
4685+ }
4686+
4687+ public boolean bucketExists(String bucket) {
4688+ return rados.lookupPool(bucket);
4689+ }
4690+
4691+ public boolean objectExists(String bucket, String object) {
4692+ Pool pool = rados.openPool(bucket);
4693+ if (pool == null)
4694+ return false;
4695+ Stat stat = pool.statObj(object);
4696+ pool.close();
4697+ return (stat != null);
4698+ }
4699+
4700+ public void createBucket(String bucket) throws IOException {
4701+ int res = rados.createPool(bucket);
4702+ if (res != 0)
4703+ throw new IOException("Unable to create bucket: " + bucket);
4704+ }
4705+
4706+ public long getSize(String bucket, String object) {
4707+ Pool pool = rados.openPool(bucket);
4708+ if (pool == null)
4709+ return -1;
4710+ Stat stat = pool.statObj(object);
4711+ pool.close();
4712+ if (stat == null)
4713+ return -1;
4714+ return stat.getSize();
4715+ }
4716+
4717+ public void deleteBucket(String bucket) throws IOException {
4718+ Pool pool = rados.openPool(bucket);
4719+ if (pool == null)
4720+ throw new IOException("Unable to delete bucket: " + bucket);
4721+ int ret = pool.delete();
4722+ pool.close();
4723+ if (ret != 0)
4724+ throw new IOException("Unable to delete bucket: " + bucket);
4725+ }
4726+
4727+ public void createObject(String bucket, String object) throws IOException {
4728+ Pool pool = rados.openPool(bucket);
4729+ if (pool == null)
4730+ throw new IOException("Unable to open bucket: " + bucket);
4731+ int ret = pool.createObj(object, true);
4732+ pool.close();
4733+ if (ret != 0)
4734+ throw new IOException("Unable to create object: " + object);
4735+ }
4736+
4737+ public StorageIO prepareForRead(String bucket, String object) throws Exception {
4738+ Pool pool = rados.openPool(bucket);
4739+ if (pool == null)
4740+ throw new IOException("Unable to open bucket: " + bucket);
4741+ return new RadosReader(pool, object);
4742+ }
4743+
4744+ public StorageIO prepareForWrite(String bucket, String object) throws Exception {
4745+ Pool pool = rados.openPool(bucket);
4746+ if (pool == null)
4747+ throw new IOException("Unable to open bucket: " + bucket);
4748+ return new RadosWriter(pool, object);
4749+ }
4750+
4751+ public int readObject(String bucket, String object, byte[] bytes, long offset) throws IOException {
4752+ Pool pool = rados.openPool(bucket);
4753+ if (pool == null)
4754+ throw new IOException("Unable to open bucket: " + bucket);
4755+ int ret = pool.readObj(object, offset, bytes, 0);
4756+ pool.close();
4757+ return ret;
4758+ }
4759+
4760+ public int readObject(String path, byte[] bytes, long offset) throws IOException {
4761+ throw new IOException("Cannot use path on this implementation\n");
4762+ }
4763+
4764+ public void deleteObject(String bucket, String object) throws IOException {
4765+ Pool pool = rados.openPool(bucket);
4766+ if (pool == null)
4767+ throw new IOException("Unable to open bucket: " + bucket);
4768+ int ret = pool.removeObj(object);
4769+ pool.close();
4770+ if (ret < 0)
4771+ throw new IOException("Unable to delete object: " + object);
4772+ }
4773+
4774+ public void deleteAbsoluteObject(String object) throws IOException {
4775+ throw new IOException("Cannot use path on this implementation\n");
4776+ }
4777+
4778+ public void putObject(String bucket, String object, byte[] base64Data, boolean append) throws IOException {
4779+ Pool pool = rados.openPool(bucket);
4780+ if (pool == null)
4781+ throw new IOException("Unable to open bucket: " + bucket);
4782+ int ret;
4783+ if (!append)
4784+ ret = pool.writeObj(object, 0, base64Data, base64Data.length);
4785+ else {
4786+ Stat stat = pool.statObj(object);
4787+ if (stat == null) {
4788+ pool.close();
4789+ throw new IOException("Unable to stat object:" + bucket + "/" + object);
4790+ }
4791+ ret = pool.writeObj(object, stat.getSize(), base64Data, base64Data.length);
4792+ }
4793+ pool.close();
4794+ if (ret < 0)
4795+ throw new IOException("Unable to put object: " + bucket + "/" + object);
4796+ }
4797+
4798+ public void renameObject(String bucket, String oldName, String newName) throws IOException {
4799+ Pool pool;
4800+ int ret;
4801+
4802+ pool = rados.openPool(bucket);
4803+ if (pool == null)
4804+ throw new IOException("Unable to open bucket: " + bucket);
4805+ ret = pool.renameObj(oldName, newName);
4806+ if (ret < 0) {
4807+ pool.close();
4808+ throw new IOException("Unable to rename object: " + oldName);
4809+ }
4810+ pool.close();
4811+ }
4812+
4813+ public void copyObject(String sourceBucket, String sourceObject, String destinationBucket, String destinationObject) throws IOException {
4814+ Pool sourcePool, destinationPool;
4815+ int ret;
4816+
4817+ sourcePool = rados.openPool(sourceBucket);
4818+ if (sourcePool == null)
4819+ throw new IOException("Unable to open bucket: " + sourceBucket);
4820+ destinationPool = rados.openPool(destinationBucket);
4821+ if (destinationPool == null) {
4822+ sourcePool.close();
4823+ throw new IOException("Unable to open bucket: " + destinationBucket);
4824+ }
4825+ ret = sourcePool.copyObj(sourceObject, destinationPool, destinationObject);
4826+ if (ret < 0) {
4827+ sourcePool.close();
4828+ destinationPool.close();
4829+ throw new IOException("Unable to copy object: " + sourceObject);
4830+ }
4831+ sourcePool.close();
4832+ destinationPool.close();
4833+ }
4834+
4835+ public String getObjectPath(String bucket, String object) {
4836+ return null;
4837+ }
4838+
4839+ public long getObjectSize(String bucket, String object) {
4840+ Pool pool;
4841+ Stat stat;
4842+ long size;
4843+
4844+ pool = rados.openPool(bucket);
4845+ if (pool == null)
4846+ return -1;
4847+ stat = pool.statObj(object);
4848+ if (stat == null) {
4849+ pool.close();
4850+ return -1;
4851+ }
4852+ size = stat.getSize();
4853+ return size;
4854+ }
4855+
4856+ public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
4857+ try {
4858+ Channel channel = request.getChannel();
4859+ Pool pool = rados.openPool(bucketName);
4860+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
4861+ if(etag != null)
4862+ httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
4863+ httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
4864+ if(contentDisposition != null)
4865+ httpResponse.addHeader("Content-Disposition", contentDisposition);
4866+ final ChunkedInput file;
4867+ isCompressed = isCompressed == null ? false : isCompressed;
4868+ if(isCompressed) {
4869+ file = new CompressedChunkedRadosObject(pool, objectName, size);
4870+ } else {
4871+ file = new ChunkedRadosObject(pool, objectName, 0, size, 5120000);
4872+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
4873+ }
4874+ if(logData != null) {
4875+ logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
4876+ logData.setBytesSent(size);
4877+ }
4878+ if(versionId != null) {
4879+ httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
4880+ }
4881+ channel.write(httpResponse);
4882+ channel.write(file).addListener(new ChannelFutureListener( ) {
4883+ @Override public void operationComplete( ChannelFuture future ) throws Exception {
4884+ Contexts.clear(request.getCorrelationId());
4885+ file.close();
4886+ if(logData != null) {
4887+ logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
4888+ WalrusBucketLogger.getInstance().addLogEntry(logData);
4889+ }
4890+ }
4891+ });
4892+ } catch(Exception ex) {
4893+ LOG.error(ex, ex);
4894+ }
4895+ }
4896+
4897+ public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long start, long end, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
4898+ try {
4899+ Channel channel = request.getChannel();
4900+ Pool pool = rados.openPool(bucketName);
4901+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
4902+ if(etag != null)
4903+ httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
4904+ httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
4905+ if(contentDisposition != null)
4906+ httpResponse.addHeader("Content-Disposition", contentDisposition);
4907+ final ChunkedInput file;
4908+ isCompressed = isCompressed == null ? false : isCompressed;
4909+ if(isCompressed) {
4910+ file = new CompressedChunkedRadosObject(pool, objectName, start, end, (int)Math.min((end - start), 8192));
4911+ } else {
4912+ file = new ChunkedRadosObject(pool, objectName, start, end, (int)Math.min((end - start), 8192));
4913+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf((end - start)));
4914+ }
4915+ httpResponse.addHeader("Content-Range", start + "-" + end + "/" + size);
4916+ if(logData != null) {
4917+ logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
4918+ logData.setBytesSent(size);
4919+ }
4920+ if(versionId != null) {
4921+ httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
4922+ }
4923+ channel.write(httpResponse);
4924+ channel.write(file).addListener(new ChannelFutureListener( ) {
4925+ @Override public void operationComplete( ChannelFuture future ) throws Exception {
4926+ Contexts.clear(request.getCorrelationId());
4927+ file.close();
4928+ if(logData != null) {
4929+ logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
4930+ WalrusBucketLogger.getInstance().addLogEntry(logData);
4931+ }
4932+ }
4933+ });
4934+ } catch(Exception ex) {
4935+ LOG.error(ex, ex);
4936+ }
4937+ }
4938+
4939+
4940+ public void sendHeaders(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, Long size, String etag,
4941+ String lastModified, String contentType, String contentDisposition, String versionId, final BucketLogData logData) {
4942+ Channel channel = request.getChannel();
4943+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
4944+ httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
4945+ if(etag != null)
4946+ httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
4947+ httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
4948+ if(contentDisposition != null)
4949+ httpResponse.addHeader("Content-Disposition", contentDisposition);
4950+ if(versionId != null) {
4951+ httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
4952+ }
4953+ if(logData != null) {
4954+ logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
4955+ }
4956+ channel.write(httpResponse).addListener(new ChannelFutureListener( ) {
4957+ @Override public void operationComplete( ChannelFuture future ) throws Exception {
4958+ logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
4959+ WalrusBucketLogger.getInstance().addLogEntry(logData);
4960+ }
4961+ });
4962+ }
4963+
4964+ private String removeLoopback(String loDevName) throws ExecutionException {
4965+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-d", loDevName});
4966+ }
4967+
4968+ private int losetup(String absoluteFileName, String loDevName) {
4969+ try
4970+ {
4971+ Runtime rt = Runtime.getRuntime();
4972+ Process proc = rt.exec(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", loDevName, absoluteFileName});
4973+ StreamConsumer error = new StreamConsumer(proc.getErrorStream());
4974+ StreamConsumer output = new StreamConsumer(proc.getInputStream());
4975+ error.start();
4976+ output.start();
4977+ int errorCode = proc.waitFor();
4978+ output.join();
4979+ LOG.info("losetup " + loDevName + " " + absoluteFileName);
4980+ LOG.info(output.getReturnValue());
4981+ LOG.info(error.getReturnValue());
4982+ return errorCode;
4983+ } catch (Throwable t) {
4984+ LOG.error(t);
4985+ }
4986+ return -1;
4987+ }
4988+
4989+ private String findFreeLoopback() throws ExecutionException {
4990+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-f"}).replaceAll("\n", "");
4991+ }
4992+
4993+ private String removeLogicalVolume(String lvName) throws ExecutionException {
4994+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvremove", "-f", lvName});
4995+ }
4996+
4997+ private String reduceVolumeGroup(String vgName, String pvName) throws ExecutionException {
4998+ return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "vgreduce", vgName, pvName});
4999+ }
5000+
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches

to status/vote changes: