Merge lp:~mcgrue/nova/volume-cleanup into lp:~hudson-openstack/nova/trunk
- volume-cleanup
- Merge into trunk
Status: | Work in progress |
---|---|
Proposed branch: | lp:~mcgrue/nova/volume-cleanup |
Merge into: | lp:~hudson-openstack/nova/trunk |
Diff against target: |
785 lines (+98/-364) 15 files modified
Authors (+1/-0) bin/nova-manage (+2/-3) doc/source/runnova/getting.started.rst (+0/-1) nova/db/api.py (+1/-35) nova/db/sqlalchemy/api.py (+0/-55) nova/db/sqlalchemy/migrate_repo/versions/042_kill_export_devices.py (+51/-0) nova/db/sqlalchemy/migration.py (+1/-1) nova/db/sqlalchemy/models.py (+0/-16) nova/exception.py (+0/-4) nova/tests/fake_flags.py (+0/-4) nova/tests/integrated/test_volumes.py (+5/-5) nova/tests/test_volume.py (+2/-80) nova/volume/driver.py (+28/-146) nova/volume/manager.py (+7/-11) nova/volume/san.py (+0/-3) |
To merge this branch: | bzr merge lp:~mcgrue/nova/volume-cleanup |
Related bugs: | |
Related blueprints: |
Volume Code cleanup
(Medium)
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Thierry Carrez (community) | ffe | Disapprove | |
Soren Hansen (community) | Needs Fixing | ||
Jay Pipes (community) | Needs Information | ||
Rick Harris (community) | Needs Fixing | ||
Review via email:
|
Commit message
Description of the change
Stage 1 of the AES removal / volume cleanup work.
- 1455. By Dan Prince
-
Cleanup the '_base' directory in libvirt tests.
- 1456. By Tushar Patil
-
Accept binary user_data in radix-64 format when you launch a new server using OSAPI. This user_data would be stored along with the other server properties in the database. Once the VM instance boots you can query for the user-data to do any custom installation of applications/
servers or do some specific job like setting up networking route table. You can query for user-data using curl with the URL:-
curl http://169.254. 169.254/ latest/ user-data - 1457. By John Tran
-
Fixed bug in which DescribeInstances was returning deleted instances. Added tests for pertinent api methods.
- 1458. By Matt Dietz
-
Fixes lp828207
The Host API reboot action was broken due to addition of a parameter to a utility method in compute/api.py
- 1459. By Rick Harris
-
Passes empty string instead of None to MySQLdb driver if the DB password isn't set.
- 1460. By Ed Leafe
-
Updated docs for the recent scheduler class changes.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Ben McGraw (mcgrue) wrote : | # |
Sorry about that, Rick. Fixed as of revision 1361.
- 1461. By Alex Meade
-
Fixes issue where ServersXMLSeria
lizer was missing a method for update actions - 1462. By Vish Ishaya
-
Makes all of the binary services launch using the same strategy.
* Removes helper methods from utils for loading flags and logging
* Changes service.serve to use Launcher
* Changes service.wait to actually wait for all the services to exit
* Changes nova-api to explicitly load flags and logging and use service.serve
* Fixes the annoying IOError when /etc/nova/nova.conf doesn't exist - 1463. By Josh Kearney
-
Currently, rescue/unrescue is only available over the admin API. Non-admin tenants also need to be able to access this functionality. This patch adds rescue functionality over an API extension.
- 1464. By dan wendlandt
-
Bugfix for lp 828429. Its still not clear to me exactly how this code path is actually invoked when nova is used, so I'm looking for input on whether we should be adding a test case for this, removing the code as unused, etc. Thanks.
- 1465. By Ryu Ishimoto
-
Added uuid column in virtual_interfaces table, and an OpenStack extension API for virtual interfaces to expose these IDs. Also set this UUID as one of the external IDs in the OVS vif driver.
- 1466. By Ed Leafe
-
Removes the incorrect hard-coded filter path.
- 1467. By Vish Ishaya
-
Next round of prep for keystone integration.
* adds middleware for authenticating ec2 signature with keystone
* adds middleware for converting keystone response into request context
* gives examples of alternative pipelines for keystone integrationNext steps:
* provide default config with no keystone integration (perhaps setting every context to admin?)
* write authmanager to keystone conversion code
* add api extension to create and destroy access/secret keys
* deprecate authmanager
* rename project to tenant - 1468. By Anthony Young
-
Assorted fixes to os-floating-ips to make it play nicely with an in-progress novaclient implementation, as well as some changes to make it more consistent with other os rest apis. Changes include:
* switch associate/
disassociate to PUTs. Previously, it was doing create calls with one-off parameter resources.
* allow graceful handling when there are no floating ips for a tenant
* allow graceful handling when disassociating an already disassociated address - 1469. By Tushar Patil
-
Added OS APIs to associate/
disassociate security groups to/from instances. I will add views to return list of security groups associated with the servers later after this branch is merged into trunk. The reason I will do this later is because my previous merge proposal (https:/
/code.launchpad .net/~tpatil/ nova/add- options- network- create- os-apis/ +merge/ 68292) is dependent on this work. In this merge proposal I have added a new extension which still uses default OS v1.1 controllers and views, but I am going to override views in this extension to send extra information like security groups.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Jay Pipes (jaypipes) wrote : | # |
Hi!
There is a merge conflict here... look for <<<<<<< TREE in the diff.
Also, is there a reason why we are removing functionality (as opposed to deprecating it)?
-jay
- 1470. By Sandy Walsh
-
Fixes utils.to_primitive (again) to handle modules, builtins and whatever other crap might be hiding in an object.
- 1471. By Alex Meade
-
Adds accessIPv4 and accessIPv6 to servers requests and responses as per the current spec.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Vish Ishaya (vishvananda) wrote : | # |
The reason for removal, is I'm rewriting the communication between compute and volume here:
https:/
maintaining AoE through this refactor seems silly since no one is using it.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Jay Pipes (jaypipes) wrote : | # |
Are we positive absolutely *nobody* is running AoE? If not, deprecation seems the way to go...
- 1472. By William Wolf
-
implemented tenant ids to be included in request uris.
- 1473. By Alex Meade
-
The FixedIpCommands
TestCase in test_nova_manage previously accessed the database. This branch stubs out the database for these tests, lowering their run time from 104 secs -> .02 secs total. I have verified the tested functionality is still being tested.
- 1474. By Soren Hansen
-
Move documentation from nova.virt.fake into nova.virt.driver.
- 1475. By Alex Meade
-
Fixes bug 831627 where nova-manage does not exit when given a non-existent network address
- 1476. By Tushar Patil
-
Our goal is to add optional parameter to the Create server OS 1.0 and 1.1 API to achieve following objectives:-
1) Specify number and order of networks to the create server API.
In the current implementation every instance you launch for a project having 3 networks assigned to it will always have 3 vnics. In this case it is not possible to have one instance with 2 vnics ,another with 1 vnic and so on. This is not flexible enough and the network resources are not used effectively.
2) Specify fixed IP address to the vnic at the boot time. When you launch a server, you can specify the fixed IP address you want to be assigned to the vnic from a particular network. If this fixed IP address is already in use, it will give exception.
Example of Server Create API request XML:
<?xml version="1.0" encoding="UTF-8"?><server xmlns="http://
docs.nttpflab. com/servers/ api/v1. 0"
name=" new-server- test" imageId="1" flavorId="1">
<metadata>
<meta key="My Server Name">Apache1</meta>
</metadata>
<personality>
<file path="/etc/banner. txt">
ICAgICAgDQoiQS BjbG91ZCBkb2VzI G5vdCBrbm93IHdo eSBp
</file>
</personality>
<networks>
<network uuid="6622436e-5289-460f- 8479-e4dcc63f16 c5" fixed_ip= "10.0.0. 3">
<network uuid="d97efefc-e071-4174- b6dd-b33af0a377 06" fixed_ip= "10.0.1. 3">
</networks>
</server>3) Networks is an optional parameter, so if you don't provide any networks to the server Create API, it will behave exactly the same as of today.
This feature is supported to all of the network models.
- 1477. By Christopher MacGown <email address hidden>
-
Implements first-pass of config-drive that adds a vfat format drive to a vm when config_drive is True (or an image id).
- 1478. By Vish Ishaya
-
This branch does the final tear out of AuthManager from the main code. The NoAuth middlewares (active by default) allow a user to specify any user and project id through headers (os_api) or access key (ec2_api).
The plan is to leave the auth manager code in but mention that it is deprecated. There is a sample paste config in ini to still allow old auth. Immediately after the diablo release we can tear out all of the Auth related code and not support the deprecated auth anymore.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Soren Hansen (soren) wrote : | # |
The conflict remains. You need to merge the current trunk into your branch and fix it.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Soren Hansen (soren) wrote : | # |
As much as I'd love AoE to *die* *die* *die*, I do see Jay's point. I'm not sure what tools we have to actually deprecate stuff, though. Logging? Release notes? What else? The second iscsi support landed, AoE was deprecated, but I guess this might not have been communicated very well to users.
- 1479. By Thierry Carrez
-
Refresh translations
- 1480. By Dan Prince
-
Move use_ipv6 into flags. Its used in multiple places (network manager and the OSAPI) and should be defined at the top level.
- 1481. By Brian Waldon
-
Fixes bug that causes 400 status code when an instance wasn't attached to a network.
- 1482. By Nachi Ueno
-
I added notifications decorator for each API call using monkey_patching.
By this merge, users can get API call notification from any modules. - 1483. By Vish Ishaya
-
Adds a use_deprecated_auth flag to make sure creds generated using nova-manage commands will work with noauth.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Ben McGraw (mcgrue) wrote : | # |
(How did bzr let me commit something with conflicts?)
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Jay Pipes (jaypipes) wrote : | # |
> (How did bzr let me commit something with conflicts?)
The same way git lets you commit something with conflicts.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Vish Ishaya (vishvananda) wrote : | # |
The diff shows what happens when the branch is merged into trunk, so if recent additions have been added to trunk it will show conflicts. Speaking of which, looks like there is still another one. Also need to bump the version number of the migrate script. I think we're on 39?
- 1484. By Ben McGraw
-
resolving conflicts.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Thierry Carrez (ttx) wrote : | # |
This should wait for Essex, based on the meeting we had on 2011-08-23 ?
Unmerged revisions
- 1484. By Ben McGraw
-
resolving conflicts.
Preview Diff
1 | === modified file 'Authors' |
2 | --- Authors 2011-08-23 04:17:57 +0000 |
3 | +++ Authors 2011-08-23 22:45:41 +0000 |
4 | @@ -10,6 +10,7 @@ |
5 | Antony Messerli <ant@openstack.org> |
6 | Armando Migliaccio <Armando.Migliaccio@eu.citrix.com> |
7 | Arvind Somya <asomya@cisco.com> |
8 | +Ben McGraw <ben@pistoncloud.com> |
9 | Bilal Akhtar <bilalakhtar@ubuntu.com> |
10 | Brian Lamar <brian.lamar@rackspace.com> |
11 | Brian Schott <bschott@isi.edu> |
12 | |
13 | === modified file 'bin/nova-manage' |
14 | --- bin/nova-manage 2011-08-23 07:21:13 +0000 |
15 | +++ bin/nova-manage 2011-08-23 22:45:41 +0000 |
16 | @@ -860,9 +860,8 @@ |
17 | msg = _('Only KVM and QEmu are supported for now. Sorry!') |
18 | raise exception.Error(msg) |
19 | |
20 | - if (FLAGS.volume_driver != 'nova.volume.driver.AOEDriver' and \ |
21 | - FLAGS.volume_driver != 'nova.volume.driver.ISCSIDriver'): |
22 | - msg = _("Support only AOEDriver and ISCSIDriver. Sorry!") |
23 | + if FLAGS.volume_driver != 'nova.volume.driver.ISCSIDriver': |
24 | + msg = _("Support only ISCSIDriver. Sorry!") |
25 | raise exception.Error(msg) |
26 | |
27 | rpc.call(ctxt, |
28 | |
29 | === modified file 'doc/source/runnova/getting.started.rst' |
30 | --- doc/source/runnova/getting.started.rst 2011-02-21 20:30:20 +0000 |
31 | +++ doc/source/runnova/getting.started.rst 2011-08-23 22:45:41 +0000 |
32 | @@ -73,7 +73,6 @@ |
33 | * dnsmasq |
34 | * vlan |
35 | * open-iscsi and iscsitarget (if you use iscsi volumes) |
36 | -* aoetools and vblade-persist (if you use aoe-volumes) |
37 | |
38 | Nova uses cutting-edge versions of many packages. There are ubuntu packages in |
39 | the nova-core trunk ppa. You can use add this ppa to your sources list on an |
40 | |
41 | === modified file 'nova/db/api.py' |
42 | --- nova/db/api.py 2011-08-22 23:35:09 +0000 |
43 | +++ nova/db/api.py 2011-08-23 22:45:41 +0000 |
44 | @@ -55,18 +55,13 @@ |
45 | sqlalchemy='nova.db.sqlalchemy.api') |
46 | |
47 | |
48 | -class NoMoreBlades(exception.Error): |
49 | - """No more available blades.""" |
50 | - pass |
51 | - |
52 | - |
53 | class NoMoreNetworks(exception.Error): |
54 | """No more available networks.""" |
55 | pass |
56 | |
57 | |
58 | class NoMoreTargets(exception.Error): |
59 | - """No more available blades""" |
60 | + """No more available targets""" |
61 | pass |
62 | |
63 | |
64 | @@ -763,25 +758,6 @@ |
65 | ################### |
66 | |
67 | |
68 | -def export_device_count(context): |
69 | - """Return count of export devices.""" |
70 | - return IMPL.export_device_count(context) |
71 | - |
72 | - |
73 | -def export_device_create_safe(context, values): |
74 | - """Create an export_device from the values dictionary. |
75 | - |
76 | - The device is not returned. If the create violates the unique |
77 | - constraints because the shelf_id and blade_id already exist, |
78 | - no exception is raised. |
79 | - |
80 | - """ |
81 | - return IMPL.export_device_create_safe(context, values) |
82 | - |
83 | - |
84 | -################### |
85 | - |
86 | - |
87 | def iscsi_target_count_by_host(context, host): |
88 | """Return count of export devices.""" |
89 | return IMPL.iscsi_target_count_by_host(context, host) |
90 | @@ -857,11 +833,6 @@ |
91 | ################### |
92 | |
93 | |
94 | -def volume_allocate_shelf_and_blade(context, volume_id): |
95 | - """Atomically allocate a free shelf and blade from the pool.""" |
96 | - return IMPL.volume_allocate_shelf_and_blade(context, volume_id) |
97 | - |
98 | - |
99 | def volume_allocate_iscsi_target(context, volume_id, host): |
100 | """Atomically allocate a free iscsi_target from the pool.""" |
101 | return IMPL.volume_allocate_iscsi_target(context, volume_id, host) |
102 | @@ -927,11 +898,6 @@ |
103 | return IMPL.volume_get_instance(context, volume_id) |
104 | |
105 | |
106 | -def volume_get_shelf_and_blade(context, volume_id): |
107 | - """Get the shelf and blade allocated to the volume.""" |
108 | - return IMPL.volume_get_shelf_and_blade(context, volume_id) |
109 | - |
110 | - |
111 | def volume_get_iscsi_target_num(context, volume_id): |
112 | """Get the target num (tid) allocated to the volume.""" |
113 | return IMPL.volume_get_iscsi_target_num(context, volume_id) |
114 | |
115 | === modified file 'nova/db/sqlalchemy/api.py' |
116 | --- nova/db/sqlalchemy/api.py 2011-08-22 23:35:09 +0000 |
117 | +++ nova/db/sqlalchemy/api.py 2011-08-23 22:45:41 +0000 |
118 | @@ -1936,28 +1936,6 @@ |
119 | |
120 | |
121 | @require_admin_context |
122 | -def export_device_count(context): |
123 | - session = get_session() |
124 | - return session.query(models.ExportDevice).\ |
125 | - filter_by(deleted=can_read_deleted(context)).\ |
126 | - count() |
127 | - |
128 | - |
129 | -@require_admin_context |
130 | -def export_device_create_safe(context, values): |
131 | - export_device_ref = models.ExportDevice() |
132 | - export_device_ref.update(values) |
133 | - try: |
134 | - export_device_ref.save() |
135 | - return export_device_ref |
136 | - except IntegrityError: |
137 | - return None |
138 | - |
139 | - |
140 | -################### |
141 | - |
142 | - |
143 | -@require_admin_context |
144 | def iscsi_target_count_by_host(context, host): |
145 | session = get_session() |
146 | return session.query(models.IscsiTarget).\ |
147 | @@ -2093,24 +2071,6 @@ |
148 | |
149 | |
150 | @require_admin_context |
151 | -def volume_allocate_shelf_and_blade(context, volume_id): |
152 | - session = get_session() |
153 | - with session.begin(): |
154 | - export_device = session.query(models.ExportDevice).\ |
155 | - filter_by(volume=None).\ |
156 | - filter_by(deleted=False).\ |
157 | - with_lockmode('update').\ |
158 | - first() |
159 | - # NOTE(vish): if with_lockmode isn't supported, as in sqlite, |
160 | - # then this has concurrency issues |
161 | - if not export_device: |
162 | - raise db.NoMoreBlades() |
163 | - export_device.volume_id = volume_id |
164 | - session.add(export_device) |
165 | - return (export_device.shelf_id, export_device.blade_id) |
166 | - |
167 | - |
168 | -@require_admin_context |
169 | def volume_allocate_iscsi_target(context, volume_id, host): |
170 | session = get_session() |
171 | with session.begin(): |
172 | @@ -2174,9 +2134,6 @@ |
173 | update({'deleted': True, |
174 | 'deleted_at': utils.utcnow(), |
175 | 'updated_at': literal_column('updated_at')}) |
176 | - session.query(models.ExportDevice).\ |
177 | - filter_by(volume_id=volume_id).\ |
178 | - update({'volume_id': None}) |
179 | session.query(models.IscsiTarget).\ |
180 | filter_by(volume_id=volume_id).\ |
181 | update({'volume_id': None}) |
182 | @@ -2277,18 +2234,6 @@ |
183 | |
184 | |
185 | @require_admin_context |
186 | -def volume_get_shelf_and_blade(context, volume_id): |
187 | - session = get_session() |
188 | - result = session.query(models.ExportDevice).\ |
189 | - filter_by(volume_id=volume_id).\ |
190 | - first() |
191 | - if not result: |
192 | - raise exception.ExportDeviceNotFoundForVolume(volume_id=volume_id) |
193 | - |
194 | - return (result.shelf_id, result.blade_id) |
195 | - |
196 | - |
197 | -@require_admin_context |
198 | def volume_get_iscsi_target_num(context, volume_id): |
199 | session = get_session() |
200 | result = session.query(models.IscsiTarget).\ |
201 | |
202 | === added file 'nova/db/sqlalchemy/migrate_repo/versions/042_kill_export_devices.py' |
203 | --- nova/db/sqlalchemy/migrate_repo/versions/042_kill_export_devices.py 1970-01-01 00:00:00 +0000 |
204 | +++ nova/db/sqlalchemy/migrate_repo/versions/042_kill_export_devices.py 2011-08-23 22:45:41 +0000 |
205 | @@ -0,0 +1,51 @@ |
206 | +# vim: tabstop=4 shiftwidth=4 softtabstop=4 |
207 | + |
208 | +# Copyright 2011 University of Southern California |
209 | +# |
210 | +# Licensed under the Apache License, Version 2.0 (the "License"); you may |
211 | +# not use this file except in compliance with the License. You may obtain |
212 | +# a copy of the License at |
213 | +# |
214 | +# http://www.apache.org/licenses/LICENSE-2.0 |
215 | +# |
216 | +# Unless required by applicable law or agreed to in writing, software |
217 | +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT |
218 | +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
219 | +# License for the specific language governing permissions and limitations |
220 | +# under the License. |
221 | + |
222 | +from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer |
223 | +from sqlalchemy import MetaData, String, Table |
224 | +from nova import log as logging |
225 | + |
226 | +meta = MetaData() |
227 | + |
228 | +# Table definition |
229 | +export_devices = Table('export_devices', meta, |
230 | + Column('created_at', DateTime(timezone=False)), |
231 | + Column('updated_at', DateTime(timezone=False)), |
232 | + Column('deleted_at', DateTime(timezone=False)), |
233 | + Column('deleted', Boolean(create_constraint=True, name=None)), |
234 | + Column('id', Integer(), primary_key=True, nullable=False), |
235 | + Column('shelf_id', Integer()), |
236 | + Column('blade_id', Integer()), |
237 | + Column('volume_id', |
238 | + Integer(), |
239 | + ForeignKey('volumes.id'), |
240 | + nullable=True), |
241 | + ) |
242 | + |
243 | + |
244 | +def downgrade(migrate_engine): |
245 | + meta.bind = migrate_engine |
246 | + try: |
247 | + export_devices.create() |
248 | + except Exception: |
249 | + logging.info(repr(export_devices)) |
250 | + logging.exception('Exception while creating table') |
251 | + raise |
252 | + |
253 | + |
254 | +def upgrade(migrate_engine): |
255 | + meta.bind = migrate_engine |
256 | + export_devices.drop() |
257 | |
258 | === modified file 'nova/db/sqlalchemy/migration.py' |
259 | --- nova/db/sqlalchemy/migration.py 2011-06-24 12:01:51 +0000 |
260 | +++ nova/db/sqlalchemy/migration.py 2011-08-23 22:45:41 +0000 |
261 | @@ -55,7 +55,7 @@ |
262 | engine = sqlalchemy.create_engine(FLAGS.sql_connection, echo=False) |
263 | meta.reflect(bind=engine) |
264 | try: |
265 | - for table in ('auth_tokens', 'zones', 'export_devices', |
266 | + for table in ('auth_tokens', 'zones', |
267 | 'fixed_ips', 'floating_ips', 'instances', |
268 | 'key_pairs', 'networks', 'projects', 'quotas', |
269 | 'security_group_instance_association', |
270 | |
271 | === modified file 'nova/db/sqlalchemy/models.py' |
272 | --- nova/db/sqlalchemy/models.py 2011-08-23 04:17:57 +0000 |
273 | +++ nova/db/sqlalchemy/models.py 2011-08-23 22:45:41 +0000 |
274 | @@ -400,22 +400,6 @@ |
275 | no_device = Column(Boolean, nullable=True) |
276 | |
277 | |
278 | -class ExportDevice(BASE, NovaBase): |
279 | - """Represates a shelf and blade that a volume can be exported on.""" |
280 | - __tablename__ = 'export_devices' |
281 | - __table_args__ = (schema.UniqueConstraint("shelf_id", "blade_id"), |
282 | - {'mysql_engine': 'InnoDB'}) |
283 | - id = Column(Integer, primary_key=True) |
284 | - shelf_id = Column(Integer) |
285 | - blade_id = Column(Integer) |
286 | - volume_id = Column(Integer, ForeignKey('volumes.id'), nullable=True) |
287 | - volume = relationship(Volume, |
288 | - backref=backref('export_device', uselist=False), |
289 | - foreign_keys=volume_id, |
290 | - primaryjoin='and_(ExportDevice.volume_id==Volume.id,' |
291 | - 'ExportDevice.deleted==False)') |
292 | - |
293 | - |
294 | class IscsiTarget(BASE, NovaBase): |
295 | """Represates an iscsi target for a given host""" |
296 | __tablename__ = 'iscsi_targets' |
297 | |
298 | === modified file 'nova/exception.py' |
299 | --- nova/exception.py 2011-08-22 23:35:09 +0000 |
300 | +++ nova/exception.py 2011-08-23 22:45:41 +0000 |
301 | @@ -346,10 +346,6 @@ |
302 | message = _("deleting volume %(volume_name)s that has snapshot") |
303 | |
304 | |
305 | -class ExportDeviceNotFoundForVolume(NotFound): |
306 | - message = _("No export device found for volume %(volume_id)s.") |
307 | - |
308 | - |
309 | class ISCSITargetNotFoundForVolume(NotFound): |
310 | message = _("No target id found for volume %(volume_id)s.") |
311 | |
312 | |
313 | === modified file 'nova/tests/fake_flags.py' |
314 | --- nova/tests/fake_flags.py 2011-07-27 16:44:14 +0000 |
315 | +++ nova/tests/fake_flags.py 2011-08-23 22:45:41 +0000 |
316 | @@ -33,11 +33,7 @@ |
317 | FLAGS['num_networks'].SetDefault(2) |
318 | FLAGS['fake_network'].SetDefault(True) |
319 | FLAGS['image_service'].SetDefault('nova.image.fake.FakeImageService') |
320 | -flags.DECLARE('num_shelves', 'nova.volume.driver') |
321 | -flags.DECLARE('blades_per_shelf', 'nova.volume.driver') |
322 | flags.DECLARE('iscsi_num_targets', 'nova.volume.driver') |
323 | -FLAGS['num_shelves'].SetDefault(2) |
324 | -FLAGS['blades_per_shelf'].SetDefault(4) |
325 | FLAGS['iscsi_num_targets'].SetDefault(8) |
326 | FLAGS['verbose'].SetDefault(True) |
327 | FLAGS['sqlite_db'].SetDefault("tests.sqlite") |
328 | |
329 | === modified file 'nova/tests/integrated/test_volumes.py' |
330 | --- nova/tests/integrated/test_volumes.py 2011-08-03 21:06:56 +0000 |
331 | +++ nova/tests/integrated/test_volumes.py 2011-08-23 22:45:41 +0000 |
332 | @@ -262,22 +262,22 @@ |
333 | |
334 | LOG.debug("Logs: %s" % driver.LoggingVolumeDriver.all_logs()) |
335 | |
336 | - # Discover_volume and undiscover_volume are called from compute |
337 | + # prepare_attach and prepare_detach are called from compute |
338 | # on attach/detach |
339 | |
340 | disco_moves = driver.LoggingVolumeDriver.logs_like( |
341 | - 'discover_volume', |
342 | + 'prepare_attach', |
343 | id=volume_id) |
344 | - LOG.debug("discover_volume actions: %s" % disco_moves) |
345 | + LOG.debug("prepare_attach actions: %s" % disco_moves) |
346 | |
347 | self.assertEquals(1, len(disco_moves)) |
348 | disco_move = disco_moves[0] |
349 | self.assertEquals(disco_move['id'], volume_id) |
350 | |
351 | last_days_of_disco_moves = driver.LoggingVolumeDriver.logs_like( |
352 | - 'undiscover_volume', |
353 | + 'prepare_detach', |
354 | id=volume_id) |
355 | - LOG.debug("undiscover_volume actions: %s" % last_days_of_disco_moves) |
356 | + LOG.debug("prepare_detach actions: %s" % last_days_of_disco_moves) |
357 | |
358 | self.assertEquals(1, len(last_days_of_disco_moves)) |
359 | undisco_move = last_days_of_disco_moves[0] |
360 | |
361 | === modified file 'nova/tests/test_volume.py' |
362 | --- nova/tests/test_volume.py 2011-08-05 14:23:48 +0000 |
363 | +++ nova/tests/test_volume.py 2011-08-23 22:45:41 +0000 |
364 | @@ -257,7 +257,7 @@ |
365 | |
366 | class DriverTestCase(test.TestCase): |
367 | """Base Test class for Drivers.""" |
368 | - driver_name = "nova.volume.driver.FakeAOEDriver" |
369 | + driver_name = "nova.volume.driver.FakeBaseDriver" |
370 | |
371 | def setUp(self): |
372 | super(DriverTestCase, self).setUp() |
373 | @@ -295,83 +295,6 @@ |
374 | self.volume.delete_volume(self.context, volume_id) |
375 | |
376 | |
377 | -class AOETestCase(DriverTestCase): |
378 | - """Test Case for AOEDriver""" |
379 | - driver_name = "nova.volume.driver.AOEDriver" |
380 | - |
381 | - def setUp(self): |
382 | - super(AOETestCase, self).setUp() |
383 | - |
384 | - def tearDown(self): |
385 | - super(AOETestCase, self).tearDown() |
386 | - |
387 | - def _attach_volume(self): |
388 | - """Attach volumes to an instance. This function also sets |
389 | - a fake log message.""" |
390 | - volume_id_list = [] |
391 | - for index in xrange(3): |
392 | - vol = {} |
393 | - vol['size'] = 0 |
394 | - volume_id = db.volume_create(self.context, |
395 | - vol)['id'] |
396 | - self.volume.create_volume(self.context, volume_id) |
397 | - |
398 | - # each volume has a different mountpoint |
399 | - mountpoint = "/dev/sd" + chr((ord('b') + index)) |
400 | - db.volume_attached(self.context, volume_id, self.instance_id, |
401 | - mountpoint) |
402 | - |
403 | - (shelf_id, blade_id) = db.volume_get_shelf_and_blade(self.context, |
404 | - volume_id) |
405 | - self.output += "%s %s eth0 /dev/nova-volumes/vol-foo auto run\n" \ |
406 | - % (shelf_id, blade_id) |
407 | - |
408 | - volume_id_list.append(volume_id) |
409 | - |
410 | - return volume_id_list |
411 | - |
412 | - def test_check_for_export_with_no_volume(self): |
413 | - """No log message when no volume is attached to an instance.""" |
414 | - self.stream.truncate(0) |
415 | - self.volume.check_for_export(self.context, self.instance_id) |
416 | - self.assertEqual(self.stream.getvalue(), '') |
417 | - |
418 | - def test_check_for_export_with_all_vblade_processes(self): |
419 | - """No log message when all the vblade processes are running.""" |
420 | - volume_id_list = self._attach_volume() |
421 | - |
422 | - self.stream.truncate(0) |
423 | - self.volume.check_for_export(self.context, self.instance_id) |
424 | - self.assertEqual(self.stream.getvalue(), '') |
425 | - |
426 | - self._detach_volume(volume_id_list) |
427 | - |
428 | - def test_check_for_export_with_vblade_process_missing(self): |
429 | - """Output a warning message when some vblade processes aren't |
430 | - running.""" |
431 | - volume_id_list = self._attach_volume() |
432 | - |
433 | - # the first vblade process isn't running |
434 | - self.output = self.output.replace("run", "down", 1) |
435 | - (shelf_id, blade_id) = db.volume_get_shelf_and_blade(self.context, |
436 | - volume_id_list[0]) |
437 | - |
438 | - msg_is_match = False |
439 | - self.stream.truncate(0) |
440 | - try: |
441 | - self.volume.check_for_export(self.context, self.instance_id) |
442 | - except exception.ProcessExecutionError, e: |
443 | - volume_id = volume_id_list[0] |
444 | - msg = _("Cannot confirm exported volume id:%(volume_id)s. " |
445 | - "vblade process for e%(shelf_id)s.%(blade_id)s " |
446 | - "isn't running.") % locals() |
447 | - |
448 | - msg_is_match = (0 <= e.message.find(msg)) |
449 | - |
450 | - self.assertTrue(msg_is_match) |
451 | - self._detach_volume(volume_id_list) |
452 | - |
453 | - |
454 | class ISCSITestCase(DriverTestCase): |
455 | """Test Case for ISCSIDriver""" |
456 | driver_name = "nova.volume.driver.ISCSIDriver" |
457 | @@ -408,7 +331,7 @@ |
458 | self.assertEqual(self.stream.getvalue(), '') |
459 | |
460 | def test_check_for_export_with_all_volume_exported(self): |
461 | - """No log message when all the vblade processes are running.""" |
462 | + """No log message when all the processes are running.""" |
463 | volume_id_list = self._attach_volume() |
464 | |
465 | self.mox.StubOutWithMock(self.volume.driver, '_execute') |
466 | @@ -431,7 +354,6 @@ |
467 | by ietd.""" |
468 | volume_id_list = self._attach_volume() |
469 | |
470 | - # the first vblade process isn't running |
471 | tid = db.volume_get_iscsi_target_num(self.context, volume_id_list[0]) |
472 | self.mox.StubOutWithMock(self.volume.driver, '_execute') |
473 | self.volume.driver._execute("ietadm", "--op", "show", |
474 | |
475 | === modified file 'nova/volume/driver.py' |
476 | --- nova/volume/driver.py 2011-08-05 14:23:48 +0000 |
477 | +++ nova/volume/driver.py 2011-08-23 22:45:41 +0000 |
478 | @@ -33,18 +33,10 @@ |
479 | FLAGS = flags.FLAGS |
480 | flags.DEFINE_string('volume_group', 'nova-volumes', |
481 | 'Name for the VG that will contain exported volumes') |
482 | -flags.DEFINE_string('aoe_eth_dev', 'eth0', |
483 | - 'Which device to export the volumes on') |
484 | flags.DEFINE_string('num_shell_tries', 3, |
485 | 'number of times to attempt to run flakey shell commands') |
486 | flags.DEFINE_string('num_iscsi_scan_tries', 3, |
487 | 'number of times to rescan iSCSI target to find volume') |
488 | -flags.DEFINE_integer('num_shelves', |
489 | - 100, |
490 | - 'Number of vblade shelves') |
491 | -flags.DEFINE_integer('blades_per_shelf', |
492 | - 16, |
493 | - 'Number of vblade blades per shelf') |
494 | flags.DEFINE_integer('iscsi_num_targets', |
495 | 100, |
496 | 'Number of iscsi target ids per host') |
497 | @@ -200,132 +192,17 @@ |
498 | """Removes an export for a logical volume.""" |
499 | raise NotImplementedError() |
500 | |
501 | - def discover_volume(self, context, volume): |
502 | - """Discover volume on a remote host.""" |
503 | - raise NotImplementedError() |
504 | - |
505 | - def undiscover_volume(self, volume): |
506 | - """Undiscover volume on a remote host.""" |
507 | - raise NotImplementedError() |
508 | - |
509 | - def check_for_export(self, context, volume_id): |
510 | - """Make sure volume is exported.""" |
511 | - raise NotImplementedError() |
512 | - |
513 | - |
514 | -class AOEDriver(VolumeDriver): |
515 | - """Implements AOE specific volume commands.""" |
516 | - |
517 | - def ensure_export(self, context, volume): |
518 | - # NOTE(vish): we depend on vblade-persist for recreating exports |
519 | - pass |
520 | - |
521 | - def _ensure_blades(self, context): |
522 | - """Ensure that blades have been created in datastore.""" |
523 | - total_blades = FLAGS.num_shelves * FLAGS.blades_per_shelf |
524 | - if self.db.export_device_count(context) >= total_blades: |
525 | - return |
526 | - for shelf_id in xrange(FLAGS.num_shelves): |
527 | - for blade_id in xrange(FLAGS.blades_per_shelf): |
528 | - dev = {'shelf_id': shelf_id, 'blade_id': blade_id} |
529 | - self.db.export_device_create_safe(context, dev) |
530 | - |
531 | - def create_export(self, context, volume): |
532 | - """Creates an export for a logical volume.""" |
533 | - self._ensure_blades(context) |
534 | - (shelf_id, |
535 | - blade_id) = self.db.volume_allocate_shelf_and_blade(context, |
536 | - volume['id']) |
537 | - self._try_execute( |
538 | - 'vblade-persist', 'setup', |
539 | - shelf_id, |
540 | - blade_id, |
541 | - FLAGS.aoe_eth_dev, |
542 | - "/dev/%s/%s" % |
543 | - (FLAGS.volume_group, |
544 | - volume['name']), |
545 | - run_as_root=True) |
546 | - # NOTE(vish): The standard _try_execute does not work here |
547 | - # because these methods throw errors if other |
548 | - # volumes on this host are in the process of |
549 | - # being created. The good news is the command |
550 | - # still works for the other volumes, so we |
551 | - # just wait a bit for the current volume to |
552 | - # be ready and ignore any errors. |
553 | - time.sleep(2) |
554 | - self._execute('vblade-persist', 'auto', 'all', |
555 | - check_exit_code=False, run_as_root=True) |
556 | - self._execute('vblade-persist', 'start', 'all', |
557 | - check_exit_code=False, run_as_root=True) |
558 | - |
559 | - def remove_export(self, context, volume): |
560 | - """Removes an export for a logical volume.""" |
561 | - (shelf_id, |
562 | - blade_id) = self.db.volume_get_shelf_and_blade(context, |
563 | - volume['id']) |
564 | - self._try_execute('vblade-persist', 'stop', |
565 | - shelf_id, blade_id, run_as_root=True) |
566 | - self._try_execute('vblade-persist', 'destroy', |
567 | - shelf_id, blade_id, run_as_root=True) |
568 | - |
569 | - def discover_volume(self, context, _volume): |
570 | - """Discover volume on a remote host.""" |
571 | - (shelf_id, |
572 | - blade_id) = self.db.volume_get_shelf_and_blade(context, |
573 | - _volume['id']) |
574 | - self._execute('aoe-discover', run_as_root=True) |
575 | - out, err = self._execute('aoe-stat', check_exit_code=False, |
576 | - run_as_root=True) |
577 | - device_path = 'e%(shelf_id)d.%(blade_id)d' % locals() |
578 | - if out.find(device_path) >= 0: |
579 | - return "/dev/etherd/%s" % device_path |
580 | - else: |
581 | - return |
582 | - |
583 | - def undiscover_volume(self, _volume): |
584 | - """Undiscover volume on a remote host.""" |
585 | - pass |
586 | - |
587 | - def check_for_export(self, context, volume_id): |
588 | - """Make sure volume is exported.""" |
589 | - (shelf_id, |
590 | - blade_id) = self.db.volume_get_shelf_and_blade(context, |
591 | - volume_id) |
592 | - cmd = ('vblade-persist', 'ls', '--no-header') |
593 | - out, _err = self._execute(*cmd, run_as_root=True) |
594 | - exported = False |
595 | - for line in out.split('\n'): |
596 | - param = line.split(' ') |
597 | - if len(param) == 6 and param[0] == str(shelf_id) \ |
598 | - and param[1] == str(blade_id) and param[-1] == "run": |
599 | - exported = True |
600 | - break |
601 | - if not exported: |
602 | - # Instance will be terminated in this case. |
603 | - desc = _("Cannot confirm exported volume id:%(volume_id)s. " |
604 | - "vblade process for e%(shelf_id)s.%(blade_id)s " |
605 | - "isn't running.") % locals() |
606 | - raise exception.ProcessExecutionError(out, _err, cmd=cmd, |
607 | - description=desc) |
608 | - |
609 | - |
610 | -class FakeAOEDriver(AOEDriver): |
611 | - """Logs calls instead of executing.""" |
612 | - |
613 | - def __init__(self, *args, **kwargs): |
614 | - super(FakeAOEDriver, self).__init__(execute=self.fake_execute, |
615 | - sync_exec=self.fake_execute, |
616 | - *args, **kwargs) |
617 | - |
618 | - def check_for_setup_error(self): |
619 | - """No setup necessary in fake mode.""" |
620 | - pass |
621 | - |
622 | - @staticmethod |
623 | - def fake_execute(cmd, *_args, **_kwargs): |
624 | - """Execute that simply logs the command.""" |
625 | - LOG.debug(_("FAKE AOE: %s"), cmd) |
626 | - return (None, None) |
627 | + def prepare_attach(self, context, volume): |
628 | + """Discover volume on a remote host.""" |
629 | + raise NotImplementedError() |
630 | + |
631 | + def prepare_detach(self, volume): |
632 | + """Undiscover volume on a remote host.""" |
633 | + raise NotImplementedError() |
634 | + |
635 | + def check_for_export(self, context, volume_id): |
636 | + """Make sure volume is exported.""" |
637 | + raise NotImplementedError() |
638 | |
639 | |
640 | class ISCSIDriver(VolumeDriver): |
641 | @@ -505,7 +382,12 @@ |
642 | '-v', property_value) |
643 | return self._run_iscsiadm(iscsi_properties, iscsi_command) |
644 | |
645 | - def discover_volume(self, context, volume): |
646 | + def prepare_attach(self, context, volume): |
647 | + #print "HODOR" |
648 | + |
649 | + #if 1 == 1: |
650 | + # return |
651 | + |
652 | """Discover volume on a remote host.""" |
653 | iscsi_properties = self._get_iscsi_properties(volume) |
654 | |
655 | @@ -557,7 +439,7 @@ |
656 | |
657 | return mount_device |
658 | |
659 | - def undiscover_volume(self, volume): |
660 | + def prepare_detach(self, volume): |
661 | """Undiscover volume on a remote host.""" |
662 | iscsi_properties = self._get_iscsi_properties(volume) |
663 | self._iscsiadm_update(iscsi_properties, "node.startup", "manual") |
664 | @@ -591,11 +473,11 @@ |
665 | """No setup necessary in fake mode.""" |
666 | pass |
667 | |
668 | - def discover_volume(self, context, volume): |
669 | + def prepare_attach(self, context, volume): |
670 | """Discover volume on a remote host.""" |
671 | return "/dev/disk/by-path/volume-id-%d" % volume['id'] |
672 | |
673 | - def undiscover_volume(self, volume): |
674 | + def prepare_detach(self, volume): |
675 | """Undiscover volume on a remote host.""" |
676 | pass |
677 | |
678 | @@ -661,11 +543,11 @@ |
679 | """Removes an export for a logical volume""" |
680 | pass |
681 | |
682 | - def discover_volume(self, context, volume): |
683 | + def prepare_attach(self, context, volume): |
684 | """Discover volume on a remote host""" |
685 | return "rbd:%s/%s" % (FLAGS.rbd_pool, volume['name']) |
686 | |
687 | - def undiscover_volume(self, volume): |
688 | + def prepare_detach(self, volume): |
689 | """Undiscover volume on a remote host""" |
690 | pass |
691 | |
692 | @@ -724,11 +606,11 @@ |
693 | """Removes an export for a logical volume""" |
694 | pass |
695 | |
696 | - def discover_volume(self, context, volume): |
697 | + def prepare_attach(self, context, volume): |
698 | """Discover volume on a remote host""" |
699 | return "sheepdog:%s" % volume['name'] |
700 | |
701 | - def undiscover_volume(self, volume): |
702 | + def prepare_detach(self, volume): |
703 | """Undiscover volume on a remote host""" |
704 | pass |
705 | |
706 | @@ -758,11 +640,11 @@ |
707 | def remove_export(self, context, volume): |
708 | self.log_action('remove_export', volume) |
709 | |
710 | - def discover_volume(self, context, volume): |
711 | - self.log_action('discover_volume', volume) |
712 | + def prepare_attach(self, context, volume): |
713 | + self.log_action('prepare_attach', volume) |
714 | |
715 | - def undiscover_volume(self, volume): |
716 | - self.log_action('undiscover_volume', volume) |
717 | + def prepare_detach(self, volume): |
718 | + self.log_action('prepare_detach', volume) |
719 | |
720 | def check_for_export(self, context, volume_id): |
721 | self.log_action('check_for_export', volume_id) |
722 | |
723 | === modified file 'nova/volume/manager.py' |
724 | --- nova/volume/manager.py 2011-06-02 21:23:05 +0000 |
725 | +++ nova/volume/manager.py 2011-08-23 22:45:41 +0000 |
726 | @@ -1,4 +1,4 @@ |
727 | -# vim: tabstop=4 shiftwidth=4 softtabstop=4 |
728 | +# vim: tabstop=4 shiftwidth=4 softtabstop=4O |
729 | |
730 | # Copyright 2010 United States Government as represented by the |
731 | # Administrator of the National Aeronautics and Space Administration. |
732 | @@ -28,17 +28,13 @@ |
733 | :volume_topic: What :mod:`rpc` topic to listen to (default: `volume`). |
734 | :volume_manager: The module name of a class derived from |
735 | :class:`manager.Manager` (default: |
736 | - :class:`nova.volume.manager.AOEManager`). |
737 | + :class:`nova.volume.manager.Manager`). |
738 | :storage_availability_zone: Defaults to `nova`. |
739 | -:volume_driver: Used by :class:`AOEManager`. Defaults to |
740 | - :class:`nova.volume.driver.AOEDriver`. |
741 | -:num_shelves: Number of shelves for AoE (default: 100). |
742 | -:num_blades: Number of vblades per shelf to allocate AoE storage from |
743 | - (default: 16). |
744 | +:volume_driver: Used by :class:`Manager`. Defaults to |
745 | + :class:`nova.volume.driver.ISCSIDriver`. |
746 | :volume_group: Name of the group that will contain exported volumes (default: |
747 | `nova-volumes`) |
748 | -:aoe_eth_dev: Device name the volumes will be exported on (default: `eth0`). |
749 | -:num_shell_tries: Number of times to attempt to run AoE commands (default: 3) |
750 | +:num_shell_tries: Number of times to attempt to run commands (default: 3) |
751 | |
752 | """ |
753 | |
754 | @@ -214,7 +210,7 @@ |
755 | if volume_ref['host'] == self.host and FLAGS.use_local_volumes: |
756 | path = self.driver.local_path(volume_ref) |
757 | else: |
758 | - path = self.driver.discover_volume(context, volume_ref) |
759 | + path = self.driver.prepare_attach(context, volume_ref) |
760 | return path |
761 | |
762 | def remove_compute_volume(self, context, volume_id): |
763 | @@ -224,7 +220,7 @@ |
764 | if volume_ref['host'] == self.host and FLAGS.use_local_volumes: |
765 | return True |
766 | else: |
767 | - self.driver.undiscover_volume(volume_ref) |
768 | + self.driver.prepare_detach(volume_ref) |
769 | |
770 | def check_for_export(self, context, instance_id): |
771 | """Make sure whether volume is exported.""" |
772 | |
773 | === modified file 'nova/volume/san.py' |
774 | --- nova/volume/san.py 2011-04-13 16:31:28 +0000 |
775 | +++ nova/volume/san.py 2011-08-23 22:45:41 +0000 |
776 | @@ -61,9 +61,6 @@ |
777 | def _build_iscsi_target_name(self, volume): |
778 | return "%s%s" % (FLAGS.iscsi_target_prefix, volume['name']) |
779 | |
780 | - # discover_volume is still OK |
781 | - # undiscover_volume is still OK |
782 | - |
783 | def _connect_to_ssh(self): |
784 | ssh = paramiko.SSHClient() |
785 | #TODO(justinsb): We need a better SSH key policy |
Looks good, just a couple of small pep8 violations:
nova/db/ sqlalchemy/ migrate_ repo/versions/ 037_kill_ export_ devices. py:38:1: E302 expected 2 blank lines, found 1 migrate_ engine) :
def downgrade(
^
Separate top-level function and class definitions with two blank lines.
Method definitions inside a class are separated by a single blank line.
Extra blank lines may be used (sparingly) to separate groups of related
functions. Blank lines may be omitted between a bunch of related
one-liners (e.g. a set of dummy implementations).
Use blank lines in functions, sparingly, to indicate logical sections.
Okay: def a():\n pass\n\n\ndef b():\n pass
Okay: def a():\n pass\n\n\n# Foo\n# Bar\n\ndef b():\n pass
E301: class Foo:\n b = 0\n def bar():\n pass sqlalchemy/ migrate_ repo/versions/ 037_kill_ export_ devices. py:47:1: E302 expected 2 blank lines, found 1 migrate_ engine) :
E302: def a():\n pass\n\ndef b(n):\n pass
E303: def a():\n pass\n\n\n\ndef b(n):\n pass
E303: def a():\n\n\n\n pass
E304: @decorator\n\ndef a():\n pass
nova/db/
def upgrade(
^
Separate top-level function and class definitions with two blank lines.
Method definitions inside a class are separated by a single blank line.
Extra blank lines may be used (sparingly) to separate groups of related
functions. Blank lines may be omitted between a bunch of related
one-liners (e.g. a set of dummy implementations).
Use blank lines in functions, sparingly, to indicate logical sections.
Okay: def a():\n pass\n\n\ndef b():\n pass
Okay: def a():\n pass\n\n\n# Foo\n# Bar\n\ndef b():\n pass
E301: class Foo:\n b = 0\n def bar():\n pass
E302: def a():\n pass\n\ndef b(n):\n pass
E303: def a():\n pass\n\n\n\ndef b(n):\n pass
E303: def a():\n\n\n\n pass
E304: @decorator\n\ndef a():\n pass