Merge lp:~benoit-canet/goamz/add-CONTRIBUTING into lp:goamz

[Benoît Canet]

Add the Outscale EC2 compatible cloud provider.

This commits define the EC2 endpoint used by Outscale in Outscale
specific regions.

Outscale only implement the EC2 API so other endpoint are left set to "".

This commit takes care of making the code panic in various constructor
when a struct will use an undefined endpoint.

https://codereview.appspot.com/98130043/

[Benoît Canet (benoit-canet)]

Reviewers: mp+218959_code.launchpad.net,

Message:
Please take a look.

Description:
Add the Outscale EC2 compatible cloud provider.

This commits define the EC2 endpoint used by Outscale in Outscale
specific regions.

Outscale only implement the EC2 API so other endpoint are left set to
"".

This commit takes care of making the code panic in various constructor
when a struct will use an undefined endpoint.

https://code.launchpad.net/~benoit-canet/goamz/add-CONTRIBUTING/+merge/218959

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/98130043/

Affected files (+61, -0 lines):
   A [revision details]
   M aws/aws.go
   M exp/sdb/sdb.go
   M exp/sns/sns.go
   M iam/iam.go
   M s3/s3.go

Index: [revision details]
=== added file '[revision details]'
--- [revision details] 2012-01-01 00:00:00 +0000
+++ [revision details] 2012-01-01 00:00:00 +0000
@@ -0,0 +1,2 @@
+Old revision:
<email address hidden>
+New revision: <email address hidden>

Index: aws/aws.go
=== modified file 'aws/aws.go'
--- aws/aws.go 2014-02-12 13:42:33 +0000
+++ aws/aws.go 2014-05-09 10:30:42 +0000
@@ -133,6 +133,47 @@
   "https://iam.amazonaws.com",
  }

+// The following regions are for the Outscale SAS cloud provider.
+// Outscale SAS only implement EC2 so other values are set to "".
+var OscEuWest3 = Region{
+ "osc-eu-west-3",
+ "https://api-ppd.outscale.com",
+ "",
+ "",
+ false,
+ false,
+ "",
+ "",
+ "",
+ "",
+}
+
+var OscEuWest1 = Region{
+ "osc-eu-west-1",
+ "https://api.eu-west-1.outscale.com",
+ "",
+ "",
+ false,
+ false,
+ "",
+ "",
+ "",
+ "",
+}
+
+var OscUsEast1 = Region{
+ "osc-us-east-1",
+ "https://api.us-east-1.outscale.com",
+ "",
+ "",
+ false,
+ false,
+ "",
+ "",
+ "",
+ "",
+}
+
  var Regions = map[string]Region{
   APNortheast.Name: APNortheast,
   APSoutheast.Name: APSoutheast,
@@ -142,6 +183,9 @@
   USWest.Name: USWest,
   USWest2.Name: USWest2,
   SAEast.Name: SAEast,
+ OscEuWest3.Name: OscEuWest3,
+ OscEuWest1.Name: OscEuWest1,
+ OscUsEast1.Name: OscUsEast1,
  }

  type Auth struct {

Index: iam/iam.go
=== modified file 'iam/iam.go'
--- iam/iam.go 2013-03-02 02:11:38 +0000
+++ iam/iam.go 2014-05-09 10:30:42 +0000
@@ -20,6 +20,9 @@

  // New creates a new IAM instance.
  func New(auth aws.Auth, region aws.Region) *IAM {
+ if region.IAMEndpoint == "" {
+ panic("The provided region has no IAM endpoint")
+ }
   return &IAM{auth, region}
  }

Index: s3/s3.go
=== modified file 's3/s3.go'
--- s3/s3.go 2014-02-12 13:42:33 +0000
+++ s3/s3.go 2014-05-09 10:30:42 +0000
@@ -68,6 +68,12 @@

  // New creates a new S3.
  func New(auth aws.Auth, region aws.Region) *S3 {
+ if region.S3Endpoint == "" {
+ panic("The provided region has no S3 endpoint")
+ }
+ if region.S3BucketEndpoint == "" {
+ panic("The provided region has no S3 bucket endpoint")
+ }
   return &S3{auth, region, 0}
  }

Index: exp/sdb/sdb.go
=== modified file 'exp/sdb/sdb.go'
--- exp/sdb/sdb.go 2013-05-23 01:06:45 +0000
+++ exp/sdb/sdb.go 2014-05-09 10:30:42 +0000
@@ -42,6 +42,9 @@

  // New creates a new SDB.
  func New(auth aws.Auth, region aw...

[Benoît Canet (benoit-canet)]

On 2014/05/09 10:52:25, benoit.canet wrote:
> Please take a look.

This provider is Outscale SAS a french company.
I must make room for a future affiliated provider Outscale INC an
US based one.

Would the region be better named OscSASEuWest3 and
"osc-sas-eu-west-3" ?

I am thinking about the prices list here.

Best regards

Benoît

https://codereview.appspot.com/98130043/

[Dimiter Naydenov (dimitern)]

@benoit-canet can you re-propose this against http://github.com/go-amz/amz ?

Unmerged revisions

51. By Benoît Canet <email address hidden> on 2014-05-09

aws: Add regions for the Outscale EC2 compatible cloud provider.

Outscale only implement EC2 so only the EC2 endpoints are defined for
Outscale's regions.

This commit takes care of making the code panic in the various
constructors if the endpoints they use are not defined.

50. By Dimiter Naydenov on 2014-02-12

ec2: Added (Un)AssignPrivateIPAddresses APIs

Added two new API calls:
- AssignPrivateIPAddresses
- UnassignPrivateIPAddresses

They allow adding or removing secondary private
IP addresses to a network interface of an instance.

This is the final part of the VPC-related goamz
changes needed to support initial container
networking in juju-core.

Added a live-only test for the two new APIs and
a couple of example response tests.

Removed ec2.*Status constants, as suggested in
a prereq review, which I omitted previously.

R=rog, niemeyer
CC=
https://codereview.appspot.com/54210047

49. By Dimiter Naydenov on 2014-02-12

ec2: Added NIC support for RunInstances

This extends RunInstances options to include
a list of NetworkInterfaceSpec options. They
allow specifying NICs to attach to instances
at launch time (either existing or new NICs).

It also adds SubnetId, VPCId, SourceDestCheck,
and NetworkInterfaces fields to the Instance
type, so Instances() and RunInstances() can
return the extended VPC-related information
for an instance.

If any NetworkInterfaceSpec options are provided,
or SubnetID is given, RunInstances() will use the
latest AWS API version (2013-10-15), otherwise it
uses the default version (2011-12-15).

Modified terminateInstances() test helper to wait
and retry when running against live EC2 servers,
to make sure the instances are really gone and
not left hanging after the test (leaking related
resources with them). In order for this to work
with the local testing server, a slight change
was made - when an instance is terminated, the
next time you fetch it with Instances(), it will
report "terminated" (thus simulating the real
state transition).

Added vpc-id and subnet-id filters to ec2test's
Instance and securityGroup types, and also changed
TestGroupFiltering and TestInstanceFiltering to
include tests for them.

Test double changed to support VPC security groups
and verify subnet ID in runInstances, as EC2 does.
Added NewInstancesVPC() method with vpcId and subnetId,
so VPC-enabled instances can be created in the test
server.

Added a TestRunInstancesVPC live test for the new
functionality (only live, because it's not worth
it to change the test double to create NICs in
runInstances()).

R=rog, niemeyer, nate.finch
CC=
https://codereview.appspot.com/60620043

48. By Dimiter Naydenov on 2014-02-12

ec2: Add VPC NetworkInterface-related APIs

Added the following new API calls:
- CreateNetworkInterface
- DeleteNetworkInterface
- NetworkInterfaces
- AttachNetworkInterface
- DetachNetworkInterface
(and related types/responses)

Modified existing calls/types:
- SecurityGroupInfo now includes VPCId field
- Add CreateSecurityGroupVPC call that does
the same as CreateSecurityGroup, but sets
the VPCId, to create a VPC group

This enables us to handle VPC NICs with
goamz and partially handle private IP
addresses for them. Next, we'll add more
VPC-related stuff to Instances.

Tested live on EC2, extended ec2test package
as needed.

Added a couple of test helpers: createSubnet,
and deleteSubnets, used to make sure we wait
for the events to happen when running against
live EC2 servers, and we don't leave stuff
around after the tests. With these the live
tests pass and clean up properly.

R=rog, niemeyer
CC=
https://codereview.appspot.com/54570048

47. By Dimiter Naydenov on 2014-02-12

ec2: Add support for VPC subnets

Added the following new API calls:
- CreateSubnet
- DeleteSubnet
- Subnets
(and related types/responses)

This is the second step on the path to
support VPC networking in goamz, next
APIs for network interfaces will be
added.

Tested live on EC2, and extended the
ec2test package as needed.

Added a deleteVPCs test helpers which
waits until a VPC is no longer in use
and can be deleted, retrying as needed
when running against live EC2 servers.
This is needed to ensure live tests do
no leave stuff behind (I've run all live
tests several times in a row to make sure
it works).

R=rog, gz, niemeyer
CC=
https://codereview.appspot.com/54690048

46. By Dimiter Naydenov on 2014-02-12

ec2: Add support for AWS VPCs

Added the following new API calls:
- CreateVPC
- DeleteVPC
- VPCs
(and the associated types/responses).

Most new code is in vpc.go and vpc_test.go.

These are needed in order to support VPC-related
operations on EC2, which will come in later follow-ups.
Implementation complexity in ec2test package for the
new calls is minimal.

Some changes were needed in order to support these
and the upcoming API calls. For the new ones, AWS API
version 2013-10-15 (latest) is used, while for the
existing calls use the previous API version, as before
(2011-12-15).

Added tests, updated test doubles and tested live on
EC2.

After running the EC2 live tests numerous times, I
realized some tests are leaking security groups, so
I added a deleteGroups() tests helper that retries
to ensure all groups are deleted. Also improved the
VPC tests to include retrying as well, so the live
tests are more stable and clean up after themselves.

R=
CC=
https://codereview.appspot.com/49930045

45. By Dimiter Naydenov on 2014-02-12

ec2: Fixed #1275406 - LocalServerSuite failure

LocalServerSuite.TestInstanceInfo fails due to
the DNSName field being empty, where as the
test expect it populated.
Changed the test to refresh the instance data
after it's started, because DNSName is empty
until the instance state is "started" (according
to AWS docs). In the test suite this is simulated
by getting the instance data again, which includes
the populated DNSName.

R=rog, gz, niemeyer
CC=
https://codereview.appspot.com/59620043

=== modified file 'aws/aws.go'
--- aws/aws.go	2014-02-12 13:42:33 +0000
+++ aws/aws.go	2014-05-09 10:44:08 +0000
@@ -133,6 +133,47 @@
 	"https://iam.amazonaws.com",
 }
 
+// The following regions are for the Outscale SAS cloud provider.
+// Outscale SAS only implement EC2 so other values are set to "".
+var OscEuWest3 = Region{
+	"osc-eu-west-3",
+	"https://api-ppd.outscale.com",
+	"",
+	"",
+	false,
+	false,
+	"",
+	"",
+	"",
+	"",
+}
+
+var OscEuWest1 = Region{
+	"osc-eu-west-1",
+	"https://api.eu-west-1.outscale.com",
+	"",
+	"",
+	false,
+	false,
+	"",
+	"",
+	"",
+	"",
+}
+
+var OscUsEast1 = Region{
+	"osc-us-east-1",
+	"https://api.us-east-1.outscale.com",
+	"",
+	"",
+	false,
+	false,
+	"",
+	"",
+	"",
+	"",
+}
+
 var Regions = map[string]Region{
 	APNortheast.Name:  APNortheast,
 	APSoutheast.Name:  APSoutheast,
@@ -142,6 +183,9 @@
 	USWest.Name:       USWest,
 	USWest2.Name:      USWest2,
 	SAEast.Name:       SAEast,
+	OscEuWest3.Name:   OscEuWest3,
+	OscEuWest1.Name:   OscEuWest1,
+	OscUsEast1.Name:   OscUsEast1,
 }
 
 type Auth struct {

=== modified file 'exp/sdb/sdb.go'
--- exp/sdb/sdb.go	2013-05-23 01:06:45 +0000
+++ exp/sdb/sdb.go	2014-05-09 10:44:08 +0000
@@ -42,6 +42,9 @@
 
 // New creates a new SDB.
 func New(auth aws.Auth, region aws.Region) *SDB {
+	if region.SDBEndpoint == "" {
+		panic("The provided region has not SDB endpoint")
+	}
 	return &SDB{auth, region, 0}
 }
 

=== modified file 'exp/sns/sns.go'
--- exp/sns/sns.go	2012-03-09 15:34:56 +0000
+++ exp/sns/sns.go	2014-05-09 10:44:08 +0000
@@ -47,6 +47,9 @@
 }
 
 func New(auth aws.Auth, region aws.Region) *SNS {
+	if region.SNSEndpoint == "" {
+		panic("The provided region has no SNS endpoint")
+	}
 	return &SNS{auth, region, 0}
 }
 

=== modified file 'iam/iam.go'
--- iam/iam.go	2013-03-02 02:11:38 +0000
+++ iam/iam.go	2014-05-09 10:44:08 +0000
@@ -20,6 +20,9 @@
 
 // New creates a new IAM instance.
 func New(auth aws.Auth, region aws.Region) *IAM {
+	if region.IAMEndpoint == "" {
+		panic("The provided region has no IAM endpoint")
+	}
 	return &IAM{auth, region}
 }
 

=== modified file 's3/s3.go'
--- s3/s3.go	2014-02-12 13:42:33 +0000
+++ s3/s3.go	2014-05-09 10:44:08 +0000
@@ -68,6 +68,12 @@
 
 // New creates a new S3.
 func New(auth aws.Auth, region aws.Region) *S3 {
+	if region.S3Endpoint == "" {
+		panic("The provided region has no S3 endpoint")
+	}
+	if region.S3BucketEndpoint == "" {
+		panic("The provided region has no S3 bucket endpoint")
+	}
 	return &S3{auth, region, 0}
 }