Merge into trunk : doc-reserve-metadata-prefix : Code : openstack-manuals

Status:	Merged
Merge reported by:	Anne Gentle
Merged at revision:	not available
Proposed branch:	lp:~justin-fathomdb/openstack-manuals/doc-reserve-metadata-prefix
Merge into:	lp:~annegentle/openstack-manuals/trunk
Diff against target:	20 lines (+10/-0) 1 file modified doc/source/docbkx/openstack-compute-api/cs-devguide.xml (+10/-0)
To merge this branch:	bzr merge lp:~justin-fathomdb/openstack-manuals/doc-reserve-metadata-prefix
Related bugs:	Link a bug report

Reviewer	Date Requested	Status
Anne Gentle	2011-03-02	Approve on 2011-03-03
Jorge L. Williams	2011-03-02	Pending
Review via email: mp+51974@code.launchpad.net

Revision history for this message

Anne Gentle (annegentle) wrote on 2011-03-03:

#

Hi Justin - I just want to make sure I understand the last sentence:

In future also, metadata with the reserved key prefix may alter the behavior of API operations.

Sounds a bit mysterious. :) Is that just for the reserved aws: key prefix or for any reserved prefix? Will we need to document the precise alterations in behavior? Can you give an example?

review: Needs Information

Revision history for this message

justinsb (justin-fathomdb) wrote on 2011-03-03:

#

The issue is that AWS have reserved keys with the "aws:" prefix, and we have no way of knowing what they're going to do in future. I tried writing something that reserved the prefix and essentially covered us for anything AWS might do.

Here's my thought process:

Suppose tomorrow we wake up and see that overnight AWS introduced two new Metadata keys: an "aws:cost-last-hour" key and a "aws:regulations=pci-compliant" key.

On the EC2 api, we would want to support those keys, so if the user had created an aws:cost-last-hour, we'd have some difficult decisions to make (overwrite the user's key, have two keys, don't include the system value?), hence it's better to just reserve the prefix.

If we start having different keys appearing based on the API we're using, I think that's confusing, and also effectively doubles our workload, so I believe we also want to reserve these keys in the OpenStack API. Another way of looking at it is that we need to map the keys to the same metadata collection behind the scenes, so if a key is reserved on one API, it needs to be reserved in all APIs. Otherwise an OpenStack API user would obliviously create a key that the EC2 API documentation warns them is a bad idea.

aws:cost-last-hour would be dynamic, so would change 'behind the scenes'. This is different from the other keys (which only change in response in user requests), hence I wrote that these keys can change any time.

aws:regulations=pci-compliant would select just racks that were operated in accordance to pci compliance standards, which might have a higher cost per hour, and these resources might be unavailable while normal compliance is available. So the behavior of an allocation changes in response to metadata. This would only be when a reserved key is present.

Now (as far as I know) most of this is at the moment 'CYA' - AWS doesn't make extensive use of the AWS prefix. However, they can in future, we can't control what they do, and so we need our documentation to remain valid even in the face of their changes.

I believe (at the moment) that this is just a documentation issue. Users _may_ still create keys with the reserved prefix (though they _shouldn't_). And at the moment there's no keys that we need to support (at least nobody has requested any, though I suspect there are some.) So I don't think anything needs to change in code, but we just need to document that creating keys with an 'aws:' prefix is "a bad idea"

I personally think we should take advantage of this functionality ourselves, by reserving an "openstack:" prefix which is comparable to the "aws:" prefix, but this is controversial.

The issue is that AWS have reserved keys with the "aws:" prefix, and we have no way of knowing what they're going to do in future.  I tried writing something that reserved the prefix and essentially covered us for anything AWS might do.

Here's my thought process:

Suppose tomorrow we wake up and see that overnight AWS introduced two new Metadata keys: an "aws:cost-last-hour" key and a "aws:regulations=pci-compliant" key.

On the EC2 api, we would want to support those keys, so if the user had created an aws:cost-last-hour, we'd have some difficult decisions to make (overwrite the user's key, have two keys, don't include the system value?), hence it's better to just reserve the prefix.

If we start having different keys appearing based on the API we're using, I think that's confusing, and also effectively doubles our workload, so I believe we also want to reserve these keys in the OpenStack API. Another way of looking at it is that we need to map the keys to the same metadata collection behind the scenes, so if a key is reserved on one API, it needs to be reserved in all APIs.  Otherwise an OpenStack API user would obliviously create a key that the EC2 API documentation warns them is a bad idea.

aws:cost-last-hour would be dynamic, so would change 'behind the scenes'.  This is different from the other keys (which only change in response in user requests), hence I wrote that these keys can change any time.

aws:regulations=pci-compliant would select just racks that were operated in accordance to pci compliance standards, which might have a higher cost per hour, and these resources might be unavailable while normal compliance is available.  So the behavior of an allocation changes in response to metadata.  This would only be when a reserved key is present.

Now (as far as I know) most of this is at the moment 'CYA' - AWS doesn't make extensive use of the AWS prefix.  However, they can in future, we can't control what they do, and so we need our documentation to remain valid even in the face of their changes.

I believe (at the moment) that this is just a documentation issue.  Users _may_ still create keys with the reserved prefix (though they _shouldn't_).  And at the moment there's no keys that we need to support (at least nobody has requested any, though I suspect there are some.)  So I don't think anything needs to change in code, but we just need to document that creating keys with an 'aws:' prefix is "a bad idea"

I personally think we should take advantage of this functionality ourselves, by reserving an "openstack:" prefix which is comparable to the "aws:" prefix, but this is controversial.

Revision history for this message

Jorge L. Williams (jorgew) wrote on 2011-03-03:

#

Download full text (5.9 KiB)

Hey Justin,

Your concern is well warranted. We should in fact reserve the "aws" prefix, and I would vote for us to reserve "openstack" as well, though I prefer "os" :-)

In the EC2 implementation this stuff is presented as metadata, but that's not how the OpenStack Compute 1.1 API works. In OpenStack Compute 1.1 metadata is reserved only for end users . We don't mess with it. This allows us to do things like place quotas on it and it keeps user data separate from ours. So the "aws" prefix will never come into play in the section that you modified. In the Compute API, the prefix is known as a vendor identifier and it's used by extensions. These prefixes are reserved specifically for vendors. Take a look at: http://wiki.openstack.org/JorgeWilliams?action=AttachFile&do=view&target=Extensions.pdf slide 17.

So essentially this is how things would would look in Compute:

{
    "server" : {
        "id" : 1234,
        "name" : "sample-server",
        "imageRef" : "https://servers.api.rackspacecloud.com/v1.1/32278/images/1234",
        "flavorRef" : "https://servers.api.rackspacecloud.com/v1.1/32278/flavors/1",
        "updated" : "2010-10-10T12:00:00Z",
        "created" : "2010-08-10T12:00:00Z",
        "hostId" : "e4d909c290d0fb1ca068ffaddf22cbd0",
        "affinityId" : "fc88bcf8394db9c8d0564e08ca6a9724188a84d1",
        "status" : "BUILD",
        "progress" : 60,
"aws-cost:cost-last-hour" : 2.25,
        "metadata" : {
            "values" : {
                "Server Label" : "Web Head 1",
                "Image Version" : "2.1"
            }
        },
        "links": [
            {
                "rel" : "self",
                "href" : "http://servers.api.openstack.org/v1.1/1234/servers/1234"
            },
            {
                "rel" : "bookmark",
                "type" : "application/vnd.openstack.compute-v1.1+xml",
                "href" : "http://servers.api.openstack.org/1234/servers/1234"
            },
            {
                "rel" : "bookmark",
                "type" : "application/vnd.openstack.compute-v1.1+json",
                "href" : "http://servers.api.openstack.org/1234/servers/1234"
            }
        ]
    }
}

Notice that cost-last-hour is part of server...not part of metadata. The aws prefix is used to identify the extension.

I think the best thing to do is to work out the description of extensions a bit more to include mention of the prefix. Then state that we reserve the aws prefix there.

Does that make sense?

-jOrGe W.

On Mar 3, 2011, at 10:47 AM, justinsb wrote:

The issue is that AWS have reserved keys with the "aws:" prefix, and we have no way of knowing what they're going to do in future. I tried writing something that reserved the prefix and essentially covered us for anything AWS might do.

Here's my thought process:

Suppose tomorrow we wake up and see that overnight AWS introduced two new Metadata keys: an "aws:cost-last-hour" key and a "aws:regulations=pci-compliant" key.

On the EC2 api, we would want to support those keys, so if the user had created an aws:cost-last-hour, we'd have some difficult decisions to make (overwrite the user's key, have two keys, don't inclu...

Hey  Justin,

Your concern is well warranted.  We should in fact reserve the "aws" prefix, and I would vote for us to reserve "openstack" as well, though I prefer "os"  :-)

In the EC2 implementation this stuff is presented as metadata, but that's not how the OpenStack Compute 1.1 API works.  In OpenStack Compute 1.1 metadata is reserved only for end users .  We don't mess with it. This allows us to do things like place quotas on it and it keeps user data separate from ours.  So the "aws" prefix will never come into play in the section that you modified. In the Compute API, the prefix is known as a vendor identifier and it's used by extensions.  These prefixes are reserved specifically for vendors.  Take a look at: http://wiki.openstack.org/JorgeWilliams?action=AttachFile&do=view&target=Extensions.pdf   slide 17.

So essentially this is how things would would look in Compute:

{
    "server" : {
        "id" : 1234,
        "name" : "sample-server",
        "imageRef" : "https://servers.api.rackspacecloud.com/v1.1/32278/images/1234",
        "flavorRef" : "https://servers.api.rackspacecloud.com/v1.1/32278/flavors/1",
        "updated" : "2010-10-10T12:00:00Z",
        "created" : "2010-08-10T12:00:00Z",
        "hostId" : "e4d909c290d0fb1ca068ffaddf22cbd0",
        "affinityId" : "fc88bcf8394db9c8d0564e08ca6a9724188a84d1",
        "status" : "BUILD",
        "progress" : 60,
"aws-cost:cost-last-hour" : 2.25,
        "metadata" : {
            "values" : {
                "Server Label" : "Web Head 1",
                "Image Version" : "2.1"
            }
        },
        "links": [
            {
                "rel" : "self",
                "href" : "http://servers.api.openstack.org/v1.1/1234/servers/1234"
            },
            {
                "rel" : "bookmark",
                "type" : "application/vnd.openstack.compute-v1.1+xml",
                "href" : "http://servers.api.openstack.org/1234/servers/1234"
            },
            {
                "rel" : "bookmark",
                "type" : "application/vnd.openstack.compute-v1.1+json",
                "href" : "http://servers.api.openstack.org/1234/servers/1234"
            }
        ]
    }
}

Notice that cost-last-hour is part of server...not part of metadata.  The aws prefix is used to identify the extension.

I think the best thing to do is to work out the description of extensions a bit more to include mention of the prefix.  Then state that we reserve the aws prefix there.