Background

Frequently doing OpenStack development I find myself wanting to query an API directly and observe the response. This is a fairly common development task, but it’s more complicated in OpenStack because there is an order in which you are supposed to make calls. The ideal flow is:

• authenticate using credentials (username/password or a range of other mechanisms)
• use the service catalog returned with authentication to find the endpoint for a service
• find the API version URL you want from the service’s endpoint
• make a request to the versioned URL

So we generally end up simply using a combination of curl and jq against a known endpoint with an existing token. This pattern has existed for so long that the --debug output of most clients is actually in curl command form. There are numerous drawbacks to this approach including:

• you have to manually refresh tokens when they expire.
• you have to know the endpoints ahead of time.
• for security reasons the actual token is no longer displayed so you can’t simply copy the outputted curl command.
• you have to remember all the curl/other tool commands for showing headers, readable output etc - YMMV on this but I always forget.

Introducing os-http

os-http is an easy to use CLI tool for making requests against OpenStack endpoints correctly. It’s designed to allow developers to debug and inspect the responses of OpenStack REST APIs without having to manage the details of authentication, service catalog and version negotiation. Its interface is 100% modelled on the excellent httpie.

I have recently added the 0.1 release to pypi and the source is available on my github though it will probably migrate to the OpenStack infrastructure if it gains adoption. It is released under the Apache 2 License.

It is still very raw and but I have been using it for some time and feel it may be useful for others. It is also in fairly desperate need of documentation - contributions welcome.

Example

Because it’s powered by os-client-config the authentication configuration is what you would expect from using openstackclient. Documentation for preparing this authentication is available from both of these projects.

1

export OS_CLOUD=devstack

There are then a number of choices you can make for service discovery:

1
2
3
4
5
6

--os-service-type <name>    Service type to request from the catalog
--os-service-name <name>    Service name to request from the catalog
--os-interface <name>       API Interface to use [public, internal, admin]
--os-region-name <name>     Region of the cloud to use
--os-endpoint-override <url>  Endpoint to use instead of the endpoint in the catalog
--os-api-version <version>  Which version of the service API to use

As is standard for OpenStack clients these options are also able to be set via the corresponding OS_ environment varibles:

1
2

export OS_SERVICE_TYPE=image
export OS_API_VERSION=2

The syntax for commands is then os-http METHOD PATH [ITEM [ITEM]]. ITEM currently only accepts headers in a key:value format but will support others in future.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80

jamie@devstack:~$ os-http get /images X-My-Header:Value
HTTP/1.1 200 OK
Date: Tue, 12 Apr 2016 01:35:31 GMT
Connection: keep-alive
Content-Type: application/json; charset=UTF-8
Content-Length: 1987
X-Openstack-Request-Id: req-3f6e07e7-cd0d-4a90-9d8a-0024a4bc347f


{
    "first": "/v2/images",
    "images": [
        {
            "checksum": "eb9139e4942121f22bbc2afc0400b2a4",
            "container_format": "ami",
            "created_at": "2016-03-08T01:56:59Z",
            "disk_format": "ami",
            "file": "/v2/images/6c13a0e6-98a3-47fb-bee4-2e356668f7d9/file",
            "id": "6c13a0e6-98a3-47fb-bee4-2e356668f7d9",
            "kernel_id": "578a708b-d0de-4a28-bcd2-8627ad15a971",
            "min_disk": 0,
            "min_ram": 0,
            "name": "cirros-0.3.4-x86_64-uec",
            "owner": "1f04217930f34b4a92fb11457783f2c0",
            "protected": false,
            "ramdisk_id": "0eaea69b-ebfd-40f3-bf86-13b8ad08462b",
            "schema": "/v2/schemas/image",
            "self": "/v2/images/6c13a0e6-98a3-47fb-bee4-2e356668f7d9",
            "size": 25165824,
            "status": "active",
            "tags": [],
            "updated_at": "2016-03-08T01:56:59Z",
            "virtual_size": null,
            "visibility": "public"
        },
        {
            "checksum": "be575a2b939972276ef675752936977f",
            "container_format": "ari",
            "created_at": "2016-03-08T01:56:57Z",
            "disk_format": "ari",
            "file": "/v2/images/0eaea69b-ebfd-40f3-bf86-13b8ad08462b/file",
            "id": "0eaea69b-ebfd-40f3-bf86-13b8ad08462b",
            "min_disk": 0,
            "min_ram": 0,
            "name": "cirros-0.3.4-x86_64-uec-ramdisk",
            "owner": "1f04217930f34b4a92fb11457783f2c0",
            "protected": false,
            "schema": "/v2/schemas/image",
            "self": "/v2/images/0eaea69b-ebfd-40f3-bf86-13b8ad08462b",
            "size": 3740163,
            "status": "active",
            "tags": [],
            "updated_at": "2016-03-08T01:56:57Z",
            "virtual_size": null,
            "visibility": "public"
        },
        {
            "checksum": "8a40c862b5735975d82605c1dd395796",
            "container_format": "aki",
            "created_at": "2016-03-08T01:56:54Z",
            "disk_format": "aki",
            "file": "/v2/images/578a708b-d0de-4a28-bcd2-8627ad15a971/file",
            "id": "578a708b-d0de-4a28-bcd2-8627ad15a971",
            "min_disk": 0,
            "min_ram": 0,
            "name": "cirros-0.3.4-x86_64-uec-kernel",
            "owner": "1f04217930f34b4a92fb11457783f2c0",
            "protected": false,
            "schema": "/v2/schemas/image",
            "self": "/v2/images/578a708b-d0de-4a28-bcd2-8627ad15a971",
            "size": 4979632,
            "status": "active",
            "tags": [],
            "updated_at": "2016-03-08T01:56:55Z",
            "virtual_size": null,
            "visibility": "public"
        }
    ],
    "schema": "/v2/schemas/images"
}

X-My-Header:Value is purely for demonstration purposes and is ignored by glance. As you can see the output is nicely formatted and in a console even includes some pygments magic for coloring.

Caveats

os-http is at version 0.1 and has many unimplemented or not quite right things. Most notably:

• There is really only support for GET and other body-less requests. Whilst you can specify PUT/POST or other to method there is currently no means to specify body data so the request will be empty. This would be easy to add but I havent used it so I haven’t implemented it - contributions welcome.

• The output is intended to be easy for a developer to consume, not for a script to parse (though this may be considered in future). It is not intended to be a replacement for the existing CLIs in scripts. The default output may change to include any additional information that could be useful to developers.

• Because os-http does requests ‘correctly’ you may find that using –os-api-version gives errors - particularly with nova. This is because for most installations the service catalog for nova points to a protected endpoint. There is ongoing work upstream to fix the service catalog in general but for now os-http doesn’t contain the hacks that clients do to work around poor setups. Using this tool may lead you to discover just how many hacks there are.

Please test it out and report any feedback or bugs.