Openstack Play Services – Juno Swift,Container, Ceilometer,Heat

Swift:

Swift is used for Object based storage. Its similar to AWS S3 service.
First, create a container to store objects:

$ swift post mycont

Upload a file to the container

$ swift upload mycont local.conf

List container and objects.

$ swift list
mycont
$ swift list mycont
local.conf

List status of created container.

$ swift stat mycont
         Account: AUTH_0fab1e00669245d189ca21d0d0023411
       Container: mycont
         Objects: 1
           Bytes: 1430
        Read ACL:
       Write ACL:
         Sync To:
        Sync Key:
   Accept-Ranges: bytes
X-Storage-Policy: Policy-0
     X-Timestamp: 1424083106.37538
      X-Trans-Id: txeee0b46398664fab8ae93-0054e1ca8e
    Content-Type: text/plain; charset=utf-8

Download the object to a local file.

$ swift download mycont local.conf

Now, delete the object and the container.

$ swift delete mycont local.conf
local.conf
$ swift delete mycont
$ swift list
$ 

Glance:

Glance is used for image management.
Display the image list.

$ glance image-list
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+
| ID                                   | Name                            | Disk Format | Container Format | Size      | Status |
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+
| dd45d30c-ca17-4ab9-83cc-2f3ab7332ba2 | cirros-0.3.2-x86_64-uec         | ami         | ami              | 25165824  | active |
| afc78c04-1506-4ce1-8b11-dca860b019b9 | cirros-0.3.2-x86_64-uec-kernel  | aki         | aki              | 4969360   | active |
| 341a5af8-a492-41fa-8074-52b7819b7866 | cirros-0.3.2-x86_64-uec-ramdisk | ari         | ari              | 3723817   | active |
| 45bfe06a-554e-41ef-96c0-0357e5c1a12c | Fedora-x86_64-20-20140618-sda   | qcow2       | bare             | 209649664 | active |
| 7a976a34-edac-44ae-828c-ee54e075d00a | ubuntu_mysql                    | qcow2       | bare             | 450012160 | active |
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+

To see details about a specific image:

$ glance image-show dd45d30c-ca17-4ab9-83cc-2f3ab7332ba2
+-----------------------+--------------------------------------+
| Property              | Value                                |
+-----------------------+--------------------------------------+
| Property 'kernel_id'  | afc78c04-1506-4ce1-8b11-dca860b019b9 |
| Property 'ramdisk_id' | 341a5af8-a492-41fa-8074-52b7819b7866 |
| checksum              | 4eada48c2843d2a262c814ddc92ecf2c     |
| container_format      | ami                                  |
| created_at            | 2015-02-16T08:09:53                  |
| deleted               | False                                |
| disk_format           | ami                                  |
| id                    | dd45d30c-ca17-4ab9-83cc-2f3ab7332ba2 |
| is_public             | True                                 |
| min_disk              | 0                                    |
| min_ram               | 0                                    |
| name                  | cirros-0.3.2-x86_64-uec              |
| owner                 | 430894167a2540cfbe7b7061df93a6b5     |
| protected             | False                                |
| size                  | 25165824                             |
| status                | active                               |
| updated_at            | 2015-02-16T08:09:55                  |
+-----------------------+--------------------------------------+

We can download or upload images to glance. We can create images from qcow2 or iso format.

$ glance image-download --file ubuntu_mysql.raw ubuntu_mysql
$ file ubuntu_mysql.raw 
ubuntu_mysql.raw: QEMU QCOW Image (v2), 1994915840 bytes

Heat: 

Heat is an Orchestration service that allows for specifying an application definition using a template format. Using this, applications can be managed as a single entity rather than managing individual components of the application like compute, storage and networking separately.

Heat templates are specified in YAML format. To checkout sample heat templates, do the following:

git clone https://github.com/openstack/heat-templates.git

There are 3 sections in a heat template. “Parameters” section describes input parameters, range checking and default values. “Resources” section defines the resources that are part of this application, resources can include any of the compute, storage and networking resources. “Output” section defines the outputs that the template can provide like the URL of the application. Along with the resources section, any initialization script that needs to be executed as part of the instance can be specified. These init-scripts are executed by cloud-init to setup the instance.

I will walk through 2 Heat examples using the example templates provided.

First example is a WordPress application template where the webserver and database are running in the same compute instance. WordPress application uses Fedora instance. I had to create a new Flavor with bigger disk size as the next available flavor was too big for my host machine.
Create a custom flavor with 512mb RAM and 4GB of disk size.

nova flavor-create m1.tiny_small 6 512 4 1 

Use following command to start the heat template. I have modified default “WordPress_Native.yaml” to allow the custom flavor that I created above.

$ heat stack-create teststack --template-file  ~/heat-templates/hot/F20/WordPress_Native_bk.yaml -P key_name=heattest -P image_id=Fedora-x86_64-20-20140618-sda
+--------------------------------------+------------+--------------------+----------------------+
| id                                   | stack_name | stack_status       | creation_time        |
+--------------------------------------+------------+--------------------+----------------------+
| 72c23784-f635-42d5-987b-ccc01fce69c7 | teststack  | CREATE_IN_PROGRESS | 2015-02-17T15:37:22Z |
+--------------------------------------+------------+--------------------+----------------------+

Lets look at heat stack list.

$ heat stack-list
+--------------------------------------+------------+-----------------+----------------------+
| id                                   | stack_name | stack_status    | creation_time        |
+--------------------------------------+------------+-----------------+----------------------+
| 72c23784-f635-42d5-987b-ccc01fce69c7 | teststack  | CREATE_COMPLETE | 2015-02-17T15:37:22Z |
+--------------------------------------+------------+-----------------+----------------------+

Lets look at instances created:

$ nova list
+--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
| ID                                   | Name                                      | Status | Task State | Power State | Networks         |
+--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
| d729f152-7115-4746-bc45-c3f2e7a0d981 | teststack-wordpress_instance-pc5i7ovai7xj | ACTIVE | -          | Running     | private=10.0.0.2 |
+--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+

To login to the heat created instance, use “ec2-user” as username with the keypair mentioned when creating the heat application.

$ ssh -i ~/Downloads/heattest.pem ec2-user@10.0.0.2
[ec2-user@teststack-wordpress-instance-pc5i7ovai7xj ~]$ 

When the cloud-init script is complete, WordPress application can be accessed from the output URL.

Now delete the heat stack instance.

$ heat stack-delete 72c23784-f635-42d5-987b-ccc01fce69c7
+--------------------------------------+------------+--------------------+----------------------+
| id                                   | stack_name | stack_status       | creation_time        |
+--------------------------------------+------------+--------------------+----------------------+
| 72c23784-f635-42d5-987b-ccc01fce69c7 | teststack  | DELETE_IN_PROGRESS | 2015-02-17T15:37:22Z |
+--------------------------------------+------------+--------------------+----------------------+

Following is another example of heat stack instance to create a WordPress application. Here, the template creates 2 instances, 1 running webserver and another running the database.

heat stack-create teststack --template-file  ~/heat-templates/hot/F20/WordPress_2_Instances_bk.yaml -P key_name=heattest -P image_id=Fedora-x86_64-20-20140618-sda

We can confirm the 2 instances created.

$ nova list
+--------------------------------------+---------------------------------------+--------+------------+-------------+------------------+
| ID                                   | Name                                  | Status | Task State | Power State | Networks         |
+--------------------------------------+---------------------------------------+--------+------------+-------------+------------------+
| 7fc2c235-0290-4a5b-80ff-bf8f8b47d69c | teststack-DatabaseServer-jiqyj733v3g3 | ACTIVE | -          | Running     | private=10.0.0.4 |
| 7d1fa991-8917-49eb-9bd5-644e554abf32 | teststack-WebServer-ic4ysgzoskov      | ACTIVE | -          | Running     | private=10.0.0.5 |
+--------------------------------------+---------------------------------------+--------+------------+-------------+------------------+

To see the cloud-init status, following files are helpful:

logs are in:
/var/log/cloud-init-output.log
install commands are in:
/var/lib/heat-cfntools/cfn-userdata

Ceilometer:

Ceilometer service is used to track resource usage and raise alarms when the threshold exceeds. This can be used for billing purposes. Ceilometer can be used in combination with Heat for features like Auto scaling which allows for dynamically creating more instances when CPU usage crosses a threshold.

To see the resources getting monitored, use the following command. I have shown only the cpu resources.

maestro@pandy-VirtualBox:~/devstack$ ceilometer meter-list
+---------------------------------+------------+-----------+-----------------------------------------------------------------------+----------------------------------+----------------------------------+
| Name                            | Type       | Unit      | Resource ID                                                           | User ID                          | Project ID                       |
+---------------------------------+------------+-----------+-----------------------------------------------------------------------+----------------------------------+----------------------------------+
| cpu                             | cumulative | ns        | bc8f97fd-9864-4217-a07b-fd85045f3f0b                                  | 9fdd3b11fd6846f6b37d038ebd90b080 | 0fab1e00669245d189ca21d0d0023411 |
| cpu                             | cumulative | ns        | ccfec6b3-57d0-4d48-8d58-c0b2b4426490                                  | 9fdd3b11fd6846f6b37d038ebd90b080 | 0fab1e00669245d189ca21d0d0023411 |
| cpu_util                        | gauge      | %         | bc8f97fd-9864-4217-a07b-fd85045f3f0b                                  | 9fdd3b11fd6846f6b37d038ebd90b080 | 0fab1e00669245d189ca21d0d0023411 |
| cpu_util                        | gauge      | %         | ccfec6b3-57d0-4d48-8d58-c0b2b4426490                                  | 9fdd3b11fd6846f6b37d038ebd90b080 | 0fab1e00669245d189ca21d0d0023411 |

To see statistics for a particular resource meter:

$ ceilometer statistics -m cpu
+--------+---------------------+---------------------+---------------+-------------+---------------+------------+-------+----------+---------------------+---------------------+
| Period | Period Start        | Period End          | Max           | Min         | Avg           | Sum        | Count | Duration | Duration Start      | Duration End        |
+--------+---------------------+---------------------+---------------+-------------+---------------+------------+-------+----------+---------------------+---------------------+
| 0      | 2015-02-16T10:22:45 | 2015-02-16T16:45:05 | 30860000000.0 | 440000000.0 | 20322083333.3 | 4.8773e+11 | 24    | 22940.0  | 2015-02-16T10:22:45 | 2015-02-16T16:45:05 |
+--------+---------------------+---------------------+---------------+-------------+---------------+------------+-------+----------+---------------------+---------------------+

References:


					

Openstack CLI interface Cheat Sheat

OpenStack services have very powerful command line interfaces, with lots of different options.  I went looking for a good command line cheat sheet, and did not find many options, so decided to created one myself. This is not a replacement for reading excellent OpenStack documentation, rather, a short summary of some basic commands. Comments, corrections, suggestions are welcome!

Keystone (Identity Service)

# List all users
keystone user-list

# List identity service catalog
keystone catalog

# Discover keystone endpoints
keystone discover

# List all services in service catalog
keystone service-list

# Create new user
keystone user-create --name --tenant-id --pass --email --enabled

# Create new tenant
keystone tenant-create --name --description --enabled

Nova (Compute Service)

# List instances, notice status of instance
nova list

# List images
nova image-list

# List flavors
nova flavor-list

# Boot an instance using flavor and image names (if names are unique)
nova boot --image --flavor
nova boot --image cirros-0.3.1-x86_64-uec --flavor m1.tiny MyFirstInstance

# Login to instance
ip netns
sudo ip netns exec ssh <user@server or use a key>
sudo ip netns exec qdhcp-6021a3b4-8587-4f9c-8064-0103885dfba2 ssh cirros@10.0.0.2

# if you are on devstack, password is “cubswin:)” without the quotes

# Show details of instance
nova show <name>
nova show MyFirstInstance

# View console log of instance
nova console-log MyFirstInstance

# Pause, suspend, stop, rescue, resize, rebuild, reboot an instance
# Pause
nova pause <name>
nova pause volumeTwoImage

# Unpause
nova unpause <name>

# Suspend
nova suspend <name>

# Unsuspend
nova resume <name>

# Stop
nova stop <name>

# Start
nova start <name>

# Rescue
nova rescue <name>

# Resize
nova resize <name> <flavor>
nova resize my-pem-server m1.small
nova resize-confirm server1

# Rebuild
nova rebuild <name> <image>
nova rebuild newtinny cirros-qcow2

# Reboot
nova reboot <name>
nova reboot newtinny

# Inject user data and files into an instance
nova boot --user-data ./userdata.txt MyUserdataInstance
nova boot --user-data userdata.txt --image cirros-qcow2 --flavor m1.tiny MyUserdataInstance2

# to validate file is there, ssh into instance, go to /var/lib/cloud look for file

# Inject a keypair into an instance and access the instance with that keypair
# Create keypair
nova keypair-add test > test.pem
chmod 600 test.pem

# Boot
nova boot --image cirros-0.3.0-x86_64 --flavor m1.small --key_name test my-first-server

# ssh into instance
sudo ip netns exec qdhcp-98f09f1e-64c4-4301-a897-5067ee6d544f ssh -i test.pem cirros@10.0.0.4

# Set metadata on an instance
nova meta volumeTwoImage set newmeta=’my meta data’

# Create an instance snapshot
nova image-create volumeTwoImage snapshotOfVolumeImage
nova image-show snapshotOfVolumeImage

# Manage security groups
# Add rules to default security group allowing ping and ssh between #instances in the default security group
nova secgroup-add-group-rule default default icmp -1 -1
nova secgroup-add-group-rule default default tcp 22 22

Glance (Image Service)

# List images you can access
glance image-list

# Delete specified image
glance image-delete <image>

# Describe a specific image
glance image-show <image>

# update image
glance image-update <image>

# Manage images
# Kernel image
glance image-create --name “cirros-threepart-kernel” --disk-format aki --container-format aki --is-public True --file ~/images/cirros-0.3.1~pre4-x86_64-vmlinuz

# Ram image
glance image-create—name “cirros-threepart-ramdisk” --disk-format ari --container-format ari --is-public True --file ~/images/cirros-0.3.1~pre4-x86_64-initrd

# 3-part image
glance image-create—name “cirros-threepart” --disk-format ami --container-format ami --is-public True --property kernel_id=$KID—property ramdisk_id=$RID --file ~/images/cirros-0.3.1~pre4-x86_64-blank.img

# Register raw image
glance image-create --name “cirros-qcow2” --disk-format qcow2 --container-format bare --is-public True --file ~/images/cirros-0.3.1~pre4-x86_64-disk.img

Neutron (Networking Service)

# Create network
neutron net-create <name>
neutron net-create my-network

# Create a subnet
neutron subnet-create <network name> <cidr>
neutron subnet-create my-network 10.0.0.0/29

# List network and subnet
neutron net-list
neutron subnet-list

# Examine details of network and subnet
neutron net-show <id or name of network>
neutron subnet-show <id or name of subnet>

Cinder (Block Storage)

# Manage volumes and volume snapshots
# Create a new volume
cinder create <size in GB> --display-name
cinder create 1 --display-name MyFirstVolume

# Boot an instance and attach to volume
nova boot—image cirros-qcow2 --flavor m1.tiny MyVolumeInstance

# List volumes, notice status of volume
cinder list

# Attach volume to instance after instance is active, and volume is available
nova volume-attach <instance-id> <volume-id> auto
nova volume-attach MyVolumeInstance /dev/vdb auto

# Login into instance, list storage devices
sudo fdisk -l

# On the instance, make filesystem on volume
sudo mkfs.ext3 /dev/vdb

# Create a mountpoint
sudo mkdir /myspace

# Mount the volume at the mountpoint
sudo mount /dev/vdc /myspace

# Create a file on the volume
sudo touch /myspace/helloworld.txt
sudo ls /myspace

# Unmount the volume
sudo umount /myspace

Swift (Object Store)

# Displays information for the account, container, or object
swift stat
swift stat <account>
swift stat <container>
swift stat <object>

# List containers
swift list

# Create a container
swift post mycontainer

# Upload file to a container
swift upload <containder name> <file name>
swift upload mycontainer myfile.txt

# List objects in container
swift list container

# Download object from container
swift download <containder name> <file name>

# Upload with chunks, for large file
swift upload -S <size> <containder name> <file name>
swift upload -S 64 container largeFile

Happy stacking!