Learn Python – Newbie

Learning Python is fun.

Here are some books

  1. Practical Programming for Total Beginners
  2. Head First Python
  3. Python Cookbook
  4. Google’s Python Class
  5. Browse | Coursera
  6. Tutorial point

If you are fully committed and dedicated it take maximum 2 weeks but to be an expertise

all you need to do is practice. I can suggest you a progression path.

  1. Discover list comprehensions
  2. Discover generators
  3. Incorporate map, reduce, filter, iter, range, xrange often into your code
  4. Discover Decorators
  5. Write recursive functions, a lot
  6. Discover itertools and functools
  7. Read Real World Haskell (read free online)
  8. Rewrite all your old Python code with tons of higher order functions, recursion, and whatnot.
  9. Annoy your cubicle mates every time they present you with a Python class. Claim it could be “better” implemented as a dictionary plus some functions. Embrace functional programming.
  10. Rediscover the Strategy pattern and then all those things from imperative code you tried so hard to forget after Haskell.
  11. Find a balance.

Download Downloads – Twisted look at the source code.

Attend http://challenge.greplin.com/ using python

Fun loving gaming site The Python Challenge. Try to import antigravity :p

Referal links:

Home python

Advertisements

Tacker Installation Openstack

what is Tacker?

Tacker is an official OpenStack project building a Generic VNF Manager (VNFM) and a NFV Orchestrator (NFVO) to deploy and operate Network Services and Virtual Network Functions (VNFs) on an NFV infrastructure platform like OpenStack. It is based on ETSI MANO Architectural Framework and provides a functional stack to Orchestrate Network Services end-to-end using VNFs.

 

High Level ArchitectureETSI MANO Tacker.JPG

To know more about architecture  click Tacker

Installation on single node setup (Devstack)

1) Pull devstack repo either master or any stable releases ( do “git clone -b stable/<stable release name>

Note: supported from the Openstack Kilo Release.

git clone https://github.com/openstack-dev/devstack

3) A sample local.conf is placed at https://raw.githubusercontent.com/openstack/tacker/master/devstack/samples/local.conf.example. Copy the local.conf to devstack root directory and customize it based on your environment settings. Update the HOST_IP to the IP address of VM or host where you are running tacker.

Note 1: Ensure local.conf file has the “enable_plugin tacker” line and it is pointing to master.

4) Run stack.sh

Installation on Multinode setup:

Prerequisites:

  • Hardware: minimum 8GB RAM, Ubuntu (version 14.04)
  • Ensure that OpenStack components Keystone, Glance, Nova, Neutron, Heat and Horizon are installed.
  • Git & Python packages should be installed

sudo apt-get install python-pip git

Steps:

  1. Create Client environment source file

export OS_PROJECT_DOMAIN_ID=default
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_NAME=admin
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=ADMIN_PASS
export OS_AUTH_URL=http://controller:35357/v3
export OS_IDENTITY_API_VERSION=3

 

Ensure entry for extensions drivers in /etc/neutron/plugins/ml2/ml2_conf.ini Restart neutron services after the below entry has been added.[ml2] extension_drivers = port_security

 

Modify heat’s policy.json file under /etc/heat/policy.json file to allow users in non-admin projects with ‘admin’ roles to create flavors."resource_types:OS::Nova::Flavor": "role:admin"

Install Tacker server

Before you install and configure Tacker server, you must create a database, service credentials, and API endpoints.

 

  1. To create the database, complete these steps:
    • Use the database access client to connect to the database server as the root user:mysql -u root -p
    • Create the tacker database:create database tacker;
    • Grant proper access to the tacker database: GRANT ALL PRIVILEGES ON tacker.* TO 'tacker'@'localhost' \
      IDENTIFIED BY 'TACKER_DBPASS';
      GRANT ALL PRIVILEGES ON tacker.* TO 'tacker'@'%' \
      IDENTIFIED BY 'TACKER_DBPASS';
        Replace ‘TACKER_DBPASS’ with a suitable password.
    • Exit the database access client.
  2. Source the admin credentials to gain access to adminonly CLI commands:source admin-openrc.sh
  3. To create the service credentials, complete these steps:
    • Create the tacker user:
    • openstack user create --domain default --password <PASSWORD> tacker
    • Replace <PASSWORD> with a suitable password
    • Add the admin role to the tacker user:
    • openstack role add --project services --user tacker admin
      Note: Project_name can be service or services. Verify the project_name under [keystone_authtoken] section in the /etc/nova/nova.conf file.
    • Create the tacker service
    • openstack service create --name tacker --description "nfv-orchestration" servicevm
    • Create the tacker service API endpoints:
      • openstack endpoint create --region RegionOne <Service Type or Service ID> public http:// <TACKER_NODE_IP> :8888
        openstack endpoint create --region RegionOne <Service Type or Service ID> admin http:// <TACKER_NODE_IP> :8888
        openstack endpoint create --region RegionOne <Service Type or Service ID> internal http:// <TACKER_NODE_IP> :8888
  4. Clone tacker repositorygit clone -b stable/liberty https://github.com/openstack/tacker
  5. 5. Install all requirements. The requirements.txt file contains a set of python-packages required to run Tacker-Servercd tacker   sudo pip install -r requirements.txt 
    Note: If OpenStack components mentioned in pre-requisites section have been installed, the below command would be sufficient. cd tacker   sudo pip install tosca-parser
  6. Install tackersudo python setup.py install
  7. Create ‘tacker’ directory in ‘/var/log’
    Note:The above referenced path ‘/var/log’ is for Ubuntu and may be different for other Operating Systems.sudo mkdir /var/log/tacker
  8. Edit tacker.conf to ensure the below entries:
    Note:

      1. In Ubuntu 14.04, the tacker.conf is located at /usr/local/etc/tacker/ and below ini sample is for Ubuntu and directory paths referred in ini may be different for other Operating Systems.
      2. Project_name can be service or services. Verify the project_name in [keystone_authtoken] section in the /etc/nova/nova.conf file.

    [DEFAULT]
    auth_strategy = keystone
    policy_file = /usr/local/etc/tacker/policy.json
    debug = True
    use_syslog = False
    state_path = /var/lib/tacker
    ...
    [keystone_ authtoken]
    project_name = services
    password = <TACKER_SERVICE_USER_PASSWORD>
    auth_url = http://<KEYSTONE_IP&gt;:35357
    identity_uri = http://<KEYSTONE_IP&gt;:5000
    auth_uri = http://<KEYSTONE_IP&gt;:5000   ...
    [agent]
    root_helper = sudo /usr/local/bin/tacker-rootwrap
    /usr/local/etc/tacker/rootwrap.conf
    ...
    [DATABASE]
    connection =
    mysql://tacker:<TACKERDB_PASSWORD>@<MYSQL_IP>:3306/tacker?charset=utf8
    ...
    [servicevm_nova]
    password = <NOVA_SERVICE_USER_PASSWORD>
    auth_url = http://<NOVA_IP&gt;:35357
    ...
    [servicev m_heat]
    heat_uri = http://<HEAT_IP&gt;:8004/v1

  9. Populate Tacker database:
    Note:The below command is for Ubuntu Operating System/usr/local/bin/tacker-db-manage --config-file /usr/local/etc/tacker/tacker.conf upgrade head

Install Tacker client

  1. Clone tacker-client repository.cd ~/
    git clone -b stable/liberty https://github.com/openstack/python-tackerclient
  2. Install tacker-client.cd python-tackerclient
    sudo python setup.py install

Install Tacker horizon

  1. Clone tacker-horizon repository.cd ~/
    git clone -b stable/liberty https://github.com/openstack/tacker-horizon
  2. Install horizon module.cd tacker-horizon
    sudo python setup.py install 
  3. Enable tacker horizon in dashboard.
    Note:The below destination path referred is for Ubuntu 14.04 and may change for other Operating Systems.sudo cp openstack_dashboard_extensions/* /usr/share/openstack-dashboard/openstack_dashboard/enabled/
  4. Restart Apache serversudo service apache2 restart

Starting Tacker server

Note:Ensure that ml2_conf.ini as per Step 4 from the pre-requisites section has been configured.
sudo python /usr/local/bin/tacker-server --config-file /usr/local/etc/tacker/tacker.conf --log-file /var/log/tacker/tacker.log &

 

Testing Tacker

Run the following tacker commands to verify whether tacker is working finetacker ext-list
tacker vnf-list
tacker device-list

 

A simple set of vnfd-create, vnf-create and vnf-update commands are shown below.

tacker vnfd-create –name ${VNFD_NAME} –vnfd-file ${VNFD_TOSCA_YAML-FILE}

tacker vnf-create –name vnf-name –vnfd-id ${VNFD_ID}

tacker vnf-update –config “${CONFIG_DATA_YAML} ${VNF_ID}

If command-line tacker works fine, try out Tacker using Horizon (NFV left menu entry)

Now Tacker is ready, start to play !!

Openstack -Delete Bulk Instances

 

Many of operation engineer look for solution how to delete instances in bulk from CLI or CURL call to reduce time effort, here is the filter method to do it, but please make sure that really you want to delete in bulk.

 

Here the best script method which i follows to do,

Source with required tenant, if you do as admin will delete all projects, so make sure of it.

Now run below CLI, which basically lists servers and grep its ID and do nova delete

nova list | awk '$2 && $2 != "ID" {print $2}' | xargs -n1 nova delete

In particular, the solution from dbxs uses the “name” field. If there are multiple instances with the same name, the “nova delete” operation will fali with:

Multiple server matches found for 'c0', use an ID to be more specific.
ERROR: Unable to delete any of the specified servers.

Openstack user data – ssh access of CentOS instance without key

You can use user-data to set a password for the user. When launching an instance you’ll paste this into the user-data:

#cloud-config
ssh_pwauth: True 
disable_root: false 
chpasswd:
  list: |
      user:password 
  expire: false

ssh_pwauth will turn on the ability to use password auth. disable_root will enable the root user. chpassword will setup a password for the user.

It’s not recommended to use this regularly. You should use this for testing. Otherwise you should be using the user + ssh-key.

This will allow you to SSH with a password. It didn’t work on Debian as the options were different in the sshd config, but should work with CentOS.

Gitlab – 502 Bad Gateway Error Troubleshooting

Lets do:

In a perfect world GitLab would now be running perfectly. Unfortunately, GitLab has surprisingly high memory requirements, so on 512MB VPSs it often chokes on the first sign in. This is because GitLab uses a lot of memory on the very first login. Since the Ubuntu 12.04 VPS has no swap space when the memory is exceeded parts of GitLab get terminated. Needless to say GitLab does not run well when parts of it are being unexpectedly terminated.

The easiest solution is just to allocate more memory to your VPS, at least for the first sign in. If you don’t want to do that, another option is to increase swap space. DigitalOcean already has a full tutorial on how to do this available here (although I would recommend adding more than just 512MB of swap). The quick fix is to run the following:

sudo dd if=/dev/zero of=/swapfile bs=1024 count=1024k
sudo mkswap /swapfile
sudo swapon /swapfile

Your swapfile is now running and active, but to set it so that it’s activated on each boot we need to edit/etc/fstab:

sudo nano /etc/fstab

Paste the following onto the bottom of the file:

/swapfile       none    swap    sw      0       0 

Now restart your VPS:

sudo restart

Wait a minute or two for your VPS to reboot, and then try GitLab again. If it doesn’t work the first time, refresh the Bad Gateway page a couple of times, and you should soon see the GitLab login page.

References:

  1. Check out the excellent Gitlab installation documentation here.
  2. And for info on the 502 bad gateway error, check this thread.

Docker – Command line cheat sheet

Early post Docker for beginners ,we talked about how to start with dockers

Here are few basic commands which we need on daily base work case with docker

Useful Docker Commands helpful for operation

1) To Build Docker image from Dockerfile

      # docker build -t thegeeklinux:1.0 .
-t for Image name and tag name:tag (If you will not give tag it will tag latest)

2) To list images :
      # docker images

3) To save docker image in tar file :
      # docker save -o /tmp/thegeeklinux.tar thegeeklinux:1.0

4) To delete docker image :
      # docker rmi -f thegeeklinux:1.0
-f forcefully

5) To load docker image from tarball :
      # docker load < /tmp/thegeeklinux.tar

6) To tag docker image :
      # docker tag thegeeklinux:1.0 localhost:5000/thegeeklinux:1.0

NOTE:localhost:5000 should be your private registry ip and port

7) To push image to private or public registry server
      # docker push localhost:5000/thegeeklinux:1.0

NOTE: replace localhost with your private registry hostname or ip.

8) To pull docker image for private or public registry

a) To pull image from private registry server
         # docker pull localhost:5000/thegeeklinux:1.0

NOTE :replace localhost with your private registry hostname or ip.

b) To pull image from public registry
         # docker pull ubuntu

Docker Container Operation

1) To RUN Docker container :
      # docker run -it thegeeklinux:1.0
-i interactive mode
-t attract terminal

      # docker run -d --name thegeeklinux -p 80:80 -h tgl -l "com.thegeeklinux=1.0" -e WORKDIR=/opt/thegeeklinux thegeeklinux:1.0

-d Detach it will run container in backgraund.
-e set environment for container
-h Hostname you can give hostname to container also
-l we can give label also to container for indentification.
-p we can open port to host from container
–name Name of the container

2) To Run command inside docker container
      # docker exec thegeeklinux ls

3) Login to container without ssh :
      # docker exec -it thegeeklinux bash

4) To check container list :
      # docker ps

5) To check container memory and cpu.
      # docker stats thegeeklinux

6) To copy contant from container to host machine or host machine to container
      # docker cp ./thegeeklinux.war tgl:/opt/tomcat/current/webapps/

7) To check container related logs :
      # docker logs -f thegeeklinux
-f It will follow log output

8) To commit container :
      # docker commit -p -a "thegeeklinux@gmail.com" -m "commit message" tgl tgl:2.0
-a Author of the container
-m commit message
-p pause container during commit

9) To export container in compressed file
      # docker export -o /tmp/dockerexport.tar thegeeklinux

NOTE : export command will not export attached volume.

10) To get container information
      # docker inspect thegeeklinux

NOTE: it will give info in json format

To get ip addres of container
       # docker inspect -f {{.NetworkSettings.IPAddress}} thegeeklinux

To get mapped port to host :
       # docker inspect -f '{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}' thegeeklinux

11) To Stop container:
      # docker stop thegeeklinux

12) To start container :
      # docker start thegeeklinux

13) To get container processes
      # docker top thegeeklinux

14) To rename the container :
      # docker rename thegeeklinux tgl

15) To restart container :
      # docker restart docker-scripts

16) To delete container :
      # docker rm -vf thegeeklinux
-v delete volume also
-f delete container forcely

Ceph with devstack – part-1

Today going to see how to integrate ceph with devstack and mapping ceph as backend for nova, glance, cinder.

Ceph is a massively scalable, open source, distributed storage system. Ceph is in the Linux kernel and is integrated with the OpenStack cloud operating system.

 

Setup Dev Environment

Install OS-specific prerequisites:

sudo apt-get update
sudo apt-get install -y python-dev libssl-dev libxml2-dev \
                        libmysqlclient-dev libxslt-dev libpq-dev git \
                        libffi-dev gettext build-essential

Exercising the Services Using Devstack

This session has only been tested on Ubuntu 14.04 (Trusty), if you don’t have create on Virtual box with 4GB RAM, 100 GB HDD.

Clone devstack:

# Create a root directory for devstack if needed
sudo mkdir -p /opt/stack
sudo chown $USER /opt/stack

git clone https://git.openstack.org/openstack-dev/devstack /opt/stack/devstack

We will run devstack with minimal local.conf settings required to enable ceph plugin along with nova & heat, disable tempest, horizon which may slow down other services  here your localrc file

#[[local|localrc]]
#######
# MISC #
########
ADMIN_PASSWORD=admin
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
#SERVICE_TOKEN = <this is generated after running stack.sh>
# Reclone each time
#RECLONE=yes
# Enable Logging
LOGFILE=/opt/stack/logs/stack.sh.log
VERBOSE=True
LOG_COLOR=True
SCREEN_LOGDIR=/opt/stack/logs
#################
# PRE-REQUISITE #
#################
ENABLED_SERVICES=rabbit,mysql,key
#########
## CEPH #
#########
enable_plugin devstack-plugin-ceph https://github.com/openstack/devstack-plugin-ceph
# DevStack will create a loop-back disk formatted as XFS to store the
# Ceph data.
CEPH_LOOPBACK_DISK_SIZE=10G
# Ceph cluster fsid
CEPH_FSID=$(uuidgen)
# Glance pool, pgs and user
GLANCE_CEPH_USER=glance
GLANCE_CEPH_POOL=images
GLANCE_CEPH_POOL_PG=8
GLANCE_CEPH_POOL_PGP=8
# Nova pool and pgs
NOVA_CEPH_POOL=vms
NOVA_CEPH_POOL_PG=8
NOVA_CEPH_POOL_PGP=8
# Cinder pool, pgs and user
CINDER_CEPH_POOL=volumes
CINDER_CEPH_POOL_PG=8
CINDER_CEPH_POOL_PGP=8
CINDER_CEPH_USER=cinder
CINDER_CEPH_UUID=$(uuidgen)
# Cinder backup pool, pgs and user
CINDER_BAK_CEPH_POOL=backup
CINDER_BAK_CEPH_POOL_PG=8
CINDER_BAKCEPH_POOL_PGP=8
CINDER_BAK_CEPH_USER=cinder-bak
# How many replicas are to be configured for your Ceph cluster
CEPH_REPLICAS=${CEPH_REPLICAS:-1}
# Connect DevStack to an existing Ceph cluster
REMOTE_CEPH=False
REMOTE_CEPH_ADMIN_KEY_PATH=/etc/ceph/ceph.client.admin.keyring
#####################
## GLANCE – IMAGE SERVICE #
###########################
ENABLED_SERVICES+=,g-api,g-reg
##################################
## CINDER – BLOCK DEVICE SERVICE #
##################################
ENABLED_SERVICES+=,cinder,c-api,c-vol,c-sch,c-bak
CINDER_DRIVER=ceph
CINDER_ENABLED_BACKENDS=ceph
###########################
## NOVA – COMPUTE SERVICE #
###########################
ENABLED_SERVICES+=,n-api,n-crt,n-cpu,n-cond,n-sch,n-net
#EFAULT_INSTANCE_TYPE=m1.micro
#Enable heat services
ENABLED_SERVICES+=,h-eng h-api h-api-cfn h-api-cw
#Enable Tempest
#ENABLED_SERVICES+=tempest’ inside ‘local.config

Now run

~/devstack$ ./stack.sh

Devstack will clone with master & ceph will be enabled & mapped as backend for cinder, glance & nova with PG pool size 8, can create own size in multiples of 2 power like 64 as your wish.

Sit back a while to clone devstack and get result as like below

 

=========================
DevStack Component Timing
=========================
Total runtime 2169
run_process 26
apt-get-update 52
pip_install 99
restart_apache_server 5
wait_for_service 20
apt-get 1653
=========================
This is your host IP address: 10.0.2.15
This is your host IPv6 address: ::1
Keystone is serving at http://10.0.2.15/identity/
The default users are: admin and demo
The password: admin

 

Check the health of ceph with root permission, see “HEALTH_OK”

pandy@malai:~/devstack$ sudo ceph -s
cluster 6f461e23-8ddd-4668-9786-92d2d305f178
health HEALTH_OK
monmap e1: 1 mons at {malai=10.0.2.15:6789/0}
election epoch 1, quorum 0 malai
osdmap e16: 1 osds: 1 up, 1 in
pgmap v24: 88 pgs, 4 pools, 33091 kB data, 12 objects
194 MB used, 7987 MB / 8182 MB avail
88 active+clean

Here you go, ceph is installed with devstack