Make HTTP Requests using Python + httplib

Python has very good package called httplib, which can send GET,POST,PUT,DELETE methods.

while dealing with REST API's you may want to test the API's.

Examples Here include Registering the User using POST .

POST /user creates user.
GET /user returns the list of users.
GET /user/uid return user details.
PUT /user/uid modifies the user details.
DELETE /user/uid deletes the user from server.

I am providing pseudo code here.

import httplib

# This creates the connection with server running on 5000 port.

h = httplib.HTTPConnection('127.0.0.1', 5000) 

def do_post():

   '''POST here'''

  hdrs = {'mobile':mobile,'email':email}

  h.request('POST', '/user', None, hdrs)

  resp = h.getresponse()

  print(resp.read())

  print(resp.status) # This prints HTTP status code.

    

def test_get():

  hdrs = {'mobile':mobile,'email':email}

  h.request('GET', '/user', None, hdrs)

  resp = h.getresponse()

  print(resp.read())

  print(resp.status) # This prints HTTP status code.

def test_put():

  hdrs = {'mobile':mobile,'email':email}

  #passing known user id 1234 here, you can change for your use case.

  h.request('PUT', '/user/1234', None, hdrs)

  resp = h.getresponse()

  print(resp.read())

  print(resp.status) # This prints HTTP status code.

we can further improve this program making multi-threaded/multi-processing and use it to test scalability and performance of your server.

Run Python + Flask Web app in Docker Container

Do you want to Dockerize your Python Flask web app. Well follow these steps. very often there will be difference of packages installed on Dev vs tester systems.
So running webapp using Docker can be easy for testing.
Follow this 10 step simple tutorial.

1. Create a Virtual Environment first .

# virtualenv dockerC
# cd dockerC
# source bin/activate

2. Install Flask

# pip install Flask

3. Install any other packages u want eg: memcached.

# pip install python-memcached

4. collect the requirements, so that we package all modules.

# pip freeze > requirements

5. create a DIR with webapp. move the requirements file to that dir.

# mkdir webapp

 #mv requirements webapp/

6. work on your Webapp using Flask.
app.py has code for entry points.

#cat app.py
from flask import Flask
app = Flask(__name__)


@app.route("/")
def get():
    return "Hello Docker"

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0', port=8000)

7. Now create Docker File called "Dockerfile" in same webapp dir
The contents has to be like this.

FROM fedora:latest
MAINTAINER Maintainer redara
RUN dnf install -y python python-pip
COPY .   /webapp
RUN pip install -r /webapp/requirements
EXPOSE 8000
WORKDIR /webapp
CMD python app.py
#eof

The Structure of our webapp dir is here.

(dockerC) [vedara@localhost webapp]# ls -ltr
total 12
-rw-r--r--. 1 vedara vedara 151 Apr 10 16:18 requirements
-rw-r--r--. 1 vedara vedara 180 Apr 10 16:46 app.py
-rw-r--r--. 1 vedara vedara 187 Apr 10 16:49 Dockerfile

description of Docker file is here
FROM uses the fedora image from docker hub.
RUN means we are installing required packages . here we are installing python,pip first
COPY .   /webapp means we are copying our current dir(. dir) which is "webapp" to /webapp dir in destination
once we copied our webapp dir, lets install required packages for Flask using RUN pip install -r /webapp/requirements
EXPOSE will expose the port on which container listens to connections. our webapp uses 8000 as port.
WORKDIR sets the working dir , so that you dont need to specify absolute paths everytime
CMD python app.py shows the cmd needed to run the flask web app.

8. Lets Build the Docker Image now. give -t which can be used to refer your image. "." means looks for Dockerfile in curr dir.

# docker build -t flask_webapp .

9. Lets run the Docker Container now by using the cmd here. giving -p tells docker to publish the containers port to host.

   #docker run  -p 8000:8000 flask_webapp 

  If you want to run as daemon specify -d option in above cmd.

10. Lets hit the URL now using curl

  #curl -i http://0.0.0.0:80

 HTTP/1.0 200 OK
 Content-Type: text/html; charset=utf-8
 Content-Length: 12
 Server: Werkzeug/0.12.1 Python/2.7.13
 Date: Mon, 10 Apr 2017 12:15:39 GMT
 Hello Docker

Thats it. your web app is running. Push it your Docker Hub using your username/password

#docker push username/tagname

Note: This demo is based on Just "Hello Docker". copy your entire webapp files using COPY . if you use database you can ship different container for Database server or Ship in Same Container.

Use the Dockerfile to copy required packages and run in same container.

Contact me for any queries: smart.ram856@gmail.com

Use C Library In Python using C Types

Do you have a C library which needs to be used for Python Devs. well, you can have python bindings which loads the C library and use those functions using C types.

Here I am giving small demo, my C library has add and multiply functions like this.
demo.c
=======

#include

int add(int,int);

float mul(float,int);

int add(int a,int b)

{

return a+b ;

}

float mul(float a,int b)

{

return a*b ;

}

Lets Compile this program as Shared Library

 # gcc -c -Wall -Werror -fpic demo.c 

The above step produces demo.o, Lets have shared lib now

On Linux

 # gcc -shared -o libcalc.so demo.o

On Mac

# gcc -shared -o libcalc.dylib demo.o

Lets Use this libcalc.so or libcalc.dynlib in Python now, using the C types. 

api.py

=========================

import ctypes

from ctypes.util import find_library

from ctypes import sizeof

'''this will find library name . In Linux you have libcalc.so , in Mac libcalc.dynlib'''

so_file_name = find_library("calc")

if so_file_name is None:

    raise Exception("libcalc.so not found")

try:

    client = ctypes.CDLL(so_file_name, ctypes.RTLD_GLOBAL, use_errno=True)

except OSError:

    raise ImportError("cannot load {0}. please set LD_LIBRARY_PATH \                              variable".format(so_file_name))

def calc_func(method,rettype,*argtypes):

    '''

    create a func belonging to calc.so

    '''

    return ctypes.CFUNCTYPE(rettype,*argtypes,use_errno=True)((method,client))

''' Define python functions for each C function in libcalc.so  

we have add,mul functions in our libcalc.so.  '''

py_add = calc_func('add',ctypes.c_int, ctypes.c_int,ctypes.c_int)

py_mul = calc_func('mul',ctypes.c_float,ctypes.c_float,ctypes.c_int)

Now we have api.py which loads the C library and bridges to python.

main.py which uses api.py in python style

==================

import api

add_output = api.py_add(2,3)

print(add_output)

mul_output = api.py_mul(float(5.5),10)

print(mul_output)

#python main.py 

Note: What if you have C++ library, well its better you use extern "C" and write a C wrapper which intern calls C++.

Generate subsets of an array using Python

Problem: If you given a problem to generate subsets of given array , we can use PowerSet concept.

lets say Array = ['Red','Orange','Blue']

the power set is defined as 2 to power of len(Array). 2^3 here.

Run the binary counter from 0 to pow(2,len(Array))

Value of Counter            Subset
    000                    -> Empty set
    001                    -> Red
    011                    -> Red,Orange
   100                     -> Blue
   101                     -> Red,Blue
   110                     -> Orange,Blue
   111                     -> Red,Orange,Blue

def gen_subsets(list_attribs):
    len_set = len(list_attribs)
    power_set = 2 ** len_set
    for i in range(power_set): # this generates the binary sequence
        subsets = ""
        for j in range(len_set): #iterate through the list_attribs positions and check bit is set
            if i & (1 << j):
                subsets +=  list_attribs[j] + ","
        print(subsets[0:-1])
        

coming up with social content site

all these days busy in building up new social content site. using python django as my framework.

learnt many things abt software project management. Things I have learnt are

1) you will never finish enhancements . once u come up with idea and do coding daily...new ideas come up and its upto you to decide which features to ship to satisfy your clients.

2) things you think easy may take more time..and some other things you  think hard may get finished
in lesser time.

3) modules that are not doable by you can be outsourced but not to cost of wasting money and time.

4) for any kind of product , people hate the word 'slow'. along with functional requirements make sure performance requirements are fulfilled.

5) technical things I have learnt are .
a) python , django framework
b) hitting db always doesnt scale. so used memcached for caching
c) divide project into subproblems..abstract the core module so that other developers can work on that and enhance the project later.
d) write wrapper for modules which are used frequently rather than duplicating the code.

and finally
e) do not re-invent the wheel again just to show that you can code complex things...I came across open source project memcached for using in memory cache to save DB hits. it saved lot of time for my project and many famous social portels like facebook use it. so its okay for me to use memcached as my in-memory cacheing

finally looking ahead for front end developer who can add colors to my website and look it attractive like heroine ileana :) lolz just kidding .

will blog again once I complete my social content site :) .