bulkanix

Running QTP tests using Python



QTP provides an interface called the automation object model. This model is essentially a COM interface providing a bunch of objects that can be used to automate QTP. The full object list is available in the QuickTest Professional Automation documentation.

Running QTP tests from the command line is useful for doing scheduled automatic testing. If you use a continuous integration system to do automatic builds of your software, you can run your QTP tests on the latest build.

The following is a Python script that is able to run a test and print out Passed or Failed. It is a direct port of example code in the documentation written in VBScript

My Own Django Blog



I have plans to move to using a Django blog that i can customize myself. I don't know if i should go down the route of implementing my own or using one that exist. Here is a list of Blogging applications written in Django. What i need is something

  • easily skinable allowing me to change the look-and-feel to the way i want it, something green and minimalistic
  • someway of transferring all the posts here to that app
  • Tagging
  • Allowing comments on posts
Seems like Banjo has all the things i want.

Maybe Blogmaker aswell. Even though Byteflow has Threaded comments it restricts comments to only registered users which i don't want.

And i may look into django-diario

But i will just play with Banjo now.

EDIT: To play with Django i need to install 5 other packages. I'm adding a new requirement to the blog application i need, easily configurable, download change settings.py with correct paths, create database, run syncdb and start server (built in server), start Blogging. I don't want to go hunting for more packages.
 django

(Ugly) Python type checking



I like Python because of the explicitness of the syntax

def add(a,b):
return a + b


explicitness is good as it really leads to code that is understandable at one glance, but quick, tell me two types that the above function will work on ?

int's and strings




>>> add(1,2)
3
>>> add('hello ','world')
hello world



the above function works on both integers and strings only because both provide the special method __add__ which gets called for the + operator.

So does this lead to implicitness ? Not really, because you should know (from programming) that you can add integers together and concatenate strings together, Python just makes this general across types.

If you wanted to say, restrict the types of the arguments to our add function above, you could do something like the following


def add(a,b):
if type(a)==type(str) and type(b)==type(str):
return a + b


Type checking is kind of ambiguous to me in a dynamic language. If i want to restrict the the ability of a function to only work with certain types or i don't know the types of the object im passing to a function then i have design issues (or no design at all).

You could rewrite the above function to do the type checking using a decorator.

EDIT: i didn't know if the following decorator was written by the original creators or not, but as i was pointed out it wasn't here is the original link

Python Cookbook Recipe

def require(arg_name, *allowed_types):
def make_wrapper(f):
if hasattr<
(f, "wrapped_args"):
wrapped_args = getattr(f, "wrapped_args")
else:
code = f.func_code
wrapped_args = list(code.co_varnames
[:code.co_argcount])

try:
arg_index = wrapped_args.index(arg_name)
except ValueError:
raise NameError, arg_name

def wrapper(*args, **kwargs):
if len(args) > arg_index:
arg = args[arg_index]
else:
arg = kwargs[arg_name]

if not isinstance(arg, allowed_types):
type_list = " or ".join("'"
+ str(allowed_type.__name__) +
"'" for allowed_type in allowed_types)
raise TypeError, "Expected argument '%s' \
to be of type %s but it was of type '%s'." \
% (arg_name, type_list,
arg.__class__.__name__)

return f(*args, **kwargs)

wrapper.wrapped_args = wrapped_args
return wrapper

return make_wrapper

@require('a',str)
@require('b',str)
def add(a,b):
return a+b




>>>add('hello ','world')
hello world
>>>add('hello',1)
Traceback (most recent call last):
File "snippet3.py", line 38, in
print add('hello',2)
File "snippet3.py", line 24, in wrapper
return f(*args, **kwargs)
File "snippet3.py", line 22, in wrapper
raise TypeError, "Expected argument '%s' to be of type %s
but it was of type '%s'." % (arg_name,
type_list, arg.__class__.__name__)
TypeError: Expected argument 'b' to be of type
'str' but it was of type 'int'.



above is code that does type checking on input arguments. I may be wrong and there may be use cases where you need to check the type of an object but the point is you should design your program so that you know all the involved types or use a language that has compile time type-checking.

Note: The above type checking is in a Django application that is live and it has users and i didn't write that decorator.



Python Module Review's



Here is what i will try to do; each week i will try to review one Python module that appears in my feed to the CheeseShop recent updates. I will try to write a mini how-to on the chosen module, rate it based on documentation available, usefulness and alternatives. The first module that i will do the review is for pysysmon 0.0.25

Why...because i can.

EDIT:

Well i downloaded the source code for pysysmon which as a file called main.py. I did and i get the following:

ImportError: No module named gobject

gobject is the GTK+ wrapper for Python and im using a Mac. So either i can try this using Ubuntu (via Parallels) or only write reviews for modules that are OS independent or Mac only. I will try to play around with pysysmon on Ubuntu because it looks interesting.

Recommendation for pysysmon creator, "please update your Cheese Shop page to describe what platforms it runs, what other packages it is dependent on". It is really not
'A curses-based system monitor' but more like 'A curses/GTK based system monitor'.

Never store passwords as clear text



Never store passwords as clear text, that is the general rule with any application that has a database backend that is used for authentication into the system. Why?

Basic authentication with a database usually works by comparing username and password combination that the user entered to values in the database table containing user details such as login name, password etc...It might be possible for a user with correct credentials to be able to inject SQL queries to the application, something like;

SELECT * FROM USERS;

assuming the user can guess or knows the table containing user data.

If passwords are in clear text then lo and behold the users now has access to all other users login name and passwords. Anyway this post is not about security of database backed applications but a post about how i overcame different versions of Python and py-bcrypt's support.

As i've posted before i've been working at a web development company as a Python programmer. It's a Zope 'shop' in the sense that there main application, developed in house a shopping cart system is developed using Zope. Anyway i just developed 'External Scripts' to do specialized stuff for customers. But recently i've been working on a 'time/job tracking' web application using web.py. I've mentioned this in a previous post. So here is the versions of Python,web.py and py-bcrypt that i used to develop the tracker;


  • Python 2.5.1

  • py-bcrypt-0.1 (Python 2.4 or higher)

  • MySQL-5



Anyway, this past week the application was put on a live production server. That has;


  • Python2.3

  • Postgres



I've also used decorators to 'decorate' functions to restrict access to certain URL paths. Take a guess in which version decorators was introduced into Python? You guessed right Python 2.4 got blessed with decorators. Guess what the lead/senior developer did ? He re-wrote most of the code to just use plain function calls instead of decorators...he re-wrote...instead of the simpler solution of installing Python >=2.4. (Converted the decorator to a plain function which is called in all other functions that was decorated with it).

I mentioned above that py-bcrypt requires Python >= 2.4 because it needs the function os.urandom which was introduced into Python 2.4. Oh slap! i've got Python 2.3 so we can't generate hashes...the solution that was suggested to me was...."write/copy urandom" i think that was one of the moments in my not so long professional career that i thought that someone more senior and with more was wrong. I would have instead installed Python 2.5 on the server, which seemed to be the 'path of least resistance'.

It seemed a daunting task. The first step i took was looking at py-bcrypt module. It contains two files;


  • __init__.py

  • _bcrytp.so



You can't edit _bcrytp.so file as it a library file. So i looked at __init__.py, which imports the os module and defines the gensalt function. From my the Python 2.5 installation on my Mac i copied the urandom implementation into __init__.py just above the line where import os . urandom is not that complex , it just tries to open /dev/urandom and reads in n number of bytes and return it. So here is what __init__.py looks like after the changes, its a hack.

25 from os import O_RDONLY,read
26
27 def urandom(n):
28     """urandom(n) -> str
29
30     Return a string of n random bytes suitable for cryptographic use.
31
32     """
33     try:
34         _urandomfd = open("/dev/urandom",'r' )
35     except (OSError, IOError):
36         raise Exception("/dev/urandom (or equivalent) not found")
37     bytes = ""
38
39     while len(bytes) < n:
40         bytes += _urandomfd.read(n-len(bytes))
41         #bytes += read(_urandomfd, n - len(bytes))
42
43     _urandomfd.close()

44     return bytes

45
46 import os

47 os.urandom = urandom
48 from _bcrypt import 


Now py-bcrypt works and passwords are hashable.

The thing that troubles me and is a question on my mind, is it worth the risk to install Python >= 2.4 on a server that contains 'live shops' ? The risk being totally blowing up the default Python installation (2.3) and bringing down the shops ? I would have probably installed a new version of Python and sandboxed it. The irony is that the senior developer was the one who chose py-crypt and told me to come up with a decorator for methods which need to be password protected. I would have thought that with his experience he would have guessed that the request for the app to go online would have come. Also if you are scared to blow the default installation of Python on the production server, WHY PUT AN IN HOUSE APPLICATION THERE?

Exporting a csv file with web.py



This is how you export a csv file and get the browser to recognize that its a csv file and popups the download window with web.py. Lets say we have a database with a table called users and you want to create a csv file that contains all the users with their names and id's here is how you do it.


1 class export:
2     def GET(self):
3         i = web.input()
4
5
6         users = web.select('users', vars=locals())
7
8         csv = []
9         csv.append("id,name\n")
10        for user in users:
11             row = []
12             row.append(user.id)
13             row.append(user.name)
14
15            csv.append(",".join(row))
16         #writer.writerow(row)
17
18         #f.close()
19
20     web.header('Content-Type','text/csv')
21     web.header('Content-disposition', 'attachment; filename=export.csv')
22     print "".join(csv)
23     return




I export the csv file in a GET method of a class called export which i map in the urls list to '/export','export'

A quick breakdown, do a database query and iterate over the IterBetter object create a row and appending a comma seperated string to the csv list. Then at the end you send the appropirate HTTP headers , the first telling the type of the file and the second setting the filename and extension.

Anyway you can download this code from http://bulkanix.pastebin.com/f1f567ea0

embryo.py and py2app awesomeness



At work i created this script that changes permissions on our application BizarShop so that it works with the new Dashboard Widget to control the starting and stopping of of the Zope server. The permissions need to change because the controller that comes with BizarShop starts the Zope server as root, which creates the lock file (Data.fs file if it doesn't exist) with root ownership. The widget on the other hand tries to control the server via normal user permissions, but the server wont start because all the files belong to root and cannot be overwritten. For example the file Z2.pid needs to be writable, so you need to change the ownership to that of the user.

So i created a python script that recursively goes through all directories under /Applications/Bizar Shop and changes all of the file/folder ownership to that of the owner current user. As you probably know to run a python script you either need to run explicitly via;

python scriptname.py

or by including a hash-bang at the start of the file to tell where python is located and then make the script executable. I thought that the user could just double click on an executable python script to run it but i was wrong. I didn't want the user to open Terminal.app and execute it manually, this is just not user friendly. I then remembered py2app. From the README file of py2app

py2app is a Python setuptools command which will allow
you to make standalone Mac OS X application bundles
and plugins from Python scripts.

py2app is similar in purpose and design to py2exe for
Windows.

So using py2app i created this installer, that also includes the widget and embryo.py, oh and embryo.py is also a nice little module, from its Google Code description;

embryo is a tiny Mac/Windows/Linux GUI toolkit for Python. It can be used to "boot-strap" the user into downloading a larger GUI toolkit such as PyGTK, PyGame, pyglet, PyOpenGL, etc.

What i used it for is, my script checks if the folder /Applications/Bizar Shop exists and if it doesn't then it assumes that BizarShop is not installed and then shows a message box saying BizarShop is not installed do you want to download. But if it does find it displays a message box letting the user now know that the Widget is about to be installed and Dashboard opens up a the install widget dialog box.

What is so cool is that py2app is very easy to use and it works! Combining this with embryo you can easily create a quick installation program for literally anything.

Oh, did i mention that embryo is created by Alex Holkner the same guy who is working on pyglet? Well now i did. Here are the links to these modules.

http://code.google.com/p/pyglet/
http://code.google.com/p/pyembryo/
http://svn.pythonmac.org/py2app/py2app/trunk/doc/index.html

Redirecting stdout to StringIO object



So how do you redirect everything written to stdout and store it somewhere but also print it out to stdout? This was asked on #python and i answered it.

To access the stdout file object you need to import the sys module. Redirecting stdout to a StringIO object works because all functions that write to stdout expect the write() method of a file-like object, which StringIO has (along with read, seek etc). So here is the code;

So here is a quick breakdown line by line:


  • Lines 1 and 2 are used to import the required modules.

  • Then we subclass StringIO and create an attribute to hold the reference to stdout.

  • In Line 9 we overwrite the write method of the StringIO baseclass which does only one additional thing of writing back out to the original stdout, then it calls the baseclasses write method to store the string again.

  • Then also overwrite the read method that does one additional thing of seeking to the start of the StringIO object and then writing it all out back to stdout.







Experimenting with Python frameworks and modules



I've been very busy since last semester got a new job as a Python (Zope) Developer. Then quitting from my previous job which i was still considered an intern. Now im in my second and last semester of university and I graduate at then end of this year!

Even though i haven't had free time to actually write a post i've had some of those times were you need to do work but cbf so you end up doing totally random stuff? Well i've had those times a plenty in which i experimented with couple of Python frameworks and modules.

First one i tried was Turbogears. Remember that major project i have were we are using Rails well that's what spurned me on to actually try a Python web application development framework. Oh and also the Python411 Podcast. Well installing Turbogears on a Mac is very easy by following this guide

  • I had Python2.5 already installed and the default version to run when python is run, so i skipped that step.
  • I had to install the easy_install using ez_setup.py
  • Then i downloaded tgsetup.py and ran it which downloads and install everything for you, except for the database wrapper.
  • The database i have installed on my Mac is MySQL 5.0 and the driver for that is MySQLdb. This was the tricky part of the whole installation process of Turbogears. So i downloaded MySQLdb and tried running sudo python setup.py install, but it failed. The solution to this failure was tricky but after a bit of Googling the problem was that setup_posix.py had the wrong path to mysql_config file which is located at /usr/local/mysql/bin/mysql_config on my Mac. Changing the path and running setup.py again worked. Then i tested it by trying to import MySQLdb from Python again and that seemed to work aswell (no Exceptions).
  • Then to actually test to see if Turbogears was correctly installed i ran this command tg-admin quickstart and entered a name for the project and package name (in this case it was wiki, creating wiki's with web application development frameworks seems to be the Hello World program of languages). Then started the built-in webserver by running the script start-wiki.py and accessed it via http://localhost:8080/ and i got the "Welcome to TurboGears" page.
  • I haven't done anything else with Turbogears as i was interested in the installation process.

AUC Python Objective C Workshop




Thursday - Day 1;

It was 29C today, very nice weather. I arrived in Sydney yesterday. The plane trip was good, compared to the fact that i kept on thinking it would be a 45 minute ride on a Mad Mouse, but it wasn't. Other than two or three "major" air pockets or turbulence, it was a good flight. Maybe watching Heroes also helped. (I did it!)

Yesterday, i just came down to my hotel checked-in left my bag and then went scouting for Cliftons Training Centre. It wasn't that hard to find, a train ride to Circular Quay Station, just a couple of minutes of walking down George St to the left side. I did this because if i was going to get lost i would have done that yesterday instead of today. I didn't get lost. The cool thing is that the centre is close to Sydney Harbor Bridge and the Opera House. So i walked a bit and took some photos with my N73. Then went back to my hotel.



The next day (today,May 3rd) I woke up at 7:30am. After i got ready i went out to Central Station. I left early as i didn't have any wireless connection in my hotel. I went out looking for a cafe where they had wireless next to the training centre. Chance had it i found Starbucks only to realise that to access the internet you need a prepaid card or a credit card and i had neither. By the time i drank my Iced Cafe Latte and it was time for me to head to the centre.

Walking into the centre going up the lift (i hate the things by the way) asking for the direction to the room. Then stepping into the room. Our tutor is James Bekkama a PhD student at CSU. After all the six students arrived it started.

The usual introduction started, with everyone saying where they are from and experience with Python and/or Objective C. Me and another student were the only two with experience, but me being the only one who was new to the Mac scene. After the introduction we where asked to install Xcode if we haven't already. I had already installed Xcode and even PyObjC. After all of us was up and running (except for one person who did some weird thing with his Mac moving /etc somewhere and aliasing it, Xcode not liking, he had to re-install Mac OS X!).

The first day was all Python. Starting off with the basics of the language, such as it being dependent on indentation for code blocks which is always a let down for people who are coming from C/C++, Java etc. Using the interactive shell, variable assignment, retrieving input from the user. Then we moved on to using Xcode for Python development. This was needed as PyObjC includes templates for Xcode which eases the development process. Setting up Xcode for Python had some steps to follow but not hard to do but easy to forget. After setting up Xcode we moved on to general language syntax. After this we started writing up a Address Book script. Starting off we a basic function based script then converting it into a more cleaner Object Oriented script.

Before i forget to mention we had morning tea during lunchtime, and lunchtime during afternoon tea. It was good fun. After lunch we started on a CGI script to create a web interface for adding entries into the Address Book. This was interesting because i couldn't get Python CGI scripts to run under Apache that is pre-installed on my MacBook. I was trying to get mod_python installed instead of just using the magic hash bang thing!

If i admit it, the first day was a bore, except for the CGI part. James does know his quite a bit about CGI (Python,Objective C in general as well) scripting, but i was more interested in the Objective C bridge. To learn to develop applications that look like "real" Mac applications but programmed with Python with no need to learn Objective-C.

Friday - Day 2;

Second day i didn't wake up that early as there is trains more or less every 2-8 minutes that goes around the City Loop in Sydney. So i arrived at the training centre ten minutes earlier for a quick email and Digg.com check (10 minutes isn't enough for a Digg ;) ) .

We started by configuring a new PyObjC project on Xcode. Again there were some steps we needed to follow. Then the fun began, with a simple example.

>>> import AppKit
>>> AppKit.NSBeep()

You can guess that all this does is a "beep" sound, very useful just like the classic "Hello World" example. No seriously, it is an easy way of checking if PyObjC is correctly installed.

Keep in mind that i've had no previous experience with Objective-C. When i got my MacBook i said to myself "might as well learn the native development framework and language. So i looked at some tutorials on Objective-C and some screen cast on YouTube about Objective-C development. When i saw NSObject or NS* i was like "what is NS", then NIB files. It was just too many things to learn and again university was like "here more work for you, we don't really want you to have a social life". So i abandoned my sojourn into Cocoa.

Now thanks to this workshop i now know what NS and NIB are. Next Step and Next Interface Builder respectively. Once everyone confirmed that PyObjC was installed and working we then started looking at the syntax mapping between Objective-C and Python. What i mean by syntax mapping is this. The way to call a method of an object in Objective-C is done like the following;

[aObject doThis:art1 withThis:arg2];

and in PyObjC this would be translated to

aObject.doThis_withThis_(arg1,arg2)

i like this better. No square brackets or anything, except the underscores, which are needed to represent the colon's in Objective-C. (Colon's in Python are used to denote the beginning of a block).

So converting any Objective-C API call to PyObjC is easy, whenever you see a colon you replace it with an underscore in Python. Here is another example of creating NSString objects in PyObjC

myString = NSString.stringWithString_(u"Hello World")

instead of a very ugly looking Objective-C version

myString = NSString.alloc().initWithString_(u"Hello World")

After this we had morning tea. Once back we started working on a Aqua interface for the Address Book we created yesterday. I must admit the Interface Builder is a very handy tool. You add your buttons and then create outlets and connect them VISUALLY (holding down ctrl?). We worked on this until lunch then we came back to more fun stuff, Bluetooth.

PyObjC can directly control Bluetooth via the NSBluetoothIO (if i remember correctly Xcode is crashing for some reason) but we just used a framework, lightblue. To do a device scan of bluetooth devices all you need to do is this,

>>> import lightblue
>>> lightblue.finddevices()

making sure Bluetooth is switched on. This call returns a list containing all devices with Bluetooth switched on and in Discoverable mode, with their MAC addresses. Using this information it is very easy to create some sort of Bluetooth proximity detector, which James did, continuously search for devices, and see if the device your looking for is in range, meaning that it was detected.

Then it was all over and time to head back. Everyone said their goodbye's and nice to "meet you". I made some good connections into the Mac development world.



CSS Layout by Rambling Soul