॥ स्वक्ष ॥

To content | To menu | To search

Tag - Systers

Entries feed - Comments feed

2010 July 26 [Monday]

Patches and the Packaging toolchain

Background assumptions::
o. A clean ubuntu hardy installation.
o. Some familiarity with command-line basics.
o. Have installed pbuilder.

That done, we can continue...

What is PACKAGING?

If you have ever used the apt-get or aptitude {Aaron has a nice writeup on aptitude Vs. apt-get} command : "sudo aptitude install <packagenames>"  on your Debian/Ubuntu system you used a tool that automatically installs a packaged software on your system. This would automatically pull all the dependencies for your Linux system to run smoothly. It saves you the pain of having to install software manually by pulling code in a tar.gz folder, extracting, etc...

Secondly, when you download a source package from a Debian/Ubuntu "deb-src" repo, its already a debian package.i.e the source files have been placed in a suitable directory tree, appropriate configuration files have been added, and the package contains scripts to help with the building and installation of the package. This is what allows you to just install from source with "apt-get -b source some-package-name" as mentioned before.

This post is different. As in, its not a path of least resistance (packaging an existing debianized package for Ubuntu). This is about creating a binary package from a source package, with customization, modification, or other tedious trouble. To do so, you do the following :

Wikipedia says,

The Debian build toolchain is a collection of software utilities used to create Debian source packages (.dsc) and Debian binary packages (.deb files) from upstream source tarballs.

These tools are used in the Debian project and also in Debian-based distributions such as Ubuntu.

One of the first steps was creating a .dsc as outlined at  https://wiki.ubuntu.com/PackagingGuide/Python

I had already pulled the stable source from LP [bzr branch lp:systers] but noticed that none of the files had copyright or license notices. I also had a number of questions on patching the current systers stable branch for releases and another part of the project as mentioned on the wiki was to work with the MM 3.0 port team. So I requested an upstream MM core-dev to comment on the following:

  • Q0. Currently systers runs only on Ubuntu-Hardy with python2.5 and systers has the habit of working only with ubuntu- LTS releases, hence Lucid could be the porting target.  I feel the tasks involved are -- convert  all .py programs compatible to python  2.6 or 2.7;  carry out code changes if any on the 2.6 or 2.7 .py files to make them work on lucid or Hardy , make package and test the package.
  • Q1. Currently systers code uses python2.5 and I'd like to know if it has to run/port compatibly on Lucid would that involve converting code to python2.7 (or python2.6)? That seems like a large task so I'd like to hear your comments. 
  • Q2. Would it involve making these python programs of systers which is currently compatible with 2.1.10 to be made compatible to mailman 3.0-- I'm not sure how different MM-2.1.10 is from MM-3.0 and assume this involves changes in code.  So what does working with the MM 3.0 port team entail?
  • Q3. According to the debian manuals the existing stable branch should contain licences for all files, to be packaged since GNU-Mailman would require this. Currently none of the stable branches or code fixes pulled from the systers repo have any license files. Am I supposed to contact the individual authors (I dont know them all) or should I just ask Jennifer/Robin ?

The reply I got was lucid (could not resist a pun) and in sum, I'd heard some of the packaging stuff before (in Debian? or was it in Ubuntu?):

  • o. Core-devs dont usually work on maintaining non-mainline stuff unless it's their own. So if you want your "feature-changes-patch" included in upstream, you'd have to be willing to maintain them long-term and that would involve a large commitment. If its not in mainline upstream, there is nobody to coordinate packaging or patches. As the developer of your "feature-changes-patch", its your baby which you should be willing to test and maintain each time mainline changes.
  • o. Ubuntu packaging of Mailman 2.1 introduces some bugs -- best compiled from source. Bringing another package (mailman-systers?) for an existing upstream package within Ubuntu could be an issue.
  • o. Mailman 2.1 and Mailman 3.0 are *entirely* different code bases. That means these wonderful systers feature changes would have to be re-implemented, not ported. Porting (as I asked above in Q0, Q1,Q2) would involve moving between python versions and this changes so many things when you are moving between two OS releases (in this case, Ubuntu hardy and lucid). Besides MM3.0 (in alpha stage) was an entirely different code base compared to 2.1 which is in feature freeze.
  • o. While it would be better to contribute code directly to MM3.0 ...BUT... using the existing code raises the license issues in Q3 above. This is something I cant answer as "legalese" is beyond me. Technically, all GPL code (for a FLOSS project) requires authors be willing to assign their work to the free software foundation, as explained in the gnu-licenses page: http://www.gnu.org/licenses/why-assign.html. IIRC, Ubuntu and Debian would require the license.txt file for packaging too. There is a list of FSF approved software licenses.

2010 July 25 [Sunday]

Selenium RC and Python client

Selenium is quite a famous testing tool and has a lot of documentation so I wont bother to repeat stuff from there, rather just focus on the bits where I feel the documentation could improve.

STEP#0.  After downloading selenium remote-control change to the directory where you extracted it. {Btw, to install selenium on your local machine, download selenium from here.}

~$ cd selenium-remote-control-1.0.3/
~/selenium-remote-control-1.0.3/selenium-server-1.0.3$ sudo java -jar selenium-server.jar
09:03:05.618 INFO - Java: Sun Microsystems Inc. 1.6.0_0-b11
09:03:05.638 INFO - OS: Linux 2.6.24-28-generic i386
09:03:05.722 INFO - v2.0 [a2], with Core v2.0 [a2]
09:03:06.304 INFO - RemoteWebDriver instances should connect to: http://192.168.1.5:4444/wd/hub
09:03:06.307 INFO - Version Jetty/5.1.x
09:03:06.311 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
09:03:06.324 INFO - Started HttpContext[/selenium-server,/selenium-server]
09:03:06.324 INFO - Started HttpContext[/,/]
09:03:06.381 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@12d15a9
09:03:06.382 INFO - Started HttpContext[/wd,/wd]
09:03:06.399 INFO - Started SocketListener on 0.0.0.0:4444
09:03:06.400 INFO - Started org.openqa.jetty.jetty.Server@228a02

I wanted to run the python script from the console and each time it would stop here and I'd be waiting and nothing would happen.....How do I run the script without a command prompt. So then, I'd interrupt it with 'ctrl C' to get a  "09:03:17.439 INFO - Shutting down...12:39:08.573 INFO - Stopping Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=4444]

That "shutting down" message was odd -- how would you run a process if it was shutting down? What the logs above dont say is "The selenium server must be running so keep that process open. Open ANOTHER terminal window and run your python script there". The documentation didnt explicitly mention that localhost must keep the server running in the background-- Its one of the most basic client-server concepts but when you are running stuff on localhost, your client and server are one and the same. Being explicit with this trivia in the documentation would have helped me not waste hundreds of hours searching the selenium website and reading irrelevant blogs which google threw up, irc, emails, etc... So, in TerminalOne,


:~/selenium-remote-control-1.0.3/selenium-server-1.0.3$ sudo java -jar selenium-server.jar
09:03:05.618 INFO - Java: Sun Microsystems Inc. 1.6.0_0-b11
09:03:05.638 INFO - OS: Linux 2.6.24-28-generic i386
09:03:05.722 INFO - v2.0 [a2], with Core v2.0 [a2]
09:03:06.304 INFO - RemoteWebDriver instances should connect to: http://192.168.1.5:4444/wd/hub
09:03:06.307 INFO - Version Jetty/5.1.x
09:03:06.311 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
09:03:06.324 INFO - Started HttpContext[/selenium-server,/selenium-server]
09:03:06.324 INFO - Started HttpContext[/,/]
09:03:06.381 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@12d15a9
09:03:06.382 INFO - Started HttpContext[/wd,/wd]
09:03:06.399 INFO - Started SocketListener on 0.0.0.0:4444
09:03:06.400 INFO - Started org.openqa.jetty.jetty.Server@228a02
09:03:17.439 INFO - Shutting down...
mom@drga:~/selenium-remote-control-1.0.3/selenium-server-1.0.3$ sudo java -jar selenium-server.jar
09:03:36.248 INFO - Java: Sun Microsystems Inc. 1.6.0_0-b11
09:03:36.252 INFO - OS: Linux 2.6.24-28-generic i386
09:03:36.270 INFO - v2.0 [a2], with Core v2.0 [a2]
09:03:36.486 INFO - RemoteWebDriver instances should connect to: http://192.168.1.5:4444/wd/hub
09:03:36.489 INFO - Version Jetty/5.1.x
09:03:36.491 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
09:03:36.493 INFO - Started HttpContext[/selenium-server,/selenium-server]
09:03:36.493 INFO - Started HttpContext[/,/]
09:03:36.523 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@12d15a9
09:03:36.523 INFO - Started HttpContext[/wd,/wd]
09:03:36.533 INFO - Started SocketListener on 0.0.0.0:4444
09:03:36.533 INFO - Started org.openqa.jetty.jetty.Server@228a02
09:04:30.236 INFO - Checking Resource aliases
09:04:30.260 INFO - Command request: getNewBrowserSession[*firefox, http://localhost:4444, ] on session null
09:04:30.283 INFO - creating new remote session
09:04:30.614 INFO - Allocated session 7cba6a7dadb243618c046ee7fb6c6bc9 for http://localhost:4444, launching...
09:04:30.778 INFO - Preparing Firefox profile...
09:04:34.676 INFO - Launching Firefox...
09:04:38.697 INFO - Got result: OK,7cba6a7dadb243618c046ee7fb6c6bc9 on session 7cba6a7dadb243618c046ee7fb6c6bc9
09:04:38.715 INFO - Command request: open[/selenium-server/tests/html/test_click_page1.html, ] on session 7cba6a7dadb243618c046ee7fb6c6bc9
09:04:38.854 INFO - Got result: XHR ERROR: URL = http://localhost:4444/selenium-server/tests/html/test_click_page1.html Response_Code = 404 Error_Message = Not+found on session 7cba6a7dadb243618c046ee7fb6c6bc9
09:04:38.863 INFO - Command request: testComplete[, ] on session 7cba6a7dadb243618c046ee7fb6c6bc9
09:04:38.863 INFO - Killing Firefox...
09:04:38.933 INFO - Got result: OK on session 7cba6a7dadb243618c046ee7fb6c6bc9
09:09:28.085 INFO - Command request: getNewBrowserSession[*firefox, http://www.irian.at, ] on session null
09:09:28.086 INFO - creating new remote session
09:09:28.088 INFO - Allocated session 97dec9f0b53545acbc9ca3624fc6cbd4 for http://www.irian.at, launching...
09:09:28.165 INFO - Preparing Firefox profile...
09:09:31.873 INFO - Launching Firefox...
09:09:35.599 INFO - Got result: OK,97dec9f0b53545acbc9ca3624fc6cbd4 on session 97dec9f0b53545acbc9ca3624fc6cbd4
09:09:35.604 INFO - Command request: open[http://www.irian.at/selenium-server/tests/html/ajax/ajax_autocompleter2_test.html, ] on session 97dec9f0b53545acbc9ca3624fc6cbd4
09:09:41.698 INFO - Got result: XHR ERROR: URL = http://www.irian.at/selenium-server/tests/html/ajax/ajax_autocompleter2_test.html Response_Code = 404 Error_Message = Not Found on session 97dec9f0b53545acbc9ca3624fc6cbd4
09:09:41.707 INFO - Command request: testComplete[, ] on session 97dec9f0b53545acbc9ca3624fc6cbd4
09:09:41.707 INFO - Killing Firefox...
09:09:41.740 INFO - Got result: OK on session 97dec9f0b53545acbc9ca3624fc6cbd4
10:18:38.252 INFO - Command request: getNewBrowserSession[*firefox, http://www.google.com/, ] on session null
10:18:38.253 INFO - creating new remote session
10:18:38.254 INFO - Allocated session c4b9c7f35ea34d428b50f2f31a6181c2 for http://www.google.com/, launching...
10:18:38.322 INFO - Preparing Firefox profile...
10:18:41.921 INFO - Launching Firefox...
10:18:45.741 INFO - Got result: OK,c4b9c7f35ea34d428b50f2f31a6181c2 on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:45.800 INFO - Command request: open[http://www.google.com/, ] on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:47.204 INFO - Got result: OK on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:47.211 INFO - Command request: type[q, hello world] on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:47.277 INFO - Got result: OK on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:47.282 INFO - Command request: click[btnG, ] on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:47.334 INFO - Got result: OK on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:47.340 INFO - Command request: waitForPageToLoad[5000, ] on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:48.359 INFO - Got result: OK on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:48.365 INFO - Command request: getTitle[, ] on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:48.411 INFO - Got result: OK,hello world - Google Search on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:48.416 INFO - Command request: testComplete[, ] on session c4b9c7f35ea34d428b50f2f31a6181c2
10:18:48.417 INFO - Killing Firefox...
10:18:48.434 INFO - Got result: OK on session c4b9c7f35ea34d428b50f2f31a6181c2


ALL this happens on TerminalOne, so keep that window open to check for the above while you are doing steps below.

STEP#1. In TerminalTwo, Change directory to the 'python-client' to run your scripts. Lets test the selenium.py script first. Btw, note that your bash file must contain the PYTHONPATH for all the directories that you run .py scripts from.

:~/selenium-remote-control-1.0.3$ ls
README.txt  selenium-php-client-driver-1.0.1  selenium-dotnet-client-driver-1.0.1  selenium-python-client-driver-1.0.1
selenium-java-client-driver-1.0.1  selenium-ruby-client-driver-1.0.1  selenium-perl-client-driver-1.0.1    selenium-server-1.0.3

:~/selenium-remote-control-1.0.3/selenium-python-client-driver-1.0.1$ python selenium.py

If it returns silently (read, No errors), it means your selenium server is working.

STEP#2
. Try testing another script, test_google.py or test_default_server.py

:~/selenium-remote-control-1.0.3/selenium-python-client-driver-1.0.1$ ls
doc    test_ajax_jsf.py  test_google.py~    selenium.py  test_ajax_jsf.pyc  test_google.pyc selenium.pyc                     test_default_server.py   test_i18n.py  selenium_test_suite_headless.py  test_default_server.pyc  test_i18n.pyc
selenium_test_suite.py  test_google.py

:~/selenium-remote-control-1.0.3/selenium-python-client-driver-1.0.1$ python test_default_server.py
Using selenium server at localhost:4444
E
======================================================================
ERROR: testLinks (__main__.TestDefaultServer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_default_server.py", line 36, in testLinks
    selenium.open("/selenium-server/tests/html/test_click_page1.html")
  File "/home/me/selenium-remote-control-1.0.3/selenium-python-client-driver-1.0.1/selenium.py", line 764, in open
    self.do_command("open", [url,])
  File "/home/me/selenium-remote-control-1.0.3/selenium-python-client-driver-1.0.1/selenium.py", line 215, in do_command
    raise Exception, data
Exception: XHR ERROR: URL = http://localhost:4444/selenium-server/tests/html/test_click_page1.html Response_Code = 404 Error_Message = Not+found

----------------------------------------------------------------------
Ran 1 test in 8.852s

FAILED (errors=1)

:~/selenium-remote-control-1.0.3/selenium-python-client-driver-1.0.1$ python test_ajax_jsf.py
Using selenium server at localhost:4444
E
======================================================================
ERROR: testKeyPress (__main__.TestAjaxJSF)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_ajax_jsf.py", line 39, in testKeyPress
    selenium.open("http://www.irian.at/selenium-server/tests/html/ajax/ajax_autocompleter2_test.html")
  File "/home/me/selenium-remote-control-1.0.3/selenium-python-client-driver-1.0.1/selenium.py", line 764, in open
    self.do_command("open", [url,])
  File "/home/me/selenium-remote-control-1.0.3/selenium-python-client-driver-1.0.1/selenium.py", line 215, in do_command
    raise Exception, data
Exception: XHR ERROR: URL = http://www.irian.at/selenium-server/tests/html/ajax/ajax_autocompleter2_test.html Response_Code = 404 Error_Message = Not Found

----------------------------------------------------------------------
Ran 1 test in 13.663s

FAILED (errors=1)

:~/selenium-remote-control-1.0.3/selenium-python-client-driver-1.0.1$ python test_google.py
.
----------------------------------------------------------------------
Ran 1 test in 10.190s

OK

When you are running the above scripts you would see Selenium throw a browser with messages but this is too fast and disappears. In the second terminal,   run your scripts from the directory you've stored them in.

 

2010 July 18 [Sunday]

Installing Sahi on Ubuntu 8.04 hardy

I was referred to sahi by ex-selenium users, so I decided to try it out.

To install sahi on ubuntu 8.04 hardy::

STEP#0.

I downloaded sahi from http://sourceforge.net/projects/sahi/files/

STEP#1.

Extracted the zip folder and changed directory. Ran the start_sahi shell script.

:~/sahi/userdata/bin$ sudo ./start_sahi.sh (The ./ is necessary because its not in the PATH as yet)
--------
SAHI_HOME: ../..
SAHI_USERDATA_DIR: ../../userdata
SAHI_EXT_CLASS_PATH:
--------
../../bin/sahi.sh: line 19: java: command not found

Since Sahi requires java installed on your ubuntu system, do "sudo apt-get install sun-java6-jd" and run the shell script again.

~/sahi/userdata/bin$ sudo ./start_sahi.sh
[sudo] password for mom:
--------
SAHI_HOME: ../..
SAHI_USERDATA_DIR: ../../userdata
SAHI_EXT_CLASS_PATH:
--------
Sahi properties file = /home/moi/sahi/config/sahi.properties
Sahi user properties file = /home/moi/sahi/userdata/config/userdata.properties
Copying profile to /home/moi/sahi/userdata/browser/ff/profiles/sahi0
Copying profile to /home/moi/sahi/userdata/browser/ff/profiles/sahi1
Copying profile to /home/moi/sahi/userdata/browser/ff/profiles/sahi2
Copying profile to /home/moi/sahi/userdata/browser/ff/profiles/sahi3
Copying profile to /home/moi/sahi/userdata/browser/ff/profiles/sahi4
Copying profile to /home/moi/sahi/userdata/browser/ff/profiles/sahi5
Copying profile to /home/moi/sahi/userdata/browser/ff/profiles/sahi6
Copying profile to /home/moi/sahi/userdata/browser/ff/profiles/sahi7
Copying profile to /home/moi/sahi/userdata/browser/ff/profiles/sahi8
Copying profile to /home/moi/sahi/userdata/browser/ff/profiles/sahi9
>>>> Sahi started. Listening on port: 9999
>>>> Configure your browser to use this server and port as its proxy
>>>> Browse any page and CTRL-ALT-DblClick on the page to bring up the Sahi Controller

STEP2:
Now visit http://sahi.co.in/w/using-sahi for the "Configure the browser" section and follow those instructions.
For FF, its "Edit > Preferences > Advanced > Network > Settings" and then you can continue with the remaining instructions for Manual Proxy Configuration.

Since “No Proxy for..” should not have localhost in it I removed it but sahi controller was not being activated. I would get the message "the proxy server is refusing connections" which results in FF browsing failing completely. Not nice.

Apparently the installation is fine and proxy has started on 9999 -- this port number is not used for any other process so technically Sahi should run. It doesnt. The second reason I didnt venture further with this is because it lacks a python client driver. Even if ruby is easy to learn, selenium scores when it provides a language driver for all the scripting languages.

- page 1 of 5