Including a Jenkins link in the crashlytics report

I've been implementing some mobile application automatic tests using appium (appium.io). We also use Jenkins (https://jenkikns.io) as continuous integration tool. We compile and test the applications as Jenkins jobs. The apps include crashlytics (https://try.crashlytics.com/) to send a report everytime the application crashes. We are thinking it would be good to know the crash report was generated by a jenkins job and have a link to the offending job, so we can access all the logs that are stored as Jenkins artifacts. This how we do it. First we obtain the job url via Jenkins environment variables. This is the python code:

jenkins_url = os.environ ("JENKINS_URL") + "/job/" + os.environ ("JOB_NAME") + \
          "/" + os.environ ("BUILD_NUMBER")

Then we instruct appium to pass this url as an environment varaible to the application as desired capabilities. iOS

args = { "args": [], "env" : { "JENKINS_URL": jenkins_url }}
desired_caps["processArguments"] = json.dumps (args)

Android

desired_caps ["optionalIntentArguments"] = '--es "android.intent.extra.JENKINS_URL" ' +  '"' + \
        jenkins_url + '"'

Then, we read the value of the variable inside application code and set it as a crashlytics key. swift (iOS):

       NSString *jenkinsUrl = environment  [@"JENKINS_URL"];
       if  (jenkinsUrl) {
            [CrashlyticsKit setObjectValue:jenkinsUrl forKey:@"JENKINS_URL"];
       }

Java (android):

Intent intent = getIntent ();
String jenkins_url = intent.getStringExtra ("android.intent.extra.JENKINS_URL");

if (jenkins_url != null) {
    Crashlytics.setString ("JENKINS_URL", jenkins_url);
}

Using two irtrans USB devices in one Ubuntu PC.

IRTrans is a little USB device to capture and send IR commands (like a remote control).

The irtrans USB device documentation states that "Individual control of multiple external transmitters is not possible". The reason for that is the provided irtrans server (irserver64) listen to two predefined ports (21000 for the irtrans protocol and 8765 for the lirc protocol). When the second server tries to bind to any of these ports it fails because the ports are already taken.

There is a workaround for this problem using docker containers. The simple idea is, we launch two servers in two separated containers and we configure these with a different IP address.

The first thing we need to do is to install docker in our Ubuntu computer. See here how to do that.

The we need to create our container. In an empty directory we copy:

•  the irserver64 file. 
•  the remotes/ directory, containing a number of files with the keypresses of the remotes we want to simulate. 
• a file called Dockerfile with the following content: 

FROM ubuntu:16.04

RUN mkdir /irtrans
RUN mkdir /irtrans/remotes

COPY irserver64 /irtrans
COPY remotes/file.rem /irtrans/remotes

WORKDIR /irtrans
ENTRYPOINT ["./irserver64"]

From this directory we can compile our container with (irtrans.1 is an arbitrary name we give to our container):

dockebuild -t irtrans.1 .

Then we need to create our docker network.

docker network create --subnet=172.18.0.0/16 mynet123=

And now we can run two containers in different ip addresses. Remember to give the container access to the serial port:

docker run --net mynet123 --ip 172.18.0.22 --device=/dev/ttyUSB0 irtrans.1 /dev/ttyUSB0
docker run --net mynet123 --ip 172.18.0.23 --device=/dev/ttyUSB2 irtrans.1 /dev/ttyUSB2

Qtest + google mock

The problem we are trying to solve is using google mocks with qtest. Since we don't use google test we need to replace the default event listener so we can fail the test with qt macros. We overwrite the OnTestPartResult. Then we create a macro to initialize all our tests the same way. unittest.h:

/**
 * \file
 * \brief Generic definitions for unit tests.
 */

#ifndef __UNIT_TESTS_H__
#define __UNIT_TESTS_H__

#include 
#include "gmock/gmock.h"

namespace unittests {

/**
 * \brief A class to replace qtest default event listener. 
 * 
 * The relevant thing here is the OnTestPartResult method. We need to change it
 * to fail the tests, at qtest level, when the expectations fail.
 *
 * This function gets called when we call the destructor of the mocks or the
 * ::testing::Mock::VerifyAndClearExpectations function.
 */
class GoogleTestEventListener : public ::testing::EmptyTestEventListener {
   virtual void OnTestStart(const ::testing::TestInfo&) {
   }

   virtual void OnTestPartResult(const ::testing::TestPartResult& test_part_result) {
        if (test_part_result.failed()) {
            QFAIL(
                QString("mock objects failed with '%1' at %2:%3")
                    .arg(QString(test_part_result.summary()))
                    .arg(test_part_result.file_name())
                    .arg(test_part_result.line_number())
                    .toLatin1().constData()
            );
        }
   }

   // Called after a test ends.
   virtual void OnTestEnd(const ::testing::TestInfo&) {
   }
};

} // namespace unittests

/**
 * \def INIT_GOOGLE_MOCKS (argc, argv)
 * \brief A macro that defines the unit test initialization. 
 *
 * We define the initialization process in one place.
 * This only really needed when using google mocks expectations.i.e. if your
 * test uses EXPECT_CALL.
 */
#define INIT_GOOGLE_MOCKS(argc, argv) { \
      ::testing::InitGoogleTest (&(argc), (argv)); \
      ::testing::TestEventListeners& listeners = ::testing::UnitTest::GetInstance()->listeners();  \
      delete listeners.Release(listeners.default_result_printer());\
      listeners.Append(new unittests::GoogleTestEventListener); }

#endif // __UNIT_TESTS_H__

And now an example on how to use it. First we have a class called classToTest. classToTest.h:

#ifndef __CLASS_TO_TEST__
#define __CLASS_TO_TEST__

#include "dependency.h"

class ClassToTest 
{
public:
    ClassToTest (Dependency *dep);
    ~ClassToTest ();

    int function1 (int param);
private:
    Dependency * _dependency;
};

#endif //__CLASS_TO_TEST__

classToTest.cpp:

#include "classToTest.h"

ClassToTest::ClassToTest (Dependency *dep)
{
    _dependency = dep;
}

ClassToTest::~ClassToTest ()
{
}

int ClassToTest::function1 (int param)
{
    return (_dependency->getValue (param));
}

The class uses the following dependency. dependency.h:

#ifndef __DEPENDENCY_H__
#define __DEPENDENCY_H__

class Dependency 
{
public:
    virtual int getValue (int inValue) = 0;

};

#endif // __DEPENDENCY_H__

This is how the test looks like: MyTest.h:

#ifndef __SECOND_TEST_H__
#define __SECOND_TEST_H__

#include 

class MyTest: public QObject
{
    Q_OBJECT
private slots:
    void test1 ();
};

#endif //__SECOND_TEST_H__

MyTest.cpp:

#include "classToTest.h"
#include "unittests.h"
#include "MyTest.h"
#include "gmock/gmock.h"

class MockDependency: public Dependency 
{
public:
    MOCK_METHOD1 (getValue, int (int inValue));
};

void MyTest::test1 ()
{
    MockDependency dep;

    EXPECT_CALL (dep, getValue (13))
        .Times (1)
        .WillOnce(::testing::Return(7));

    ClassToTest C (&dep);

    QCOMPARE (C.function1 (13), 7);
}

int main (int argc, char **argv)
{
    INIT_GOOGLE_MOCKS (argc, argv);

    MyTest *theTest = new MyTest ();
    return (QTest::qExec (theTest, argc, argv));
    delete theTest;
}

Finally, I've included the cmake/ctest script in case you want to compile the example. CMakeLists.txt:

cmake_minimum_required (VERSION 2.8.11)

enable_testing ()
project (ppral CXX)

add_subdirectory (googletest-master/googlemock)

set(CMAKE_INCLUDE_CURRENT_DIR ON)
set(CMAKE_AUTOMOC ON)

find_package (Qt5Core REQUIRED)
find_package (Qt5Test REQUIRED)

# Add the include directories for the Qt 5 Widgets module to
# the compile lines.
include_directories(${Qt5Core_INCLUDE_DIRS} )
include_directories(googletest-master/googlemock/include)
include_directories(googletest-master/googletest/include)
# Add compiler flags for building executables (-fPIE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${Qt5Core_EXECUTABLE_COMPILE_FLAGS} ")

qt5_wrap_cpp (classToTest.cpp classToTest.h Dependecy.h unittests.h
 MyTest.h MyTest.cpp)

add_executable (MyTest classToTest.cpp classToTest.h MyTest.h
    MyTest.cpp unittests.h dependency.h)
target_link_libraries (MyTest ${Qt5Core_LIBRARIES} ${Qt5Test_LIBRARIES} 
    gmock gtest)
add_test (NAME MyTest COMMAND MyTest) 

wxPython double slider widget to enter a range.

I had this problem to enter a range of times in a wxPython application. After some research I could not found a widget that does what I need. The closest I could get was a Slider with SL_SELRANGE
style. Unfortunatelly this option is only available in Windows.

So I decide to create a small widget with two sliders to select the maximum and minimum time. If the minimum goes over the maximum the maximum is updated and viceversa. The selected values are updated as labels.

This is the code:

#!/usr/bin/python
# -*- coding: utf-8 -*-

import wx

class RangeSlider (wx.Panel):
    MAX_VALUE = 1000

    def __init__ (self, parent, minTime, maxTime):
        super(RangeSlider, self).__init__(parent, wx.ID_ANY)

        self.minTime = minTime
        self.maxTime = maxTime

        sizer = wx.FlexGridSizer (rows=2, cols = 3, vgap = 5, hgap = 5)
        self.sldMax = wx.Slider(self, value=RangeSlider.MAX_VALUE, minValue=0,
                maxValue=self.MAX_VALUE,
                style=wx.SL_HORIZONTAL )
        self.sldMin = wx.Slider (self, value=0, minValue=0,
                maxValue=self.MAX_VALUE,
                style =wx.SL_HORIZONTAL )

        self.sldMax.Bind(wx.EVT_SCROLL, self.OnSliderScroll)
        self.sldMin.Bind (wx.EVT_SCROLL, self.OnSliderScroll2)

        self.txtMax = wx.StaticText(self, label= self.formatTime (self.maxTime))
        self.txtMin = wx.StaticText(self, label=self.formatTime (self.minTime))

        lab1 = wx.StaticText (self, label="Min " + self.formatTime (self.minTime))
        lab2 = wx.StaticText (self, label="Max " + self.formatTime (self.maxTime))

        sizer.Add (lab1, 0, wx.LEFT, 10)
        sizer.Add (self.sldMax, 1, wx.EXPAND)
        sizer.Add (lab2, 0, wx.RIGHT, 10)
        sizer.Add (self.txtMin, 1, wx.ALIGN_CENTER)
        sizer.Add (self.sldMin, 1, wx.EXPAND)
        sizer.Add (self.txtMax, 1, wx.ALIGN_CENTER)
        sizer.AddGrowableCol (1, 1)

        self.SetSizer (sizer)

    def formatTime (self, t):
        return "%02d:%02d:%02d" % (t / 3600, (t%3600)/60, t%60)

    def formatValue (self, v):
        t = v * (self.maxTime - self.minTime) / 1000
        return self.formatTime (t)


    def OnSliderScroll(self, e):

        val = self.sldMax.GetValue()

        valMin = self.sldMin.GetValue ()
        if valMin > val:
            self.sldMin.SetValue (val)
            self.txtMin.SetLabel (self.formatValue(val))

        self.txtMax.SetLabel(self.formatValue(val))

    def OnSliderScroll2 (self, e):
        val = self.sldMin.GetValue()

        valMax = self.sldMax.GetValue ()
        if valMax < val:
            self.sldMax.SetValue (val)
            self.txtMax.SetLabel (self.formatValue(val))

        self.txtMin.SetLabel(self.formatValue(val))


class Example(wx.Frame):

    def __init__(self, parent, minTime, maxTime):
        """ The time is seconds """
        super(Example, self).__init__(parent, wx.ID_ANY)


        self.InitUI(minTime, maxTime)


    def InitUI(self, minTime, maxTime):

        rangeSlider = RangeSlider (self,minTime, maxTime)

        self.SetTitle('RangeSlider')
        self.Centre()
        self.Show(True)

def main():

    ex = wx.App()
    Example(None, 0, 7200)
    ex.MainLoop()

if __name__ == '__main__':
    main()

Timedoctor Visualizer. An application to analyze timedoctor files.

Recently I needed to analyze some linux execution tasks and interrupt traces and I could not find a tool to do what I wanted so I created my own.

The timedoctor files show the begining and end time of each task executed by a CPU. This is a quite usefull tool to analyze errors in embedded systems, like deadlock, priority inversions...

The tool has two main windows the first one shows the existing tasks in one file and allows to sort, select and deselect each of them. More than one file can be analyzed at the same time.

The other windows allows to visualize the time of execution of the selected tasks in the time. The window allows to pan, zoom and save an image with the graph.

The application was implemented in python and it uses wxPython for the user interface and matplotlib for the plots. I've made it run on Linux, Windows and Mac. The tool can be downloaded from here:

http://github.com/rgonzalez72/tdv

Managing id tags in mp3 files with eyed3.

This is a problem I solved some time ago but after updating to the lastest library I discovered the interface has changed. Setting the tags is quite straightforward following the instructions in here: http://eyed3.nicfit.net/. It wasn't so easy for me to figure out how to attach and image to an mp3 file.
This is the solution I found:

import eyed3

audiofile = eyed3.load (fileName)
if audiofile.tag == None:
    audiofile.tag = eyed3.id3.Tag ()
    audiofile.file_info = eyed3.id3.FileInfo (fileName)


audiofile.tag.artist = artist
audiofile.tag.title = unicode (title)
audiofile.tag.genre = unicode (genre)
img_fp = open (file_path, "rb")

# The type of image, the content of the jpg file, mime type, description
audiofile.tag.images.set(eyed3.id3.frames.ImageFrame.FRONT_COVER,
                img_fp.read(), "image/jpeg", u" ")
audiofile.tag.save (filename=fileName, version=eyed3.id3.ID3_V2_3)

Heads up Omaha. An artificial inteligence example.

This entry describes the implementation of the Bot I entered to the "Heads up Omaha" competition (http://theaigames.com/competitions/heads-up-omaha). This is basically a competion between IA robots playing poker (Omaha). I was a bit surprised to make to the semifinals considering the little efford I put on it so I decided to explain the simple solution I found.

 I put the code in here:
 https://github.com/rgonzalez72/omaha

This is how I did it step by step. I wrote the bot using object oriented python.

Basic classes 

I created a number of classes to deal with the poker card. They are Suit, Height, Card, Hand and Maze. These classes are simple enough and they don't require further explanation.

Hand evaluation

The class Evaluate performs the evaluation of the hands. I copied the evaluation functions from the "getting started" code of the competion. The trick here is representing every hand and the rank as a 32 bit map in a way they can be compared. This is done in the eval5 function.

There is another function called eval9 that calculates the best score considering the four card in the hand and the five on the table  according to the Omaha rules. That is why we need the getCouples and getTrios functions in the Hand class.

Probabilities calculation 

This section describes how the probabilities of my hand is calculated. We'll I don't, I just estimate them after running a number of random games and counting the winning and losing hands. This is done in the calProbabilies function.

Fuzzy logic implementation

We have all the basic poker functionallity implemented so far. What we need now is to define the strategy. I decided to use fuzzy logic for the simple reason that I had never used it before and wanted to become familiar with it. First I was considering using some library but I could not be sure the library would be available in the competition servers so I implemented my basic fuzzy logic module.

The module defines fuzzy variables and operators. The variables can be triangles or trapezoids. The inputSet and outputSet classes are arrays of variables.

Fuzzy logic rules

Finally, with all the elements described, we can't define the rules that make our program "intelligent".

Type of game 

The first set of input rules we define is the number of chips in the pot that can be FEW, AVERAGE, LOTS. Depending on the number of chips we change the way we play that can be SUICIDAL, AGGRESIVE, CAUTIOUS and CONSERVATIVE. The rules are defined in GameTypeCalculator.

This strategy proved to be not too good and was later dropped the Bot is always AGGRESIVE.

ActionCalculator

The action calculator uses two inputSet: the game type and the hand probability. The hand probability can be VERY GOOD, GOOD, REGULAR and BAD. The goodness of the hand based on the probability is different in the phases of the game, that is why we have defined different ranges for pre-flop, flop, turn and river. The output set is the action: FOLD, CALL, RAISE MIN, RAISE MED, RAISE MAX. Please see ActionCalculator class and its children for details.

And that is, the rest of code is used to deal with the Bot interface. We've implemented a reasonable poker player in 1200 lines of code.

Report the duration using html

Another problem about creating automatic reports in html. In this case, we need to show the duration of cetain events graphically. It is expressed in a percentange of a total. The first idea is to use a progress bar as defined by html5:

def printProgress2 (percent):
    progStr = str(percent) + " "
    progStr += '<progress value="' + \
               str(percent) + '" max="100"></progress>'
    return progStr

And the result is something like:

33 <progress value="33" max="100"></progress>

This is not to bad but I don't totally like the animations that show some navigators. So my next approach is to use a html5 rectangle. This is the function:

def printProgress1 (percent):
    progStr = " "
    v = percent * 150 / 100
    progStr += '<svg width="160" height="25">'
    progStr += '<rect x="5" y="3" width="' + \
               str(v) + '" height="19" style="fill:rgb(148, 148, 210)"/>'
    progStr += '<text x="6" y="20">' + str (percent) + "%</text>"
    progStr += '<,/svg>'
    return progStr

And the result is something like:

<svg width="160" height="25"><rect x="5" y="3" width="4" height="19" 
   style="fill:rgb(148, 148, 210)"/><text x="6" y="20">3%</text></svg>

This is not bad but now the problem is that not all the navigators like html5 and outlook does not like it either. So the idea I came out with is to use the width of the table cells and change their background color to show the duration:

def getDurationHtml (self, percent):
        """ This function returns the contents of the duration cell based in the percentage. """
        durationStr = '<td><table><tr><td width="45">' + str (percent) + '%</td>'
        first = percent
        second = 100 - first
        durationStr += '<td width="' + str (first) + '" bgcolor="#7070C0"></td>'
        if second > 0:
            durationStr += '<td width="' + str (second) + '"></td>'
        durationStr += '</tr></table>'
        return durationStr

And this is the result:

<table><tr><td width="45">33%</td><td width="33" bgcolor="#7070C0"></td>
  <td width="67"> </td></tr></table>

Color gradation for reports (from red to green)

A little piece of code I wrote to represent a color gradation: red means very bad, green very good and everything in between. The trick here is to use HSV system instead of RGB. You keep the S and V to your favourite value and change the H from 0 to 120 degrees. Here is the function:

import colorsys

def get_color (percentage):

    V=1.0
    S=0.6
    H= (float(percentage) * 120.0) / 36000.0 

    R, G, B = colorsys.hsv_to_rgb (H, S, V)

    R = int (R * 255.0)
    G = int (G * 255.0)
    B = int (B * 255.0)

    octets = [R, G, B]

    color = "#{0:02X}{1:02X}{2:02X}".format (*octets)  

    return  color

Where have I been running?

A small project to extract the geographical information from my GPS training device and present it using the Google Maps API. I want to see a map with all the places I've run. To do so I export the activities from "Garmin Training Center" in TCX format version 2. Then I use the first longitude and latitude of the first point of each activity. To consider them to be different they must be separated, at least, for 1 kilometer. Finally I create a html file with the Google Maps API and a marker for each starting point. Easy.

I borrowed some code from pyTrainer.

A file, called startingPoints.py, that creates a data structure to manages list of points:

import math

class Point (object):
    def __init__ (self, lat, lon):
        self._lat = lat
        self._lon = lon

    def getLatitude (self):
        return self._lat

    def getLongitude (self):
        return self._lon

    def __str__ (self):
        retVal = str(self._lat) + ',' + str(self._lon)
        return retVal

    def distanceInKm (self, other):
        d2r = math.pi / 180.0

        dlong = (self._lon - other._lon) * d2r
        dlat = (self._lat - other._lat) * d2r
        a = math.pow (math.sin (dlat / 2.0), 2) + math.cos (self._lat * d2r) * math.cos (other._lat * d2r) \
            * math.pow(math.sin (dlong/2.0), 2)
        c = 2.0 * math.atan2 (math.sqrt(a), math.sqrt(1.0 -a))
        d = 6367.0 * c
        return d

class PointList (object):
    def __init__ (self):
        self._list = []

    def appendPoint (self, point):
        found = False

        for p in self._list:
            if p.distanceInKm (point) < 1.0:
                found = True
                break

        if not found:
            self._list.append (point)

    def __str__ (self):
        retVal = ""
        for p in self._list:
            retVal += str (p) + "\n"
        return retVal

    def __getitem__ (self, key):
        return self._list [key]

    def len (self):
        return len (self._list)

    def getParameters (self):
        # Returns max, min & average
        maxLat = -50000.0
        minLat = 50000.0
        maxLon = -50000.0
        minLon = 50000.0
        avLon = 0.0
        avLat = 0.0
        total = float (len (self._list))

        for s in self._list:
            lon = s.getLongitude ()
            lat = s.getLatitude ()

            if lon > maxLon:
                maxLon = lon
            if lon < minLon:
                minLon = lon
            if lat > maxLat:
                maxLat = lat
            if lat < minLat:
                minLat = lat
            avLon += lon / total
            avLat += lat / total

        return maxLat, minLat, maxLon, minLon, avLat, avLon

Another file to generate the map file:

#!/usr/bin/env python
# -*- coding: iso-8859-1 -*-

class PrintMap (object):
    
    def __init__ (self):
        pass

    def printMap (self, filename, startingPoints):

        maxLat, minLat, maxLon, minLon, avLat, avLon = startingPoints.getParameters ()

        fp = open (filename, "w")

        fp.write (
"""<!DOCTYPE html>
<html>
<head>
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no" />
    <style type="text/css">
        html { height: 100% }
        body { height: 100%; margin: 0; padding: 0 }
        #map_canvas { height: 100% }
    </style>
    <script type="text/javascript" 
        src="https://maps.googleapis.com/maps/api/js?sensor=false">    
    </script>
    <script type="text/javascript">
    function initialize ()
    {
        var mapOptions = {
            center: new google.maps.LatLng (""")
        fp.write (str(avLat) + ',' + str(avLon))
        fp.write ("""),
            zoom: 12,
            mapTypeId: google.maps.MapTypeId.ROADMAP
        };
        var map = new google.maps.Map (document.getElementById ("map_canvas"), mapOptions);""")

        fp.write ("var swlatlng = new google.maps.LatLng (%f, %f);\n" % (minLat, minLon))
        fp.write ("var nelatlng = new google.maps.LatLng (%f, %f);\n" % (maxLat, maxLon))
        fp.write ('''var boundsBox = new google.maps.LatLngBounds(swlatlng, nelatlng );\n
                    map.fitBounds(boundsBox);\n''')

        no_of_points = startingPoints.len ()
        for i in range (0, no_of_points):
            p = startingPoints [i]
            markerName = "point" + str (i)

            content = " var startmarker = new google.maps.Marker ({\n" + \
                "position: new google.maps.LatLng (" + \
                str (p.getLatitude ()) + "," + str (p.getLongitude ()) + \
                "),\n" + \
                "map: map,\n" + \
                'title: "' + markerName + '"})\n\n'
            fp.write (content)

        fp.write ("""
    }
    google.maps.event.addDomListener(window, 'load', initialize);
    </script>
</head>
<body>
    <div id="map_canvas"/></div>
</body>
</html>""")

And finally a file to parse the TCX files:

#!/usr/bin/env python
# -*- coding: iso-8859-1 -*-

import logging
import sys
import traceback
from lxml import etree
from startingPoints import Point, PointList
from printMap import PrintMap


class parseTCX (object):
    def __init__ (self):
        pass

    def validate(self, xmldoc, schema):
        logging.debug(">>")
        xmlschema_doc = etree.parse("./" + schema)
        xmlschema = etree.XMLSchema(xmlschema_doc)
        logging.debug("<<")
        return xmlschema.validate(xmldoc)

    def parseFile (self, filename):
        startingPoints = PointList ()
        logging.debug ("Parsing file " +  filename)
        try:
            xmldoc = etree.parse(filename)
            valid_xml = self.validate(xmldoc, "schemas/GarminTrainingCenterDatabase_v2.xsd")
            if (valid_xml):
                logging.debug ("It is a valid file " +  filename)
                activities = xmldoc.findall(".//{http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2}Activity")
                for activity in activities:
                    point = activity.find(".//{http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2}Trackpoint")
                    if point != None:
                        pos = point.find(".//{http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2}Position")
                        if pos != None:
                            lat = pos.find(".//{http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2}LatitudeDegrees")
                            lon = pos.find(".//{http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2}LongitudeDegrees")
                            startingPoints.appendPoint (Point (float(lat.text), float(lon.text)))
        except:
            logging.debug("Traceback: %s" % traceback.format_exc())

        return startingPoints

if __name__ == "__main__":
    logging.basicConfig(format='%(asctime)-6s: %(name)s - %(levelname)s - %(message)s', level=logging.DEBUG,
            filename="parseTCX.log")

    filename = sys.argv[1]

    P = parseTCX ()
    startingPoints = P.parseFile (filename)

    M = PrintMap ()
    M.printMap ("mapa.html", startingPoints)

A serial terminal in Python using curses

I have written this small program to interact with a device using Python. It's basically a terminal that has been written with python and curses. What is the need to write another terminal program? In my case, to have it extended. It'll be extended to parse the output and control other systems. Here is the code:

#!/usr/bin/env python

import serial
import logging 
import threading
import time
import curses
import Queue

serial_port='/dev/ttyUSB0'
g_exit_threads = False

class SerialPortThread (threading.Thread):
    def __init__ (self, port, logger, queue):
        threading.Thread.__init__ (self)
        self._port = port
        self._logger = logger
        self._queue = queue

    def run (self):
        global g_exit_threads
        self._logger.info ("SerialPortThread.run. ");
        while (not g_exit_threads):
            #line = self._serial_port.readline ()
            #print line,
            c = self._port.read ()
            if (len(c) > 0):
                self._queue.put (c)
        self._logger.info ("SerialPortThread. Exiting ");
            
            
        
class KeyboardThread (threading.Thread):
    def __init__ (self, screen, logger, port):
        threading.Thread.__init__ (self)
        self._screen = screen
        self._logger = logger
        self._port = port

    def run (self):
        global g_exit_threads
        self._logger.info ("KeyboardThread.Run")
        status = 0
        while (not g_exit_threads):
            try: 
                key = self._screen.getkey ()
                self._logger.info ("KeyboardThread.run. Event: >>" + key + "<<")
                self._port.write (key) 
                if status == 0:
                    if ord(key) == 27:
                        status = 1
                elif status == 1:
                    if ord(key) == 79:
                        status = 2
                    else:
                        status = 0
                elif status == 2:
                    if ord(key) == 80:
                        self._logger.info ("KeyboardThread. Exiting")
                        g_exit_threads = True
                        status = 0
                    else:
                        status = 0

            except:
                time.sleep (0.1);
        
class OutputThread (threading.Thread):

    def __init__ (self, screen, logger, queue):
        threading.Thread.__init__ (self)
        self._screen = screen
        self._logger = logger
        self._queue = queue

    def run (self):
        global g_exit_threads
        self._logger.info ("OutputThread.Run")
        y, x = self._screen.getmaxyx ()
        pos = 2
        while (not g_exit_threads):
            try:
                e = self._queue.get (timeout=0.1)
                self._logger.info ("OutputThread: " + str(ord(e[0])) )
                if e == '\n':
                    pos = 1
                    self._screen.scroll ()
                else:
                    self._logger.info ("Char: " + str(ord(e)))
                    if (ord(e) == 8):
                        if pos > 2:
                            pos -= 1
                        self._screen.addstr (y-1, pos, ' ')
                    elif (ord(e) == 7):
                        curses.flash ()
                    else:
                        self._screen.addstr (y-1, pos, e)
                        pos +=1
            except:
                pass

        self._logger.info ("OutputThread. Exiting")



if __name__ == "__main__":
    logging.basicConfig(format='%(asctime)-6s: %(name)s - %(levelname)s - %(message)s', level=logging.DEBUG,
            filename="console.log")

    queue = Queue.Queue ()

    screen = curses.initscr ()
    curses.noecho ()
    curses.curs_set (0)
    curses.raw ()
    screen.keypad (0)
    screen.clear ()
    screen.nodelay (True)
    y, x = screen.getmaxyx ()
    screen.setscrreg (1, y-1)
    screen.scrollok (True)

    port = serial.Serial (serial_port, 115200, timeout= 0.1);

    threads = []
    thread1 = SerialPortThread (port, logging, queue)
    threads.append (thread1)

    thread2 = KeyboardThread (screen, logging, port)
    threads.append (thread2)

    thread3 = OutputThread (screen, logging, queue)
    threads.append (thread3)


    for t in threads:
        t.setDaemon (True)
        t.start ()

    for t in threads:
        t.join ()

    # Wait for all the thread to finish
    #while threading.active_count () > 0:
    #    time.sleep (0.1)

    curses.endwin ()

Log diff script

I wanted to share a little a script I wrote to make a diff between two log files. The basic idea is to compare the log messages but ignore certain fields that are different every time, like the time stamp. What I do is remove every digit (0-9) and hex digit (a-f A-F) from every line and then call difflib to make the comparison. Then the original lines are presented in and html format. So here is the script:

#!/usr/bin/python



import difflib

import sys

import re

import itertools



def charJunk (ch):

    ch  = re.sub (r"[A-Fa-f0-9]", "", ch)



    if len(ch) == 0:

        return True

    else:

        return False



def filter (line):

    return  re.sub (r"[A-Fa-f0-9]", "", line)



def printResultsHtml (result, file1Name, file2Name):

    text1 = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" \n\

          "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> \n\

            \n\

            <html> \n\

            \n\

            <head> \n\

                <meta http-equiv="Content-Type" \n\

                    content="text/html; charset=ISO-8859-1" /> \n\

                <title></title> \n\

                <style type="text/css"> \n\

                    table.diff {font-family:Courier; border:medium;} \n\

                    .diff_header {background-color:#e0e0e0} \n\

                    td.diff_header {text-align:right} \n\

                    .diff_next {background-color:#c0c0c0} \n\

                    .diff_add {background-color:#aaffaa} \n\

                    .diff_chg {background-color:#ffff77} \n\

                    .diff_sub {background-color:#ffaaaa} \n\

                </style> \n\

            </head>\n\

            <body>\n'



    table_header = '<table class="diff" id="difflib_chg_to0__top"\n\

           cellspacing="0" cellpadding="0" rules="groups" >\n\

        <colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>\n\

        <colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>\n\

        <thead><tr><th class="diff_next"><br /></th><th colspan="2"\n\

        class="diff_header">\n'



    print text1



    print table_header, 

    print file1Name,

    print '</th><th class="diff_next"><br /></th><th colspan="2" class="diff_header">',

    print file2Name,

    print '</th></tr></thead>',





    index1 = 0

    index2 = 0

    for line in result:

        if line.startswith ('-'):

            print '<tr><td class="diff_next"></td><td class="diff_header">' + str (index1 +1) + '</td>',

            print '<td nowrap="nowrap"><span class="diff_sub">' + lines1[index1] + '</span></td>',

            print '<td class="diff_next"></td><td class="diff_header">' + '</td>',

            print '<td nowrap="nowrap">' + '</td></tr>',

            index1 += 1

        elif line.startswith ('+'):

            print '<tr><td class="diff_next"></td><td class="diff_header">'  + '</td>',

            print '<td nowrap="nowrap">' + '</td>',

            print '<td class="diff_next"></td><td class="diff_header">' + str(index2 + 1) +  '</td>',

            print '<td nowrap="nowrap"><span class="diff_add">' + lines2[index2] + '</span></td></tr>',

            index2 += 1

        elif line.startswith ('?'):

            pass

        else:

            if index1 >= len (lines1):

                print "Reeeeeeeeediox"

                break

            else:

                print '<tr><td class="diff_next"></td><td class="diff_header">' + str (index1 +1) + '</td>',

                print '<td nowrap="nowrap">' + lines1[index1] + '</td>',

                print '<td class="diff_next"></td><td class="diff_header">' + str (index2 +1) + '</td>',

                print '<td nowrap="nowrap">' + lines2[index2] + '</td></tr>',

                index1 += 1

                index2 += 1

    print text2





def printResultsHtml2 (result, file1Name, file2Name):

    text1 = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" \n\

          "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> \n\

            \n\

            <html> \n\

            \n\

            <head> \n\

                <meta http-equiv="Content-Type" \n\

                    content="text/html; charset=ISO-8859-1" /> \n\

                <title></title> \n\

                <style type="text/css"> \n\

                    table.diff {font-family:Courier; border:medium;} \n\

                    .diff_header {background-color:#e0e0e0} \n\

                    td.diff_header {text-align:right} \n\

                    .diff_next {background-color:#c0c0c0} \n\

                    .diff_add {background-color:#aaffaa} \n\

                    .diff_chg {background-color:#ffff77} \n\

                    .diff_sub {background-color:#ffaaaa} \n\

                    #one \n\

                {border:1px solid red;overflow:hidden; \n\

                float: left; \n\

                width: 48%;overflow-x: scroll; \n\

                } \n\

            #two \n\

                {border:1px solid red;overflow:hidden; \n\

                float: left; \n\

                width: 48%;overflow-x: scroll; \n\

                } \n\

            .c1 {width: 200px;} \n\

            #wrapper \n\

                { \n\

                float: left; \n\

                float/**/: none; \n\

                } \n\

            /* easy clearing */ \n\

            #wrapper:after \n\

                { \n\

                content: ' + "'.';  \n" +\

                'display: block;  \n\

                height: 0;  \n\

                clear: both;  \n\

                visibility: hidden; \n\

                } \n\

            #wrapper \n\

                { \n\

                display: inline-block; \n\

                } \n\

            /*\*/ \n\

            #wrapper \n\

                { \n\

                display: block; \n\

                } \n\

            /* end easy clearing */ \n\

                </style> \n\

            </head>\n\

            <body>\n'



    text2= '<table class="diff" summary="Legends">\n\

        <tr> <th colspan="2"> Legends </th> </tr>\n\

        <tr> <td> <table border="" summary="Colors">\n\

                      <tr><th> Colors </th> </tr>\n\

                      <tr><td class="diff_add"> Added </td></tr>\n\

                      <tr><td class="diff_chg">Changed</td> </tr>\n\

                      <tr><td class="diff_sub">Deleted</td> </tr>\n\

                  </table></td>\n\

             <td> <table border="" summary="Links">\n\

                      <tr><th colspan="2"> Links </th> </tr>\n\

                      <tr><td>(f)irst change</td> </tr>\n\

                      <tr><td>(n)ext change</td> </tr>\n\

                      <tr><td>(t)op</td> </tr>\n\

                  </table></td> </tr>\n\

    </table>\n\

    </body>\n\

    </html>\n'



    table_header = '<table class="diff" id="difflib_chg_to0__top"\n\

           cellspacing="0" cellpadding="0" rules="groups" >\n\

        <colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>\n\

        <colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>\n\

        <thead><tr><th class="diff_next"><br /></th><th colspan="2"\n\

        class="diff_header">\n'

    table_header2 = '<table class="diff" id="difflib_chg_to0__top"\n\

           cellspacing="0" cellpadding="0" rules="groups" >\n\

        <colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>\n\

        <colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>\n\

        <thead><tr><th class="diff_next"><br /></th><th colspan="2"\n\

        class="diff_header">\n'



    print text1



    print '<div id="wrapper"><div id="one">'

    print table_header, 

    print file1Name,

    print '</th></tr></thead>',



    r1, r2 = itertools.tee (result)



    index1 = 0

    index2 = 0

    for line in r1:

        if line.startswith ('-'):

            print '<tr><td class="diff_next"></td><td class="diff_header">' + str (index1 +1) + '</td>',

            print '<td nowrap="nowrap"><span class="diff_sub">' + lines1[index1] + '</span></td></tr>',

            index1 += 1

        elif line.startswith ('+'):

            print '<tr><td class="diff_next"></td><td class="diff_header">  '  + '</td>',

            print '<td nowrap="nowrap">+  ' + '</td></tr>',

            index2 += 1

        elif line.startswith ('?'):

            pass

        else:

            if index1 >= len (lines1):

                print "Reeeeeeeeediox"

                break

            else:

                print '<tr><td class="diff_next"></td><td class="diff_header">' + str (index1 +1) + '</td>',

                print '<td nowrap="nowrap">' + lines1[index1] + '</td></tr>',

                index1 += 1

                index2 += 1

    print '</table></div><div id="two">'

    print table_header2, 

    print file2Name,

    print '</th></tr></thead>',



    index1 = 0

    index2 = 0

    for line in r2:

        if line.startswith ('-'):

            print '<tr><td class="diff_next"></td><td class="diff_header">' + '</td>',

            print '<td nowrap="nowrap">- ' +  '</td></tr>',

            index1 += 1

        elif line.startswith ('+'):

            print '<tr><td class="diff_next"></td><td class="diff_header">'  + str(index2 +1 ) + '</td>',

            print '<td nowrap="nowrap"><span class="diff_add">' + lines2[index2] + '</span></td></tr>',

            index2 += 1

        elif line.startswith ('?'):

            pass

        else:

            if index1 >= len (lines1):

                print "Reeeeeeeeediox"

                break

            else:

                print '<tr><td class="diff_next"></td><td class="diff_header">' + str (index2 +1) + '</td>',

                print '<td nowrap="nowrap">' + lines2[index2] + '</td></tr>',

                index1 += 1

                index2 += 1



    print '</table></div></div>'





def printResults (result):

    index1 = 0

    index2 = 0

    for line in result:

        print (index1, index2),

        if line.startswith ('-'):

            print '- ' + lines1[index1],

            index1 += 1

        elif line.startswith ('+'):

            print '+ ' + lines2[index2],

            index2 += 1

        elif line.startswith ('?'):

            pass

        else:

            if index1 >= len (lines1):

                break

            else:

                print '  ' + lines1[index1],

                index1 += 1

                index2 += 1



def printResults2 (result):

    for line in result:

        print line,





file1 = open (sys.argv[1])

file2 = open (sys.argv[2])

    

lines1 = file1.read().splitlines (1)

lines2 = file2.read().splitlines (1)



filteredLines1 = []

filteredLines2 = []



for line in lines1:

    filteredLines1.append (filter (line))

for line in lines2:

    filteredLines2.append ( filter (line))



d = difflib.Differ ()



results = d.compare (filteredLines1, filteredLines2)



printResultsHtml2 (results, sys.argv[1], sys.argv[2])

Managing tags from a mp3 with eyeD3

Removing all the tags from a mp3 file

I've been using eyeD3 to write some tags in the mp3 files from a python script. Then I had the problem to remove all the tags from a file. According to the documentation you should do something like:

tag.link("/some/file.mp3")
tag.remove()
tag.update()

but it didn't worked for me so I decided to look at the library code and I found the following solution:

tag = eyeD3.Tag()
tag.link (fileName)
frameList = []
for frame in tag.frames:
        frameList.append (frame.header.id)
for l in frameList:
        tag.frames.removeFramesByID(l)
 tag.update()

Converting an iPod podcast file into regular audio file

The iPod uses some special tags to signal the files that are podcast or audio-books and they're stored and played in a different way. I've used eyeD3 to change files I download from certain podcast and use then as regular audio files.
The first thing is to remove an undocumented tag called PCST, then the genre needs to be changed from podcast to something else. Here is the code:

tag = eyeD3.Tag()
tag.link(fileName)
tag.frames.removeFramesByID("PCST")
g = eyeD3.Genre (None, 'Radio 3')
tag.setGenre (g)
tag.update()

Converting an audio file into an iPod podcast

We only have to invert the process. The content of the PCST tag is expected to be
"00 00 00 04 00 00 00 00 00 00" (*). Here is the code:

tag = eyeD3.Tag()
tag.link (fileName)
frameHeader = eyeD3.FrameHeader()
frameHeader.id = 'PCST'
pcstValue = struct.pack ('BBBBBBBBBB', 0, 0, 0, 4, 0, 0, 0, 0, 0, 0)
f = eyeD3.createFrame(frameHeader, pcstValue, eyeD3.TagHeader())
tag.frames.addFrame(f)

g = eyeD3.Genre (None, 'Podcast')
tag.setGenre (g)
tag.update()

Python's urllib2 and podomatic

I'm creating for myself a small tool to download some podcasts. To do so, I use python and the urllib2 library. Everything went well with a number of sites until yesterday.

I discover this great podcast so I wanted to include it in the tool. The problem is that when I try to download the mp3 file I get a "httpError 403: Forbidden". This puzzles me because the web browser can access to file with no problem.

I started wireshark to look into the requests. I couldn't see any significant difference. So after a few tries I discover the issue was the User Agent field of the header. The library was sending something like:

User-agent: Python-urllib/2.6 /r/n

So I decided to change it. This is the code that does the trick:

 
        user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
        headers ={'User-agent': user_agent}
        try:
            req = urllib2.Request (url, headers=headers)
            response = urllib2.urlopen (req)
        except urllib2.URLError, e:
            print ("Error: %(e)s with url: %(u)s" % {'e':e , 'u':url})    

Now the question is why do they configure the server like that?

Downloading a flash stream audio and convert it to a mp3

I've created this python script to download a flash stream audio and then convert it to mp3. My idea was to download a radio program so I can listen to it later on a not-connected portable device. This script can't be used for live streams.

The script uses 3 programs rtmpdump, ffmpeg and lame. rtmpdump is a util to download a stream using rtmp protocol. It can be easily compiled; it depends on libssl and zlib. ffmpeg and lame should be available for any LInux distribution, I compiled them for MAC.

The first step is to download the contents with rtmpdump. It's not a good idea to download everything in one go; it might take a lot of time. I've implemented a multithread mechanism instead. The number of threads and the number of seconds to download can be configured. Each downloaded audio chunk is stored in a file.

The second step is convert all the chunk to pcm raw format and concatenate all of them in a single file. The ffmpeg program is used to do so.

Finally, the raw file is converted to mp3 using lame.

The script uses a considerable amount of disk space for the temporary files. I think the script can be used, with a little modification to download video streams but I haven't tried that.

Here is the code:

#!/usr/bin/env python

import sys
import subprocess
import os
import threading

def executeCommand (cmd):
    p1 = subprocess.Popen (cmd, stdout = subprocess.PIPE, 
            stderr = subprocess.PIPE)
    out = p1.communicate()
    return out

class ChunkDownloaderThread (threading.Thread):
    def __init__ (self, downloader):
        threading.Thread.__init__(self)
        self._downloader = downloader

    def run (self):
        exitLoop = False
        while not exitLoop:
            cmd = self._downloader.getNextThreadCmd ()
            if cmd == None:
                exitLoop = True
            else:
                executeCommand (cmd[0])
                sys.stdout.write ( str(cmd[1]) + ' ')
                sys.stdout.flush ()

class Downloader (object):
    _rtmpDumpProgram = 'rtmpdump'
    _ffmpegProgram = 'ffmpeg'
    _lameProgram= 'lame'
    _tmpFile = 'tempFile'
    _tmpExtension = '.flv'
    _rawExtension = '.raw'

    def __init__ (self, chunkSize, noOfThreads):
        """ The class constructor. 
            chunkSize: the size in seconds of each file chunk. 
            noOfThreads: the number of simultaneaous threads to use. """
        self._chunkSize = chunkSize
        self._noOfThreads = noOfThreads
        


    def cleanTempFiles (self):
        extensions = [self._tmpExtension, self._rawExtension ]
        for e in extensions:
            fileName = self._tmpFile + e
            if os.path.isfile (fileName):
                os.remove (fileName)

        index = 0
        exitLoop = False
        while not exitLoop:
            fileName = self._tmpFile + str(index) + self._tmpExtension
            if os.path.isfile (fileName):
                os.remove (fileName)
            else:
                exitLoop = True
            index += 1
            


    def prepareDownloadCmd (self, url, destinationFile):
        cmd = [self._rtmpDumpProgram]
        cmd +=  ['-r', url, '-o', destinationFile]
        return cmd

    def findDuration (self, url):
        duration = 0.0
        fileName = self._tmpFile + self._tmpExtension
        cmd = self.prepareDownloadCmd (url, fileName)

        # Download just one second
        cmd += ['-B', '1']
        output = executeCommand(cmd)[1]
        output = output.split ('\n')
        for line in output:
            durationStr = 'duration' 
            infoStr = 'INFO:'
            pos = line.find (infoStr)
            if pos > -1:
                pos = line.find (durationStr)
                if  pos > -1:
                    duration = line[pos + len(durationStr):].strip()
                    duration = float(duration)
                    break
        self.cleanTempFiles()
        return duration
        
    def downloadChunk (self, url, tempFile, firstSecond=0.0, lastSecond=0.0):
        cmd = self.prepareDownloadCmd (url, tempFile)
        if firstSecond != 0.0:
            cmd += ['-A', str(firstSecond)]
        if lastSecond != 0.0:
            cmd += ['-B', str(lastSecond)]
        return cmd 

    def getNextThreadCmd (self):
        retVal = None
        self.theLock.acquire (True)
        if self.currentChunk < self.totalChunks: 
            auxIndex = self.currentChunk 
            cmd = self.downloadChunk (self.url, self.chunkList[auxIndex][2],
                            self.chunkList[auxIndex][0],
                            self.chunkList[auxIndex][1])
            retVal  = [cmd, auxIndex]
            self.currentChunk += 1
            
        self.theLock.release ()

        return retVal

    def downloadFile (self, url):
        duration = self.findDuration (url)

        self.chunkList = []
        chSize = float (self._chunkSize)
        begin = 0.0
        end = chSize
        index = 0
        while (begin < duration):
            if end >= duration:
                end = 0.0
            fileName = self._tmpFile + str(index) + self._tmpExtension
            self.chunkList.append ([begin, end, fileName])
            begin += chSize
            end += chSize
            index += 1

        self.totalChunks = index
        self.currentChunk = 0
        self.url = url
        self.theLock = threading.Lock()
        index = 0

        sys.stdout.write ('Total ' + str(self.totalChunks) + '\n')
        sys.stdout.flush ()

        threads = []
        for i in range (0, self._noOfThreads):
            t = ChunkDownloaderThread (self)
            threads.append (t)

        for t in threads:
            t.start()
        for t in threads:
            t.join()

        sys.stdout.write ('\n')
        sys.stdout.flush ()
        return self.totalChunks

    def concatenateFile (self, totalChunks):
        # Flv to raw command
        cmd = [self._ffmpegProgram, '-i']
        completeCmd = ['-vn', '-f', 'u16le', '-acodec', 'pcm_s16le', 
            '-ac', '2', '-ab', '128k', '-ar', '44100', '-']#, '<', '/dev/null']
        tempRawFile = self._tmpFile + self._rawExtension
        f = open (tempRawFile, "wb")
        sys.stdout.write ('Concatenate: \n')
        sys.stdout.flush ()
        for i in range (0, totalChunks):
            flvFile = self._tmpFile + str(i) + self._tmpExtension
            toExe = cmd + [flvFile] + completeCmd

            output = executeCommand (toExe)
            f.write (output[0])
            sys.stdout.write (str (i) + ' ')
            sys.stdout.flush ()
            # Delete the chunk to save disk space
            os.remove (flvFile)
        sys.stdout.write ('\n')
        sys.stdout.flush ()

        f.close ()


    def convertToMp3 (self, destination):
        tempRawFile = self._tmpFile + self._rawExtension
        cmd = [self._lameProgram, '-r', '-s', '44.1', '--preset', 'cd', 
            tempRawFile, destination]
        sys.stdout.write ('Converting to mp3\n')
        sys.stdout.flush ()
        executeCommand (cmd)
        sys.stdout.write ('Done\n')
        sys.stdout.flush ()

    def downloadAndConvertFile (self, url, destination):
        totalChunks = self.downloadFile (url)
        self.concatenateFile (totalChunks)
        self.convertToMp3 (destination)
        self.cleanTempFiles ()


if __name__ == '__main__':
    D = Downloader(60, 15)

    D.downloadAndConvertFile (sys.argv[1], sys.argv[2])

wxPython example. A jigsaw puzzle (3). MS Windows a double buffer

This is a continuation of the previous two entries. I finally got the chance of testing the puzzle on windows and it didn't show properly it flicked a lot. Then I learn that the Windows platform doesn't implement a double buffer so I have to do it.
The idea is painting to a bitmap and then copy it to the DC on the OnPaint event. This solution improved a lot the way application was shown it didn't removed he flicks completely.
After some research I found the solution by capturing the EVT_ERASE_BACKGROUND event and do nothing with it. This fixed all the issues. It also important to call the Update method after the Refresh.

The new board.py code is:

#!/usr/bin/python  

import wx
import stateDB
import rectangle

class Board(wx.Panel):
    boardId = 77
    def __init__ (self, parent, pieces, count, seconds, allowRotation=True):
        wx.Panel.__init__(self, parent, style = wx.NO_FULL_REPAINT_ON_RESIZE)

        self.db = stateDB.StateDB()

            
        self.pieces = pieces
        self.dragged = None
        self.count = count
        self.completed = False

        self.seconds = seconds
        self.timer = wx.Timer (self, wx.ID_ANY) 
        self.Bind (wx.EVT_TIMER, self.OnTimer, self.timer)
        self.timer.Start (1000, False)
        self.printStatus ()

        self.Bind (wx.EVT_PAINT, self.OnPaint)
        self.Bind (wx.EVT_ERASE_BACKGROUND, self.OnEraseBackground)
        self.Bind (wx.EVT_SIZE, self.OnSize)
        self.Bind (wx.EVT_LEFT_DOWN, self.OnDown)
        self.Bind (wx.EVT_LEFT_UP, self.OnUp)
        self.Bind (wx.EVT_MOTION, self.OnMouseMotion)
        if allowRotation:
            self.Bind (wx.EVT_RIGHT_UP, self.OnRight)

    def SaveStatus(self):
        self.db.initiateDB ()
        self.db.saveEverything (Board.boardId, self.count, self.seconds, \
                self.pieces)

    def printStatus (self):
        mins = self.seconds%(60*60)/60
        minStr = str(mins)
        if (mins < 10):
            minStr = '0' + minStr
        secs = self.seconds%60
        secStr = str(secs)
        if (secs < 10):
            secStr = '0' + secStr
        timeStr = str(self.seconds/(60*60)) + ':' + minStr + ':' + secStr

        if self.completed: 
            status = 'Completed: ' + timeStr
        else:
            status = str(self.count) + ': ' + timeStr
        self.GetParent().statusbar.SetStatusText (status)

    def OnTimer (self, e):
        if not self.completed:
            self.seconds += 1
            self.printStatus ()

    def OnEraseBackground (self, e):
 """ To avoid flickering in windows"""
        pass

    def OnPaint (self, e):
        dc = wx.BufferedPaintDC(self, self._Buffer, wx.BUFFER_VIRTUAL_AREA)

    def draw (self, dc):
        dc.Clear ()
        for p in self.pieces:
            p.drawPiece (dc)

    def UpdateDrawing (self):
        dc = wx.MemoryDC ()
        dc.SelectObject (self._Buffer)
        self.draw (dc)
        del dc
        self.Refresh ()
        self.Update ()

    def OnSize (self, e):
        s = self.ClientSize 
        self._Buffer = wx.EmptyBitmap (*s)
        self.UpdateDrawing ()
 

    def OnDown (self, e):
        mousePos = e.GetPosition()

        # The iteration is reversed so the piece on top is choosen 
        indexes = range(len(self.pieces))
        indexes.reverse()
        for i in indexes:
            if self.pieces[i].checkPointInPiece (mousePos):
                # Put the selected at the end of the list so it's painted the last
                p = self.pieces[i] 
                self.pieces.append(p)
                del self.pieces[i]
                self.dragged = len(self.pieces) -1
                break

    def OnUp (self, e):
        if self.dragged != None:
            mousePos = e.GetPosition()
            for i in range(len(self.pieces)):
                if (self.dragged != i):
                    if self.pieces[self.dragged].checkPieceMatch (\
                            self.pieces[i]):
                        del self.pieces[i]
                        #self.Refresh()
                        self.UpdateDrawing ()
                        self.count += 1
                        self.printStatus ()
                        self.SaveStatus ()
                        if len(self.pieces) == 1:
                           if (self.pieces[0].getOrientation() ==\
                                   rectangle.Orientation.or0):
                               self.db.RemoveDBFile ()
                               self.completed = True
                               self.printStatus ()

                        break
            self.dragged = None

    def OnMouseMotion (self, e):
        if self.dragged != None:
            mousePos = e.GetPosition()
            self.pieces[self.dragged].movePiece (mousePos)
            #self.Refresh()
            self.UpdateDrawing ()

    def OnRight (self, e):
        mousePos = e.GetPosition()
        indexes = range(len(self.pieces))
        indexes.reverse()
        for i in indexes:
            if self.pieces[i].checkPointInPiece (mousePos):
                self.pieces[i].incrementOrientation()
                newPos = self.pieces[i].calculateCenterPosition()
                self.pieces[i].changeRelPosition (newPos)
                self.pieces[i].checkPointInPiece (mousePos)
                #self.Refresh()
                self.UpdateDrawing ()
                if len(self.pieces) == 1:
                   if (self.pieces[0].getOrientation() == \
                           rectangle.Orientation.or0):
                       self.completed = True
                       self.printStatus ()
                break

class Puzzle (wx.Frame):
    def __init__ (self, parent, id, title, pieces, boardSize, count,
            seconds, allowRotation=True):
        wx.Frame.__init__(self, parent, id, title, size=boardSize)

        self.statusbar = self.CreateStatusBar()
        self.board = Board(self, pieces, count, seconds, allowRotation)

        self.Centre()
        self.Show(True)

wxPython example. A jigsaw puzzle (2)

Continuing with the previous post I created another puzzle, but this time the pieces has the shape of the mainland Spain provinces. I think is a fun way to learn geography and can be used for any map.



The code is basically what described in the previous post. The only difference is the way the pieces masks are created. I did it manually with gimp. For example:



The main file of the application, which I call prov.py, has the following code:

#!/usr/bin/python  

import board
import stateDB
import rectangle
import piece
import wx
import random

# Id, image, mask, size, top-left positions, relations
provinceList = [
    [1, 'coruna.png', 'corunaMask.png', (106, 96), (0,0), [2, 3] ],
    [2, 'lugo.png', 'lugoMask.png', (72, 119), (82,0), [1, 3, 4, 5, 37 ]], 
    [3, 'pontevedra.png', 'pontevedraMask.png', (70, 79), (17, 73), [1,2,4]],
    [4, 'orense.png', 'orenseMask.png', (97, 70), (60,94), [2, 3, 25, 37]],
    [5, 'asturias.png', 'asturiasMask.png', (159, 66), (134,11), [2,6, 37]],
    [6, 'cantabria.png', 'cantabriaMask.png', (104,64), (274, 25), [5,7, 37, 
        40, 41]],
    [7, 'vizcaya.png', 'vizcayaMask.png', (68, 35), (356,32), [6, 8, 9, 40]],
    [8, 'guipuzcoa.png', 'guipuzcoaMask.png', (50, 42), (412,34), [7, 9, 10]],
    [9, 'alava.png','alavaMask.png', (62, 60), (372, 52), [8, 7, 10, 39, 40 ]],
    [10, 'navarra.png', 'navarraMask.png', (102, 116), (423, 38), [8,9, 11, 
        12, 39]],
    [11, 'huesca.png', 'huescaMask.png', (112, 126), (511,69), [10, 12, 14 ]], 
    [12, 'zaragoza.png', 'zaragozaMask.png', (160, 147), (441, 85), [11, 10,
        13, 14, 17, 18, 38]], 
    [13, 'teruel.png', 'teruelMask.png', (132, 122), (464, 197), [12, 17, 18,
        33, 34, 35]], 
    [14, 'lleida.png', 'lleidaMask.png', (93,133), (596, 69), [ 11, 12, 15, 
        16, 17]], 
    [15, 'girona.png', 'gironaMask.png', (94, 72), (683,89), [ 14, 16 ]],
    [16, 'barcelona.png', 'barcelonaMask.png', (85, 96), (662, 108), [14, 15, 
        17]], 
    [17, 'tarra.png', 'tarraMask.png', (93,91), (589,169), [16, 14, 13, 12, 
        34]], 
    [18, 'guada.png', 'guadaMask.png', (127, 95), (354, 202), [ 13, 12, 19,
        35, 38, 43]],
    [19, 'madrid.png', 'madridMask.png', (101,100), (287, 219), [18, 20, 35, 
        43, 44]],
    [20, 'avila.png', 'avilaMask.png',  (103,91), (212,216), [19, 21, 24, 42, 
        43, 44]],
    [21, 'caceres.png', 'caceresMask.png', (165, 119), (96, 270), [20, 22, 
        24, 44]], 
    [22, 'badajoz.png', 'badajozMask.png', (177,125), (106, 352), [21, 23, 26,
        44, 45, 47]],
    [23, 'huelva.png', 'huelvaMask.png', (91,112), (87, 456), [22, 26 ]],
    [24, 'salamanca.png', 'salamancaMask.png',(113, 94), (141,198), [20, 21, 
        25, 42 ]],
    [25, 'zamora.png', 'zamoraMask.png',(111, 84), (136,127), [24, 4, 37, 42 ]],
    [26, 'sevilla.png', 'sevillaMask.png',(128, 116), (153, 456), [22, 23, 
        27, 28, 47 ]],
    [27, 'cadiz.png', 'cadizMask.png', (96,84), (154, 555), [26, 28]],
    [28, 'malaga.png', 'malagaMask.png', (122,84), (221, 532), [27, 26, 29, 
        47]],
    [29, 'granada.png', 'granadaMask.png', (138,117), (303, 466), [28,30, 31,
        36, 46, 47]],
    [30, 'almeria.png', 'almeriaMask.png', (105,103), (381, 480), [29, 31]],
    [31, 'murcia.png', 'murciaMask.png', (108, 114), (434, 410), [29, 32, 36]],
    [32, 'alicante.png', 'alicanteMask.png', (81,92), (520,391), [31, 33, 36]],
    [33, 'valencia.png', 'valenciaMask.png', (95,121), (485,292), [32, 13, 34,
        35, 36]],
    [34, 'castellon.png', 'castellonMask.png', (86,89), (526,242), [33, 13, 
        17]],
    [35, 'cuenca.png', 'cuencaMask.png', (132, 118), (376, 256), [18, 19, 13,
        33, 36, 44, 45]],
    [36, 'albacete.png', 'albaceteMask.png', (129,119), (397, 354), [35, 29,
        36, 32, 33, 31, 45, 46]],
    [37, 'leon.png', 'leonMask.png', (143, 102), (138, 50), [2, 4, 25, 5, 6, 
        41, 42]],
    [38, 'soria.png', 'soriaMask.png', (107, 89), (358,136), [12, 18, 39, 40, 
        43 ]],
    [39, 'rioja.png', 'riojaMask.png', (83,55), (383, 96), [38, 9, 10, 40]],
    [40, 'burgos.png', 'burgosMask.png', (110,141), (306,52), [39, 38, 6, 7, 
        9, 41, 42, 43]],
    [41, 'palencia.png', 'palenciaMask.png', (70,100), (262,65), [37, 6, 40, 
        42 ]],
    [42, 'valla.png', 'vallaMask.png', (96,98), (235, 125), [37, 40, 41, 20,
        25, 24, 43]],
    [43, 'segovia.png', 'segoviaMask.png', (92,78), (280, 185), [42, 40, 38, 
        18, 19, 20]],
    [44, 'toledo.png', 'toledoMask.png', (160, 83), (233, 287), [35,21, 22,
        19, 20, 45]],
    [45, 'creal.png', 'crealMask.png', (158, 103), (255, 344), [44, 36, 35,
        22, 46, 47 ]],
    [46, 'Jaen.png', 'JaenMask.png', (122, 98), (306, 430), [45, 36, 29, 47]],
    [47, 'cordoba.png', 'cordobaMask.png', (109,129), (218,412), [46, 45, 26, 
        28, 29, 22]]
    ]


def createPieceList (provinceList, boardSize):
    pieces = []

    for entry in provinceList:
        im = wx.Image (entry[1])
        mask = wx.Image (entry[2])
        im.SetMaskFromImage (mask, 255, 255, 255)

        rels = []
        # Calculate the relations
        for related in entry[5]:
            index = related -1
            difX = provinceList [index][4][0] - entry[4][0]
            difY = provinceList [index][4][1] - entry[4][1]
            rel = [related, (difX, difY), 0, 0]
            rels.append (rel)

        rec = rectangle.Rectangle (entry[0], (im, mask), entry[3], 
                rels, 0, 0)
        p = piece.Piece (entry[0])
        p.addRectangle (rec, (0, 0))
        pieces.append (p)

    for i in range (len(pieces)):
        pieces[i].setPosition ( (random.randint (10, boardSize[0] - 300),\
                random.randint(10, boardSize[1] - 300)))
#pieces[i].setOrientation (rectangle.getRandomOrientation())

    # Re-order the list so we don't always get the same provinces on top
    random.shuffle (pieces)

    return pieces

boardSize = (1000, 800)
app = wx.App ()
db = stateDB.StateDB()
count = -1
seconds = 0

if db.isThereADBFile ():
    count, seconds, pieces = db.readBoard (board.Board.boardId)

if count <=0:
    pieces= createPieceList (provinceList, boardSize)
    count = 0

board.Puzzle(None, -1, 'Spain mainland provinces', pieces, boardSize, count,
        seconds, False)
app.MainLoop()

wxPython example. A jigsaw puzzle

I've been teaching myself a bit about python and sqlite. To do so I have created a puzzle. The thing is pretty basic so far but it's playable. You can use any picture and the pieces are created automatically. The pieces are dragged by pressing the mouse left button and dropped when released. They are rotated with the right button.

I've tested it in Linux and Mac and it should work in Windows as well. You may need to install python and wxWidgets. The Mac version wxPython is only supported for 32bits so you must define the foillowing variable before executing:

export VERSIONER_PYTHON_PREFER_32_BIT=yes





The basic element of the puzzle is a rectangle. A rectangle has an image, a mask and they store the relative position of other rectangles. They can be moved around and rotated. The following code must be included in a file called rectangle.py.

import math
import wx
import random

class Orientation (object):
    or0 = 0
    or90 = 1
    or180 = 2
    or270 = 3

def rotateImage (orientation, im):
    imAux = im
    for i in range (orientation):
        imAux = imAux.Rotate90 (True)

    return imAux

def rotateRelPosition (orientation, pos, compX, compY):
    posAux = pos

    if orientation == Orientation.or90:
        posAux =(-posAux[1]  - compY, posAux[0])
    elif orientation == Orientation.or180:
        posAux =(-posAux[0]  - compX, -posAux[1] - compY)
    elif orientation == Orientation.or270:
        posAux =(posAux[1], - posAux[0]  - compX)

#for i in range (orientation):
#       posAux = (-posAux[1], posAux[0])
    return posAux

def rotateSize (orientation, size):
    sizeAux = size
    if (orientation == Orientation.or90 or orientation == Orientation.or270):
        sizeAux = (sizeAux[1], sizeAux[0])
    return sizeAux


def nextOrientation (orientation):
    if orientation == Orientation.or270:
        orientation = Orientation.or0
    else:
        orientation +=1
    return orientation

def getRandomOrientation ():
    return random.randint (Orientation.or0, Orientation.or270)

class Rectangle (object):
    def __init__ (self, id, im, size, relations, compX, compY):
        self.id = id
        self.image = im[0]
        self.mask = im[1]
        self.size = size
        self.relations = relations
        self.position = (0,0)
        self.orientation = Orientation.or0
        self.compX = compX
        self.compY = compY

    def __str__ (self):
        rectStr = "Rectangle " + str(self.id) + " size: " + str(self.size) + \
               ", pos: " + str(self.position) + ", comp: " + \
                str((self.compX, self.compY))+ ", or: " + \
                str(self.orientation) + ", rel: " + str(self.relations)
        return rectStr

    def setPosition (self, pos):
        self.position = pos

    def setOrientation (self, ort):
        self.orientation = ort

    def getCompX (self):
        self.compX

    def getCompY (self):
        self.compY

    def incrementOrientation (self):
        self.orientation = nextOrientation (self.orientation)

    def removeRelations(self, id):
        index = 0
        for index in range(len(self.relations)):
            if self.relations[index][0] == id:
                del self.relations[index]
                break

    def getId (self):
        return self.id

    def getImage (self):
        return self.image

    def getMask (self):
        return self.mask

    def getSize (self):
        return self.size

    def getPosition (self):
        return self.position

    def getOrientation (self):
        return self.orientation

    def getAbsolutePosition (self, absPosition):
        relPos = rotateRelPosition (self.orientation, self.position, 
                self.compX, self.compY)
        return (( absPosition[0] + relPos[0],\
                absPosition[1] + relPos[1]))

    def drawRectangle (self, absPosition, dc):
        absPos = self.getAbsolutePosition (absPosition)
        im = rotateImage (self.orientation, self.image)
        dc.DrawBitmap ( wx.BitmapFromImage(im), absPos[0],\
                absPos[1], True)

    def changeRelPosition (self, shift):
        self.position = ( self.position[0] - shift[0], \
                self.position[1] - shift[1])

    def checkPointInRectangle (self, absPosition, point):
        """ Returns the relative position of the point. """
        retVal = False

        relPos = (0, 0)
        absPos = self.getAbsolutePosition (absPosition)

        rotatedSize = rotateSize (self.orientation, self.size)

        if (point[0] >= absPos[0]) and \
           (point[0] < absPos[0] + rotatedSize[0]) and \
           (point[1] >= absPos[1]) and \
           (point[1] < absPos[1] + rotatedSize[1]):
               retVal = True
               relPos = (point[0] - absPos[0] + self.position[0], 
                       point[1] - absPos[1] + self.position[1])

        return (retVal, relPos)

    def checkRectagleMatch (self, absPosition, id, pos):
        """ Checks if another rectangle matches.
            id, the id of the other rectangle
            pos, the absolute position of the other rectangle. 
            The function returns the relative position of the other rectangle
            with respect to this piece."""
        retVal = False
        relPos = (0, 0)
        absPos = self.getAbsolutePosition (absPosition)
        for rel in self.relations:
            if rel[0] == id:
                rotatedRel = rotateRelPosition (self.orientation, rel[1],\
                        rel[2], rel[3])
                absPos = (absPos[0] + rotatedRel[0], absPos[1] + rotatedRel[1])

                distX = pos[0] - absPos[0]
                distY = pos[1] - absPos[1]

                if (math.sqrt((distX * distX) + (distY * distY)) < 4.0):
                    retVal = True
                    relPos= (self.position[0] + rel[1][0],\
                        self.position[1] + rel[1][1])

                break
        return retVal, relPos

The next element of the puzzle are the pieces. A piece contains one or more rectangles. The number of pieces gets reduces as the puzzle is solved. The code should be put in a file called piece.py.

import rectangle
import wx

class Piece(object):
    def __init__ (self, id):
        self.id = id
        self.pos = (0,0)
        self.mousePos = (0, 0)
        self.rectangles = []
        self.orientation = rectangle.Orientation.or0

    def __str__ (self):
        pieceStr = 'Piece ' + str(self.id) + ' pos: ' + str(self.pos) + \
                   ' mousePos: ' + str(self.mousePos) + ' or: '+\
                    str(self.orientation) + ' , rect:\n'
        for rect in self.rectangles:
            pieceStr += str(rect) + '\n'
        return pieceStr

    def getId (self):
        return self.id

    def getPosition (self):
        return (self.pos)

    def setPosition (self, pos):
        self.pos = pos

    def setOrientation (self, ort):
        self.orientation = ort
        for index in range(len(self.rectangles)):
            self.rectangles[index].setOrientation (ort)

    def addRectangle (self, rect, relPos):
        for index in range(len(self.rectangles)):
            self.rectangles[index].removeRelations (rect.getId())

        for index in range(len(self.rectangles)):
            rect.removeRelations(self.rectangles[index].getId())

        rect.setPosition (relPos)
        self.rectangles.append (rect)

    def getNoOfRectangles (self):
        return len (self.rectangles)

    def getRectangle (self, i):
        return self.rectangles[i]

    def getRectangleId (self, i):
        return self.rectangles[i].getId ()

    def getRectangleAbsPos (self, i):
        return self.rectangles[i].getAbsolutePosition (self.pos)

    def changeRelPosition (self, shift):
        for rect in self.rectangles:
            rect.changeRelPosition (shift)

    def calculateCenterPosition (self):
        maxLeft = 0 
        maxRight = 0
        maxTop = 0
        maxBottom = 0
        for rect in self.rectangles:
            x = rect.getPosition()[0]
            if maxLeft > x:
                maxLeft = x
            elif maxRight <  x:
                maxRight = x
            y = rect.getPosition()[1]
            if maxTop > y:
                maxTop = y
            elif maxBottom < y:
                maxBottom = y

        c = ((maxLeft + maxRight) / 2, (maxTop + maxBottom)/2)
        return c
            


    def drawPiece (self, dc):
        for rect in self.rectangles:
            rect.drawRectangle (self.pos, dc)

    def checkPointInPiece (self, point):
        retVal = False
        for rect in self.rectangles:
            ret, relPos = rect.checkPointInRectangle (self.pos, point)
            if ret:
                retVal = True
                self.mousePos = relPos
                break

        return retVal

    def checkPieceMatch (self, otherPiece):
        retVal = False

        if (self.orientation == otherPiece.getOrientation()):
            for i in range(otherPiece.getNoOfRectangles()):
                otherRect = otherPiece.getRectangle (i)
                for rect in self.rectangles:
                    ret, relPos = rect.checkRectagleMatch (self.pos,
                        otherRect.getId(), 
                        otherPiece.getRectangleAbsPos (i))
                    if ret:
                        retVal = True
                        # Calculate the relative position of the other piece
                        posOtherPiece = (relPos[0] - otherRect.getPosition()[0],
                            relPos[1] - otherRect.getPosition()[1])

                        # Include the rectangles in this piece
                        for j in range(otherPiece.getNoOfRectangles()):
                            otherRect = otherPiece.getRectangle(j)
                            self.addRectangle (otherRect, 
                                (posOtherPiece[0] + otherRect.getPosition()[0],
                                posOtherPiece[1] + otherRect.getPosition()[1]))


                        break

        return retVal

    def movePiece (self, newPos):
        self.pos = (newPos[0] - self.mousePos[0], 
                newPos[1] - self.mousePos[1])

    def incrementOrientation (self):
        self.orientation = rectangle.nextOrientation (self.orientation) 
        for i in range(len(self.rectangles)):
            self.rectangles[i].setOrientation(self.orientation)

    def getOrientation (self):
        return self.orientation

The board.py file contains the Board class that creates the gui panel and manages the events.

import wx
import stateDB
import rectangle

class Board(wx.Panel):
    boardId = 77
    def __init__ (self, parent, pieces, count, seconds, allowRotation=True):
        wx.Panel.__init__(self, parent)
                
        self.db = stateDB.StateDB()
            
        self.pieces = pieces
        self.dragged = None
        self.count = count
        self.completed = False

        self.seconds = seconds
        self.timer = wx.Timer (self, wx.ID_ANY) 
        self.Bind (wx.EVT_TIMER, self.OnTimer, self.timer)
        self.timer.Start (1000, False)
        self.printStatus ()

        self.Bind (wx.EVT_PAINT, self.OnPaint)
        self.Bind (wx.EVT_LEFT_DOWN, self.OnDown)
        self.Bind (wx.EVT_LEFT_UP, self.OnUp)
        self.Bind (wx.EVT_MOTION, self.OnMouseMotion)
        if allowRotation:
            self.Bind (wx.EVT_RIGHT_UP, self.OnRight)

    def SaveStatus(self):
        self.db.initiateDB ()
        self.db.saveEverything (Board.boardId, self.count, self.seconds, \
                self.pieces)

    def printStatus (self):
        mins = self.seconds%(60*60)/60
        minStr = str(mins)
        if (mins < 10):
            minStr = '0' + minStr
        secs = self.seconds%60
        secStr = str(secs)
        if (secs < 10):
            secStr = '0' + secStr
        timeStr = str(self.seconds/(60*60)) + ':' + minStr + ':' + secStr

        if self.completed: 
            status = 'Completed: ' + timeStr
        else:
            status = str(self.count) + ': ' + timeStr
        self.GetParent().statusbar.SetStatusText (status)

    def OnTimer (self, e):
        if not self.completed:
            self.seconds += 1
            self.printStatus ()

    def OnPaint (self, e):
        dc = wx.PaintDC(self)

        for p in self.pieces:
            p.drawPiece (dc)

    def OnDown (self, e):
        mousePos = e.GetPosition()

        # The iteration is reversed so the piece on top is choosen 
        indexes = range(len(self.pieces))
        indexes.reverse()
        for i in indexes:
            if self.pieces[i].checkPointInPiece (mousePos):
                # Put the selected at the end of the list so it's painted the last
                p = self.pieces[i] 
                self.pieces.append(p)
                del self.pieces[i]
                self.dragged = len(self.pieces) -1
                break

    def OnUp (self, e):
        if self.dragged != None:
            mousePos = e.GetPosition()
            for i in range(len(self.pieces)):
                if (self.dragged != i):
                    if self.pieces[self.dragged].checkPieceMatch (\
                            self.pieces[i]):
                        del self.pieces[i]
                        self.Refresh()
                        self.count += 1
                        self.printStatus ()
                        self.SaveStatus ()
                        if len(self.pieces) == 1:
                           if (self.pieces[0].getOrientation() ==\
                                   rectangle.Orientation.or0):
                               self.db.RemoveDBFile ()
                               self.completed = True
                               self.printStatus ()

                        break
            self.dragged = None

    def OnMouseMotion (self, e):
        if self.dragged != None:
            mousePos = e.GetPosition()
            self.pieces[self.dragged].movePiece (mousePos)
            self.Refresh()

    def OnRight (self, e):
        mousePos = e.GetPosition()
        indexes = range(len(self.pieces))
        indexes.reverse()
        for i in indexes:
            if self.pieces[i].checkPointInPiece (mousePos):
                self.pieces[i].incrementOrientation()
                newPos = self.pieces[i].calculateCenterPosition()
                self.pieces[i].changeRelPosition (newPos)
                self.pieces[i].checkPointInPiece (mousePos)
                self.Refresh()
                if len(self.pieces) == 1:
                   if (self.pieces[0].getOrientation() == \
                           rectangle.Orientation.or0):
                       self.completed = True
                       self.printStatus ()
                break

class Puzzle (wx.Frame):
    def __init__ (self, parent, id, title, pieces, boardSize, count,
            seconds, allowRotation=True):
        wx.Frame.__init__(self, parent, id, title, size=boardSize)

        self.statusbar = self.CreateStatusBar()
        self.board = Board(self, pieces, count, seconds, allowRotation)

        self.Centre()
        self.Show(True)

The state of the puzzle is stored in a sqlite database. The access to database is implemented in a file called stateDB.py.

import sqlite3
import os
import piece
import rectangle
import wx


class StateDB (object):
    # Names of the tables
    boardTable = 'board'
    pieceTable = 'piece'
    rectangleTable = 'rectangle'
    relationTable = 'relation'

    # The name of the DB file
    DBFileName = '.puzzle.db'

    # field names
    ekey= 'key'
    BTcount = 'count'
    BTseconds = 'seconds'

    PTboard = 'board'
    PTposx = 'posx'
    PTposy = 'posy'
    PTorientation = 'orientation'

    RTpiece = 'piece'
    RTsizex = 'sizex'
    RTsizey = 'sizey'
    RTposx = 'posx'
    RTposy = 'posy'
    RTcompx = 'compx'
    RTcompy = 'compy'
    RTimage = 'image'
    RTmask = 'mask'

    RTsource = 'source'
    RTdest = 'dest'
    RTrelx = 'relx'
    RTrely = 'rely'
    RelTCompX = 'compX'
    RelTCompY = 'compY'

    def __init__ (self):
        self.con = None
        self.cur = None

    def OpenDB (self):
        self.con = sqlite3.connect (StateDB.DBFileName)
        self.cur = self.con.cursor ()

    def CloseDB (self):
        self.cur.close()
        self.con.commit()
        self.con.close()
        self.cur = None
        self.con = None

    def RemoveDBFile (self):
        if os.path.exists (StateDB.DBFileName):
            os.remove (StateDB.DBFileName)

    def isThereADBFile (self):
        return os.path.exists (StateDB.DBFileName)

    def CreateTables (self):
        createBT = 'CREATE TABLE ' + StateDB.boardTable + ' (' + \
                StateDB.ekey + ' INTEGER PRIMARY KEY, ' + \
                StateDB.BTcount + ' INTEGER,' +\
                StateDB.BTseconds + ' INTEGER)'
        self.cur.execute (createBT)

        createPT = 'CREATE TABLE ' + StateDB.pieceTable + ' (' + \
                StateDB.ekey + ' INTEGER PRIMARY KEY, ' + \
                StateDB.PTboard + ' INTEGER, ' + \
                StateDB.PTposx + ' INTEGER, ' + \
                StateDB.PTposy + ' INTEGER, ' + \
                StateDB.PTorientation + ' INTEGER, ' + \
                'FOREIGN KEY (' + StateDB.PTboard + ') REFERENCES ' + \
                StateDB.boardTable + '(' + StateDB.ekey + '))'
        self.cur.execute (createPT)

        createRT = 'CREATE TABLE ' + StateDB.rectangleTable + '(' + \
                   StateDB.ekey + ' INTEGER PRIMARY KEY, ' + \
                   StateDB.RTpiece + ' INTEGER, ' + \
                   StateDB.RTsizex + ' INTEGER, ' + \
                   StateDB.RTsizey + ' INTEGER, ' + \
                   StateDB.RTposx + ' INTEGER, ' + \
                   StateDB.RTposy + ' INTEGER, ' + \
                   StateDB.RTcompx + ' INTEGER, ' + \
                   StateDB.RTcompy + ' INTEGER, ' + \
                   StateDB.RTimage + ' BLOB, ' + \
                   StateDB.RTmask + ' BLOB, ' + \
                   'FOREIGN KEY (' + StateDB.RTpiece + ') REFERENCES ' + \
                   StateDB.pieceTable + '(' + StateDB.ekey + '))'
        self.cur.execute (createRT)

        createRT = 'CREATE TABLE ' + StateDB.relationTable + '(' + \
                   StateDB.ekey + ' INTEGER PRIMARY KEY, ' + \
                   StateDB.RTsource + ' INTEGER, ' + \
                   StateDB.RTdest + ' INTEGER, ' + \
                   StateDB.RTrelx + ' INTEGER, ' + \
                   StateDB.RTrely + ' INTEGER, ' + \
                   StateDB.RelTCompX + ' INTEGER, ' + \
                   StateDB.RelTCompY + ' INTEGER, ' + \
                   'FOREIGN KEY (' + StateDB.RTsource + ') REFERENCES ' + \
                   StateDB.rectangleTable + '(' + StateDB.ekey + '), ' + \
                   'FOREIGN KEY (' + StateDB.RTdest + ') REFERENCES ' + \
                   StateDB.rectangleTable + '(' + StateDB.ekey + '))'
        self.cur.execute (createRT)

    def initiateDB (self):
        self.RemoveDBFile ()
        self.OpenDB ()
        self.CreateTables ()
        self.CloseDB ()

    def saveBoard (self, id, count, seconds):
        query = 'INSERT INTO ' + StateDB.boardTable + ' VALUES (?,?,?)'
        self.cur.execute (query, (id, count, seconds))

    def saveRelation (self, rectId, rel):
        query = ' INSERT INTO ' + StateDB.relationTable + '(' + \
                StateDB.RTsource + ',' + StateDB.RTdest + ',' + \
                StateDB.RTrelx + ',' + StateDB.RTrely + ', ' + \
                StateDB.RelTCompX + ', ' + StateDB.RelTCompY + ') ' + \
                ' VALUES (?,?,?,?,?,?) '
        entry = (rectId, rel[0], rel[1][0], rel[1][1],rel[2],rel[3])
        self.cur.execute (query, entry)


    def saveRectangle (self, pieceId, rect):
        query = ' INSERT INTO ' + StateDB.rectangleTable + \
                ' VALUES (?,?,?,?,?,?,?,?,?,?) '
        size = rect.getSize ()
        pos = rect.getPosition()
        imData = rect.getImage().GetData()
        maskData = rect.getMask().GetData()
        entry = (rect.getId(), pieceId, size[0], size[1], pos[0], pos[1], 
                rect.compX, rect.compY, sqlite3.Binary(imData),
                sqlite3.Binary(maskData))
        self.cur.execute (query, entry)
        for rel in rect.relations:
            self.saveRelation (rect.getId(), rel)

    def savePiece (self, boardId, piece):
        query = ' INSERT INTO ' + StateDB.pieceTable + ' VALUES (?,?,?,?,?) '
        pos = piece.getPosition()
        entry = (piece.getId(), boardId,pos[0], pos[1], piece.getOrientation())
        self.cur.execute (query, entry)
        for i in range(piece.getNoOfRectangles()):
            self.saveRectangle (piece.getId(), piece.getRectangle(i))

    def saveEverything (self, id, count, seconds, pieces):
        self.OpenDB ()
        self.saveBoard (id, count, seconds)
        for piece in pieces:
            self.savePiece (id, piece)
        self.CloseDB ()

    def readRelations (self, rectId):
        relations = []
        query = 'SELECT * FROM ' + StateDB.relationTable + ' WHERE ' + \
                StateDB.RTsource + '=' + str(rectId)
        self.cur.execute(query)
        relList = self.cur.fetchall ()
        for entry in relList:
            r = [entry[2], (entry[3], entry[4]), entry[5], entry[6]]
            relations.append (r)
        return relations

    def readRectangles (self, pieceId):
        rectangles = []
        query = 'SELECT * FROM ' + StateDB.rectangleTable + ' WHERE ' + \
                StateDB.RTpiece + '=' + str(pieceId)
        self.cur.execute(query)
        rectList = self.cur.fetchall ()
        for entry in rectList:
            im = wx.ImageFromData (entry[2], entry[3], entry[8])
            mask = wx.ImageFromData (entry[2], entry[3], entry[9])
            im.SetMaskFromImage (mask, 255, 255, 255)
            rels = self.readRelations (entry[0])
            r = rectangle.Rectangle (entry[0], (im, mask),\
                    (entry[2], entry[3]), rels, entry[6], entry[7])
            pos = (entry[4], entry[5])
            rectangles.append((r, pos))
        return rectangles

    def readPieces (self, boardId):
        pieces = []
        query = 'SELECT * FROM ' + StateDB.pieceTable + ' WHERE ' + \
                StateDB.PTboard + '=' + str(boardId)
        self.cur.execute (query)
        pieceList = self.cur.fetchall ()
        for entry in pieceList:
            p = piece.Piece (entry[0]) 
            p.setPosition ((entry[2], entry[3]))
            rectangles = self.readRectangles (entry[0])
            for r in rectangles:
                p.addRectangle (r[0], r[1])
            p.setOrientation (entry[4])
            pieces.append (p)
        return pieces

    def readBoard (self, id):
        self.OpenDB()
        count = -1
        pieces = []

        query = 'SELECT ' + StateDB.BTcount + ', ' + StateDB.BTseconds +\
                ' FROM ' + StateDB.boardTable +\
                ' WHERE ' + StateDB.ekey + '=' + str(id)
        self.cur.execute (query)
        countList = self.cur.fetchall ()
        if countList != None:
            count = countList[0][0]
            seconds = countList[0][1]
            pieces = self.readPieces(id)
        self.CloseDB()

        return count, seconds, pieces

Finally, the main file, puzzle.py, takes an image creates all the pieces automatically and then starts the GUI.

#!/usr/bin/python  

import wx
import random
import piece
import rectangle
import stateDB
import board


def createMask (sizeX, sizeY, overlap, dims, pos):
    theBuffer = bytearray()

    w = 255
    b = 0

    leftLimit = (sizeX/2) - overlap/2
    rightLimit = (sizeX/2) +overlap/2
    topLimit = (sizeY/2) - overlap/2
    bottomLimit = (sizeY/2) + overlap/2


    even = (((pos[0] + pos[1])% 2) == 0)

    circleShift = 2
    radiusSquare = ((overlap+circleShift)*(overlap+circleShift))/4

    for j in range(sizeY ):
        for i in range (sizeX ):
            setWhite = False

            setWhite = False
            if (pos[0] < dims[0] -1):
                if even:
                    if (i >= sizeX - overlap):
                        newJ = topLimit + (overlap /2) - j
                        newI = sizeX - (overlap/2) - circleShift -i
                        if (newI * newI + newJ * newJ) > radiusSquare:
                            setWhite = True
                else:
                    if (i >= sizeX - overlap):
                        setWhite = True
                    if (i < sizeX - overlap) and (i >= sizeX - overlap*2):
                        newJ = topLimit + (overlap /2) - j
                        newI = sizeX - (3*overlap/2) + circleShift -i
                        if (newI * newI + newJ * newJ) <= radiusSquare:
                            setWhite = True

            if (pos[0] > 0):
                if not even:
                    if (i < overlap): 
                        setWhite = True
                    if (i >= overlap) and (i < overlap *2):
                        newI = (3*overlap/2) - circleShift -i
                        newJ = topLimit + (overlap/2) - j
                        if (newI * newI + newJ * newJ) <= radiusSquare:
                            setWhite = True
                else:
                    if (i < overlap): 
                        newI = (overlap/2) + circleShift - i
                        newJ = topLimit + (overlap/2) -j
                        if (newI * newI + newJ * newJ) > radiusSquare:
                            setWhite = True

            if (pos[1] < dims[1] - 1):
                if not even:
                    if (j >= sizeY - overlap): 
                        newI = leftLimit + (overlap/2) -i
                        newJ = sizeY - (overlap/2) - circleShift - j
                        if (newI * newI + newJ * newJ) > radiusSquare:
                            setWhite = True
                else: 
                    if (j >= sizeY - overlap):
                        setWhite = True
                    if (j < sizeY - overlap) and (j >= sizeY - overlap*2):
                        newI = leftLimit + (overlap/2) -i
                        newJ = sizeY - (3*overlap/2) + circleShift - j
                        if (newI * newI + newJ * newJ) <= radiusSquare:
                            setWhite = True

            if (pos[1] > 0):
                if not even:
                    if (j < overlap):
                        newI = leftLimit + (overlap/2) -i
                        newJ = (overlap/2) +circleShift - j
                        if (newI * newI + newJ * newJ) > radiusSquare:
                            setWhite = True
                else:
                    if (j < overlap): 
                        setWhite = True
                    if (j >= overlap) and (j < overlap * 2):
                        newI = leftLimit + (overlap/2) -i
                        newJ = (3*overlap/2) - circleShift - j
                        if (newI * newI + newJ * newJ) <= radiusSquare:
                            setWhite = True
            if setWhite:
                color = w
            else:
                color = b
            theBuffer.append (color)
            theBuffer.append (color)
            theBuffer.append (color)

    return wx.ImageFromData (sizeX , sizeY, theBuffer)

def createPieceList (imFile, boardSize):
    sizeX = 50
    sizeY = 50
    overlap = 12
    im = wx.Image (imFile)
    imW = im.GetWidth()
    imH = im.GetHeight()
    pieces = []

    noOfRows = imH/sizeY
    noOfCols = imW/sizeX


    id = 0
    for j in range (noOfRows):
        for i in range (noOfCols):
            # SubImage for each rectangle
            iPos = 0
            if i > 0:
                iPos = i * sizeX - overlap
            jPos = 0
            if j > 0:
                jPos = j * sizeY - overlap

            curSizeX = sizeX + overlap
            if i > 0  and i < noOfCols -1:
                curSizeX += overlap
            curSizeY = sizeY + overlap
            if j > 0  and j < noOfRows -1:
                curSizeY += overlap


            r = wx.Rect (iPos, jPos, curSizeX, curSizeY)
            imAux = im.GetSubImage (r)

            mask = createMask (curSizeX, curSizeY, overlap, \
                    (noOfCols, noOfRows), (i, j))
            imAux.SetMaskFromImage (mask, 255, 255, 255)


            rels = []
            if (i > 0):
                if (i==1):
                    rels.append ([id-1,(-(sizeX-overlap), 0), -overlap, 0]) 
                elif (i == (noOfCols - 1)):
                    rels.append ([id-1,(-(sizeX), 0), overlap, 0]) 
                else:
                    rels.append ([id-1,(-(sizeX), 0), 0, 0]) 
            if (i < (noOfCols -1)):
                if (i==0):
                    rels.append ([id+1, (sizeX-overlap, 0), overlap, 0])
                elif (i == (noOfCols -2)):
                    rels.append ([id+1, (sizeX, 0), -overlap, 0])
                else:
                    rels.append ([id+1, (sizeX, 0), 0, 0])

            if (j > 0):
                if (j==1):
                    rels.append([id-noOfCols, (0, -(sizeY-overlap)), 0,
                            -overlap])
                elif (j == (noOfRows -1)):
                    rels.append([id-noOfCols, (0, -(sizeY)), 0, overlap])
                else:
                    rels.append([id-noOfCols, (0, -(sizeY)), 0, 0])
            if (j < (noOfRows - 1)): 
                if (j==0):
                    rels.append([id+noOfCols, (0, sizeY-overlap), 0, overlap])
                elif (j == (noOfRows -2)):
                    rels.append([id+noOfCols, (0, sizeY), 0, -overlap])
                else:
                    rels.append([id+noOfCols, (0, sizeY), 0, 0])

            compX, compY = calculateCompensantion (i, j, noOfRows, noOfCols,
                    overlap)


            rec = rectangle.Rectangle (id, (imAux, mask),
                    (curSizeX, curSizeY), rels, compX, compY)
            p = piece.Piece (id)
            p.addRectangle (rec, (0,0))
            pieces.append (p)
            id +=1

    for i in range (len(pieces)):
        pieces[i].setPosition ( (random.randint (10, boardSize[0] -300),\
                random.randint(10, boardSize[1] -300)))
        pieces[i].setOrientation (rectangle.getRandomOrientation())

    return pieces

def calculateCompensantion (i, j, noOfRows, noOfCols, overlap):
    compX = 0
    compY = 0
    if i==0:
        compX = -overlap
    elif i== (noOfCols -1):
        compX = -overlap

    if j==0:
        compY = -overlap
    elif j == (noOfRows -1):
        compY = -overlap

    return compX, compY

boardSize = (1000, 800)
app = wx.App ()
db = stateDB.StateDB()
count = -1
seconds = 0


if db.isThereADBFile ():
    count, seconds, pieces = db.readBoard (board.Board.boardId)

if count <=0:
    pieces= createPieceList ('im2.png', boardSize)
    count = 0


board.Puzzle(None, -1, 'Puzzle', pieces, boardSize, count, seconds)
app.MainLoop()