Managing tags from a mp3 with eyeD3

Removing all the tags from a mp3 file

I've been using eyeD3 to write some tags in the mp3 files from a python script. Then I had the problem to remove all the tags from a file. According to the documentation you should do something like:

tag.link("/some/file.mp3")
tag.remove()
tag.update()

but it didn't worked for me so I decided to look at the library code and I found the following solution:

tag = eyeD3.Tag()
tag.link (fileName)
frameList = []
for frame in tag.frames:
        frameList.append (frame.header.id)
for l in frameList:
        tag.frames.removeFramesByID(l)
 tag.update()

Converting an iPod podcast file into regular audio file

The iPod uses some special tags to signal the files that are podcast or audio-books and they're stored and played in a different way. I've used eyeD3 to change files I download from certain podcast and use then as regular audio files.
The first thing is to remove an undocumented tag called PCST, then the genre needs to be changed from podcast to something else. Here is the code:

tag = eyeD3.Tag()
tag.link(fileName)
tag.frames.removeFramesByID("PCST")
g = eyeD3.Genre (None, 'Radio 3')
tag.setGenre (g)
tag.update()

Converting an audio file into an iPod podcast

We only have to invert the process. The content of the PCST tag is expected to be
"00 00 00 04 00 00 00 00 00 00" (*). Here is the code:

tag = eyeD3.Tag()
tag.link (fileName)
frameHeader = eyeD3.FrameHeader()
frameHeader.id = 'PCST'
pcstValue = struct.pack ('BBBBBBBBBB', 0, 0, 0, 4, 0, 0, 0, 0, 0, 0)
f = eyeD3.createFrame(frameHeader, pcstValue, eyeD3.TagHeader())
tag.frames.addFrame(f)

g = eyeD3.Genre (None, 'Podcast')
tag.setGenre (g)
tag.update()

Python's urllib2 and podomatic

I'm creating for myself a small tool to download some podcasts. To do so, I use python and the urllib2 library. Everything went well with a number of sites until yesterday.

I discover this great podcast so I wanted to include it in the tool. The problem is that when I try to download the mp3 file I get a "httpError 403: Forbidden". This puzzles me because the web browser can access to file with no problem.

I started wireshark to look into the requests. I couldn't see any significant difference. So after a few tries I discover the issue was the User Agent field of the header. The library was sending something like:

User-agent: Python-urllib/2.6 /r/n

So I decided to change it. This is the code that does the trick:

 
        user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
        headers ={'User-agent': user_agent}
        try:
            req = urllib2.Request (url, headers=headers)
            response = urllib2.urlopen (req)
        except urllib2.URLError, e:
            print ("Error: %(e)s with url: %(u)s" % {'e':e , 'u':url})    

Now the question is why do they configure the server like that?

Downloading a flash stream audio and convert it to a mp3

I've created this python script to download a flash stream audio and then convert it to mp3. My idea was to download a radio program so I can listen to it later on a not-connected portable device. This script can't be used for live streams.

The script uses 3 programs rtmpdump, ffmpeg and lame. rtmpdump is a util to download a stream using rtmp protocol. It can be easily compiled; it depends on libssl and zlib. ffmpeg and lame should be available for any LInux distribution, I compiled them for MAC.

The first step is to download the contents with rtmpdump. It's not a good idea to download everything in one go; it might take a lot of time. I've implemented a multithread mechanism instead. The number of threads and the number of seconds to download can be configured. Each downloaded audio chunk is stored in a file.

The second step is convert all the chunk to pcm raw format and concatenate all of them in a single file. The ffmpeg program is used to do so.

Finally, the raw file is converted to mp3 using lame.

The script uses a considerable amount of disk space for the temporary files. I think the script can be used, with a little modification to download video streams but I haven't tried that.

Here is the code:

#!/usr/bin/env python

import sys
import subprocess
import os
import threading

def executeCommand (cmd):
    p1 = subprocess.Popen (cmd, stdout = subprocess.PIPE, 
            stderr = subprocess.PIPE)
    out = p1.communicate()
    return out

class ChunkDownloaderThread (threading.Thread):
    def __init__ (self, downloader):
        threading.Thread.__init__(self)
        self._downloader = downloader

    def run (self):
        exitLoop = False
        while not exitLoop:
            cmd = self._downloader.getNextThreadCmd ()
            if cmd == None:
                exitLoop = True
            else:
                executeCommand (cmd[0])
                sys.stdout.write ( str(cmd[1]) + ' ')
                sys.stdout.flush ()

class Downloader (object):
    _rtmpDumpProgram = 'rtmpdump'
    _ffmpegProgram = 'ffmpeg'
    _lameProgram= 'lame'
    _tmpFile = 'tempFile'
    _tmpExtension = '.flv'
    _rawExtension = '.raw'

    def __init__ (self, chunkSize, noOfThreads):
        """ The class constructor. 
            chunkSize: the size in seconds of each file chunk. 
            noOfThreads: the number of simultaneaous threads to use. """
        self._chunkSize = chunkSize
        self._noOfThreads = noOfThreads
        


    def cleanTempFiles (self):
        extensions = [self._tmpExtension, self._rawExtension ]
        for e in extensions:
            fileName = self._tmpFile + e
            if os.path.isfile (fileName):
                os.remove (fileName)

        index = 0
        exitLoop = False
        while not exitLoop:
            fileName = self._tmpFile + str(index) + self._tmpExtension
            if os.path.isfile (fileName):
                os.remove (fileName)
            else:
                exitLoop = True
            index += 1
            


    def prepareDownloadCmd (self, url, destinationFile):
        cmd = [self._rtmpDumpProgram]
        cmd +=  ['-r', url, '-o', destinationFile]
        return cmd

    def findDuration (self, url):
        duration = 0.0
        fileName = self._tmpFile + self._tmpExtension
        cmd = self.prepareDownloadCmd (url, fileName)

        # Download just one second
        cmd += ['-B', '1']
        output = executeCommand(cmd)[1]
        output = output.split ('\n')
        for line in output:
            durationStr = 'duration' 
            infoStr = 'INFO:'
            pos = line.find (infoStr)
            if pos > -1:
                pos = line.find (durationStr)
                if  pos > -1:
                    duration = line[pos + len(durationStr):].strip()
                    duration = float(duration)
                    break
        self.cleanTempFiles()
        return duration
        
    def downloadChunk (self, url, tempFile, firstSecond=0.0, lastSecond=0.0):
        cmd = self.prepareDownloadCmd (url, tempFile)
        if firstSecond != 0.0:
            cmd += ['-A', str(firstSecond)]
        if lastSecond != 0.0:
            cmd += ['-B', str(lastSecond)]
        return cmd 

    def getNextThreadCmd (self):
        retVal = None
        self.theLock.acquire (True)
        if self.currentChunk < self.totalChunks: 
            auxIndex = self.currentChunk 
            cmd = self.downloadChunk (self.url, self.chunkList[auxIndex][2],
                            self.chunkList[auxIndex][0],
                            self.chunkList[auxIndex][1])
            retVal  = [cmd, auxIndex]
            self.currentChunk += 1
            
        self.theLock.release ()

        return retVal

    def downloadFile (self, url):
        duration = self.findDuration (url)

        self.chunkList = []
        chSize = float (self._chunkSize)
        begin = 0.0
        end = chSize
        index = 0
        while (begin < duration):
            if end >= duration:
                end = 0.0
            fileName = self._tmpFile + str(index) + self._tmpExtension
            self.chunkList.append ([begin, end, fileName])
            begin += chSize
            end += chSize
            index += 1

        self.totalChunks = index
        self.currentChunk = 0
        self.url = url
        self.theLock = threading.Lock()
        index = 0

        sys.stdout.write ('Total ' + str(self.totalChunks) + '\n')
        sys.stdout.flush ()

        threads = []
        for i in range (0, self._noOfThreads):
            t = ChunkDownloaderThread (self)
            threads.append (t)

        for t in threads:
            t.start()
        for t in threads:
            t.join()

        sys.stdout.write ('\n')
        sys.stdout.flush ()
        return self.totalChunks

    def concatenateFile (self, totalChunks):
        # Flv to raw command
        cmd = [self._ffmpegProgram, '-i']
        completeCmd = ['-vn', '-f', 'u16le', '-acodec', 'pcm_s16le', 
            '-ac', '2', '-ab', '128k', '-ar', '44100', '-']#, '<', '/dev/null']
        tempRawFile = self._tmpFile + self._rawExtension
        f = open (tempRawFile, "wb")
        sys.stdout.write ('Concatenate: \n')
        sys.stdout.flush ()
        for i in range (0, totalChunks):
            flvFile = self._tmpFile + str(i) + self._tmpExtension
            toExe = cmd + [flvFile] + completeCmd

            output = executeCommand (toExe)
            f.write (output[0])
            sys.stdout.write (str (i) + ' ')
            sys.stdout.flush ()
            # Delete the chunk to save disk space
            os.remove (flvFile)
        sys.stdout.write ('\n')
        sys.stdout.flush ()

        f.close ()


    def convertToMp3 (self, destination):
        tempRawFile = self._tmpFile + self._rawExtension
        cmd = [self._lameProgram, '-r', '-s', '44.1', '--preset', 'cd', 
            tempRawFile, destination]
        sys.stdout.write ('Converting to mp3\n')
        sys.stdout.flush ()
        executeCommand (cmd)
        sys.stdout.write ('Done\n')
        sys.stdout.flush ()

    def downloadAndConvertFile (self, url, destination):
        totalChunks = self.downloadFile (url)
        self.concatenateFile (totalChunks)
        self.convertToMp3 (destination)
        self.cleanTempFiles ()


if __name__ == '__main__':
    D = Downloader(60, 15)

    D.downloadAndConvertFile (sys.argv[1], sys.argv[2])

wxPython example. A jigsaw puzzle (3). MS Windows a double buffer

This is a continuation of the previous two entries. I finally got the chance of testing the puzzle on windows and it didn't show properly it flicked a lot. Then I learn that the Windows platform doesn't implement a double buffer so I have to do it.
The idea is painting to a bitmap and then copy it to the DC on the OnPaint event. This solution improved a lot the way application was shown it didn't removed he flicks completely.
After some research I found the solution by capturing the EVT_ERASE_BACKGROUND event and do nothing with it. This fixed all the issues. It also important to call the Update method after the Refresh.

The new board.py code is:

#!/usr/bin/python  

import wx
import stateDB
import rectangle

class Board(wx.Panel):
    boardId = 77
    def __init__ (self, parent, pieces, count, seconds, allowRotation=True):
        wx.Panel.__init__(self, parent, style = wx.NO_FULL_REPAINT_ON_RESIZE)

        self.db = stateDB.StateDB()

            
        self.pieces = pieces
        self.dragged = None
        self.count = count
        self.completed = False

        self.seconds = seconds
        self.timer = wx.Timer (self, wx.ID_ANY) 
        self.Bind (wx.EVT_TIMER, self.OnTimer, self.timer)
        self.timer.Start (1000, False)
        self.printStatus ()

        self.Bind (wx.EVT_PAINT, self.OnPaint)
        self.Bind (wx.EVT_ERASE_BACKGROUND, self.OnEraseBackground)
        self.Bind (wx.EVT_SIZE, self.OnSize)
        self.Bind (wx.EVT_LEFT_DOWN, self.OnDown)
        self.Bind (wx.EVT_LEFT_UP, self.OnUp)
        self.Bind (wx.EVT_MOTION, self.OnMouseMotion)
        if allowRotation:
            self.Bind (wx.EVT_RIGHT_UP, self.OnRight)

    def SaveStatus(self):
        self.db.initiateDB ()
        self.db.saveEverything (Board.boardId, self.count, self.seconds, \
                self.pieces)

    def printStatus (self):
        mins = self.seconds%(60*60)/60
        minStr = str(mins)
        if (mins < 10):
            minStr = '0' + minStr
        secs = self.seconds%60
        secStr = str(secs)
        if (secs < 10):
            secStr = '0' + secStr
        timeStr = str(self.seconds/(60*60)) + ':' + minStr + ':' + secStr

        if self.completed: 
            status = 'Completed: ' + timeStr
        else:
            status = str(self.count) + ': ' + timeStr
        self.GetParent().statusbar.SetStatusText (status)

    def OnTimer (self, e):
        if not self.completed:
            self.seconds += 1
            self.printStatus ()

    def OnEraseBackground (self, e):
 """ To avoid flickering in windows"""
        pass

    def OnPaint (self, e):
        dc = wx.BufferedPaintDC(self, self._Buffer, wx.BUFFER_VIRTUAL_AREA)

    def draw (self, dc):
        dc.Clear ()
        for p in self.pieces:
            p.drawPiece (dc)

    def UpdateDrawing (self):
        dc = wx.MemoryDC ()
        dc.SelectObject (self._Buffer)
        self.draw (dc)
        del dc
        self.Refresh ()
        self.Update ()

    def OnSize (self, e):
        s = self.ClientSize 
        self._Buffer = wx.EmptyBitmap (*s)
        self.UpdateDrawing ()
 

    def OnDown (self, e):
        mousePos = e.GetPosition()

        # The iteration is reversed so the piece on top is choosen 
        indexes = range(len(self.pieces))
        indexes.reverse()
        for i in indexes:
            if self.pieces[i].checkPointInPiece (mousePos):
                # Put the selected at the end of the list so it's painted the last
                p = self.pieces[i] 
                self.pieces.append(p)
                del self.pieces[i]
                self.dragged = len(self.pieces) -1
                break

    def OnUp (self, e):
        if self.dragged != None:
            mousePos = e.GetPosition()
            for i in range(len(self.pieces)):
                if (self.dragged != i):
                    if self.pieces[self.dragged].checkPieceMatch (\
                            self.pieces[i]):
                        del self.pieces[i]
                        #self.Refresh()
                        self.UpdateDrawing ()
                        self.count += 1
                        self.printStatus ()
                        self.SaveStatus ()
                        if len(self.pieces) == 1:
                           if (self.pieces[0].getOrientation() ==\
                                   rectangle.Orientation.or0):
                               self.db.RemoveDBFile ()
                               self.completed = True
                               self.printStatus ()

                        break
            self.dragged = None

    def OnMouseMotion (self, e):
        if self.dragged != None:
            mousePos = e.GetPosition()
            self.pieces[self.dragged].movePiece (mousePos)
            #self.Refresh()
            self.UpdateDrawing ()

    def OnRight (self, e):
        mousePos = e.GetPosition()
        indexes = range(len(self.pieces))
        indexes.reverse()
        for i in indexes:
            if self.pieces[i].checkPointInPiece (mousePos):
                self.pieces[i].incrementOrientation()
                newPos = self.pieces[i].calculateCenterPosition()
                self.pieces[i].changeRelPosition (newPos)
                self.pieces[i].checkPointInPiece (mousePos)
                #self.Refresh()
                self.UpdateDrawing ()
                if len(self.pieces) == 1:
                   if (self.pieces[0].getOrientation() == \
                           rectangle.Orientation.or0):
                       self.completed = True
                       self.printStatus ()
                break

class Puzzle (wx.Frame):
    def __init__ (self, parent, id, title, pieces, boardSize, count,
            seconds, allowRotation=True):
        wx.Frame.__init__(self, parent, id, title, size=boardSize)

        self.statusbar = self.CreateStatusBar()
        self.board = Board(self, pieces, count, seconds, allowRotation)

        self.Centre()
        self.Show(True)

wxPython example. A jigsaw puzzle (2)

Continuing with the previous post I created another puzzle, but this time the pieces has the shape of the mainland Spain provinces. I think is a fun way to learn geography and can be used for any map.



The code is basically what described in the previous post. The only difference is the way the pieces masks are created. I did it manually with gimp. For example:



The main file of the application, which I call prov.py, has the following code:

#!/usr/bin/python  

import board
import stateDB
import rectangle
import piece
import wx
import random

# Id, image, mask, size, top-left positions, relations
provinceList = [
    [1, 'coruna.png', 'corunaMask.png', (106, 96), (0,0), [2, 3] ],
    [2, 'lugo.png', 'lugoMask.png', (72, 119), (82,0), [1, 3, 4, 5, 37 ]], 
    [3, 'pontevedra.png', 'pontevedraMask.png', (70, 79), (17, 73), [1,2,4]],
    [4, 'orense.png', 'orenseMask.png', (97, 70), (60,94), [2, 3, 25, 37]],
    [5, 'asturias.png', 'asturiasMask.png', (159, 66), (134,11), [2,6, 37]],
    [6, 'cantabria.png', 'cantabriaMask.png', (104,64), (274, 25), [5,7, 37, 
        40, 41]],
    [7, 'vizcaya.png', 'vizcayaMask.png', (68, 35), (356,32), [6, 8, 9, 40]],
    [8, 'guipuzcoa.png', 'guipuzcoaMask.png', (50, 42), (412,34), [7, 9, 10]],
    [9, 'alava.png','alavaMask.png', (62, 60), (372, 52), [8, 7, 10, 39, 40 ]],
    [10, 'navarra.png', 'navarraMask.png', (102, 116), (423, 38), [8,9, 11, 
        12, 39]],
    [11, 'huesca.png', 'huescaMask.png', (112, 126), (511,69), [10, 12, 14 ]], 
    [12, 'zaragoza.png', 'zaragozaMask.png', (160, 147), (441, 85), [11, 10,
        13, 14, 17, 18, 38]], 
    [13, 'teruel.png', 'teruelMask.png', (132, 122), (464, 197), [12, 17, 18,
        33, 34, 35]], 
    [14, 'lleida.png', 'lleidaMask.png', (93,133), (596, 69), [ 11, 12, 15, 
        16, 17]], 
    [15, 'girona.png', 'gironaMask.png', (94, 72), (683,89), [ 14, 16 ]],
    [16, 'barcelona.png', 'barcelonaMask.png', (85, 96), (662, 108), [14, 15, 
        17]], 
    [17, 'tarra.png', 'tarraMask.png', (93,91), (589,169), [16, 14, 13, 12, 
        34]], 
    [18, 'guada.png', 'guadaMask.png', (127, 95), (354, 202), [ 13, 12, 19,
        35, 38, 43]],
    [19, 'madrid.png', 'madridMask.png', (101,100), (287, 219), [18, 20, 35, 
        43, 44]],
    [20, 'avila.png', 'avilaMask.png',  (103,91), (212,216), [19, 21, 24, 42, 
        43, 44]],
    [21, 'caceres.png', 'caceresMask.png', (165, 119), (96, 270), [20, 22, 
        24, 44]], 
    [22, 'badajoz.png', 'badajozMask.png', (177,125), (106, 352), [21, 23, 26,
        44, 45, 47]],
    [23, 'huelva.png', 'huelvaMask.png', (91,112), (87, 456), [22, 26 ]],
    [24, 'salamanca.png', 'salamancaMask.png',(113, 94), (141,198), [20, 21, 
        25, 42 ]],
    [25, 'zamora.png', 'zamoraMask.png',(111, 84), (136,127), [24, 4, 37, 42 ]],
    [26, 'sevilla.png', 'sevillaMask.png',(128, 116), (153, 456), [22, 23, 
        27, 28, 47 ]],
    [27, 'cadiz.png', 'cadizMask.png', (96,84), (154, 555), [26, 28]],
    [28, 'malaga.png', 'malagaMask.png', (122,84), (221, 532), [27, 26, 29, 
        47]],
    [29, 'granada.png', 'granadaMask.png', (138,117), (303, 466), [28,30, 31,
        36, 46, 47]],
    [30, 'almeria.png', 'almeriaMask.png', (105,103), (381, 480), [29, 31]],
    [31, 'murcia.png', 'murciaMask.png', (108, 114), (434, 410), [29, 32, 36]],
    [32, 'alicante.png', 'alicanteMask.png', (81,92), (520,391), [31, 33, 36]],
    [33, 'valencia.png', 'valenciaMask.png', (95,121), (485,292), [32, 13, 34,
        35, 36]],
    [34, 'castellon.png', 'castellonMask.png', (86,89), (526,242), [33, 13, 
        17]],
    [35, 'cuenca.png', 'cuencaMask.png', (132, 118), (376, 256), [18, 19, 13,
        33, 36, 44, 45]],
    [36, 'albacete.png', 'albaceteMask.png', (129,119), (397, 354), [35, 29,
        36, 32, 33, 31, 45, 46]],
    [37, 'leon.png', 'leonMask.png', (143, 102), (138, 50), [2, 4, 25, 5, 6, 
        41, 42]],
    [38, 'soria.png', 'soriaMask.png', (107, 89), (358,136), [12, 18, 39, 40, 
        43 ]],
    [39, 'rioja.png', 'riojaMask.png', (83,55), (383, 96), [38, 9, 10, 40]],
    [40, 'burgos.png', 'burgosMask.png', (110,141), (306,52), [39, 38, 6, 7, 
        9, 41, 42, 43]],
    [41, 'palencia.png', 'palenciaMask.png', (70,100), (262,65), [37, 6, 40, 
        42 ]],
    [42, 'valla.png', 'vallaMask.png', (96,98), (235, 125), [37, 40, 41, 20,
        25, 24, 43]],
    [43, 'segovia.png', 'segoviaMask.png', (92,78), (280, 185), [42, 40, 38, 
        18, 19, 20]],
    [44, 'toledo.png', 'toledoMask.png', (160, 83), (233, 287), [35,21, 22,
        19, 20, 45]],
    [45, 'creal.png', 'crealMask.png', (158, 103), (255, 344), [44, 36, 35,
        22, 46, 47 ]],
    [46, 'Jaen.png', 'JaenMask.png', (122, 98), (306, 430), [45, 36, 29, 47]],
    [47, 'cordoba.png', 'cordobaMask.png', (109,129), (218,412), [46, 45, 26, 
        28, 29, 22]]
    ]


def createPieceList (provinceList, boardSize):
    pieces = []

    for entry in provinceList:
        im = wx.Image (entry[1])
        mask = wx.Image (entry[2])
        im.SetMaskFromImage (mask, 255, 255, 255)

        rels = []
        # Calculate the relations
        for related in entry[5]:
            index = related -1
            difX = provinceList [index][4][0] - entry[4][0]
            difY = provinceList [index][4][1] - entry[4][1]
            rel = [related, (difX, difY), 0, 0]
            rels.append (rel)

        rec = rectangle.Rectangle (entry[0], (im, mask), entry[3], 
                rels, 0, 0)
        p = piece.Piece (entry[0])
        p.addRectangle (rec, (0, 0))
        pieces.append (p)

    for i in range (len(pieces)):
        pieces[i].setPosition ( (random.randint (10, boardSize[0] - 300),\
                random.randint(10, boardSize[1] - 300)))
#pieces[i].setOrientation (rectangle.getRandomOrientation())

    # Re-order the list so we don't always get the same provinces on top
    random.shuffle (pieces)

    return pieces

boardSize = (1000, 800)
app = wx.App ()
db = stateDB.StateDB()
count = -1
seconds = 0

if db.isThereADBFile ():
    count, seconds, pieces = db.readBoard (board.Board.boardId)

if count <=0:
    pieces= createPieceList (provinceList, boardSize)
    count = 0

board.Puzzle(None, -1, 'Spain mainland provinces', pieces, boardSize, count,
        seconds, False)
app.MainLoop()

wxPython example. A jigsaw puzzle

I've been teaching myself a bit about python and sqlite. To do so I have created a puzzle. The thing is pretty basic so far but it's playable. You can use any picture and the pieces are created automatically. The pieces are dragged by pressing the mouse left button and dropped when released. They are rotated with the right button.

I've tested it in Linux and Mac and it should work in Windows as well. You may need to install python and wxWidgets. The Mac version wxPython is only supported for 32bits so you must define the foillowing variable before executing:

export VERSIONER_PYTHON_PREFER_32_BIT=yes





The basic element of the puzzle is a rectangle. A rectangle has an image, a mask and they store the relative position of other rectangles. They can be moved around and rotated. The following code must be included in a file called rectangle.py.

import math
import wx
import random

class Orientation (object):
    or0 = 0
    or90 = 1
    or180 = 2
    or270 = 3

def rotateImage (orientation, im):
    imAux = im
    for i in range (orientation):
        imAux = imAux.Rotate90 (True)

    return imAux

def rotateRelPosition (orientation, pos, compX, compY):
    posAux = pos

    if orientation == Orientation.or90:
        posAux =(-posAux[1]  - compY, posAux[0])
    elif orientation == Orientation.or180:
        posAux =(-posAux[0]  - compX, -posAux[1] - compY)
    elif orientation == Orientation.or270:
        posAux =(posAux[1], - posAux[0]  - compX)

#for i in range (orientation):
#       posAux = (-posAux[1], posAux[0])
    return posAux

def rotateSize (orientation, size):
    sizeAux = size
    if (orientation == Orientation.or90 or orientation == Orientation.or270):
        sizeAux = (sizeAux[1], sizeAux[0])
    return sizeAux


def nextOrientation (orientation):
    if orientation == Orientation.or270:
        orientation = Orientation.or0
    else:
        orientation +=1
    return orientation

def getRandomOrientation ():
    return random.randint (Orientation.or0, Orientation.or270)

class Rectangle (object):
    def __init__ (self, id, im, size, relations, compX, compY):
        self.id = id
        self.image = im[0]
        self.mask = im[1]
        self.size = size
        self.relations = relations
        self.position = (0,0)
        self.orientation = Orientation.or0
        self.compX = compX
        self.compY = compY

    def __str__ (self):
        rectStr = "Rectangle " + str(self.id) + " size: " + str(self.size) + \
               ", pos: " + str(self.position) + ", comp: " + \
                str((self.compX, self.compY))+ ", or: " + \
                str(self.orientation) + ", rel: " + str(self.relations)
        return rectStr

    def setPosition (self, pos):
        self.position = pos

    def setOrientation (self, ort):
        self.orientation = ort

    def getCompX (self):
        self.compX

    def getCompY (self):
        self.compY

    def incrementOrientation (self):
        self.orientation = nextOrientation (self.orientation)

    def removeRelations(self, id):
        index = 0
        for index in range(len(self.relations)):
            if self.relations[index][0] == id:
                del self.relations[index]
                break

    def getId (self):
        return self.id

    def getImage (self):
        return self.image

    def getMask (self):
        return self.mask

    def getSize (self):
        return self.size

    def getPosition (self):
        return self.position

    def getOrientation (self):
        return self.orientation

    def getAbsolutePosition (self, absPosition):
        relPos = rotateRelPosition (self.orientation, self.position, 
                self.compX, self.compY)
        return (( absPosition[0] + relPos[0],\
                absPosition[1] + relPos[1]))

    def drawRectangle (self, absPosition, dc):
        absPos = self.getAbsolutePosition (absPosition)
        im = rotateImage (self.orientation, self.image)
        dc.DrawBitmap ( wx.BitmapFromImage(im), absPos[0],\
                absPos[1], True)

    def changeRelPosition (self, shift):
        self.position = ( self.position[0] - shift[0], \
                self.position[1] - shift[1])

    def checkPointInRectangle (self, absPosition, point):
        """ Returns the relative position of the point. """
        retVal = False

        relPos = (0, 0)
        absPos = self.getAbsolutePosition (absPosition)

        rotatedSize = rotateSize (self.orientation, self.size)

        if (point[0] >= absPos[0]) and \
           (point[0] < absPos[0] + rotatedSize[0]) and \
           (point[1] >= absPos[1]) and \
           (point[1] < absPos[1] + rotatedSize[1]):
               retVal = True
               relPos = (point[0] - absPos[0] + self.position[0], 
                       point[1] - absPos[1] + self.position[1])

        return (retVal, relPos)

    def checkRectagleMatch (self, absPosition, id, pos):
        """ Checks if another rectangle matches.
            id, the id of the other rectangle
            pos, the absolute position of the other rectangle. 
            The function returns the relative position of the other rectangle
            with respect to this piece."""
        retVal = False
        relPos = (0, 0)
        absPos = self.getAbsolutePosition (absPosition)
        for rel in self.relations:
            if rel[0] == id:
                rotatedRel = rotateRelPosition (self.orientation, rel[1],\
                        rel[2], rel[3])
                absPos = (absPos[0] + rotatedRel[0], absPos[1] + rotatedRel[1])

                distX = pos[0] - absPos[0]
                distY = pos[1] - absPos[1]

                if (math.sqrt((distX * distX) + (distY * distY)) < 4.0):
                    retVal = True
                    relPos= (self.position[0] + rel[1][0],\
                        self.position[1] + rel[1][1])

                break
        return retVal, relPos

The next element of the puzzle are the pieces. A piece contains one or more rectangles. The number of pieces gets reduces as the puzzle is solved. The code should be put in a file called piece.py.

import rectangle
import wx

class Piece(object):
    def __init__ (self, id):
        self.id = id
        self.pos = (0,0)
        self.mousePos = (0, 0)
        self.rectangles = []
        self.orientation = rectangle.Orientation.or0

    def __str__ (self):
        pieceStr = 'Piece ' + str(self.id) + ' pos: ' + str(self.pos) + \
                   ' mousePos: ' + str(self.mousePos) + ' or: '+\
                    str(self.orientation) + ' , rect:\n'
        for rect in self.rectangles:
            pieceStr += str(rect) + '\n'
        return pieceStr

    def getId (self):
        return self.id

    def getPosition (self):
        return (self.pos)

    def setPosition (self, pos):
        self.pos = pos

    def setOrientation (self, ort):
        self.orientation = ort
        for index in range(len(self.rectangles)):
            self.rectangles[index].setOrientation (ort)

    def addRectangle (self, rect, relPos):
        for index in range(len(self.rectangles)):
            self.rectangles[index].removeRelations (rect.getId())

        for index in range(len(self.rectangles)):
            rect.removeRelations(self.rectangles[index].getId())

        rect.setPosition (relPos)
        self.rectangles.append (rect)

    def getNoOfRectangles (self):
        return len (self.rectangles)

    def getRectangle (self, i):
        return self.rectangles[i]

    def getRectangleId (self, i):
        return self.rectangles[i].getId ()

    def getRectangleAbsPos (self, i):
        return self.rectangles[i].getAbsolutePosition (self.pos)

    def changeRelPosition (self, shift):
        for rect in self.rectangles:
            rect.changeRelPosition (shift)

    def calculateCenterPosition (self):
        maxLeft = 0 
        maxRight = 0
        maxTop = 0
        maxBottom = 0
        for rect in self.rectangles:
            x = rect.getPosition()[0]
            if maxLeft > x:
                maxLeft = x
            elif maxRight <  x:
                maxRight = x
            y = rect.getPosition()[1]
            if maxTop > y:
                maxTop = y
            elif maxBottom < y:
                maxBottom = y

        c = ((maxLeft + maxRight) / 2, (maxTop + maxBottom)/2)
        return c
            


    def drawPiece (self, dc):
        for rect in self.rectangles:
            rect.drawRectangle (self.pos, dc)

    def checkPointInPiece (self, point):
        retVal = False
        for rect in self.rectangles:
            ret, relPos = rect.checkPointInRectangle (self.pos, point)
            if ret:
                retVal = True
                self.mousePos = relPos
                break

        return retVal

    def checkPieceMatch (self, otherPiece):
        retVal = False

        if (self.orientation == otherPiece.getOrientation()):
            for i in range(otherPiece.getNoOfRectangles()):
                otherRect = otherPiece.getRectangle (i)
                for rect in self.rectangles:
                    ret, relPos = rect.checkRectagleMatch (self.pos,
                        otherRect.getId(), 
                        otherPiece.getRectangleAbsPos (i))
                    if ret:
                        retVal = True
                        # Calculate the relative position of the other piece
                        posOtherPiece = (relPos[0] - otherRect.getPosition()[0],
                            relPos[1] - otherRect.getPosition()[1])

                        # Include the rectangles in this piece
                        for j in range(otherPiece.getNoOfRectangles()):
                            otherRect = otherPiece.getRectangle(j)
                            self.addRectangle (otherRect, 
                                (posOtherPiece[0] + otherRect.getPosition()[0],
                                posOtherPiece[1] + otherRect.getPosition()[1]))


                        break

        return retVal

    def movePiece (self, newPos):
        self.pos = (newPos[0] - self.mousePos[0], 
                newPos[1] - self.mousePos[1])

    def incrementOrientation (self):
        self.orientation = rectangle.nextOrientation (self.orientation) 
        for i in range(len(self.rectangles)):
            self.rectangles[i].setOrientation(self.orientation)

    def getOrientation (self):
        return self.orientation

The board.py file contains the Board class that creates the gui panel and manages the events.

import wx
import stateDB
import rectangle

class Board(wx.Panel):
    boardId = 77
    def __init__ (self, parent, pieces, count, seconds, allowRotation=True):
        wx.Panel.__init__(self, parent)
                
        self.db = stateDB.StateDB()
            
        self.pieces = pieces
        self.dragged = None
        self.count = count
        self.completed = False

        self.seconds = seconds
        self.timer = wx.Timer (self, wx.ID_ANY) 
        self.Bind (wx.EVT_TIMER, self.OnTimer, self.timer)
        self.timer.Start (1000, False)
        self.printStatus ()

        self.Bind (wx.EVT_PAINT, self.OnPaint)
        self.Bind (wx.EVT_LEFT_DOWN, self.OnDown)
        self.Bind (wx.EVT_LEFT_UP, self.OnUp)
        self.Bind (wx.EVT_MOTION, self.OnMouseMotion)
        if allowRotation:
            self.Bind (wx.EVT_RIGHT_UP, self.OnRight)

    def SaveStatus(self):
        self.db.initiateDB ()
        self.db.saveEverything (Board.boardId, self.count, self.seconds, \
                self.pieces)

    def printStatus (self):
        mins = self.seconds%(60*60)/60
        minStr = str(mins)
        if (mins < 10):
            minStr = '0' + minStr
        secs = self.seconds%60
        secStr = str(secs)
        if (secs < 10):
            secStr = '0' + secStr
        timeStr = str(self.seconds/(60*60)) + ':' + minStr + ':' + secStr

        if self.completed: 
            status = 'Completed: ' + timeStr
        else:
            status = str(self.count) + ': ' + timeStr
        self.GetParent().statusbar.SetStatusText (status)

    def OnTimer (self, e):
        if not self.completed:
            self.seconds += 1
            self.printStatus ()

    def OnPaint (self, e):
        dc = wx.PaintDC(self)

        for p in self.pieces:
            p.drawPiece (dc)

    def OnDown (self, e):
        mousePos = e.GetPosition()

        # The iteration is reversed so the piece on top is choosen 
        indexes = range(len(self.pieces))
        indexes.reverse()
        for i in indexes:
            if self.pieces[i].checkPointInPiece (mousePos):
                # Put the selected at the end of the list so it's painted the last
                p = self.pieces[i] 
                self.pieces.append(p)
                del self.pieces[i]
                self.dragged = len(self.pieces) -1
                break

    def OnUp (self, e):
        if self.dragged != None:
            mousePos = e.GetPosition()
            for i in range(len(self.pieces)):
                if (self.dragged != i):
                    if self.pieces[self.dragged].checkPieceMatch (\
                            self.pieces[i]):
                        del self.pieces[i]
                        self.Refresh()
                        self.count += 1
                        self.printStatus ()
                        self.SaveStatus ()
                        if len(self.pieces) == 1:
                           if (self.pieces[0].getOrientation() ==\
                                   rectangle.Orientation.or0):
                               self.db.RemoveDBFile ()
                               self.completed = True
                               self.printStatus ()

                        break
            self.dragged = None

    def OnMouseMotion (self, e):
        if self.dragged != None:
            mousePos = e.GetPosition()
            self.pieces[self.dragged].movePiece (mousePos)
            self.Refresh()

    def OnRight (self, e):
        mousePos = e.GetPosition()
        indexes = range(len(self.pieces))
        indexes.reverse()
        for i in indexes:
            if self.pieces[i].checkPointInPiece (mousePos):
                self.pieces[i].incrementOrientation()
                newPos = self.pieces[i].calculateCenterPosition()
                self.pieces[i].changeRelPosition (newPos)
                self.pieces[i].checkPointInPiece (mousePos)
                self.Refresh()
                if len(self.pieces) == 1:
                   if (self.pieces[0].getOrientation() == \
                           rectangle.Orientation.or0):
                       self.completed = True
                       self.printStatus ()
                break

class Puzzle (wx.Frame):
    def __init__ (self, parent, id, title, pieces, boardSize, count,
            seconds, allowRotation=True):
        wx.Frame.__init__(self, parent, id, title, size=boardSize)

        self.statusbar = self.CreateStatusBar()
        self.board = Board(self, pieces, count, seconds, allowRotation)

        self.Centre()
        self.Show(True)

The state of the puzzle is stored in a sqlite database. The access to database is implemented in a file called stateDB.py.

import sqlite3
import os
import piece
import rectangle
import wx


class StateDB (object):
    # Names of the tables
    boardTable = 'board'
    pieceTable = 'piece'
    rectangleTable = 'rectangle'
    relationTable = 'relation'

    # The name of the DB file
    DBFileName = '.puzzle.db'

    # field names
    ekey= 'key'
    BTcount = 'count'
    BTseconds = 'seconds'

    PTboard = 'board'
    PTposx = 'posx'
    PTposy = 'posy'
    PTorientation = 'orientation'

    RTpiece = 'piece'
    RTsizex = 'sizex'
    RTsizey = 'sizey'
    RTposx = 'posx'
    RTposy = 'posy'
    RTcompx = 'compx'
    RTcompy = 'compy'
    RTimage = 'image'
    RTmask = 'mask'

    RTsource = 'source'
    RTdest = 'dest'
    RTrelx = 'relx'
    RTrely = 'rely'
    RelTCompX = 'compX'
    RelTCompY = 'compY'

    def __init__ (self):
        self.con = None
        self.cur = None

    def OpenDB (self):
        self.con = sqlite3.connect (StateDB.DBFileName)
        self.cur = self.con.cursor ()

    def CloseDB (self):
        self.cur.close()
        self.con.commit()
        self.con.close()
        self.cur = None
        self.con = None

    def RemoveDBFile (self):
        if os.path.exists (StateDB.DBFileName):
            os.remove (StateDB.DBFileName)

    def isThereADBFile (self):
        return os.path.exists (StateDB.DBFileName)

    def CreateTables (self):
        createBT = 'CREATE TABLE ' + StateDB.boardTable + ' (' + \
                StateDB.ekey + ' INTEGER PRIMARY KEY, ' + \
                StateDB.BTcount + ' INTEGER,' +\
                StateDB.BTseconds + ' INTEGER)'
        self.cur.execute (createBT)

        createPT = 'CREATE TABLE ' + StateDB.pieceTable + ' (' + \
                StateDB.ekey + ' INTEGER PRIMARY KEY, ' + \
                StateDB.PTboard + ' INTEGER, ' + \
                StateDB.PTposx + ' INTEGER, ' + \
                StateDB.PTposy + ' INTEGER, ' + \
                StateDB.PTorientation + ' INTEGER, ' + \
                'FOREIGN KEY (' + StateDB.PTboard + ') REFERENCES ' + \
                StateDB.boardTable + '(' + StateDB.ekey + '))'
        self.cur.execute (createPT)

        createRT = 'CREATE TABLE ' + StateDB.rectangleTable + '(' + \
                   StateDB.ekey + ' INTEGER PRIMARY KEY, ' + \
                   StateDB.RTpiece + ' INTEGER, ' + \
                   StateDB.RTsizex + ' INTEGER, ' + \
                   StateDB.RTsizey + ' INTEGER, ' + \
                   StateDB.RTposx + ' INTEGER, ' + \
                   StateDB.RTposy + ' INTEGER, ' + \
                   StateDB.RTcompx + ' INTEGER, ' + \
                   StateDB.RTcompy + ' INTEGER, ' + \
                   StateDB.RTimage + ' BLOB, ' + \
                   StateDB.RTmask + ' BLOB, ' + \
                   'FOREIGN KEY (' + StateDB.RTpiece + ') REFERENCES ' + \
                   StateDB.pieceTable + '(' + StateDB.ekey + '))'
        self.cur.execute (createRT)

        createRT = 'CREATE TABLE ' + StateDB.relationTable + '(' + \
                   StateDB.ekey + ' INTEGER PRIMARY KEY, ' + \
                   StateDB.RTsource + ' INTEGER, ' + \
                   StateDB.RTdest + ' INTEGER, ' + \
                   StateDB.RTrelx + ' INTEGER, ' + \
                   StateDB.RTrely + ' INTEGER, ' + \
                   StateDB.RelTCompX + ' INTEGER, ' + \
                   StateDB.RelTCompY + ' INTEGER, ' + \
                   'FOREIGN KEY (' + StateDB.RTsource + ') REFERENCES ' + \
                   StateDB.rectangleTable + '(' + StateDB.ekey + '), ' + \
                   'FOREIGN KEY (' + StateDB.RTdest + ') REFERENCES ' + \
                   StateDB.rectangleTable + '(' + StateDB.ekey + '))'
        self.cur.execute (createRT)

    def initiateDB (self):
        self.RemoveDBFile ()
        self.OpenDB ()
        self.CreateTables ()
        self.CloseDB ()

    def saveBoard (self, id, count, seconds):
        query = 'INSERT INTO ' + StateDB.boardTable + ' VALUES (?,?,?)'
        self.cur.execute (query, (id, count, seconds))

    def saveRelation (self, rectId, rel):
        query = ' INSERT INTO ' + StateDB.relationTable + '(' + \
                StateDB.RTsource + ',' + StateDB.RTdest + ',' + \
                StateDB.RTrelx + ',' + StateDB.RTrely + ', ' + \
                StateDB.RelTCompX + ', ' + StateDB.RelTCompY + ') ' + \
                ' VALUES (?,?,?,?,?,?) '
        entry = (rectId, rel[0], rel[1][0], rel[1][1],rel[2],rel[3])
        self.cur.execute (query, entry)


    def saveRectangle (self, pieceId, rect):
        query = ' INSERT INTO ' + StateDB.rectangleTable + \
                ' VALUES (?,?,?,?,?,?,?,?,?,?) '
        size = rect.getSize ()
        pos = rect.getPosition()
        imData = rect.getImage().GetData()
        maskData = rect.getMask().GetData()
        entry = (rect.getId(), pieceId, size[0], size[1], pos[0], pos[1], 
                rect.compX, rect.compY, sqlite3.Binary(imData),
                sqlite3.Binary(maskData))
        self.cur.execute (query, entry)
        for rel in rect.relations:
            self.saveRelation (rect.getId(), rel)

    def savePiece (self, boardId, piece):
        query = ' INSERT INTO ' + StateDB.pieceTable + ' VALUES (?,?,?,?,?) '
        pos = piece.getPosition()
        entry = (piece.getId(), boardId,pos[0], pos[1], piece.getOrientation())
        self.cur.execute (query, entry)
        for i in range(piece.getNoOfRectangles()):
            self.saveRectangle (piece.getId(), piece.getRectangle(i))

    def saveEverything (self, id, count, seconds, pieces):
        self.OpenDB ()
        self.saveBoard (id, count, seconds)
        for piece in pieces:
            self.savePiece (id, piece)
        self.CloseDB ()

    def readRelations (self, rectId):
        relations = []
        query = 'SELECT * FROM ' + StateDB.relationTable + ' WHERE ' + \
                StateDB.RTsource + '=' + str(rectId)
        self.cur.execute(query)
        relList = self.cur.fetchall ()
        for entry in relList:
            r = [entry[2], (entry[3], entry[4]), entry[5], entry[6]]
            relations.append (r)
        return relations

    def readRectangles (self, pieceId):
        rectangles = []
        query = 'SELECT * FROM ' + StateDB.rectangleTable + ' WHERE ' + \
                StateDB.RTpiece + '=' + str(pieceId)
        self.cur.execute(query)
        rectList = self.cur.fetchall ()
        for entry in rectList:
            im = wx.ImageFromData (entry[2], entry[3], entry[8])
            mask = wx.ImageFromData (entry[2], entry[3], entry[9])
            im.SetMaskFromImage (mask, 255, 255, 255)
            rels = self.readRelations (entry[0])
            r = rectangle.Rectangle (entry[0], (im, mask),\
                    (entry[2], entry[3]), rels, entry[6], entry[7])
            pos = (entry[4], entry[5])
            rectangles.append((r, pos))
        return rectangles

    def readPieces (self, boardId):
        pieces = []
        query = 'SELECT * FROM ' + StateDB.pieceTable + ' WHERE ' + \
                StateDB.PTboard + '=' + str(boardId)
        self.cur.execute (query)
        pieceList = self.cur.fetchall ()
        for entry in pieceList:
            p = piece.Piece (entry[0]) 
            p.setPosition ((entry[2], entry[3]))
            rectangles = self.readRectangles (entry[0])
            for r in rectangles:
                p.addRectangle (r[0], r[1])
            p.setOrientation (entry[4])
            pieces.append (p)
        return pieces

    def readBoard (self, id):
        self.OpenDB()
        count = -1
        pieces = []

        query = 'SELECT ' + StateDB.BTcount + ', ' + StateDB.BTseconds +\
                ' FROM ' + StateDB.boardTable +\
                ' WHERE ' + StateDB.ekey + '=' + str(id)
        self.cur.execute (query)
        countList = self.cur.fetchall ()
        if countList != None:
            count = countList[0][0]
            seconds = countList[0][1]
            pieces = self.readPieces(id)
        self.CloseDB()

        return count, seconds, pieces

Finally, the main file, puzzle.py, takes an image creates all the pieces automatically and then starts the GUI.

#!/usr/bin/python  

import wx
import random
import piece
import rectangle
import stateDB
import board


def createMask (sizeX, sizeY, overlap, dims, pos):
    theBuffer = bytearray()

    w = 255
    b = 0

    leftLimit = (sizeX/2) - overlap/2
    rightLimit = (sizeX/2) +overlap/2
    topLimit = (sizeY/2) - overlap/2
    bottomLimit = (sizeY/2) + overlap/2


    even = (((pos[0] + pos[1])% 2) == 0)

    circleShift = 2
    radiusSquare = ((overlap+circleShift)*(overlap+circleShift))/4

    for j in range(sizeY ):
        for i in range (sizeX ):
            setWhite = False

            setWhite = False
            if (pos[0] < dims[0] -1):
                if even:
                    if (i >= sizeX - overlap):
                        newJ = topLimit + (overlap /2) - j
                        newI = sizeX - (overlap/2) - circleShift -i
                        if (newI * newI + newJ * newJ) > radiusSquare:
                            setWhite = True
                else:
                    if (i >= sizeX - overlap):
                        setWhite = True
                    if (i < sizeX - overlap) and (i >= sizeX - overlap*2):
                        newJ = topLimit + (overlap /2) - j
                        newI = sizeX - (3*overlap/2) + circleShift -i
                        if (newI * newI + newJ * newJ) <= radiusSquare:
                            setWhite = True

            if (pos[0] > 0):
                if not even:
                    if (i < overlap): 
                        setWhite = True
                    if (i >= overlap) and (i < overlap *2):
                        newI = (3*overlap/2) - circleShift -i
                        newJ = topLimit + (overlap/2) - j
                        if (newI * newI + newJ * newJ) <= radiusSquare:
                            setWhite = True
                else:
                    if (i < overlap): 
                        newI = (overlap/2) + circleShift - i
                        newJ = topLimit + (overlap/2) -j
                        if (newI * newI + newJ * newJ) > radiusSquare:
                            setWhite = True

            if (pos[1] < dims[1] - 1):
                if not even:
                    if (j >= sizeY - overlap): 
                        newI = leftLimit + (overlap/2) -i
                        newJ = sizeY - (overlap/2) - circleShift - j
                        if (newI * newI + newJ * newJ) > radiusSquare:
                            setWhite = True
                else: 
                    if (j >= sizeY - overlap):
                        setWhite = True
                    if (j < sizeY - overlap) and (j >= sizeY - overlap*2):
                        newI = leftLimit + (overlap/2) -i
                        newJ = sizeY - (3*overlap/2) + circleShift - j
                        if (newI * newI + newJ * newJ) <= radiusSquare:
                            setWhite = True

            if (pos[1] > 0):
                if not even:
                    if (j < overlap):
                        newI = leftLimit + (overlap/2) -i
                        newJ = (overlap/2) +circleShift - j
                        if (newI * newI + newJ * newJ) > radiusSquare:
                            setWhite = True
                else:
                    if (j < overlap): 
                        setWhite = True
                    if (j >= overlap) and (j < overlap * 2):
                        newI = leftLimit + (overlap/2) -i
                        newJ = (3*overlap/2) - circleShift - j
                        if (newI * newI + newJ * newJ) <= radiusSquare:
                            setWhite = True
            if setWhite:
                color = w
            else:
                color = b
            theBuffer.append (color)
            theBuffer.append (color)
            theBuffer.append (color)

    return wx.ImageFromData (sizeX , sizeY, theBuffer)

def createPieceList (imFile, boardSize):
    sizeX = 50
    sizeY = 50
    overlap = 12
    im = wx.Image (imFile)
    imW = im.GetWidth()
    imH = im.GetHeight()
    pieces = []

    noOfRows = imH/sizeY
    noOfCols = imW/sizeX


    id = 0
    for j in range (noOfRows):
        for i in range (noOfCols):
            # SubImage for each rectangle
            iPos = 0
            if i > 0:
                iPos = i * sizeX - overlap
            jPos = 0
            if j > 0:
                jPos = j * sizeY - overlap

            curSizeX = sizeX + overlap
            if i > 0  and i < noOfCols -1:
                curSizeX += overlap
            curSizeY = sizeY + overlap
            if j > 0  and j < noOfRows -1:
                curSizeY += overlap


            r = wx.Rect (iPos, jPos, curSizeX, curSizeY)
            imAux = im.GetSubImage (r)

            mask = createMask (curSizeX, curSizeY, overlap, \
                    (noOfCols, noOfRows), (i, j))
            imAux.SetMaskFromImage (mask, 255, 255, 255)


            rels = []
            if (i > 0):
                if (i==1):
                    rels.append ([id-1,(-(sizeX-overlap), 0), -overlap, 0]) 
                elif (i == (noOfCols - 1)):
                    rels.append ([id-1,(-(sizeX), 0), overlap, 0]) 
                else:
                    rels.append ([id-1,(-(sizeX), 0), 0, 0]) 
            if (i < (noOfCols -1)):
                if (i==0):
                    rels.append ([id+1, (sizeX-overlap, 0), overlap, 0])
                elif (i == (noOfCols -2)):
                    rels.append ([id+1, (sizeX, 0), -overlap, 0])
                else:
                    rels.append ([id+1, (sizeX, 0), 0, 0])

            if (j > 0):
                if (j==1):
                    rels.append([id-noOfCols, (0, -(sizeY-overlap)), 0,
                            -overlap])
                elif (j == (noOfRows -1)):
                    rels.append([id-noOfCols, (0, -(sizeY)), 0, overlap])
                else:
                    rels.append([id-noOfCols, (0, -(sizeY)), 0, 0])
            if (j < (noOfRows - 1)): 
                if (j==0):
                    rels.append([id+noOfCols, (0, sizeY-overlap), 0, overlap])
                elif (j == (noOfRows -2)):
                    rels.append([id+noOfCols, (0, sizeY), 0, -overlap])
                else:
                    rels.append([id+noOfCols, (0, sizeY), 0, 0])

            compX, compY = calculateCompensantion (i, j, noOfRows, noOfCols,
                    overlap)


            rec = rectangle.Rectangle (id, (imAux, mask),
                    (curSizeX, curSizeY), rels, compX, compY)
            p = piece.Piece (id)
            p.addRectangle (rec, (0,0))
            pieces.append (p)
            id +=1

    for i in range (len(pieces)):
        pieces[i].setPosition ( (random.randint (10, boardSize[0] -300),\
                random.randint(10, boardSize[1] -300)))
        pieces[i].setOrientation (rectangle.getRandomOrientation())

    return pieces

def calculateCompensantion (i, j, noOfRows, noOfCols, overlap):
    compX = 0
    compY = 0
    if i==0:
        compX = -overlap
    elif i== (noOfCols -1):
        compX = -overlap

    if j==0:
        compY = -overlap
    elif j == (noOfRows -1):
        compY = -overlap

    return compX, compY

boardSize = (1000, 800)
app = wx.App ()
db = stateDB.StateDB()
count = -1
seconds = 0


if db.isThereADBFile ():
    count, seconds, pieces = db.readBoard (board.Board.boardId)

if count <=0:
    pieces= createPieceList ('im2.png', boardSize)
    count = 0


board.Puzzle(None, -1, 'Puzzle', pieces, boardSize, count, seconds)
app.MainLoop()

Monitoring processes with Python

We've been working on some scripts to monitor the behavior of a process. The idea is run a process for days and register its memory and CPU usage. We also want to run in both Linux and Windows.

In the Linux environment the top command is used to find out CPU usage and the proc filesystem to obtain the memory usage. The ps command is used to obtain the list of processes.

The win32 extensions for python library is being used for Windows. I've found a bit difficult to work with this library and that's is the main reason I'm writing this post.

Finding out the process ID

The first thing is to find out process to monitor. We are assuming the process to monitor is the lastest descendent of the current process. We have created a function to list all the processes along with their parent proces and then another to obtain the last descendent.

def getListOfProcessesLinux (logger):
    processList  = []
    cmd = 'ps -eo pid,ppid'
    output = executeCommand (cmd, logger, True)
    lines = output.split ('\n')
    skipFirstLine = True
    for line in lines:
        if skipFirstLine:
            skipFirstLine = False
        else:
            tempList = line.split ();
            if (len (tempList) >= 2):
                pid = int (tempList[0].strip())
                ppid = int (tempList[1].strip())
                processList.append ([pid, ppid]);
    return processList

def getListOfProcessesWindows (logger):
    processList  = []
    object = win32pdhutil.find_pdh_counter_localized_name("Process")
    object2 = win32pdhutil.find_pdh_counter_localized_name('ID Process')
    object3 = win32pdhutil.find_pdh_counter_localized_name('Creating Process ID')

    # Get the list of processes
    try:
        junk, instances = win32pdh.EnumObjectItems(None,None, object, win32pdh.PERF_DETAIL_WIZARD)
    except Exception, e:
        print ('exception' +  str(e))

    for instance in instances:
            hq = win32pdh.OpenQuery() # initializes the query handle 
            path = win32pdh.MakeCounterPath( (None,object,instance, None, 0,object2) )
            counter_handle=win32pdh.AddCounter(hq, path)
            win32pdh.CollectQueryData(hq) #collects data for the counter 
            type, pid = win32pdh.GetFormattedCounterValue(counter_handle, win32pdh.PDH_FMT_LONG)

            path2 = win32pdh.MakeCounterPath( (None, object, instance, None, 0, object3) )
            counter_handle2 = win32pdh.AddCounter (hq, path2)
            win32pdh.CollectQueryData(hq) #collects data for the counter 
            type, ppid = win32pdh.GetFormattedCounterValue(counter_handle2, win32pdh.PDH_FMT_LONG)
            win32pdh.CloseQuery(hq)
            processList.append ([pid, ppid])
    return processList

def getListOfProcesses (logger, isLinux):
    """ This function returns a list of pairs pid ppid for all the processes in the system. 
        The function has different implementations for Linux and windows. """
    if isLinux:
        return getListOfProcessesLinux (logger);
    else:
        return getListOfProcessesWindows (logger);

def getLastDescendent (pid, logger, isLinux):
    """ This function returns the last descendent of the given process. """
    descendent = pid
    processes = getListOfProcesses (logger, isLinux)
    exitLoop = False
    while (not exitLoop):
        # Look for the first descendent of the process. The case of more than one descendent is not considered
        found = False
        for entry in processes:
            if entry[1] == descendent:
                found = True
                descendent = entry[0]
                break

        if not found:
            exitLoop = True
    return descendent 

Monitoring the memory usage

This is the code for the memory usage:

def getVmPeakLinux (pid, logger):    try:
        cmd = 'cat /proc/' + pid + '/status'
        out = executeCommand (cmd, logger, True)        pos = out.find('VmPeak:')

        memStr = out [pos + 7:pos + 17]
        memStr = memStr.strip()    except Exception, detail:
        memStr = '0'
        logger.error ('getVmPeakLinux. Cannot get memory information. ' + str(deta
il))
    return memStr

def getVmPeakWindows (pid, logger):
    try:
        pid = int (pid)
        han = win32api.OpenProcess (win32con.PROCESS_ALL_ACCESS, False, pid)
        info = win32process.GetProcessMemoryInfo (han)

        memStr = str(info['PeakPagefileUsage'])
    except Exception,detail:
        memStr  = '0'
        logger.error ('getVmPeakWindows. Cannot get memory information. ' + str(de
tail))

    return memStr

def getVmPeak (pid, logger, isLinux):
    """ This function receives a pid and returns a string indicating the peak of t
he process virtual memory 
        size. 
        The function has different implementations for Linux and windows."""
    if isLinux:
        return (getVmPeakLinux (pid, logger))
    else:
        return (getVmPeakWindows (pid, logger))


def getVmSizeLinux (pid, logger):    try:
        cmd = 'cat /proc/' + pid + '/status'
        out = executeCommand (cmd, logger, True)
        pos = out.find('VmSize:')
        memStr = out [pos + 7:pos + 17]
        memStr = memStr.strip()
    except Exception, detail:
        memStr = '0'
        logger.error ('getVmSizeLinux. Cannot get memory information. ' + str(deta
il))    return memStr


def getVmSizeWindows (pid, logger):
    try:
        pid = int(pid)
        han = win32api.OpenProcess (win32con.PROCESS_ALL_ACCESS, False, pid)
        info = win32process.GetProcessMemoryInfo (han)

        memStr = str(info['PagefileUsage'])
    except Exception,detail:
        memStr  = '0'
        logger.error ('getVmSizeWindows. Cannot get memory information. ' + str(de
tail) )
    return memStr

def getVmSize (pid, logger, isLinux):
    """ This function receives a pid and returns a string indicating the current p
rocess virtual memory size. 
        The function has different implementations for Linux and windows."""
    if isLinux:
        return (getVmSizeLinux (pid, logger))
    else:
        return (getVmSizeWindows (pid, logger))

Monitoring the CPU usage

This is the actual code:

def getIdleTimeLinux (logger):
    retVal = 110.0
    cmd = 'top -b -d 2 -n 2'

    output = executeCommand (cmd, logger, True)
    pos = output.find ('%id')
    if pos > 0:
        output = output [pos + 3:]
        pos = output.find ('%id')
        if pos > 0:
            percentStr = output [pos - 5:pos]
            percentStr = percentStr.strip()
            retVal = float(percentStr)
        else:
            logger.error ('Cannot find the second idle')
    else:
        logger.error ('Cannot find the first idle')

    return retVal

def getIdleTimeWindows (logger):
    retVal = 110.0
    object = win32pdhutil.find_pdh_counter_localized_name("Processor")

    # I can't translate with find_pdh_counter_localized_name the name "% Processor time"
    items, instances = win32pdh.EnumObjectItems (None, None, object, win32pdh.PERF_DETAIL_WIZARD)

    hq = win32pdh.OpenQuery()
    path = win32pdh.MakeCounterPath ((None, object, '_Total', None, -1, items[0]))
    counter_handle = win32pdh.AddCounter (hq, path)
    win32pdh.CollectQueryData(hq)
    time.sleep (2)
    win32pdh.CollectQueryData(hq)
    type, retVal = win32pdh.GetFormattedCounterValue (counter_handle, win32pdh.PDH_FMT_LONG)
    win32pdh.CloseQuery(hq)

    return (100.0 - retVal)

def getIdleTime (logger, isLinux):
    """ This function measures the current CPU usage. It returns the percentage of idle time for a certain
        time period.
        The function has different implementations for Linux and windows. """
    if isLinux:
        return getIdleTimeLinux (logger)
    else:
        return getIdleTimeWindows (logger)

The way to find out the platform we're running is:

if os.name=='posix':
    isLinux=True
else:
    isLinux=False