I haven't been able to find an understandable explanation of how to actually use Python's itertools.groupby() function. What I'm trying to do is this:

• Take a list - in this case, the children of an objectified lxml element
• Divide it into groups based on some criteria
• Then later iterate over each of these groups separately.

I've reviewed the documentation, and the examples, but I've had trouble trying to apply them beyond a simple list of numbers.

So, how do I use of itertools.groupby()? Is there another technique I should be using? Pointers to good "prerequisite" reading would also be appreciated.

Can you show us your code?

The example on the Python docs is quite straightforward:

groups = []
uniquekeys = []
for k, g in groupby(data, keyfunc):
    groups.append(list(g))      # Store group iterator as a list
    uniquekeys.append(k)

So in your case, data is a list of nodes, keyfunc is where the logic of your criteria function goes and then groupby() groups the data.

You must be careful to sort the data by the criteria before you call groupby or it won't work. groupby method actually just iterates through a list and whenever the key changes it creates a new group.

As Sebastjan said, you first have to sort your data. This is important.

The part I didn't get is that in the example construction

groups = []
uniquekeys = []
for k, g in groupby(data, keyfunc):
   groups.append(list(g))    # Store group iterator as a list
   uniquekeys.append(k)

k is the current grouping key, and g is an iterator that you can use to iterate over the group defined by that grouping key. In other words, the groupby iterator itself returns iterators.

Here's an example of that, using clearer variable names:

from itertools import groupby
things = [("animal", "bear"), ("animal", "duck"), ("plant", "cactus"), ("vehicle", "speed boat"), ("vehicle", "school bus")]
for key, group in groupby(things, lambda x: x[0]):
    for thing in group:
        print "A %s is a %s." % (thing[1], key)
    print " "

This will give you the output:

A bear is a animal.
A duck is a animal.

A cactus is a plant.

A speed boat is a vehicle.
A school bus is a vehicle.

In this example, things is a list of tuples where the first item in each tuple is the group the second item belongs to.

The groupby() function takes two arguments: (1) the data to group and (2) the function to group it with.

Here, lambda x: x[0] tells groupby() to use the first item in each tuple as the grouping key.

In the above for statement, groupby returns three (key, group iterator) pairs - once for each unique key. You can use the returned iterator to iterate over each individual item in that group.

Here's a slightly different example with the same data, using a list comprehension:

for key, group in groupby(things, lambda x: x[0]):
    listOfThings = " and ".join([thing[1] for thing in group])
    print key + "s:  " + listOfThings + "."

This will give you the output:

animals: bear and duck.
plants: cactus.
vehicles: speed boat and school bus.

I have created a PHP-script to update a webserver that is live inside a local directory. I'm migrating the script into Python. It works fine for the most part, but after a PUT command the size of the file appears to change. Thus, the size of the file is different from that of the file on the server.

Once I download again the file from the FTP server, the only difference is the CR/LF mark. This annoys me because the same script is comparing the size of the files to update. Also, in case it means anything, the script works perfectly in PHP vía ftp_put.

from ftplib import FTP
ftpserver = "myserver"
ftpuser = "myuser"
ftppass = "mypwd"
locfile =  "g:/test/style.css"
ftpfile =  "/temp/style.css"
try:
    ftp = FTP(ftpserver, ftpuser, ftppass)
except:
    exit ("Cannot connect")
f = open (locfile, "r")
try:
    ftp.delete (ftpfile)
except:
    pass
# ftp.sendcmd ("TYPE I")
# ftp.storlines("STOR %s" % ftpfile, f)
ftp.storbinary("STOR %s" % ftpfile, f)
f.close()
ftp.dir (ftpfile)
ftp.quit()

Any suggestions?

Do you need to open the locfile in binary using rb?

f = open (locfile, "rb")

Well if you go under the properties of your file in Windows or a *nix environment, you will notice two sizes. One is the sector size, and one is the actual size. The sector size is the number of sectors in bytes that are used up on your hard disk. That is because two files cannot be in the same sector with most modern file systems, so if your file fills up half of the sector the whole sector is marked as filled.

So you might be comparing the sector file size to the actual file size on the FTP server or vice versa.

Small files take up a whole node on the filesystem whatever size that is.

My host tends to report all small files as 4kb in ftp but in a shell gives an accurate size so it might be a 'feature' common to ftp clients.

I'm trying to decode the result of the Python os.wait() function. This returns, according to the Python docs:

a tuple containing its pid and exit status indication: a 16-bit number, whose low byte is the signal number that killed the process, and whose high byte is the exit status (if the signal number is zero); the high bit of the low byte is set if a core file was produced.

How do I decode the exit status indication (which is an integer) to obtain the high and low byte? To be specific, how do I implement the decode function used in the following code snippet:

(pid,status) = os.wait()
(exitstatus, signum) = decode(status)

This will do what you want:

signum = status & 0xff
exitstatus = (status & 0xff00) >> 8

You can unpack the status using bit-shifting and masking operators.

low = status & 0x00FF
high = (status & 0xFF00) >> 8

I'm not a Python programmer, so I hope got the syntax correct.

The folks before me've nailed it, but if you really want it on one line, you can do this:

(signum, exitstatus) = (status & 0xFF, (status >> 8) & 0xFF)

EDIT: Had it backwards.

To answer your general question, you can use bit manipulation techniques:

pid, status = os.wait()
exitstatus, signum = status & 0xFF, (status & 0xFF00) >> 8

However, there are also built-in functions for interpreting exit status values:

pid, status = os.wait()
exitstatus, signum = os.WEXITSTATUS( status ), os.WTERMSIG( status )

See also:

• os.WCOREDUMP()
• os.WIFCONTINUED()
• os.WIFSTOPPED()
• os.WIFSIGNALED()
• os.WIFEXITED()
• os.WSTOPSIG()

I'm starting work on a hobby project with a python codebase and would like to set up some form of continuous integration (i.e. running a battery of test-cases each time a check-in is made and sending nag e-mails to responsible persons when the tests fail) similar to CruiseControl or TeamCity.

I realize I could do this with hooks in most VCSes, but that requires that the tests run on the same machine as the version control server, which isn't as elegant as I would like. Does anyone have any suggestions for a small, user-friendly, open-source continuous integration system suitable for a Python codebase?

One possibility is Hudson. It's written in Java, but there's integration with Python projects:

Hudson embraces Python

I've never tried it myself, however.

(Update, Sept. 2011: After a trademark dispute Hudson has been renamed to Jenkins.)

We run Buildbot - Trac at work, I haven't used it too much since my code base isn't part of the release cycle yet. But we run the tests on different environments (OSX/Linux/Win) and it sends emails --and it's written in python.

Second the Buildbot - Trac integration. You can find more information about the integration on the Buildbot website. At my previous job, we wrote and used the plugin they mention (tracbb). What the plugin does is rewriting all of the Buildbot urls so you can use Buildbot from within Trac. (http://example.com/tracbb).

The really nice thing about Buildbot is that the configuration is written in Python. You can integrate your own Python code directly to the configuration. It's also very easy to write your own BuildSteps to execute specific tasks.

We used BuildSteps to get the source from SVN, pull the dependencies, publish test results to WebDAV, etcetera.

I wrote an X10 interface so we could send signals with build results. When the build failed, we switched on a red lava lamp. When the build succeeded, a green lava lamp switched on. Good times :-)

I've got a menu in Python. That part was easy. I'm using raw_input() to get the selection from the user.

The problem is that raw_input (and input) require the user to press Enter after they make a selection. Is there any way to make the program act immediately upon a keystroke? Here's what I've got so far:

import sys
print """Menu
1) Say Foo
2) Say Bar"""
answer = raw_input("Make a selection> ")
if "1" in answer: print "foo"
elif "2" in answer: print "bar"

It would be great to have something like

print menu
while lastKey = "":
    lastKey = check_for_recent_keystrokes()
if "1" in lastKey: #do stuff...

On Linux:

• set raw mode
• select and read the keystroke
• restore normal settings

import sys
import select
import termios
import tty
def getkey():
    old_settings = termios.tcgetattr(sys.stdin)
    tty.setraw(sys.stdin.fileno())
    select.select([sys.stdin], [], [], 0)
    answer = sys.stdin.read(1)
    termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_settings)
    return answer
print """Menu
1) Say Foo
2) Say Bar"""
answer=getkey()
if "1" in answer: print "foo"
elif "2" in answer: print "bar"

On Windows:

import msvcrt
answer=msvcrt.getch()

Wow, that took forever. Ok, here's what I've ended up with

#!C:\python25\python.exe
import msvcrt
print """Menu
1) Say Foo 
2) Say Bar"""
while 1:
    char = msvcrt.getch()
    if char == chr(27): #escape
        break
    if char == "1":
        print "foo"
        break
    if char == "2":
        print "Bar"
        break

It fails hard using IDLE, the python...thing...that comes with python. But once I tried it in DOS (er, CMD.exe), as a real program, then it ran fine.

No one try it in IDLE, unless you have Task Manager handy.

I've already forgotten how I lived with menus that arn't super-instant responsive.

The reason msvcrt fails in IDLE is because IDLE is not accessing the library that runs msvcrt. Whereas when you run the program natively in cmd.exe it works nicely. For the same reason that your program blows up on Mac and Linux terminals.

But I guess if you're going to be using this specifically for windows, more power to ya.

When I try to print a Unicode string in a Windows console, I get a UnicodeEncodeError: 'charmap' codec can't encode character .... error. I assume this is because the Windows console does not accept Unicode-only characters. What's the best way around this? Is there any way I can make Python automatically print a ? instead of failing in this situation?

Edit: I'm using Python 2.5.

Note: @LasseV.Karlsen answer with the checkmark is sort of outdated (from 2008). Please use the solutions/answers/suggestions below with care!!

@JFSebastian answer is more relevant as of today (6 Jan 2016).

Note: This answer is sort of outdated (from 2008). Please use the solution below with care!!

Here is a page that details the problem and a solution (search the page for the text Wrapping sys.stdout into an instance):

PrintFails - Python Wiki

Here's a code excerpt from that page:

$ python -c 'import sys, codecs, locale; print sys.stdout.encoding; \
    sys.stdout = codecs.getwriter(locale.getpreferredencoding())(sys.stdout); \
    line = u"\u0411
"; print type(line), len(line); \
    sys.stdout.write(line); print line'
  UTF-8
  <type 'unicode'> 2
  Б
  Б
  $ python -c 'import sys, codecs, locale; print sys.stdout.encoding; \
    sys.stdout = codecs.getwriter(locale.getpreferredencoding())(sys.stdout); \
    line = u"\u0411
"; print type(line), len(line); \
    sys.stdout.write(line); print line' | cat
  None
  <type 'unicode'> 2
  Б
  Б

There's some more information on that page, well worth a read.

The cause of your problem is NOT the Win console not willing to accept Unicode (as it does this since I guess Win2k by default). It is the default system encoding. Try this code and see what it gives you:

import sys
sys.getdefaultencoding()

if it says ascii, there's your cause ;-) You have to create a file called sitecustomize.py and put it under python path (I put it under /usr/lib/python2.5/site-packages, but that is differen on Win - it is c:\python\lib\site-packages or something), with the following contents:

import sys
sys.setdefaultencoding('utf-8')

and perhaps you might want to specify the encoding in your files as well:

# -*- coding: UTF-8 -*-
import sys,time

Edit: more info can be found in excellent the Dive into Python book

Is it possible to write a doctest unit test that will check that an exception is raised?
For example, if I have a function foo(x) that is supposed to raise an exception if x<0, how would I write the doctest for that?

Yes. You can do it. The doctest module documentation and Wikipedia has an example of it.

   >>> x
   Traceback (most recent call last):
     ...
   NameError: name 'x' is not defined

This is a difficult and open-ended question I know, but I thought I'd throw it to the floor and see if anyone had any interesting suggestions.

I have developed a code-generator that takes our python interface to our C++ code (generated via SWIG) and generates code needed to expose this as WebServices. When I developed this code I did it using TDD, but I've found my tests to be brittle as hell. Because each test essentially wanted to verify that for a given bit of input code (which happens to be a C++ header) I'd get a given bit of outputted code I wrote a small engine that reads test definitions from XML input files and generates test cases from these expectations.

The problem is I dread going in to modify the code at all. That and the fact that the unit tests themselves are a: complex, and b: brittle.

So I'm trying to think of alternative approaches to this problem, and it strikes me I'm perhaps tackling it the wrong way. Maybe I need to focus more on the outcome, IE: does the code I generate actually run and do what I want it to, rather than, does the code look the way I want it to.

Has anyone got any experiences of something similar to this they would care to share?

I started writing up a summary of my experience with my own code generator, then went back and re-read your question and found you had already touched upon the same issues yourself, focus on the execution results instead of the code layout/look.

Problem is, this is hard to test, the generated code might not be suited to actually run in the environment of the unit test system, and how do you encode the expected results?

I've found that you need to break down the code generator into smaller pieces and unit test those. Unit testing a full code generator is more like integration testing than unit testing if you ask me.

Yes, results are the ONLY thing that matters. The real chore is writing a framework that allows your generated code to run independently... spend your time there.

If you are running on *nux you might consider dumping the unittest framework in favor of a bash script or makefile. on windows you might consider building a shell app/function that runs the generator and then uses the code (as another process) and unittest that.

A third option would be to generate the code and then build an app from it that includes nothing but a unittest. Again you would need a shell script or whatnot to run this for each input. As to how to encode the expected behavior, it occurs to me that it could be done in much the same way as you would for the C++ code just using the generated interface rather than the C++ one.

Recall that "unit testing" is only one kind of testing. You should be able to unit test the internal pieces of your code generator. What you're really looking at here is system level testing (a.k.a. regression testing). It's not just semantics... there are different mindsets, approaches, expectations, etc. It's certainly more work, but you probably need to bite the bullet and set up an end-to-end regression test suite: fixed C++ files -> SWIG interfaces -> python modules -> known output. You really want to check the known input (fixed C++ code) against expected output (what comes out of the final Python program). Checking the code generator results directly would be like diffing object files...

What do I need to look at to see if I'm on Windows, Unix, etc?

>>> import os
>>> print os.name
posix
>>> import platform
>>> platform.system()
'Linux'
>>> platform.release()
'2.6.22-15-generic'

See: platform — Access to underlying platform’s identifying data

Dang -- lbrandy beat me to the punch, but that doesn't mean I can't provide you with the system results for Vista!

>>> import os
>>> os.name
'nt'
>>> import platform
>>> platform.system()
'Windows'
>>> platform.release()
'Vista'

For the record here's the results on Mac:

>>> import os
>>> os.name
'posix'
>>> import platform
>>> platform.system()
'Darwin'
>>> platform.release()
'8.11.1'

In many places, (1,2,3) and [1,2,3] can be used interchangeably.

When should I use one or the other, and why?

The list [1,2,3] is dynamic and flexible but that flexibility comes at a speed cost.

The tuple (1,2,3) is fixed (immutable) and therefore faster.

From the Python FAQ:

Lists and tuples, while similar in many respects, are generally used in fundamentally different ways. Tuples can be thought of as being similar to Pascal records or C structs; they're small collections of related data which may be of different types which are operated on as a group. For example, a Cartesian coordinate is appropriately represented as a tuple of two or three numbers.

Lists, on the other hand, are more like arrays in other languages. They tend to hold a varying number of objects all of which have the same type and which are operated on one-by-one.

Generally by convention you wouldn't choose a list or a tuple just based on its (im)mutability. You would choose a tuple for small collections of completely different pieces of data in which a full-blown class would be too heavyweight, and a list for collections of any reasonable size where you have a homogeneous set of data.

Tuples are a quick\flexible way to create composite data-types. Lists are containers for, well, lists of objects.

For example, you would use a List to store a list of student details in a class.

Each student detail in that list may be a 3-tuple containing their roll number, name and test score.

 `[(1,'Mark',86),(2,'John',34)...]`

Also, because tuples are immutable they can be used as keys in dictionaries.

As others have mentioned, Lists and tuples are both containers which can be used to store python objects. Lists are extensible and their contents can change by assignment, on the other hand tuples are immutable.

Also, lists cannot be used as keys in a dictionary whereas tuples can.

If you can find a solution that works with tuples, use them, as it forces immutability which kind of drives you down a more functional path. You almost never regret going down the functional/immutable path.

Whenever I need to pass in a collection of items to a function, if I want the function to not change the values passed in - I use tuples.

Else if I want to have the function to alter the values, I use list.

Always if you are using external libraries and need to pass in a list of values to a function and are unsure about the integrity of the data, use a tuple.

The notion of tuples are highly expressive:

• Pragmatically, they are great for packing and unpacking values (x,y=coord).

• In combination with dictionaries (hash tables), they allow forms of mapping that would otherwise require many levels of association. For example, consider marking that (x,y) has been found.

  // PHP
  if (!isset($found[$x])) {
      $found[$x] = Array();
      $found[$x][$y] = true;
  } else if (!isset($found[$x][$y])) {
      $found[$x][$y] = true;
  }
  
  # Python
  found[(x,y)] = True # parens added for clarity
  

• Lists should be used with the expectation of operations on its contents (hence the various mentions of immutability). One will want to pop, push, splice, slice, search, insert before, insert after, etc with a list.

• Tuples should be a low-level representation of an object, where simple comparisons are made, or operations such as extracting the n'th element or n elements in a predictable fashion, such as the coordinates example given earlier.

• Lastly, lists are not hashable, so the type of mapping done with dictionaries (hash tables in Perl, associative arrays in PHP) must be done with tuples.

  Here's a simple example of tuples and dictionaries, together at last:

  """
  couple is a tuple of two people
  doesLike is a dictionary mapping couples to True or False
  """
  couple = "john", "jane"
  doesLike = dict()
  doesLike[couple] = True
  doesLike["jane", "john"] = False # unrequited love :'(
  

I learned Swing back in the day but now I've moved to Python and want to make some apps with GUIs. I haven't had the time to learn a new GUI API so I've been using Jython, but I would prefer to use CPython.

It would be great if I can have one simple markup that allows me to switch GUI libraries. It would be even better if I can use the same markup language across languages so I can quickly make GUIs for any language I'm using. Does anyone know of such a markup/library?

I've seen markups like Glade and wxWidget's markup (I forget the name). They're partly what I'm looking for (making a GUI without coding it in a language) but they're intertwined with a specific library. And neither are really nice looking or friendly to human editting.

erm.. HTML? (trying to be funny here... while we wait for real answers..)

The wxWidgets (formerly known as wxWindows) library might be what you're looking for. There's a particularly good port for Python, wxPython, as well as versions for different languages -- C#, C++, Perl and Ruby come to mind -- and for various GUIs: Win32, Mac OS X, GTK+, X11, Motif, WinCE. The library's been around for a while and is pretty solid.

Not sure if this is what you're looking for, but there's Glade (or Windows download) which is a designer for GTK+. It generates an XML file which can then be used to build the GUI in a number of different languages.

Qt (pronounced "cute" by its creators[1]) is a cross-platform application development framework, widely used for the development of GUI programs.

Qt uses C++ with several non-standard extensions implemented by an additional pre-processor that generates standard C++ code before compilation. Qt can also be used in several other programming languages; bindings exist for Ada (QtAda)[4], C# (Qyoto/Kimono)[5], Java (Qt Jambi)[6], Pascal, Perl, PHP (PHP-Qt), Ruby (RubyQt), and Python (PyQt). It runs on all major platforms, and has extensive internationalization support. Non-GUI features include SQL database access, XML parsing, thread management, network support and a unified cross-platform API for file handling.

XML User Interface Language. Don't know much about it so not sure if it meets your desires. Post back with your experience if you play with it.

I read a little on XML User Interface Language (XUL) and it looks really robust and well supported. The main problem for me is it's tied to the Gecko rendering engine so it's cross platform the way wxWidgets, QT and GTK+ are cross platform. Also, there Python bindings don't seem as good as those other libraries.

GladeXML and XRC seem like better markups

@Cristian and Antony Cramp: while XUL is a nice choice for UI development (cross-platform, open-source licensed, used in Firefox and other major software), it's certainly not language agnostic. You're tied to Gecko, with JS for scripting. There is experimental support for Python scripting, but it's only experimental, AFAIK.

You can define the UI in XUL+JS and use back-end components written in C++ though (there are Python and Java bridges available too).

I seriously doubt you're going to find a markup language for GIU's that's not tied to a specific library. For such a thing to exist, there would need to be a standardized GUI markup language, with several implementations.

How do you express an integer as a binary number with Python literals?

I was easily able to find the answer for hex:

    >>> 0x12AF
    4783
    >>> 0x100
    256

and octal:

    >>> 01267
    695
    >>> 0100
    64

How do you use literals to express binary in Python?

Summary of Answers

• Python 2.5 and earlier: can express binary using int('01010101111',2) but not with a literal.
• Python 2.5 and earlier: there is no way to express binary literals.
• Python 2.6 beta: You can do like so: 0b1100111 or 0B1100111.
• Python 2.6 beta: will also allow 0o27 or 0O27 (second character is the letter O) to represent an octal.
• Python 3.0 beta: Same as 2.6, but will no longer allow the older 027 syntax for octals.

As far as I can tell Python, up through 2.5, only supports hexadecimal & octal literals. I did find some discussions about adding binary to future versions but nothing definite.

I am pretty sure this is one of the things due to change in Python 3.0 with perhaps bin() to go with hex() and oct().

EDIT: lbrandy's answer is correct in all cases.

>>> print int('01010101111',2)
687
>>> print int('11111111',2)
255

Another way.

I've read that it is possible to add a method to an existing object (e.g. not in the class definition) in Python, I think this is called Monkey Patching (or in some cases Duck Punching). I understand that it's not always a good decision to do so. But, how might one do this?

UPDATE 8/04/2008 00:21:01 EST:

That looks like a good answer John Downey, I tried it but it appears that it ends up being not a true method.

Your example defines the new patch function with an argument of self, but if you write actual code that way, the now patched class method asks for an argument named self (it doesn't automagically recognize it as the object to which it is supposed to bind, which is what would happen if defined within the class definition), meaning you have to call class.patch(obj) instead of just class.patch() if you want the same functionality as a true method.

It looks like Python isn't really treating it as a method, but more just as a variable which happens to be a function (and as such is callable). Is there any way to attach an actual method to a class?

Oh, and Ryan, that isn't exactly what I was looking for (it isn't a builtin functionality), but it is quite cool nonetheless.

In Python monkey patching generally works by overwriting a class or functions signature with your own. Below is an example from the Zope Wiki:

from SomeOtherProduct.SomeModule import SomeClass
def speak(self):
   return "ook ook eee eee eee!"
SomeClass.speak = speak

That code will overwrite/create a method called speak on the class. In Jeff Atwood's recent post on monkey patching. He shows an example in C# 3.0 which is the current language I use for work.

I don't know Python syntax, but I know Ruby can do it, and it is rather trivial. Let's say you want to add a method to Array that prints the length to standard out:

class Array
  def print_length
    puts length
  end
end

If you don't want to modify the whole class, you can just add the method to a single instance of the array, and no other arrays will have the method:

array = [1, 2, 3]
def array.print_length
  puts length
end

Just be aware of the issues involved in using this feature. Jeff Atwood actually wrote about it not too long ago.

In Python, there is a difference between functions and bound methods.

>>> def foo():
...     print "foo"
...
>>> class A:
...     def bar( self ):
...         print "bar"
...
>>> a = A()
>>> foo
<function foo at 0x00A98D70>
>>> a.bar
<bound method A.bar of <__main__.A instance at 0x00A9BC88>>
>>>

Bound methods have been "bound" (how descriptive) to an instance, and that instance will be passed as the first argument whenever the method is called.

Callables that are attributes of a class (as opposed to an instance) are still unbound, though, so you can modify the class definition whenever you want:

>>> def fooFighters( self ):
...     print "fooFighters"
...
>>> A.fooFighters = fooFighters
>>> a2 = A()
>>> a2.fooFighters
<bound method A.fooFighters of <__main__.A instance at 0x00A9BEB8>>
>>> a2.fooFighters()
fooFighters

Previously defined instances are updated as well (as long as they haven't overridden the attribute themselves):

>>> a.fooFighters()
fooFighters

The problem comes when you want to attach a method to a single instance:

>>> def barFighters( self ):
...     print "barFighters"
...
>>> a.barFighters = barFighters
>>> a.barFighters()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: barFighters() takes exactly 1 argument (0 given)

The function is not automatically bound when it's attached directly to an instance:

>>> a.barFighters
<function barFighters at 0x00A98EF0>

To bind it, we can use the MethodType function in the types module:

>>> import types
>>> a.barFighters = types.MethodType( barFighters, a )
>>> a.barFighters
<bound method ?.barFighters of <__main__.A instance at 0x00A9BC88>>
>>> a.barFighters()
barFighters

This time other instances of the class have not been affected:

>>> a2.barFighters()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: A instance has no attribute 'barFighters'

More information can be found by reading about descriptors and metaclass programming.

What you're looking for is setattr I believe. Use this to set an attribute on an object.

>>> def printme(s): print repr(s)
>>> class A: pass
>>> setattr(A,'printme',printme)
>>> a = A()
>>> a.printme() # s becomes the implicit 'self' variable
< __ main __ . A instance at 0xABCDEFG>

When asked to create system XYZ and you ask to do it in Python over PHP or Ruby, what are the main features you can mention when they require you to explain it?

Focus on the shorter time needed for development/prototype and possibly easier maintenance (none of this may apply against Ruby).

This is one of those cases that really boil down to personal preference or situational details. If you're more comfortable and experienced with Python, then say so. Are they asking you to justify it because they're more comfortable with one of the other environments? After you're done, will the system be passed off to someone else for long-term maintenance?

If they ask you to use a technology or language that you're not as familiar with, then make sure they know up-front that it's going to take you longer.

I would consider that using python on a new project is completely dependent on what problem you are trying to solve with python. If you want someone to agree with you that you should use python, then show them how python's features apply specifically to that problem.

In the case of web development with python, talk about WSGI and other web libraries and frameworks you could use that would make your life easier. One note for python is that most of the frameworks for python web development can be plugged right into any current project. With ruby on rails, you're practically working in a DSL that anyone who uses your project will have to learn. If they know python, then they can figure out what you are doing with django, etc in a day.

I'm only talking about web development because it appears that's what you are going to be working on seeing ruby, python and PHP in the same list. The real message that's important is applying to whatever it is you like about python directly to some problem you are trying to solve.

There are several ways to iterate over a result set. What are the tradeoff of each?

The canonical way is to use the built-in cursor iterator.

curs.execute('select * from people')
for row in curs:
    print row

You can use fetchall() to get all rows at once.

for row in curs.fetchall():
    print row

It can be convenient to use this to create a Python list containing the values returned:

curs.execute('select first_name from people')
names = [row[0] for row in curs.fetchall()]

This can be useful for smaller result sets, but can have bad side effects if the result set is large.

• You have to wait for the entire result set to be returned to your client process.

• You may eat up a lot of memory in your client to hold the built-up list.

• It may take a while for Python to construct and deconstruct the list which you are going to immediately discard anyways.

If you know there's a single row being returned in the result set you can call fetchone() to get the single row.

curs.execute('select max(x) from t')
maxValue = curs.fetchone()[0]

Finally, you can loop over the result set fetching one row at a time. In general, there's no particular advantage in doing this over using the iterator.

row = curs.fetchone()
while row:
    print row
    row = curs.fetchone()

I am using the Photoshop's javascript API to find the fonts in a given PSD.

Given a font name returned by the API, I want to find the actual physical font file that that font name corresponds to on the disc.

This is all happening in a python program running on OSX so I guess I'm looking for one of:

• Some Photoshop javascript
• A Python function
• An OSX API that I can call from python

open up a terminal (Applications->Utilities->Terminal) and type this in:

locate InsertFontHere

This will spit out every file that has the name you want.

Warning: there may be alot to wade through.

I haven't been able to find anything that does this directly. I think you'll have to iterate through the various font folders on the system: /System/Library/Fonts, /Library/Fonts, and there can probably be a user-level directory as well ~/Library/Fonts.

Unfortunately the only API that isn't deprecated is located in the ApplicationServices framework, which doesn't have a bridge support file, and thus isn't available in the bridge. If you're wanting to use ctypes, you can use ATSFontGetFileReference after looking up the ATSFontRef.

Cocoa doesn't have any native support, at least as of 10.5, for getting the location of a font.

I tried to follow a couple of googled up tutorials on setting up mod_python, but failed every time. Do you have a good, step-by step, rock-solid howto?

My dev box is OS X, production - Centos.

There are two main ways of running Python on Apache. The simplest would be to use CGI and write normal Python scripts while the second is using a web framework like Django or Pylons.

Using CGI is straightforward. Make sure your Apache config file has a cgi-bin set up. If not, follow their documentation (http://httpd.apache.org/docs/2.0/howto/cgi.html). At that point all you need to do is place your Python scripts in the cgi-bin directory and the standard output will become the HTTP response. Refer to Python's documentation for further info (https://docs.python.org/library/cgi.html).

If you want to use a web framework you'll need to setup mod_python or FastCGI. These steps are dependent on which framework you want to use. Django provides clear instructions on how to setup mod_python and Django with Apache (http://www.djangoproject.com/documentation/modpython/)

Yes, mod_python is pretty confusing to set up. Here's how I did it.

In httpd.conf:

LoadModule python_module modules/mod_python.so

<Directory "/serverbase/htdocs/myapp">
  AddHandler mod_python .py
  PythonHandler myapp
  PythonDebug On

and in your application directory:

$ /serverbase/htdocs/myapp$ ls -l
total 16
-r-xr-xr-x 1 root sys        6484 May 21 15:54 myapp.py

Repeat the configuration for each python program you wish to have running under mod_python.

Are you running Python on UNIX or Windows?

An alternative to mod_python and FastCGI is mod_wsgi. You can find out more at modwsgi

I have built and installed this on Solaris without problems. I had previously tried mod_python but ran into problems with shared libraries as part of the build. There are good install docs available.

What is the best way to retrieve mp3 metadata in python? I've seen a couple frameworks out there, but I'm unsure as to which would be the best to use.... Any ideas?

This toolkit may do what you need. I can't say if it's the "best", but really, if it does what you need, that's all that matters, right?

HTH

A simple example from the book Dive Into Python works ok for me, this is the download link, the example is fileinfo.py. Don't know if it's the best, but it can do the basic job.

The entire book is available online here.

It can depend on exactly what you want to do in addition to reading the metadata. If it is just simply the bitrate / name etc. that you need, and nothing else, something lightweight is probably best.

If you're manipulating the mp3 past that PyMedia may be suitable.

There are quite a few, whatever you do get, make sure and test it out on plenty of sample media. There are a few different versions of ID3 tags in particular, so make sure it's not too out of date.

Personally I've used this small MP3Info class with luck. It is quite old though.

http://www.omniscia.org/~vivake/python/MP3Info.py

What you're after is the ID3 module. It's very simple and will give you exactly what you need. Just copy the ID3.py file into your site-packages directory and you'll be able to do something like the following:

from ID3 import *
try:
  id3info = ID3('file.mp3')
  print id3info
  # Change the tags
  id3info['TITLE'] = "Green Eggs and Ham"
  id3info['ARTIST'] = "Dr. Seuss"
  for k, v in id3info.items():
    print k, ":", v
except InvalidTagError, message:
  print "Invalid ID3 tag:", message

I've used mutagen to edit tags in media files before. The nice thing about mutagen is that it can handle other formats, such as mp4, FLAC etc. I've written several scripts with a lot of success using this API.

What is the library? Is there a full implementation? How is the library used? Where is its website?

The lxml package supports xpath. It seems to work pretty well, although I've had some trouble with the self:: axis. There's also Amara, but I haven't used it personally.

PyXML works well.

You didn't say what platform you're using, however if you're on Ubuntu you can get it with sudo apt-get install python-xml. I'm sure other Linux distros have it as well.

If you're on a Mac, xpath is already installed but not immediately accessible. You can set PY_USE_XMLPLUS in your environment or do it the Python way before you import xml.xpath:

if sys.platform.startswith('darwin'):
    os.environ['PY_USE_XMLPLUS'] = '1'

In the worst case you may have to build it yourself. This package is no longer maintained but still builds fine and works with modern 2.x Pythons. Basic docs are here.

The latest version of elementtree supports XPath pretty well. Not being an XPath expert I can't say for sure if the implementation is full but it has satisfied most of my needs when working in Python. I've also use lxml and PyXML and I find etree nice because it's a standard module.

NOTE: I've since found lxml and for me it's definitely the best XML lib out there for Python. It does XPath nicely as well (though again perhaps not a full implementation).

I have a cross-platform (Python) application which needs to generate a JPEG preview of the first page of a PDF.

On the Mac I am spawning sips. Is there something similarly simple I can do on Windows?

You can use ImageMagick's convert utility for this, see some examples in http://studio.imagemagick.org/pipermail/magick-users/2002-May/002636.html :

Convert taxes.pdf taxes.jpg 

Will convert a two page PDF file into [2] jpeg files: taxes.jpg.0, taxes.jpg.1

I can also convert these JPEGS to a thumbnail as follows:

convert -size 120x120 taxes.jpg.0 -geometry 120x120 +profile '*' thumbnail.jpg

I can even convert the PDF directly to a jpeg thumbnail as follows:

convert -size 120x120 taxes.pdf -geometry 120x120 +profile '*' thumbnail.jpg

This will result in a thumbnail.jpg.0 and thumbnail.jpg.1 for the two pages.

Is the PC likely to have Acrobat installed? I think Acrobat installs a shell extension so previews of the first page of a PDF document appear in Windows Explorer's thumbnail view. You can get thumbnails yourself via the IExtractImage COM API, which you'll need to wrap. VBAccelerator has an example in C# that you could port to Python.

ImageMagick delegates the PDF->bitmap conversion to GhostScript anyway, so here's a command you can use (it's based on the actual command listed by the ps:alpha delegate in ImageMagick, just adjusted to use JPEG as output):

gs -q -dQUIET -dPARANOIDSAFER -dBATCH -dNOPAUSE -dNOPROMPT \
-dMaxBitmap=500000000 -dLastPage=1 -dAlignToPixels=0 -dGridFitTT=0 \
-sDEVICE=jpeg -dTextAlphaBits=4 -dGraphicsAlphaBits=4 -r72x72 \
-sOutputFile=$OUTPUT -f$INPUT

where $OUTPUT and $INPUT are the output and input filenames. Adjust the 72x72 to whatever resolution you need. (Obviously, strip out the backslashes if you're writing out the whole command as one line.)

This is good for two reasons:

1. You don't need to have ImageMagick installed anymore. Not that I have anything against ImageMagick (I love it to bits), but I believe in simple solutions.
2. ImageMagick does a two-step conversion. First PDF->PPM, then PPM->JPEG. This way, the conversion is one-step.

Other things to consider: with the files I've tested, PNG compresses better than JPEG. If you want to use PNG, change the -sDEVICE=jpeg to -sDEVICE=png16m.

What is the best way to go about calling a function given a string with the function's name in a Python program. For example, let's say that I have a module foo, and I have a string whose contents are "bar". What is the best way to go about calling foo.bar()?

I need to get the return value of the function, which is why I don't just use eval. I figured out how to do it by using eval to define a temp function that returns the result of that function call, but I'm hoping that there is a more elegant way to do this.

Assuming module foo with method bar:

import foo
methodToCall = getattr(foo, 'bar')
result = methodToCall()

As far as that goes, lines 2 and 3 can be compressed to:

result = getattr(foo, 'bar')()

if that makes more sense for your use case. You can use getattr in this fashion on class instance bound methods, module-level methods, class methods... the list goes on.

Patrick's solution is probably the cleanest. If you need to dynamically pick up the module as well, you can import it like:

m = __import__ ('foo')
func = getattr(m,'bar')
func()

Python works on multiple platforms and can be used for desktop and web applications, thus I conclude that there is some way to compile it into an executable for Mac, Windows and Linux.

The problem being I have no idea where to start or how to write a GUI with it, can anybody shed some light on this and point me in the right direction please?

First you will need some GUI library with Python bindings and then (if you want) some program that will convert your python scripts into standalone executables.

Cross-platform GUI libraries with Python bindings (Windows, Linux, Mac)

Of course, there are many, but the most popular that I've seen in wild are:

• Tkinter - based on Tk GUI toolkit (de-facto standard GUI library for python, free for commercial projects)
• WxPython - based on WxWidgets (very popular, free for commercial projects)
• PyQt - based on Qt (also very popular and more stable than WxWidgets but costly license for commercial projects)

Complete list is at http://wiki.python.org/moin/GuiProgramming

Single executable (Windows)

• py2exe - Probably the most popular out there (PyInstaller is also gaining in popularity)

Single executable (Linux)

• Freeze - works the same way like py2exe but targets Linux platform

Single executable (Mac)

• py2app - again, works like py2exe but targets Mac OS

You don't need to compile python for Mac/Windows/Linux. It is an interpreted language, so you simply need to have the Python interpreter installed on the system of your choice (it is available for all three platforms).

As for a GUI library that works cross platform, Python's Tk/Tcl widget library works very well, and I believe is sufficiently cross platform.

Tkinter is the python interface to Tk/Tcl

From the python project webpage:

Tkinter is not the only GuiProgramming toolkit for Python. It is however the most commonly used one, and almost the only one that is portable between Unix, Mac and Windows

Since python is installed on nearly every non-Windows OS by default now, the only thing you really need to make sure of is that all of the non-standard libraries you use are installed.

Having said that, it is possible to build executables that include the python interpreter, and any libraries you use. This is likely to create a large executable, however.

MacOS X even includes support in the Xcode IDE for creating full standalone GUI apps. These can be run by any user running OS X.

An alternative tool to py2exe is bbfreeze which generates executables for windows and linux. It's newer than py2exe and handles eggs quite well. I've found it magically works better without configuration for a wide variety of applications.

I'm not sure that this is the best way to do it, but when I'm deploying Ruby GUI apps (not Python, but has the same "problem" as far as .exe's are concerned) on Windows, I just write a short launcher in C# that calls on my main script. It compiles to an executable, and I then have an application executable.