I'm trying to decode the result of the Python os.wait() function. This returns, according to the Python docs:

a tuple containing its pid and exit status indication: a 16-bit number, whose low byte is the signal number that killed the process, and whose high byte is the exit status (if the signal number is zero); the high bit of the low byte is set if a core file was produced.

How do I decode the exit status indication (which is an integer) to obtain the high and low byte? To be specific, how do I implement the decode function used in the following code snippet:

(pid,status) = os.wait()
(exitstatus, signum) = decode(status)

To answer your general question, you can use bit manipulation techniques:

pid, status = os.wait()
exitstatus, signum = status & 0xFF, (status & 0xFF00) >> 8

However, there are also built-in functions for interpreting exit status values:

pid, status = os.wait()
exitstatus, signum = os.WEXITSTATUS( status ), os.WTERMSIG( status )

See also:

• os.WCOREDUMP()
• os.WIFCONTINUED()
• os.WIFSTOPPED()
• os.WIFSIGNALED()
• os.WIFEXITED()
• os.WSTOPSIG()

The folks before me've nailed it, but if you really want it on one line, you can do this:

(signum, exitstatus) = (status & 0xFF, (status >> 8) & 0xFF)

EDIT: Had it backwards.

This will do what you want:

signum = status & 0xff
exitstatus = (status & 0xff00) >> 8

You can unpack the status using bit-shifting and masking operators.

low = status & 0x00FF
high = (status & 0xFF00) >> 8

I'm not a Python programmer, so I hope got the syntax correct.

I've read that it is possible to add a method to an existing object (e.g. not in the class definition) in Python, I think this is called Monkey Patching (or in some cases Duck Punching). I understand that it's not always a good decision to do so. But, how might one do this?

UPDATE 8/04/2008 00:21:01 EST:

That looks like a good answer John Downey, I tried it but it appears that it ends up being not a true method.

Your example defines the new patch function with an argument of self, but if you write actual code that way, the now patched class method asks for an argument named self (it doesn't automagically recognize it as the object to which it is supposed to bind, which is what would happen if defined within the class definition), meaning you have to call class.patch(obj) instead of just class.patch() if you want the same functionality as a true method.

It looks like Python isn't really treating it as a method, but more just as a variable which happens to be a function (and as such is callable). Is there any way to attach an actual method to a class?

Oh, and Ryan, that isn't exactly what I was looking for (it isn't a builtin functionality), but it is quite cool nonetheless.

In Python, there is a difference between functions and bound methods.

>>> def foo():
...     print "foo"
...
>>> class A:
...     def bar( self ):
...         print "bar"
...
>>> a = A()
>>> foo
<function foo at 0x00A98D70>
>>> a.bar
<bound method A.bar of <__main__.A instance at 0x00A9BC88>>
>>>

Bound methods have been "bound" (how descriptive) to an instance, and that instance will be passed as the first argument whenever the method is called.

Callables that are attributes of a class (as opposed to an instance) are still unbound, though, so you can modify the class definition whenever you want:

>>> def fooFighters( self ):
...     print "fooFighters"
...
>>> A.fooFighters = fooFighters
>>> a2 = A()
>>> a2.fooFighters
<bound method A.fooFighters of <__main__.A instance at 0x00A9BEB8>>
>>> a2.fooFighters()
fooFighters

Previously defined instances are updated as well (as long as they haven't overridden the attribute themselves):

>>> a.fooFighters()
fooFighters

The problem comes when you want to attach a method to a single instance:

>>> def barFighters( self ):
...     print "barFighters"
...
>>> a.barFighters = barFighters
>>> a.barFighters()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: barFighters() takes exactly 1 argument (0 given)

The function is not automatically bound when it's attached directly to an instance:

>>> a.barFighters
<function barFighters at 0x00A98EF0>

To bind it, we can use the MethodType function in the types module:

>>> import types
>>> a.barFighters = types.MethodType( barFighters, a )
>>> a.barFighters
<bound method ?.barFighters of <__main__.A instance at 0x00A9BC88>>
>>> a.barFighters()
barFighters

This time other instances of the class have not been affected:

>>> a2.barFighters()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: A instance has no attribute 'barFighters'

More information can be found by reading about descriptors and metaclass programming.

I don't know Python syntax, but I know Ruby can do it, and it is rather trivial. Let's say you want to add a method to Array that prints the length to standard out:

class Array
  def print_length
    puts length
  end
end

If you don't want to modify the whole class, you can just add the method to a single instance of the array, and no other arrays will have the method:

array = [1, 2, 3]
def array.print_length
  puts length
end

Just be aware of the issues involved in using this feature. Jeff Atwood actually wrote about it not too long ago.

In Python monkey patching generally works by overwriting a class or functions signature with your own. Below is an example from the Zope Wiki:

from SomeOtherProduct.SomeModule import SomeClass
def speak(self):
   return "ook ook eee eee eee!"
SomeClass.speak = speak

That code will overwrite/create a method called speak on the class. In Jeff Atwood's recent post on monkey patching. He shows an example in C# 3.0 which is the current language I use for work.

What you're looking for is setattr I believe. Use this to set an attribute on an object.

>>> def printme(s): print repr(s)
>>> class A: pass
>>> setattr(A,'printme',printme)
>>> a = A()
>>> a.printme() # s becomes the implicit 'self' variable
< __ main __ . A instance at 0xABCDEFG>

I haven't been able to find an understandable explanation of how to actually use Python's itertools.groupby() function. What I'm trying to do is this:

• Take a list - in this case, the children of an objectified lxml element
• Divide it into groups based on some criteria
• Then later iterate over each of these groups separately.

I've reviewed the documentation, and the examples, but I've had trouble trying to apply them beyond a simple list of numbers.

So, how do I use of itertools.groupby()? Is there another technique I should be using? Pointers to good "prerequisite" reading would also be appreciated.

As Sebastjan said, you first have to sort your data. This is important.

The part I didn't get is that in the example construction

groups = []
uniquekeys = []
for k, g in groupby(data, keyfunc):
   groups.append(list(g))    # Store group iterator as a list
   uniquekeys.append(k)

k is the current grouping key, and g is an iterator that you can use to iterate over the group defined by that grouping key. In other words, the groupby iterator itself returns iterators.

Here's an example of that, using clearer variable names:

from itertools import groupby
things = [("animal", "bear"), ("animal", "duck"), ("plant", "cactus"), ("vehicle", "speed boat"), ("vehicle", "school bus")]
for key, group in groupby(things, lambda x: x[0]):
    for thing in group:
        print "A %s is a %s." % (thing[1], key)
    print " "

This will give you the output:

A bear is a animal.
A duck is a animal.

A cactus is a plant.

A speed boat is a vehicle.
A school bus is a vehicle.

In this example, things is a list of tuples where the first item in each tuple is the group the second item belongs to.

The groupby() function takes two arguments: (1) the data to group and (2) the function to group it with.

Here, lambda x: x[0] tells groupby() to use the first item in each tuple as the grouping key.

In the above for statement, groupby returns three (key, group iterator) pairs - once for each unique key. You can use the returned iterator to iterate over each individual item in that group.

Here's a slightly different example with the same data, using a list comprehension:

for key, group in groupby(things, lambda x: x[0]):
    listOfThings = " and ".join([thing[1] for thing in group])
    print key + "s:  " + listOfThings + "."

This will give you the output:

animals: bear and duck.
plants: cactus.
vehicles: speed boat and school bus.

Can you show us your code?

The example on the Python docs is quite straightforward:

groups = []
uniquekeys = []
for k, g in groupby(data, keyfunc):
    groups.append(list(g))      # Store group iterator as a list
    uniquekeys.append(k)

So in your case, data is a list of nodes, keyfunc is where the logic of your criteria function goes and then groupby() groups the data.

You must be careful to sort the data by the criteria before you call groupby or it won't work. groupby method actually just iterates through a list and whenever the key changes it creates a new group.

In many places, (1,2,3) and [1,2,3] can be used interchangeably.

When should I use one or the other, and why?

From the Python FAQ:

Lists and tuples, while similar in many respects, are generally used in fundamentally different ways. Tuples can be thought of as being similar to Pascal records or C structs; they're small collections of related data which may be of different types which are operated on as a group. For example, a Cartesian coordinate is appropriately represented as a tuple of two or three numbers.

Lists, on the other hand, are more like arrays in other languages. They tend to hold a varying number of objects all of which have the same type and which are operated on one-by-one.

Generally by convention you wouldn't choose a list or a tuple just based on its (im)mutability. You would choose a tuple for small collections of completely different pieces of data in which a full-blown class would be too heavyweight, and a list for collections of any reasonable size where you have a homogeneous set of data.

The notion of tuples are highly expressive:

• Pragmatically, they are great for packing and unpacking values (x,y=coord).

• In combination with dictionaries (hash tables), they allow forms of mapping that would otherwise require many levels of association. For example, consider marking that (x,y) has been found.

  // PHP
  if (!isset($found[$x])) {
      $found[$x] = Array();
      $found[$x][$y] = true;
  } else if (!isset($found[$x][$y])) {
      $found[$x][$y] = true;
  }
  
  # Python
  found[(x,y)] = True # parens added for clarity
  

• Lists should be used with the expectation of operations on its contents (hence the various mentions of immutability). One will want to pop, push, splice, slice, search, insert before, insert after, etc with a list.

• Tuples should be a low-level representation of an object, where simple comparisons are made, or operations such as extracting the n'th element or n elements in a predictable fashion, such as the coordinates example given earlier.

• Lastly, lists are not hashable, so the type of mapping done with dictionaries (hash tables in Perl, associative arrays in PHP) must be done with tuples.

  Here's a simple example of tuples and dictionaries, together at last:

  """
  couple is a tuple of two people
  doesLike is a dictionary mapping couples to True or False
  """
  couple = "john", "jane"
  doesLike = dict()
  doesLike[couple] = True
  doesLike["jane", "john"] = False # unrequited love :'(
  

Tuples are a quick\flexible way to create composite data-types. Lists are containers for, well, lists of objects.

For example, you would use a List to store a list of student details in a class.

Each student detail in that list may be a 3-tuple containing their roll number, name and test score.

 `[(1,'Mark',86),(2,'John',34)...]`

Also, because tuples are immutable they can be used as keys in dictionaries.

The list [1,2,3] is dynamic and flexible but that flexibility comes at a speed cost.

The tuple (1,2,3) is fixed (immutable) and therefore faster.

Whenever I need to pass in a collection of items to a function, if I want the function to not change the values passed in - I use tuples.

Else if I want to have the function to alter the values, I use list.

Always if you are using external libraries and need to pass in a list of values to a function and are unsure about the integrity of the data, use a tuple.

As others have mentioned, Lists and tuples are both containers which can be used to store python objects. Lists are extensible and their contents can change by assignment, on the other hand tuples are immutable.

Also, lists cannot be used as keys in a dictionary whereas tuples can.

If you can find a solution that works with tuples, use them, as it forces immutability which kind of drives you down a more functional path. You almost never regret going down the functional/immutable path.

This is a difficult and open-ended question I know, but I thought I'd throw it to the floor and see if anyone had any interesting suggestions.

I have developed a code-generator that takes our python interface to our C++ code (generated via SWIG) and generates code needed to expose this as WebServices. When I developed this code I did it using TDD, but I've found my tests to be brittle as hell. Because each test essentially wanted to verify that for a given bit of input code (which happens to be a C++ header) I'd get a given bit of outputted code I wrote a small engine that reads test definitions from XML input files and generates test cases from these expectations.

The problem is I dread going in to modify the code at all. That and the fact that the unit tests themselves are a: complex, and b: brittle.

So I'm trying to think of alternative approaches to this problem, and it strikes me I'm perhaps tackling it the wrong way. Maybe I need to focus more on the outcome, IE: does the code I generate actually run and do what I want it to, rather than, does the code look the way I want it to.

Has anyone got any experiences of something similar to this they would care to share?

I started writing up a summary of my experience with my own code generator, then went back and re-read your question and found you had already touched upon the same issues yourself, focus on the execution results instead of the code layout/look.

Problem is, this is hard to test, the generated code might not be suited to actually run in the environment of the unit test system, and how do you encode the expected results?

I've found that you need to break down the code generator into smaller pieces and unit test those. Unit testing a full code generator is more like integration testing than unit testing if you ask me.

Recall that "unit testing" is only one kind of testing. You should be able to unit test the internal pieces of your code generator. What you're really looking at here is system level testing (a.k.a. regression testing). It's not just semantics... there are different mindsets, approaches, expectations, etc. It's certainly more work, but you probably need to bite the bullet and set up an end-to-end regression test suite: fixed C++ files -> SWIG interfaces -> python modules -> known output. You really want to check the known input (fixed C++ code) against expected output (what comes out of the final Python program). Checking the code generator results directly would be like diffing object files...

Yes, results are the ONLY thing that matters. The real chore is writing a framework that allows your generated code to run independently... spend your time there.

If you are running on *nux you might consider dumping the unittest framework in favor of a bash script or makefile. on windows you might consider building a shell app/function that runs the generator and then uses the code (as another process) and unittest that.

A third option would be to generate the code and then build an app from it that includes nothing but a unittest. Again you would need a shell script or whatnot to run this for each input. As to how to encode the expected behavior, it occurs to me that it could be done in much the same way as you would for the C++ code just using the generated interface rather than the C++ one.

Is it possible to write a doctest unit test that will check that an exception is raised?
For example, if I have a function foo(x) that is supposed to raise an exception if x<0, how would I write the doctest for that?

Yes. You can do it. The doctest module documentation and Wikipedia has an example of it.

   >>> x
   Traceback (most recent call last):
     ...
   NameError: name 'x' is not defined

I am using the Photoshop's javascript API to find the fonts in a given PSD.

Given a font name returned by the API, I want to find the actual physical font file that that font name corresponds to on the disc.

This is all happening in a python program running on OSX so I guess I'm looking for one of:

• Some Photoshop javascript
• A Python function
• An OSX API that I can call from python

open up a terminal (Applications->Utilities->Terminal) and type this in:

locate InsertFontHere

This will spit out every file that has the name you want.

Warning: there may be alot to wade through.

Unfortunately the only API that isn't deprecated is located in the ApplicationServices framework, which doesn't have a bridge support file, and thus isn't available in the bridge. If you're wanting to use ctypes, you can use ATSFontGetFileReference after looking up the ATSFontRef.

Cocoa doesn't have any native support, at least as of 10.5, for getting the location of a font.

I haven't been able to find anything that does this directly. I think you'll have to iterate through the various font folders on the system: /System/Library/Fonts, /Library/Fonts, and there can probably be a user-level directory as well ~/Library/Fonts.

What is the best way to go about calling a function given a string with the function's name in a Python program. For example, let's say that I have a module foo, and I have a string whose contents are "bar". What is the best way to go about calling foo.bar()?

I need to get the return value of the function, which is why I don't just use eval. I figured out how to do it by using eval to define a temp function that returns the result of that function call, but I'm hoping that there is a more elegant way to do this.

Patrick's solution is probably the cleanest. If you need to dynamically pick up the module as well, you can import it like:

m = __import__ ('foo')
func = getattr(m,'bar')
func()

Assuming module foo with method bar:

import foo
methodToCall = getattr(foo, 'bar')
result = methodToCall()

As far as that goes, lines 2 and 3 can be compressed to:

result = getattr(foo, 'bar')()

if that makes more sense for your use case. You can use getattr in this fashion on class instance bound methods, module-level methods, class methods... the list goes on.