[PEAK] Missing pickling support for generic methods

[Sylvain Prat]

Hello,

I recently encountered a pickling problem with peak.rules. The generic
methods do not pickle: they raise a "TypeError: can't pickle lock objects".
I've dugg into the code with a colleague and we found that the Dispatching
object that registers the implementations of the method is pickled and
since it contains a lock, the pickling fails.

It's a little annoying because it prevents us from using generic methods as
action callbacks (actions performed when e.g. we click on a link in a web
page) in Nagare Web applications because Nagare pickles them in the user
session.

I'm not sure how the problem can be fixed. I'm wondering if the Dispatching
object should be pickled (or not) and if regenerations have to be triggered
on pickling/depickling.

Can someone help? Thanks in advance!

Here is a test case that triggers the problem:

-----

from peak.rules import when
import cPickle as pickle


class Test(object):
def run(self, arg):
return 42

@when(run, 'arg == 0')
def run_3(self, arg):
return 0


if __name__ == '__main__':
run = Test().run

assert run(3) == 42
assert run(0) == 0

s = pickle.dumps(run) # raises a "TypeError: can't pickle lock objects"
run = pickle.loads(s)

assert run(3) == 42
assert run(0) == 0

[Hervé Coatanhay]

Hi,

As a note, the `lock` issue occurs with stackless python, which allows
instancemethod pickling. In standard cpython it is "TypeError: can't pickle
instancemethod objects" obviously.

--
Herv?

Post by Sylvain Prat
Hello,
I recently encountered a pickling problem with peak.rules. The generic
methods do not pickle: they raise a "TypeError: can't pickle lock objects".
I've dugg into the code with a colleague and we found that the Dispatching
object that registers the implementations of the method is pickled and
since it contains a lock, the pickling fails.
It's a little annoying because it prevents us from using generic methods
as action callbacks (actions performed when e.g. we click on a link in a
web page) in Nagare Web applications because Nagare pickles them in the
user session.
I'm not sure how the problem can be fixed. I'm wondering if the
Dispatching object should be pickled (or not) and if regenerations have to
be triggered on pickling/depickling.
Can someone help? Thanks in advance!
-----
from peak.rules import when
import cPickle as pickle
return 42
@when(run, 'arg == 0')
return 0
run = Test().run
assert run(3) == 42
assert run(0) == 0
s = pickle.dumps(run) # raises a "TypeError: can't pickle lock objects"
run = pickle.loads(s)
assert run(3) == 42
assert run(0) == 0
--
Sylvain PRAT
_______________________________________________
PEAK mailing list
http://www.eby-sarna.com/mailman/listinfo/peak

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.eby-sarna.com/pipermail/peak/attachments/20130202/985ae223/attachment.html

[PJ Eby]

On Fri, Feb 1, 2013 at 6:32 PM, Herv? Coatanhay

Post by Hervé Coatanhay
Hi,
As a note, the `lock` issue occurs with stackless python, which allows
instancemethod pickling. In standard cpython it is "TypeError: can't pickle
instancemethod objects" obviously.

Try defining the method *outside* the class, then copying it inside, e.g.:

@abstract
def run(self, arg):
pass

class Test(object):
run = run
...

This should cause the pickler to see the generic function as a global
function object rather than a standalone function. Essentially, the
problem is that the function shouldn't be being pickled as a
standalone object with state, it should be pickled as a thing imported
from a particular module, so that when the pickle is loaded it just
imports the function rather than *recreating* the function.

If you're doing this a lot, you might be better off with a custom
reducer for methods, something like:

class MethodWrapper(object):
def __init__(self, inst, name):
self.reduction = (getattr, (inst, name))
def __reduce__(self):
return self.reduction

Pickling a 'MethodWrapper(inst, methodname)' will create a pickle that
is loaded by fetching the named attribute from the (pickled) instance.

(Better still, write a version of this that you can register in the
copy_reg.dispatch_table as a custom handler for instance methods
within your program.)

In general, the workaround details are going to be highly specific to
the pickle implementation used; pickle and cPickle do things
differently, and if Stackless pickles methods, I imagine there are
even more differences.