Decreasing the size of cPickle objects
I am running code that creates large objects, containing multiple
user-defined classes, which I must then serialize for later use. From what
I can tell, only pickling is versatile enough for my requirements. I've
been using cPickle to store them but the objects it generates are
approximately 40G in size, from code that runs in 500 mb of memory. Speed
of serialization isn't an issue, but size of the object is. Are there any
tips or alternate processes I can use to make the pickles smaller?
No comments:
Post a Comment