|
Search | Today's Posts | Mark Forums Read |
|
Thread Tools | Search this Thread |
|
|||
|
|||
In the low level API, it seems that when I call the function CmtGetRecordDataByRecId() a few hundred thousand times, something leaks memory. It's roughly a few hundred KB/s in the daemon I wrote. I'm using Python, which has a reference counted garbage collector, and all of the memory profiling seems to indicate that there's no uncontrolled memory allocation going on in my code.
I'm not quite sure yet if it's a problem with my code, or the library. I don't have access to Valgrind on Windows, so I'm not sure how I would go about conclusively proving it one way or the other. Is this a known problem? Is there some way we could figure out if this function is leaking memory? Thanks, guys. |
|
|||
|
|||
FYI, I reproduced the problem in less than fifty lines of C, without any dynamic memory allocation. This rules out Python garbage collection as well as a memory leak in my code.
You can find the source here: https://gist.github.com/jakogut/f429540cf8487ff24115 As I said, I believe that I found a workaround for the issue in my project, but I think this shows conclusively that there is a problem with a leak in this function. I'd be happy to help further with this issue if necessary. |
|
|||
|
|||
It appears that my workaround wasn't as resilient as I had thought.
I was refreshing the handle to the low-level API libraries after a certain number of calls, hoping Python's garbage collector would delete and free the loaded libraries after the reference count decreased to zero. However, the ctypes module does not garbage collect and unload libraries by default until the interpreter exits, because any open references to the library in question will crash the interpreter in short order. This means that my workaround was not actually freeing the memory being leaked, which was causing the machine hosting the code to run out of memory every two or three days. I found that I could manually unload the library with a call to: Quote:
Is there any update on fixing the leak in the library itself? |
|
|||
|
|||
Thank you for the update. I checked and we do not have any news on this right now. It was reviewed at the time but nothing came up. However, following your post here we will reevaluate and review this again in our lab and will post an update once we have it. Thanks!
|
|
|||
|
|||
I did find a better workaround for this issue. I isolated my wrapper class for CommitCRM's low level interface into its own process, and access it using Pyro4 (an RPC/data serialization library).
This way I can terminate and recreate the process to free up the used memory, it doesn't crash the interpreter, and even if it did, it wouldn't be a problem. |
«
Previous Thread
|
Next Thread
»
Thread Tools | Search this Thread |
|
All times are GMT -6. The time now is 08:39 AM.