[sage-devel] Fwd: [sage-trac] #26795: Some memory leaks

[Dima Pasechnik]

An expert in Cython/Python classes needed: __destruct__ is not called for a
reason I don't understand.

There is class Graph with a method convexity_properties

which calls cdef'd class ConvexityProperties in a separate pyx file.
A destructor for the latter is defined as
def __destruct__ but is never called. What is going on? What is the correct
design here?

Dima


---------- Forwarded message ---------
From: sage-trac <***@sagemath.org>
Date: Mon, 3 Dec 2018 15:02
Subject: Re: [sage-trac] #26795: Some memory leaks
To:


#26795: Some memory leaks
-------------------------------------+-------------------------------------
Reporter: jmantysalo | Owner:
Type: defect | Status: needs_work
Priority: major | Milestone: sage-8.5
Component: memleak | Resolution:
Keywords: | Merged in:
Authors: Dima Pasechnik | Reviewers:
Report Upstream: N/A | Work issues:
Branch: | Commit:
public/packages/glpkbackendmemleak |
c42c3bc27da534544dc62a953d5457d6c27532b5
Dependencies: | Stopgaps:
-------------------------------------+-------------------------------------

{{{
import gc
i = 0
GG = graphs()
_ = next(GG); _ = next(GG);
if i > 20000: break
_ = gc.collect()
print get_memory_usage(), P.order()
i += 1
_ = P.convexity_properties()
}}}
??? There is an explicit `__destruct__` in class `ConvexityProperties`.

however, `__destruct__` is not called, as one can see by putting a
`print()` there.

--
Ticket URL: <https://trac.sagemath.org/ticket/26795#comment:26>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple,
Mathematica, and MATLAB

[Nils Bruin]

Is there something new in cython that makes __destruct__ a special method?
I thought __dealloc__ was the place to undo __cinit__ stuff.

Post by Dima Pasechnik
An expert in Cython/Python classes needed: __destruct__ is not called for
a reason I don't understand.
There is class Graph with a method convexity_properties
which calls cdef'd class ConvexityProperties in a separate pyx file.
A destructor for the latter is defined as
def __destruct__ but is never called. What is going on? What is the
correct design here?
Dima
---------- Forwarded message ---------
Date: Mon, 3 Dec 2018 15:02
Subject: Re: [sage-trac] #26795: Some memory leaks
#26795: Some memory leaks
-------------------------------------+-------------------------------------
Type: defect | Status: needs_work
Priority: major | Milestone: sage-8.5
public/packages/glpkbackendmemleak |
c42c3bc27da534544dc62a953d5457d6c27532b5
-------------------------------------+-------------------------------------

{{{
import gc
i = 0
GG = graphs()
_ = next(GG); _ = next(GG);
if i > 20000: break
_ = gc.collect()
print get_memory_usage(), P.order()
i += 1
_ = P.convexity_properties()
}}}
??? There is an explicit `__destruct__` in class `ConvexityProperties`.

however, `__destruct__` is not called, as one can see by putting a
`print()` there.
--
Ticket URL: <https://trac.sagemath.org/ticket/26795#comment:26>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple,
Mathematica, and MATLAB

[Dima Pasechnik]

It is old code. And there is no __cinit__, only __init__

Post by Nils Bruin
Is there something new in cython that makes __destruct__ a special method?
I thought __dealloc__ was the place to undo __cinit__ stuff.

Post by Dima Pasechnik
An expert in Cython/Python classes needed: __destruct__ is not called
for a reason I don't understand.
There is class Graph with a method convexity_properties
which calls cdef'd class ConvexityProperties in a separate pyx file.
A destructor for the latter is defined as
def __destruct__ but is never called. What is going on? What is the
correct design here?
Dima
---------- Forwarded message ---------
Date: Mon, 3 Dec 2018 15:02
Subject: Re: [sage-trac] #26795: Some memory leaks
#26795: Some memory leaks
-------------------------------------+-------------------------------------
Type: defect | Status: needs_work
Priority: major | Milestone: sage-8.5
public/packages/glpkbackendmemleak |
c42c3bc27da534544dc62a953d5457d6c27532b5
-------------------------------------+-------------------------------------

{{{
import gc
i = 0
GG = graphs()
_ = next(GG); _ = next(GG);
if i > 20000: break
_ = gc.collect()
print get_memory_usage(), P.order()
i += 1
_ = P.convexity_properties()
}}}
??? There is an explicit `__destruct__` in class `ConvexityProperties`.

however, `__destruct__` is not called, as one can see by putting a
`print()` there.
--
Ticket URL: <https://trac.sagemath.org/ticket/26795#comment:26>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple,
Mathematica, and MATLAB

--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[Vincent Delecroix]

Old or new, C-allocator is __cinit__ and C-deallocator is
__dealloc__. The __destruct__ method is not special in
any way (beyond the fact that it starts and ends by
two underscores).

Post by Dima Pasechnik
It is old code. And there is no __cinit__, only __init__

Post by Nils Bruin
Is there something new in cython that makes __destruct__ a special method?
I thought __dealloc__ was the place to undo __cinit__ stuff.

Post by Dima Pasechnik
An expert in Cython/Python classes needed: __destruct__ is not called
for a reason I don't understand.
There is class Graph with a method convexity_properties
which calls cdef'd class ConvexityProperties in a separate pyx file.
A destructor for the latter is defined as
def __destruct__ but is never called. What is going on? What is the
correct design here?
Dima
---------- Forwarded message ---------
Date: Mon, 3 Dec 2018 15:02
Subject: Re: [sage-trac] #26795: Some memory leaks
#26795: Some memory leaks
-------------------------------------+-------------------------------------
Type: defect | Status: needs_work
Priority: major | Milestone: sage-8.5
public/packages/glpkbackendmemleak |
c42c3bc27da534544dc62a953d5457d6c27532b5
-------------------------------------+-------------------------------------

{{{
import gc
i = 0
GG = graphs()
_ = next(GG); _ = next(GG);
if i > 20000: break
_ = gc.collect()
print get_memory_usage(), P.order()
i += 1
_ = P.convexity_properties()
}}}
??? There is an explicit `__destruct__` in class `ConvexityProperties`.

however, `__destruct__` is not called, as one can see by putting a
`print()` there.
--
Ticket URL: <https://trac.sagemath.org/ticket/26795#comment:26>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple,
Mathematica, and MATLAB

--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[Dima Pasechnik]

Thanks everyone, __dealloc__ works marvellously.
That poor __desrtuct__() was never called (except by a sort of
meaningless doctesting), since it got there in 2011...
Actually, I wonder if there is a regular way to doctest destructors
like this one.


On Mon, Dec 3, 2018 at 4:19 PM Vincent Delecroix

Post by Vincent Delecroix
Old or new, C-allocator is __cinit__ and C-deallocator is
__dealloc__. The __destruct__ method is not special in
any way (beyond the fact that it starts and ends by
two underscores).

Post by Dima Pasechnik
It is old code. And there is no __cinit__, only __init__

Post by Nils Bruin
Is there something new in cython that makes __destruct__ a special method?
I thought __dealloc__ was the place to undo __cinit__ stuff.

Post by Dima Pasechnik
An expert in Cython/Python classes needed: __destruct__ is not called
for a reason I don't understand.
There is class Graph with a method convexity_properties
which calls cdef'd class ConvexityProperties in a separate pyx file.
A destructor for the latter is defined as
def __destruct__ but is never called. What is going on? What is the
correct design here?
Dima
---------- Forwarded message ---------
Date: Mon, 3 Dec 2018 15:02
Subject: Re: [sage-trac] #26795: Some memory leaks
#26795: Some memory leaks
-------------------------------------+-------------------------------------
Type: defect | Status: needs_work
Priority: major | Milestone: sage-8.5
public/packages/glpkbackendmemleak |
c42c3bc27da534544dc62a953d5457d6c27532b5
-------------------------------------+-------------------------------------

{{{
import gc
i = 0
GG = graphs()
_ = next(GG); _ = next(GG);
if i > 20000: break
_ = gc.collect()
print get_memory_usage(), P.order()
i += 1
_ = P.convexity_properties()
}}}
??? There is an explicit `__destruct__` in class `ConvexityProperties`.

however, `__destruct__` is not called, as one can see by putting a
`print()` there.
--
Ticket URL: <https://trac.sagemath.org/ticket/26795#comment:26>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple,
Mathematica, and MATLAB

--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[Jori Mäntysalo]

An expert in Cython/Python classes needed: __destruct__ is not called
for a reason I don't understand.

[ Resolution: Should have been __dealloc__ ]

I grepped the source, and this was the only __destruct__(). Of course
there might be other misnamed methods. By

egrep -R 'def __d[^ (]+' src/sage -o --no-filename | colrm 1 4 | sort | uniq -c | sort -rn

I found only __delete__, but that seems not to be like this one was. (Btw,
why schemes/elliptic_curves/ell_tate_curve.py has __delta() etc, not
_delta() and so?)

 > {{{
 > import gc
 >
 > i = 0
 > GG = graphs()
 > _ = next(GG); _ = next(GG);
 >     if i > 20000: break
 >         _ = gc.collect()
 >         print get_memory_usage(), P.order()
 >     i += 1
 >     _ = P.convexity_properties()
 > }}}

I did some other tests, and found no more leaks. It may make sense to run
this in a more systematic way after leaks in MILP interface has been
resolved. And it's not a bad idea to do similar tests in, say, matrix
functions over finite field etc.

[Simon King]

Hi!

Post by Jori Mäntysalo
I grepped the source, and this was the only __destruct__(). Of course
there might be other misnamed methods. By
egrep -R 'def __d[^ (]+' src/sage -o --no-filename | colrm 1 4 | sort | uniq -c | sort -rn
I found only __delete__, but that seems not to be like this one was. (Btw,
why schemes/elliptic_curves/ell_tate_curve.py has __delta() etc, not
_delta() and so?)

__delete__ probably should be __del__ (which is a magical Python
method).

Best regards,
Simon