parallel processing - A memory issue with multiprocessing of Python: How to release the memory of process who ended their tasks -


when run multiprocessing several pools, how quit process after tasks , drop memory usage? include close() , join(). saw process still holds memory after jobs. how drop memory?

for example, code plays below

from multiprocessing import pool import pandas pd      def sub_test(aux):     return sum(aux.a) 

at below 4 pools generated , holds several copies of aux.

def test(aux):     pool = pool(processes=4)     pool.map(sub_test, aux)     pool.close()     pool.join() 

i expect process release memory after close , join, still holds.

#here main function aux = pd.dataframe() aux['a'] = [1,1,1]  x in range(10):    test(aux) 

is there commands handle issue?


problems solved.

pool.close() , pool.join() work properly.


Comments