Dear All,
My doubts :-
----------------
Is there any way to store the data (temporarily) in the SWAP SPACE instead
of RAM ?
When RAM overflows then only SWAP area will be utilized by the OS.
At the same time the whole system started to slowdown to do another jobs due
insufficient memory in RAM.
Is there any way to store some data in SWAP (but not in RAM) and get the
data from the SWAP partition whenever we need it in Python or C language ?
yup. It may be slower to access the swap area rather than RAM, but it is
much faster than access the data of files which are stored in the hard disk.
Explanation about my doubt/need :-
------------------------------------------------
I am running one computation program in python / CDAT [1].
In that program we need to access the data in numpy array size of (122 x 360
x 720). [i.e time x latitude x longitude in shape ]
By accessing two different data in the same shape (above), we are creating
15 different array variables in the shape of (10 x 360 x 720).
I have created the file object to extract the each latitude,longitude of
data,
to compute the 15 different calculations and store it in the temporary numpy
array.
Finally store the results into netCDF file format.
It takes 2 days to complete this job and eats 3GB RAM memory.
One of the reasons is ' 2 x 360 x 720 ( = 518400 ) times needs to access
python file object to retrieve the data of corresponding latitude,
longitude'.
So accessing the file object 518400 times to get the data makes the delay
time to complete our tasks...
Even though if we used threads, parallel python and cluster the system(s)
needs to access the file object 518400 times in my case.
Instead of these if we stored the data into swap space means, the accessing
the data should be faster.
[ Read my doubts again here ]
I may be wrong. Please throw me your suggestions.
Thanks in advance.
[1] http://www2-pcmdi.llnl.gov/
--
Regards,
Arulalan.T
Project Associate
Centre for Atmospheric Sciences
Indian Institute of Technology Delhi
My Experiments In Gnu/Linux ! : http://tuxcoder.wordpress.com
Kanchi Linux User Group Rocks ! : http://kanchilug.wordpress.com