2. What can we measure? with what tools?#
2.1. Figures given by Aster#
From the start of a calculation, JEVEUX prints several information concerning memory management into the « message » file:
MEMOIRE IMPOSEE POUR JEVEUX: 3145728 OCTETS (3,000 MEGAOCTETS)
MEMOIRE DONNEE PAR "MEMDIS ": 3145728 OCTETS
MEMOIRE PRISE: 3145728 OCTETS (3,000 MEGAOCTETS)
LIMITE MEMOIRE DYNAMIQUE: 1258291200 OCTETS (1200,000 MEGAOCTETS)
In this example, the important information is on the first and last lines: static memory JEVEUX is limited to 3 MB and dynamic memory is limited to 1200 MB. By default, all objects are allocated to dynamic memory and static memory is limited to 1 MB. So if we ask for 100 MB of total memory in ASTK, we will have 1 MB on the first line and 99 MB on the last line.
At the end of the calculation, JEVEUX prints other information concerning the use of dynamic memory:
STATISTIQUES CONCERNANT The ALLOCATION DYNAMIQUE:
TAILLE CUMULEE MAXIMUM 577 MB,
DONT 3 MB POUR LA ZONE GEREE PAR JEVEUX.
TAILLE CUMULEE LIBEREE 1173 MB.
NOMBRE TOTAL from ALLOCATIONS: 487.
NOMBRE TOTAL FROM LIBERATIONS: 487.
0 APPELS TO THE MECANISME OF LIBERATION.
TAILLE MEMOIRE CUMULEE RECUPEREE: 0 MB.
In the previous example, we can see (a posteriori) that the dynamic memory allocated to JEVEUX (1200 MB) is oversized: With 577 MB, the calculation would have gone through without triggering the « release mechanism ». It is likely that this calculation can be done with less memory than 577 MB.
2.2. The Unix « top » command#
The Unix top command gives information (refreshed at regular intervals) about the progress of a process.
The 3 columns VIRT, RES, SHR concern the use of memory.
VIRT: total virtual memory used by the process.
RES: "resident" memory (not swapped)
SHR: "shared" memory
But other parameters are available: DATA, SWAP,…
For more details, consult Man Top
2.2.1. The /proc file system#
System UNIX creates for each process with the number xxxxx, a (temporary) directory with the name /proc/xxxxx.
In this directory, we find in particular the status file. This file, updated periodically by the system, gives information on the use of memory by the process. Interesting items have names starting with Vm (Virtual Memory).
For example, we will find:
VMpeak: 174912 kB
VM Size: 168,384 kB
VMLck: 0 kB
VM HWM: 33728 kB
VM RSS: 33664 kB
VMdata: 159424 kB
VMStk: 1344 kB
VMExe: 384 kB
VMlib: 3520 kB
VM PTE: 704 kB
A small experiment on the Bull machine showed that the 3 parameters:
VMsize
Vm RSS
VMdata
incremented (and unincremented) after Fortran 90’s ALLOCATE/DEALLOCATE.
They can therefore be used to « track » over time the sum of the variables dynamically allocated by the program.
For more information: http://okki666.free.fr/docmaster/articles/linux070.htm
2.3. Measuring memory usage in a Fortan program#
Since version 9.3.11, we know after each order the instantaneous consumption of JEVEUX, but also the peak reached so far:
USAGE OF THE MEMOIRE JEVEUX
# - MEMOIRE DYNAMIQUE CONSOMMEE: 112.76 MB (MAXIMUM ATTEINT: 558.45 MB)
In addition, an interesting indication to look at the end of the execution is the minimum memory required to run the calculation (value MAXIMUM ATTEINT of the MEMOIRE UTILISEE):
# - MEMOIRE UTILISEE: 7.54 MB (MAXIMUM ATTEINT: 110.47 MB)
2.4. Measuring memory usage in a Python program#
On the web, we found a small piece of Python code (see below) that uses the /proc/xxxxx/status file presented in the previous paragraph.
In a Python program, if we want to know the memory consumption of a piece of code, we can do:
mav=memory ()
...
Code snippet to be instrumentalized
...
map=memory ()
print "Increase in memory used (in bytes):", map — mav
#-----------------------------------------------------------------
# Python script to measure memory usage:
#-----------------------------------------------------------------
Import OS
_proc_status = '/proc/%d/status'% os.getpid ()
_scale = {'kB': 1024.0, 'mB': 1024.0*1024.0,
'KB': 1024.0, 'MB': 1024.0*1024.0}
def _VMb (VMKey):
"'Private.
"'
global _proc_status, _scale
<pid># get nickname file /proc/ /status
Try:
t = open (_proc_status)
v = t.read ()
t.close ()
except:
return 0.0 # Non-Linux?
# get VMKey line e.g. 'Vm RSS: 9999 kB\n... '
i = v.index (VMKey)
v = v [i:] .split (None, 3) # whitespace
If len (v) < 3:
return 0.0 # invalid format?
# convert Vm value to bytes
return float (v [1]) * _scale [v [2]]
def memory (since=0.0):
"'Return memory usage in bytes.
"'
return _VMb ('vmSize: ') - since
def resident (since=0.0):
"'Return resident memory usage in bytes.
"'
return _VMb ('Vm RSS: ') - since
def stacksize (since=0.0):
"'Return stack size in bytes.
"'
return _VMb ('VMStk: ') - since
#-----------------------------------------------------------------