What can we measure? with what tools?
=========================================




Figures given by Aster
-------------------------

From the start of a calculation, JEVEUX prints several information concerning memory management into the "message" file:


.. code-block:: text

    MEMOIRE IMPOSEE POUR JEVEUX: 3145728 OCTETS (3,000 MEGAOCTETS)

    MEMOIRE DONNEE PAR "MEMDIS ": 3145728 OCTETS

    MEMOIRE PRISE: 3145728 OCTETS (3,000 MEGAOCTETS)

    LIMITE MEMOIRE DYNAMIQUE: 1258291200 OCTETS (1200,000 MEGAOCTETS)

In this example, the important information is on the first and last lines: static memory JEVEUX is limited to 3 MB and dynamic memory is limited to 1200 MB. By default, all objects are allocated to dynamic memory and static memory is limited to 1 MB. So if we ask for 100 MB of total memory in ASTK, we will have 1 MB on the first line and 99 MB on the last line.

At the end of the calculation, JEVEUX prints other information concerning the use of dynamic memory:

.. code-block:: text


    STATISTIQUES CONCERNANT The ALLOCATION DYNAMIQUE:

    TAILLE CUMULEE MAXIMUM 577 MB,

    DONT 3 MB POUR LA ZONE GEREE PAR JEVEUX.

    TAILLE CUMULEE LIBEREE 1173 MB.

    NOMBRE TOTAL from ALLOCATIONS: 487.

    NOMBRE TOTAL FROM LIBERATIONS: 487.

    0 APPELS TO THE MECANISME OF LIBERATION.

    TAILLE MEMOIRE CUMULEE RECUPEREE: 0 MB.

In the previous example, we can see (a posteriori) that the dynamic memory allocated to JEVEUX (1200 MB) is oversized: With 577 MB, the calculation would have gone through without triggering the "release mechanism". It is likely that this calculation can be done with less memory than 577 MB.

.. code-block:: text




The Unix "top" command
----------------------

The Unix top command gives information (refreshed at regular intervals) about the progress of a process.

The 3 columns VIRT, RES, SHR concern the use of memory.


.. code-block:: text

    VIRT: total virtual memory used by the process.

    RES: "resident" memory (not swapped)

    SHR: "shared" memory

But other parameters are available: DATA, SWAP,...

For more details, consult Man Top

.. code-block:: text




The /proc file system
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

System UNIX creates for each process with the number xxxxx, a (temporary) directory with the name /proc/xxxxx.

In this directory, we find in particular the status file. This file, updated periodically by the system, gives information on the use of memory by the process. Interesting items have names starting with Vm (Virtual Memory).

For example, we will find:


.. code-block:: text

    VMpeak: 174912 kB

    VM Size: 168,384 kB

    VMLck: 0 kB

    VM HWM: 33728 kB

    VM RSS: 33664 kB

    VMdata: 159424 kB

    VMStk: 1344 kB

    VMExe: 384 kB

    VMlib: 3520 kB

    VM PTE: 704 kB

A small experiment on the Bull machine showed that the 3 parameters:


.. code-block:: text

    VMsize

    Vm RSS

    VMdata

incremented (and unincremented) after Fortran 90's ALLOCATE/DEALLOCATE.

They can therefore be used to "track" over time the sum of the variables dynamically allocated by the program.

For more information: http://okki666.free.fr/docmaster/articles/linux070.htm

.. code-block:: text




Measuring memory usage in a Fortan program
------------------------------------------------------

.. code-block:: text

Since version 9.3.11, we know after each order the instantaneous consumption of JEVEUX, but also the peak reached so far:

.. code-block:: text


    USAGE OF THE MEMOIRE JEVEUX

    # - MEMOIRE DYNAMIQUE CONSOMMEE: 112.76 MB (MAXIMUM ATTEINT: 558.45 MB)

In addition, an interesting indication to look at the end of the execution is the minimum memory required to run the calculation (value MAXIMUM ATTEINT of the MEMOIRE UTILISEE):


.. code-block:: text

    # - MEMOIRE UTILISEE: 7.54 MB (MAXIMUM ATTEINT: 110.47 MB)




Measuring memory usage in a Python program
------------------------------------------------------

On the web, we found a small piece of Python code (see below) that uses the /proc/xxxxx/status file presented in the previous paragraph.

In a Python program, if we want to know the memory consumption of a piece of code, we can do:


.. code-block:: text

    mav=memory ()

    ...

    Code snippet to be instrumentalized

    ...

    map=memory ()

    print "Increase in memory used (in bytes):", map — mav


    #-----------------------------------------------------------------

    # Python script to measure memory usage:

    #-----------------------------------------------------------------

    Import OS


    _proc_status = '/proc/%d/status'% os.getpid ()


    _scale = {'kB': 1024.0, 'mB': 1024.0*1024.0,

    'KB': 1024.0, 'MB': 1024.0*1024.0}


    def _VMb (VMKey):

    "'Private.

    "'

    global _proc_status, _scale

    <pid># get nickname file /proc/ /status

    Try:

    t = open (_proc_status)

    v = t.read ()

    t.close ()

    except:

    return 0.0 # Non-Linux?

    # get VMKey line e.g. 'Vm RSS: 9999 kB\n... '

    i = v.index (VMKey)

    v = v [i:] .split (None, 3) # whitespace

    If len (v) < 3:

    return 0.0 # invalid format?

    # convert Vm value to bytes

    return float (v [1]) * _scale [v [2]]


    def memory (since=0.0):

    "'Return memory usage in bytes.

    "'

    return _VMb ('vmSize: ') - since


    def resident (since=0.0):

    "'Return resident memory usage in bytes.

    "'

    return _VMb ('Vm RSS: ') - since


    def stacksize (since=0.0):

    "'Return stack size in bytes.

    "'

    return _VMb ('VMStk: ') - since

    #-----------------------------------------------------------------