From Documentation
Jump to: navigation, search

Allocating Arrays

Static arrays in FORTRAN 90 are created on the stack which has a fixed size(2Gb).

Dynamic arrays in FORTRAN 90 are created on the heap, can have any size (provided there is space left on the heap), but need to be allocated and deleted by the program.

But with c/C++ static and dynamic arrays can have any size (provided there is space left on the heap).

c/C++ examples

Program listings and command lines to compile and run c and C++ examples follow:

static c

// file scA.c
 
 int main () {
 
   long int sz2G=2000000000;
   long int myarray[sz2G];        // c static array
 
    myarray[sz2G-1]=sz2G;
    printf("myarray[sz2G-1]=%ld\n",myarray[sz2G-1]);
 
    return 0;
 }


$CC scA.c
./a.out
myarray[sz2G-1]=2000000000

dynamic c

 // file dcA.c
 
 int main () {
 
   long int sz2G=2000000000;       // c dynamic array
   long int * myarray = malloc(sizeof(long int) *sz2G);
 
    myarray[sz2G-1]=sz2G;
    printf("myarray[sz2G-1]=%ld\n",myarray[sz2G-1]);
 
    return 0;
 }
$CC dcA.c
./a.out
myarray[sz2G-1]=2000000000

static C++

 // file sCPPA.C
 
 #include <iostream>
 using namespace std;
 
 int main () {
 
   long int sz2G=2000000000;
   long int myarray[sz2G];        // C++ static array
 
    myarray[sz2G-1]=sz2G;
    cout << "myarray[sz2G-1]=" << myarray[sz2G-1] << endl;
 
    return 0;
 }
 $CXX sCPPA.C
./a.out
myarray[sz2G-1]=2000000000

dynamic C++

 // file dCPPA.C
 
 #include <iostream>
 using namespace std;
 
 int main () {
 
   long int sz2G=2000000000;
   long int * myarray = new long int[sz2G];  // C++ dynamic array
 
    myarray[sz2G-1]=sz2G;
    cout << "myarray[sz2G-1]=" << myarray[sz2G-1] << endl;
 
    return 0;
 }
$CXX dCPPA.C
./a.out
myarray[sz2G-1]=2000000000

Fortran example

Declaring large arrays in FORTRAN programs and using the Intel compiler often leads to compile errors like:

 "relocation truncated to fit r_x86_64_pc32 against .bss"

Even with the flag '-mcmodel large' in the compile command the error will persist, because there is a limit to the amount of data you can have allocated in static arrays for x86_64 architecture. This data segment is limited to 2GB. The data in COMMON plus any other statically declared arrays PLUS your code must fit in 2GB. (the problem does not occur with gfortran because it allocates static arrays on the heap)

In order to overcome this problem use allocatable arrays declared in a module together with a short subroutine which allocates the arrays.

We illustrate these concepts by starting with a program 'fortx.f90' which work fine for values of parameter 'Nmx' less than 108. Thus, if you compile it with

      integer, parameter :: Nmx=107

it works, but for any higher values it fails with error message:

 "relocation truncated to fit: R_X86_64_PC32 against symbol '...'"

To overcome this problem, we first copy program fortx.f90 into a new file, say, forty.f90, and write a module called 'mp_data' and a subroutine called 'allocate_arrays' at the top of the new file. We move all the declarations of large arrays into the module, and declare them as 'allocatable'.

In the subroutine 'allocate_arrays' we do the allocations and in the main program we add the statements:

     USE mp_data
     ...
     call allocate_arrays
     ...

Below you will find the listings for the files 'fortx.f90', 'forty.f90' and the 'HIST' file containing the commands to compile these files and submit jobs:

Original program fortx.f90

 ! file fortx.f90
 
       program FORTX
 
       implicit none
 
       integer, parameter :: wp = kind(0.0q0)  ! Quad precision
 
 !     integer, parameter :: Nmx=107  ! compilation fails for 108
       integer, parameter :: Nmx=108
 
       INTEGER   :: I,J,K,L
 
       real(wp) zero, one, mem, giga
 
       real(wp) fac(0:Nmx),fci(0:Nmx)
       real(wp) BETA2(Nmx,Nmx)
       real(wp) BETA3(Nmx,Nmx,Nmx)
       real(wp) BETA4(Nmx,Nmx,Nmx,Nmx)
 
       giga = 1000000000.0
       mem  = 2*(2+Nmx) + Nmx*Nmx + Nmx*Nmx*Nmx + Nmx*Nmx*Nmx*Nmx
 
       mem = 16*mem/giga
 
       write(6,1010) mem
  1010 format("mem=",f12.4)
 
       zero=0.0_wp
       one =1.0_wp
 
       fac(0)=one
       fci(0)=one
 
       do j=1,Nmx
         fac(j)=fac(j-1)*j
         fci(j)=one/fac(j)
       end do
 
       I=Nmx/2
       J=I-1
       K=J-1
       L=K-1
 
       write(6,1000) I,J,K,L
  1000 format(/"I,J,K,L = ",4i3)
 
       write(6,1001) I,fac(I)
  1001 format(/"fac(",i3.3,") =",e16.8)
 
       BETA2(I,J)      =fac(I)*fac(J)*fci(I+J)
       BETA3(I,J,K)    =fac(I)*fac(J)*fac(K)*fci(I+J)*fci(J+K)
       BETA4(I,J,K,L)  =fac(I)*fac(J)*fac(K)*fac(L)*fci(I+J)*fci(J+K)*fci(K+L)
 
       write(6,1002) I,J,BETA2(I,J)
  1002 format(/"BETA2(",i3.3,",",i3.3,") =",e16.8)
 
       write(6,1003) I,J,K,BETA3(I,J,K)
  1003 format(/"BETA3(",i3.3,",",i3.3,",",i3.3,") =",e16.8)
 
       write(6,1004) I,J,K,L,BETA4(I,J,K,L)
  1004 format(/"BETA4(",i3.3,",",i3.3,",",i3.3,",",i3.3,") =",e16.8)
 
       write(6,1005)
  1005 format(/"DONE"//)
 
       stop
       end

Modified program forty.f90

 ! file forty.f90
 
       module mp_data
 
       implicit none
 
       integer, parameter :: wp = kind(0.0q0)  ! Quad precision
       integer, parameter :: Nmx =108
 
       integer            :: AllocateStatus
 
 !     real(wp) zero, one, mem, giga
 !
 !     real(wp) fac(0:Nmx),fci(0:Nmx)
 !     real(wp) BETA2(Nmx,Nmx)
 !     real(wp) BETA3(Nmx,Nmx,Nmx)
 !     real(wp) BETA4(Nmx,Nmx,Nmx,Nmx)
 
       real(wp) zero, one
 
       real(wp), dimension (:), allocatable        :: fac,fci
       real(wp), dimension (:,:), allocatable      :: BETA2
       real(wp), dimension (:,:,:), allocatable    :: BETA3
       real(wp), dimension (:,:,:,:), allocatable  :: BETA4
 
       end module mp_data
 
       subroutine allocate_arrays
 
       USE mp_data
       implicit none
 
       allocate(fac(0:Nmx),STAT=AllocateStatus)
       IF (AllocateStatus /= 0) STOP "Not enough memory for fac "
 
      allocate(fci(0:Nmx),STAT=AllocateStatus)
       IF (AllocateStatus /= 0) STOP "Not enough memory for fci "
 
      allocate(BETA2(Nmx,Nmx),STAT=AllocateStatus)
       IF (AllocateStatus /= 0) STOP "Not enough memory for BETA2"
 
      allocate(BETA3(Nmx,Nmx,Nmx),STAT=AllocateStatus)
       IF (AllocateStatus /= 0) STOP "Not enough memory for BETA3"
 
      allocate(BETA4(Nmx,Nmx,Nmx,Nmx),STAT=AllocateStatus)
       IF (AllocateStatus /= 0) STOP "Not enough memory for BETA4"
 
       print *,"Done with allocations"
 
       return
       END
 
 ! ----------------------------------------------------------------------
 
       program FORTY
       USE mp_data
 
       implicit none
 
       INTEGER :: I,J,K,L
 
       zero=0.0_wp
       one =1.0_wp
 
       call allocate_arrays
 
       fac(0)=one
       fci(0)=one
 
       do j=1,Nmx
         fac(j)=fac(j-1)*j
         fci(j)=one/fac(j)
       end do
 
       I=Nmx/2
       J=I-1
       K=J-1
       L=K-1
 
       write(6,1000) I,J,K,L
  1000 format(/"I,J,K,L = ",4i3)
 
       write(6,1001) I,fac(I)
  1001 format(/"fac(",i3.3,") =",e16.8)
 
       BETA2(I,J)      =fac(I)*fac(J)*fci(I+J)
       BETA3(I,J,K)    =fac(I)*fac(J)*fac(K)*fci(I+J)*fci(J+K)
       BETA4(I,J,K,L)  =fac(I)*fac(J)*fac(K)*fac(L)*fci(I+J)*fci(J+K)*fci(K+L)
 
       write(6,1002) I,J,BETA2(I,J)
  1002 format(/"BETA2(",i3.3,",",i3.3,") =",e16.8)
 
       write(6,1003) I,J,K,BETA3(I,J,K)
  1003 format(/"BETA3(",i3.3,",",i3.3,",",i3.3,") =",e16.8)
 
       write(6,1004) I,J,K,L,BETA4(I,J,K,L)
  1004 format(/"BETA4(",i3.3,",",i3.3,",",i3.3,",",i3.3,") =",e16.8)
 
       write(6,1005)
  1005 format(/"DONE"//)
 
       stop
       end

HIST file with command lines

 
# file HIST
 #
 # Set Nmx=107
 
         vi fortx.f90
 
 and set Nmx=107 in file fortx.f90
 
 # Compile (copy/paste command line)
 
 $FC fortx.f90
 
 # Execute code (it should work)
 
         ./a.out
 
 # Set Nmx=108
 
         vi fortx.f90
 
 and set Nmx=108 in file fortx.f90
 
 # Re-compile and look at file AOUT1
 
 ifort fortx.f90 >& AOUT1
 
 
 ============================================================================
 
 # File forty.f90 was copied from fortx.f90  and modified as described earlier.
 
 # Compile forty.f90
 
 ifort forty.f90 -mcmodel medium -shared-intel
 
 
 
 ============================================================================
 
 
 # The executable might require more than 2.0G of memory:
 
 # This will pruduce an error:
 
 sqsub -r 5m  --mpp=4.0G  -o OUTPUTFILE4 ./a.out
 
 # This should work on cluster hound:
 
 sqsub -r 5m  --mpp=80.0G  -o OUTPUTFILE80  ./a.out
 
 ============================================================================