4D Doc Center : Arrays and Memory

4D v13.4

Arrays and Memory

Unlike the data you store on disk using tables and records, an array is always held in memory in its entirety.

For example, if all US zip codes were entered in the [Zip Codes] table, it would contain about 100,000 records. In addition, that table would include several fields: the zip code itself and the corresponding city, county, and state. If you select only the zip codes from California, the 4D database engine creates the corresponding selection of records within the [Zip Codes] table, and then loads the records only when they are needed (i.e., when they are displayed or printed). In order words, you work with an ordered series of values (of the same type for each field) that is partially loaded from the disk into the memory by the database engine of 4D.

Doing the same thing with arrays would be prohibitive for the following reasons:

In order to maintain the four information types (zip code, city, county, state), you would have to maintain four large arrays in memory.
Because an array is always held in memory in its entirety, you would have to keep all the zip codes information in memory throughout the whole working session, even though the data is not always in use.
Again, because an array is always held in memory in its entirety, each time the database is started and then quit, the four arrays would have to be loaded and then saved on the disk, even though the data is not used or modified during the working session.

Conclusion: Arrays are intended to hold reasonable amounts of data for a short period of time. On the other hand, because arrays are held in memory, they are easy to handle and quick to manipulate.

However, in some circumstances, you may need to work with arrays holding hundreds or thousands of elements. The following table lists the formulas used to calculate the amount of memory used for each array type:

Array Type	Formula for determining Memory Usage in Bytes
Boolean	(31+number of elements)\8
Date	(1+number of elements) * 6
String (Unicode mode)	(1+number of elements) * (Sum of the size of each text)
String (ASCII compatibility mode)	(1+number of elements) * Declared length (+1 of odd, +2 if even)
Integer	(1+number of elements) * 2
Long Integer	(1+number of elements) * 4
Picture	(1+number of elements) * 4 + Sum of the size of each picture
Pointer	(1+number of elements) * 16
Real	(1+number of elements) * 8
Text (Unicode mode)	(1+number of elements) * (Sum of the size of each text)
Text (ASCII compatibility mode)	(1+number of elements) * 6 + Sum of the size of each text
Two-dimensional	(1+number of elements) * 12 + Sum of the size of each array

Notes:

The size of a text in memory is calculated using this formula: ((Length + 1) * 2)
A few additional bytes are required to keep track of the selected element, the number of elements, and the array itself.

When working with very large arrays, the best way to handle full memory situations is to surround the creation of the arrays with an ON ERR CALL project method. Example:

  ` You are going to run a batch operation the whole night

  ` that requires the creation of large arrays. Instead of risking

  ` occurrences of errors in the middle of the night, put

  ` the creation of the arrays at the beginning of the operation

  ` and test the errors at this moment:

 gError:=0 ` Assume no error

 ON ERR CALL("ERROR HANDLING") ` Install a method for catching errors

 ARRAY STRING(63;asThisArray;50000) ` Roughly 3125K in ASCII mode

 ARRAY REAL(arThisAnotherArray;50000) ` 488K

 ON ERR CALL("") ` No longer need to catch errors

 If(gError=0)

  ` The arrays could be created

  ` and let's pursue the operation

 Else

    ALERT("This operation requires more memory!")

 End if

  ` Whatever the case, we no longer need the arrays

 CLEAR VARIABLE(asThisArray)

 CLEAR VARIABLE(arThisAnotherArray)

The ERROR HANDLING project method is listed here:

  ` ERROR HANDLING project method

 gError:=Error ` Just return the error code