Managing the log file

A continuously-used database is always recording changes, and record additions or deletions. Performing regular backups of data is important but does not allow (in case of incident) restoring data entered since the last backup. To respond to this need, 4D now offers a specific tool: the log file. This file allows ensuring permanent security of database data.

In addition, 4D works continuously with a data cache in memory. Any changes made to the data of the database are stored temporarily in the cache before being written to the hard disk. This accelerates the operation of applications; in fact, accessing memory is faster than accessing the hard disk. If an incident occurs in the database before the data stored in the cache could be written to the disk, you must include the current log file in order to restore the database entirely.

Finally, 4D has functions that analyze the contents of the log file, making it possible to rollback the operations carried out on the data of the database. These functions area available in the MSC: refer to the Activity analysis page and the Rollback page.

The log file generated by 4D contains all operations performed on the data of journaled tables of the database, which are logged sequentially. By default, all the tables are journaled, i.e. included in the log file, but you can deselect individual tables using the Include in Log File property.

As such, each operation performed by a user causes two simultaneous actions: the first one in the database (instruction is executed normally) and the second one in the log file (the description of the operation is recorded). The log file is created independently without disturbing or slowing down the work of the user. A database can only work with one log file at a time. The log file records the following types of operations:

• Opening and closing of the data file,
• Opening and closing of the process (contexts),
• Adding of records or BLOBs,
• Modifying of records,
• Deleting of records,
• Creating and closing of transactions,

For more information about these actions, refer to the Activity analysis page. 

4D manages the log file. It takes into account all operations that affect the data file equally, regardless of any manipulations performed by a user, 4D methods, the SQL engine, 4D plug-ins (4D Write, 4D View, etc.), or a Web browser.

The following illustration sums up how the log file works:

The current log file is automatically saved with the current data file. This mechanism has two distinct advantages:

• Its avoids saturating the disk volume where the log file is stored. Without a backup, the log file would get bigger and bigger with use, and would eventually use all available disk space. For each data file backup, 4D or 4D Server closes the current log file and immediately starts a new, empty file, thereby avoiding the risk of saturation. The old log file is then archived and eventually destroyed depending on the mechanism for managing the backup sets.
• It keeps log files corresponding to backups in order to be able to parse or repair a database at a later point in time. The integration of a log file can only be done in the database to which it corresponds. It is important, in order to be able to properly integrate a log file into a backup, to have backups and log files archived simultaneously.

By default, any database created with 4D uses a log file: the creation of this file is part of the automatic backup strategy (refer to Automatic backup strategy). The log file is named DataFileName.journal and is placed in the folder containing the database structure.

You can find out if your database uses a log file at any time: just check whether the Use Log File option is selected on the Backup/Configuration page of the Database Settings (see Log management). If you deselected this option, or if you use a database without a log file and wish to set up a backup strategy with a log file, you will have to create one. 

To create a log file:

1. On the Backup/Configuration page of the Database Settings, check the Use Log File option.
   The program displays a standard open file or new log file dialog box. By default, the log file is named DataFileName.journal.
2. Keep the default name or rename it, and then select the file location.
   If you have at least two hard drives, it is recommended that you place the log file on a disk other than the one containing the database. If the database hard drive is lost, you can still recall your log file.
3. Click Save.
   The disk and the name of the open log file are now displayed in the “Use Log File” area of the dialog box. You can click on this area in order to display a pop-up menu containing the series of folders on the disk.
4. Validate the Database Settings dialog box.

Note: Starting with 4D v14, new internal mechanisms for generating and using a log file are implemented in order to make this file more resistant. To use a log file, all of the journaled tables must have a primary key. If this is not the case (in a converted database for example), a warning dialog box indicates that it is not possible to activate the log file as long as primary keys have not been correctly set for all journaled tables. In order to make your application compliant, you can use the Creating a database structure. This assistant checks the state of each table and allows you to set a primary key.

In order for you to be able to create a log file directly, the database must be in one of the following situations (except for compliance issues related to primary keys, see above):

• The data file is blank,
• You just performed a backup of the database and no changes have yet been made to the data.

In all other cases, when you validate the Database Settings dialog box, an alert dialog box will appear to inform you that it is necessary to perform a backup. If you click OK, the backup begins immediately, then the log file is activated. If you click Cancel, the request is saved but the creation of the log file is postponed and it will actually be created only after the next backup of the database. This precaution is indispensable because, in order to restore a database after any incidents, you will need a copy of the database into which the operations recorded in the log file will be integrated. 

Without having to do anything else, all operations performed on the data are logged in this file and it will be used in the future when the database is opened. 

You must create another log file if you create a new data file. You must set or create another log file if you open another data file that is not linked to a log file (or if the log file is missing).

If you would like to stop logging operations to the current log file, simply deselect the Use Log File option on the Backup/Configuration page of the Database Settings.

4D then displays an alert message to remind you that this action prevents you from taking advantage of the security that the log file provides:

If you click Stop, the current log file is immediate closed (the Database Settings dialog box does not need to be validated afterwards). 

If you wish to close the current log file because it is too large, you must perform a data file backup, which will cause the log file to be backed up as well.

Notes for 4D Server:

• The New log file command automatically closes the current log file and starts a new one.
• If for some reason the log file becomes unavailable during a working session, error 1274 is generated and 4D Server does not allow users to write data anymore. When the log file is available again, it is necessary to do a backup.