From cd6ecf6829e813c91dd38eeafefb647775cb2314 Mon Sep 17 00:00:00 2001 From: Tom Lane Date: Mon, 8 Nov 2004 18:01:28 +0000 Subject: [PATCH] Recommend that archive_command be coded to not overwrite existing files. Add explicit documentation of the recovery configuration settings. Other minor improvements in the PITR docs. Simon Riggs, some editorialization by Tom Lane. --- doc/src/sgml/backup.sgml | 237 ++++++++++++++++++++++++++++++++------- 1 file changed, 194 insertions(+), 43 deletions(-) diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml index f2c06c8b2a..70c2f6a85b 100644 --- a/doc/src/sgml/backup.sgml +++ b/doc/src/sgml/backup.sgml @@ -1,5 +1,5 @@ Backup and Restore @@ -416,7 +416,7 @@ tar -cf backup.tar /usr/local/pgsql/data Since we can string together an indefinitely long sequence of WAL files for replay, continuous backup can be had simply by continuing to archive the WAL files. This is particularly valuable for large databases, where - making a full backup may take an unreasonable amount of time. + it may not be convenient to take a full backup frequently. @@ -443,7 +443,7 @@ tar -cf backup.tar /usr/local/pgsql/data As with the plain filesystem-backup technique, this method can only support restoration of an entire database cluster, not a subset. - Also, it requires a lot of archival storage: the base backup is bulky, + Also, it requires a lot of archival storage: the base backup may be bulky, and a busy system will generate many megabytes of WAL traffic that have to be archived. Still, it is the preferred backup technique in many situations where high reliability is needed. @@ -503,10 +503,11 @@ tar -cf backup.tar /usr/local/pgsql/data character in the command. The simplest useful command is something like -archive_command = 'cp %p /mnt/server/archivedir/%f' +archive_command = 'cp -i %p /mnt/server/archivedir/%f </dev/null' which will copy archivable WAL segments to the directory - /mnt/server/archivedir. + /mnt/server/archivedir. (This is an example, not a + recommendation, and may not work on all platforms.) @@ -522,18 +523,53 @@ archive_command = 'cp %p /mnt/server/archivedir/%f' It is important that the archive command return zero exit status if and only if it succeeded. Upon getting a zero result, PostgreSQL will assume that the WAL segment file has been - successfully archived, and it may be overwritten with new data very - soon thereafter. However, a nonzero status tells + successfully archived, and will remove or recycle it. + However, a nonzero status tells PostgreSQL that the file was not archived; it will try again periodically until it succeeds. + + The archive command should generally be designed to refuse to overwrite + any pre-existing archive file. This is an important safety feature to + preserve the integrity of your archive in case of administrator error + (such as sending the output of two different servers to the same archive + directory). + It is advisable to test your proposed archive command to ensure that it + indeed does not overwrite an existing file, and that it returns + nonzero status in this case. We have found that cp -i does + this correctly on some platforms but not others. If the chosen command + does not itself handle this case correctly, you should add a command + to test for pre-existence of the archive file. For example, something + like + +archive_command = 'test ! -f .../%f && cp %p .../%f' + + works correctly on most Unix variants. + + + + While designing your archiving setup, consider what will happen if + the archive command fails repeatedly because some aspect requires + operator intervention or the archive runs out of space. For example, this + could occur if you write to tape without an autochanger; when the tape + fills, nothing further can be archived until the tape is swapped. + You should ensure that any error condition or request to a human operator + is reported appropriately so that the situation can be + resolved relatively quickly. The pg_xlog/ directory will + continue to fill with WAL segment files until the situation is resolved. + + Speed of the archiving command is not important, so long as it can keep up - with the average rate at which your server generates WAL data. It is okay - if the archiving process falls a little behind (or even a lot behind, if - you don't mind the pg_xlog/ directory filling up with - not-yet-archived segment files). + with the average rate at which your server generates WAL data. Normal + operation continues even if the archiving process falls a little behind. + If archiving falls significantly behind, this will increase the amount of + data that would be lost in the event of a disaster. It will also mean that + the pg_xlog/ directory will contain large numbers of + not-yet-archived segment files, which could eventually exceed available + disk space. You are advised to monitor the archiving process to ensure that + it is working as you intend. @@ -545,11 +581,11 @@ archive_command = 'cp %p /mnt/server/archivedir/%f' before a WAL segment file is completely filled and ready to archive. One possible way to handle this is to set up a cron job that periodically (once a minute, perhaps) identifies the current WAL - segment file and saves it someplace safe. The combination of the archived - WAL segments and the saved current segment will then be enough to ensure - you can always restore to within a minute of current time. This behavior - is not presently built into PostgreSQL because we did not - want to complicate the definition of the PostgreSQL because + we did not want to complicate the definition of the by requiring it to keep track of successively archived, but different, copies of the same WAL file. The is only invoked on finished @@ -622,7 +658,7 @@ SELECT pg_stop_backup(); Be certain that your backup dump includes all of the files underneath - the database cluster directory (e.g., /usr/local/pgsql/data). + the database cluster directory (e.g., /usr/local/pgsql/data). If you are using tablespaces that do not reside underneath this directory, be careful to include them as well (and be sure that your backup dump archives symbolic links as links, otherwise the restore will mess up @@ -631,10 +667,10 @@ SELECT pg_stop_backup(); You may, however, omit from the backup dump the files within the - pg_xlog/ subdirectory of the cluster directory. This + pg_xlog/ subdirectory of the cluster directory. This slight complication is worthwhile because it reduces the risk of mistakes when restoring. This is easy to arrange if - pg_xlog/ is a symbolic link pointing to someplace outside + pg_xlog/ is a symbolic link pointing to someplace outside the cluster directory, which is a common setup anyway for performance reasons. @@ -706,7 +742,7 @@ SELECT pg_stop_backup(); under the cluster data directory and under the root directories of any tablespaces you are using. (If there are recent, unarchived WAL segment files in - pg_xlog/ that you want to use during restore, move these aside + pg_xlog/ that you want to use during restore, move these aside instead of removing them.) @@ -715,32 +751,32 @@ SELECT pg_stop_backup(); Restore the database files from your backup dump. Be careful that they are restored with the right ownership (the database system user, not root!) and with the right permissions. If you are using tablespaces, - you may want to verify that the symbolic links in pg_tblspc/ + you may want to verify that the symbolic links in pg_tblspc/ were correctly restored. - Remove any files present in pg_xlog/; these came from the + Remove any files present in pg_xlog/; these came from the backup dump and are therefore probably obsolete rather than current. - If you didn't archive pg_xlog/ at all, then re-create it, + If you didn't archive pg_xlog/ at all, then re-create it, and be sure to re-create the subdirectory - pg_xlog/archive_status/ as well. + pg_xlog/archive_status/ as well. If you had unarchived WAL segment files that you saved aside in step 1, - copy them into pg_xlog/. (It's best to copy them, not move + copy them into pg_xlog/. (It's best to copy them, not move them back in, so that you still have the unmodified files if the worst happens and you have to start over.) - Create a recovery command file recovery.conf in the cluster + Create a recovery command file recovery.conf in the cluster data directory, as discussed below. You may also want to temporarily - modify pg_hba.conf to prevent ordinary users from connecting + modify pg_hba.conf to prevent ordinary users from connecting until you are sure the recovery has worked. @@ -749,7 +785,7 @@ SELECT pg_stop_backup(); Start the postmaster. The postmaster will go into recovery mode and proceed to read through the archived WAL files it needs. Upon completion of the recovery process, the postmaster will rename - recovery.conf to recovery.done (to prevent + recovery.conf to recovery.done (to prevent accidentally re-entering recovery mode in case of a crash later) and then commence normal database operations. @@ -758,7 +794,7 @@ SELECT pg_stop_backup(); Inspect the contents of the database to ensure you have recovered to where you want to be. If not, return to step 1. If all is well, - let in your users by restoring pg_hba.conf to normal. + let in your users by restoring pg_hba.conf to normal. @@ -767,10 +803,10 @@ SELECT pg_stop_backup(); The key part of all this is to set up a recovery command file that describes how you want to recover and how far the recovery - should run. You can use recovery.conf.sample (normally - installed in the installation share/ directory) as a + should run. You can use recovery.conf.sample (normally + installed in the installation share/ directory) as a prototype. The one thing that you absolutely must specify in - recovery.conf is the restore_command, + recovery.conf is the restore_command, which tells how to get back archived WAL file segments. Like the archive_command, this is a shell command string. It may contain %f, @@ -783,7 +819,7 @@ SELECT pg_stop_backup(); restore_command = 'cp /mnt/server/archivedir/%f %p' which will copy previously archived WAL segments from the directory - /mnt/server/archivedir. You could of course use something + /mnt/server/archivedir. You could of course use something much more complicated, perhaps even a shell script that requests the operator to mount an appropriate tape. @@ -799,10 +835,10 @@ restore_command = 'cp /mnt/server/archivedir/%f %p' WAL segments that cannot be found in the archive will be sought in - pg_xlog/; this allows use of recent un-archived segments. + pg_xlog/; this allows use of recent un-archived segments. However segments that are available from the archive will be used in - preference to files in pg_xlog/. The system will not - overwrite the existing contents of pg_xlog/ when retrieving + preference to files in pg_xlog/. The system will not + overwrite the existing contents of pg_xlog/ when retrieving archived files. @@ -812,13 +848,11 @@ restore_command = 'cp /mnt/server/archivedir/%f %p' get given the available WAL segments). But if you want to recover to some previous point in time (say, right before the junior DBA dropped your main transaction table), just specify the required stopping point in - recovery.conf. You can specify the stop point either by - date/time or completion of a specific transaction ID. The stop - specification can be inclusive or exclusive. As of this writing + recovery.conf. You can specify the stop point, known as the + recovery target, either by date/time or by completion of a + specific transaction ID. As of this writing only the date/time option is very usable, since there are no tools - to help you identify which transaction ID to use. Keep in mind - that while transaction IDs are asigned sequentially at transaction - start, transactions can complete in a different numeric order. + to help you identify with any accuracy which transaction ID to use. Note that the stop point must be after the ending time of the backup @@ -827,6 +861,123 @@ restore_command = 'cp /mnt/server/archivedir/%f %p' recover to such a time, you must go back to your previous base backup and roll forward from there.) + + + Recovery Settings + + + These settings can only be made in the + recovery.conf file, and apply only for the + duration of the recovery. They must be reset for any subsequent + recovery you wish to perform. They cannot be changed once recovery + has begun. + + + + + + restore_command (string) + + + The shell command to execute to retrieve an archived segment of + the WAL file series. This parameter is required. + Any %f in the string is + replaced by the name of the file to retrieve from the archive, + and any %p is replaced by the absolute path to copy + it to on the server. + Write %% to embed an actual % character + in the command. + + + It is important for the command to return a zero exit status only if + it succeeds. The command will be asked for file names + that are not present in the archive; + it must return nonzero when so asked. + Examples: + +restore_command = 'cp /mnt/server/archivedir/%f "%p"' +restore_command = 'copy /mnt/server/archivedir/%f "%p"' # Windows + + + + + + + recovery_target_time + (timestamp) + + + + This parameter specifies the timestamp up to which recovery + will proceed. + At most one of recovery_target_time and + can be specified. + The default is to recover to the end of the WAL log. + The precise stopping point is also influenced by + . + + + + + + recovery_target_xid (string) + + + This parameter specifies the transaction ID up to which recovery + will proceed. Keep in mind + that while transaction IDs are assigned sequentially at transaction + start, transactions can complete in a different numeric order. + The transactions that will be recovered are those that committed + before (and optionally including) the specified one. + At most one of recovery_target_xid and + can be specified. + The default is to recover to the end of the WAL log. + The precise stopping point is also influenced by + . + + + + + + recovery_target_inclusive + (boolean) + + + + Specifies whether we stop just after the specified recovery target + (true), or just before the recovery target + (false). + Applies to both + and , whichever one is + specified for this recovery. This indicates whether transactions + having exactly the target commit time or ID, respectively, will + be included in the recovery. Default is true. + + + + + + recovery_target_timeline + (string) + + + + Specifies recovering into a particular timeline. The default is + to recover along the same timeline that was current when the + base backup was taken. You would only need to set this parameter + in complex re-recovery situations, where you need to return to + a state that itself was reached after a point-in-time recovery. + See for discussion. + + + + + + + + @@ -892,7 +1043,7 @@ restore_command = 'cp /mnt/server/archivedir/%f %p' that was current when the base backup was taken. If you want to recover into some child timeline (that is, you want to return to some state that was itself generated after a recovery attempt), you need to specify the - target timeline in recovery.conf. You cannot recover into + target timeline in recovery.conf. You cannot recover into timelines that branched off earlier than the base backup. -- GitLab