Re: PITR, checkpoint, and local relations

> I really dislike the notion of turning off checkpointing. What if the
> backup process dies or gets stuck (eg, it's waiting for some
> operator to
> change a tape, but the operator has gone to lunch)? IMHO, backup
> systems that depend on breaking the system's normal
> operational behavior
> are broken. It should be sufficient to force a checkpoint when you
> start and when you're done --- altering normal operation in between is
> a bad design.

But you have to prevent log files reusing while you copy data files.
That's why I asked are 3 commands from pg_copy required and couldn't
be backup accomplished by issuing single command

ALTER SYSTEM BACKUP <dir | stdout (to copy data to client side)>

(even from pgsql) so backup process would die with entire system -:)
As for tape changing, maybe we could use some timeout and then just
stop backup process.

Vadim

"Mikheev, Vadim" <vmikheev(at)SECTORBASE(dot)COM> writes:
>> It should be sufficient to force a checkpoint when you
>> start and when you're done --- altering normal operation in between is
>> a bad design.

> But you have to prevent log files reusing while you copy data files.

No, I don't think so. If you are using PITR then you presumably have
some process responsible for archiving off log files on a continuous
basis. The backup process should leave that normal operational behavior
in place, not muck with it.

regards, tom lane

Tom Lane wrote:
> "Mikheev, Vadim" <vmikheev(at)SECTORBASE(dot)COM> writes:
> >> It should be sufficient to force a checkpoint when you
> >> start and when you're done --- altering normal operation in between is
> >> a bad design.
>
> > But you have to prevent log files reusing while you copy data files.
>
> No, I don't think so. If you are using PITR then you presumably have
> some process responsible for archiving off log files on a continuous
> basis. The backup process should leave that normal operational behavior
> in place, not muck with it.

But what if you normally continuous LOG to tape, and now you want to
backup to tape. You can't use the same tape drive for both operations.
Is that typical? I know sites that had only one tape drive that did
that.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

On Fri, Aug 02, 2002 at 08:52:27PM -0400, Bruce Momjian wrote:

> But what if you normally continuous LOG to tape, and now you want to
> backup to tape. You can't use the same tape drive for both operations.
> Is that typical? I know sites that had only one tape drive that did
> that.

I have seen such installations. They always seemed like a real false
economy to me. Tape drives are not so expensive that, if you really
need to ensure your data is well and truly safe, you can't afford two
of them. But that's just my 2 cents. (Or, I guess in this case, 4
cents.)

A

--
----
Andrew Sullivan 87 Mowat Avenue
Liberty RMS Toronto, Ontario Canada
<andrew(at)libertyrms(dot)info> M6K 3E3
+1 416 646 3304 x110

When I've seen this done, I've seen DLT's used as they allow for
multiple channels to be streamed to tape at the same time. If your tape
device does not allow for multiple, concurrent input streams, you're
going to have to obtain multiple drives.

Please keep in mind, my DLT experience is limited.

Greg

On Tue, 2002-08-06 at 13:35, Andrew Sullivan wrote:
> On Fri, Aug 02, 2002 at 08:52:27PM -0400, Bruce Momjian wrote:
>
> > But what if you normally continuous LOG to tape, and now you want to
> > backup to tape. You can't use the same tape drive for both operations.
> > Is that typical? I know sites that had only one tape drive that did
> > that.
>
> I have seen such installations. They always seemed like a real false
> economy to me. Tape drives are not so expensive that, if you really
> need to ensure your data is well and truly safe, you can't afford two
> of them. But that's just my 2 cents. (Or, I guess in this case, 4
> cents.)
>
> A
>
> --
> ----
> Andrew Sullivan 87 Mowat Avenue
> Liberty RMS Toronto, Ontario Canada
> <andrew(at)libertyrms(dot)info> M6K 3E3
> +1 416 646 3304 x110
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://archives.postgresql.org

> -----Original Message-----
> From: Bruce Momjian [mailto:pgman(at)candle(dot)pha(dot)pa(dot)us]
> Sent: Friday, August 02, 2002 8:52 PM
> To: Tom Lane
> Cc: Mikheev, Vadim; richt(at)multera(dot)com; J. R. Nield; PostgreSQL Hacker
> Subject: Re: [HACKERS] PITR, checkpoint, and local relations
>
>
> Tom Lane wrote:
> > "Mikheev, Vadim" <vmikheev(at)SECTORBASE(dot)COM> writes:
> > >> It should be sufficient to force a checkpoint when you
> > >> start and when you're done --- altering normal operation in
> between is
> > >> a bad design.
> >
> > > But you have to prevent log files reusing while you copy data files.
> >
> > No, I don't think so. If you are using PITR then you presumably have
> > some process responsible for archiving off log files on a continuous
> > basis. The backup process should leave that normal operational behavior
> > in place, not muck with it.
>
> But what if you normally continuous LOG to tape, and now you want to
> backup to tape. You can't use the same tape drive for both operations.
> Is that typical? I know sites that had only one tape drive that did
> that.
Our implementation of pg_copy did not archive to tape. This adds a lot of
complications so I thought just make a disk to disk copy and then the disk
copy could be archived to table at the users discretion.
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
> + If your life is a hard drive, | 830 Blythe Avenue
> + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
>

> -----Original Message-----
> From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us]
> Sent: Friday, August 02, 2002 8:06 PM
> To: Mikheev, Vadim
> Cc: richt(at)multera(dot)com; J. R. Nield; Bruce Momjian; PostgreSQL Hacker
> Subject: Re: [HACKERS] PITR, checkpoint, and local relations
>
>
> "Mikheev, Vadim" <vmikheev(at)SECTORBASE(dot)COM> writes:
> >> It should be sufficient to force a checkpoint when you
> >> start and when you're done --- altering normal operation in between is
> >> a bad design.
>
> > But you have to prevent log files reusing while you copy data files.
>
> No, I don't think so. If you are using PITR then you presumably have
> some process responsible for archiving off log files on a continuous
> basis. The backup process should leave that normal operational behavior
> in place, not muck with it.
You want the log files necessary for recovering the database to be in the
backup copy -- don't you?
>
> regards, tom lane
>

Richard Tucker <richt(at)multera(dot)com> writes:
> But you have to prevent log files reusing while you copy data files.

>> No, I don't think so. If you are using PITR then you presumably have
>> some process responsible for archiving off log files on a continuous
>> basis. The backup process should leave that normal operational behavior
>> in place, not muck with it.

> You want the log files necessary for recovering the database to be in the
> backup copy -- don't you?

Why? As far as I can see, this entire feature only makes sense in the
context where you are continuously archiving log files to someplace
(let's say tape, for purposes of discussion). Every so often you make a
backup, and what that does is it lets you recycle the log-archive tapes
older than the start of the backup. You still need the log segments
newer than the start of the backup, and you might as well just keep the
tapes that they're going to be on anyway. Doing it the way you propose
(ie, causing a persistent change in the behavior of the log archiving
process) simply makes the whole operation more complex and more fragile,
without any actual gain in functionality that I can detect.

regards, tom lane