IDS Forum

Re: Logical Log files MISSING

Posted By: Martin Fuerderer
Date: Tuesday, 14 October 2008, at 10:22 a.m.

In Response To: Re: Logical Log files MISSING (VIKAS HIVARKAR)

Hi,

a) a file with a name like
"ONTL_proddb_1_Mon_Oct_13_22_46_09_2008"
is a temporary file that ontape creates while
doing backup. ( Such a file will be renamed to the
proper backup file name once the backup has
finished successfully. This is to avoid having a
seemingly correct backup file lying around from
an unsuccessful (and aborted) backup attempt ...)

b) with your command "ls -ltr | tail -1 | ... " you get
only the very last file of the listing in the directory.
On a busy system one execution of "ontape -a -d"
may backup more than one log file, especially
when the next log file gets full while the last one is
backed up. At the end of a log backup ontape
checks if there are (meanwhile) more logs to
backup. If so it just continues to backup logs.
Finally (when ontape caught up or log filling
slowed down), there will no full log to be backed
up when ontape finished the last one, and then
it will exit.
Now assume "ontape -a -d" backed up 5 log files
in a row, and when it exits you get only the last
of them with your "ls -ltr | tail -1 | ...", then only the
last one is copied to the remote machine and the
other 4 log files are missing (on the remote machine).

c) If your alarm script just finished the "ontape -a -d"
and in the same (split) second a new log fills and
the alarm program starts a new log backup, then
this new "ontape -a -d" creates its temporary file
(with a name like the above in a)). In this moment
the "old log backup" is now doing its "ls -ltr | tail -1 | ..."
and then the newest file will not be the one it just
backed up, but it will be the temp file of the next
"ontape -a -d" that has just started. Then your script
copies that temp file to the remote machine (instead
of the last log file that it has backed up itself).
This is why you get some of these temp files on
the remote machine (and miss even more proper
backup files on the remote machine).

So, the scripting of copying log files to the remote
machine is not that simple ... You need to put "a bit"
more thought into it ... ;)

But as long as you see all the log backup files on the
production machine, everything is OK. There's nothing
"messy" with the log backup itself. :)

Regards,
Martin
--
Martin Fuerderer
IBM Informix Development Munich, Germany
Information Management

IBM Deutschland Research & Development GmbH
Chairman of the Supervisory Board: Martin Jetter
Board of Management: Erich Baier
Corporate Seat: Boeblingen, Germany
Reg.-Gericht: Amtsgericht Stuttgart, HRB 243294

ids-bounces@iiug.org wrote on 14.10.2008 13:42:27:

> Hi,
>
> I've started capturing the logs for "ontape -a -d" from alarmprogram.sh
> $BACKUP_CMD 1>>$LOG 2>&1
>
> Fortunately since last level 0 backup we have not missed any logical log

> archive file and that's just my manual obervation also no any message
yet as
> mentioned by Martin in the $LOG file.
> I want to know that why a alarm is not raised when the logical log file
> archive is missed, we have set the ALARMADMIN=3
>
> Another Issue
> I've started the logical log archive file copy using scp from my prod
server
> to a remote host as DR ( i know this is not the way to do DR but...)
> for this i've added the scp command in the alarmprogram.sh script just
after
>
> if ( `test x${BACKUPLOGS} = xY` ) then
> # $BACKUP_CMD 2>&1 >> /dev/null
>
> $BACKUP_CMD 1>>$LOG 2>&1
>
> EXIT_STATUS=$?
>
> printf " '$BACKUP_CMD' has been executed and returned CODE=%s\n\n"
> $EXIT_STATUS >> $MAILBODY
>
> else
>
> echo "Script will not backup the logical logs."
>
> LOGFILE=`ls -ltr $LOGPATH |tail -1 | awk -F" " '{print $9}'`
>
> scp -q $LOGPATH/$LOGFILE $RHOST:$LOGPATH >>$LOG 2>>$ERRLOG &
>
> fi
>
> I tested the above logic on a less heavy transaction server for 7 days
and
> found that all the logs are copying with out any issue or missing file (

> ofcourse i dont know why nothing is written to the $LOG file ).
>
> I've implemented this on the prod server but now when i see this forthe
prod
> server, there are many log files missing on the remote server ( firstly
i
> thought there is an issue with scp but now as files are missing on
> prod server
> It is realy hard to figure out what ALL is wrong )
>
> I see some logical log files on the Remote host server ( logical
logfiles scp
> from prod server)with name such as
>
> -rw-r----- 1 informix informix 20578304 Oct 13 22:46
proddb_1_Log0000021398
> -rw-r----- 1 informix informix 20578304 Oct 13 22:46
proddb_1_Log0000021396
> -rw-r----- 1 informix informix 12713984 Oct 13 22:46
> ONTL_proddb_1_Mon_Oct_13_22_46_09_2008
> -rw-r----- 1 informix informix 20578304 Oct 13 22:46
proddb_1_Log0000021394
> -rw-r----- 1 informix informix 18481152 Oct 13 22:46
> ONTL_proddb_1_Mon_Oct_13_22_45_53_2008
> -rw-r----- 1 informix informix 20578304 Oct 13 22:46
proddb_1_Log0000021392
> -rw-r----- 1 informix informix 20578304 Oct 13 22:45
proddb_1_Log0000021390
> -rw-r----- 1 informix informix 20578304 Oct 13 22:45
proddb_1_Log0000021389
>
> What are these ONTL_proddb_1_Mon_Oct_13_22_46_09_2008 files?
> All the logs from 21389 to 21398 are available on the prod server.
>
> Have i configured the scp with alarmprogarm.sh in a bad manner or isthis
some
> thing very messy..
>
> Regards,
> Vikas
>
> ----------------------------------
> ------------
>
> Hi,
>
> most likely (> 98%), the problem is what Davis described
> in his answer (see below). If you want to be 100% sure,
> then you have to modify your alarm program script (or
> whatever script/program actually executes the ontape
> command) to capture the output of the ontape processes
> in a file. If you see messages indicating that some
> temporary file cannot be renamed and/or removed
> because it does not exist (errno 2), then this is the
> described problem.
>
> Besides taking a level-0 archive when the situation of
> log file backup loss occured, you may consider the
> following to improve the situation:
>
> a) in your script/program add some code to first
> check whether an "ontape -a -d" command is already
> being executed (e.g. using "ps ..." output). If so, then
> do not start a new "ontape -a -d". The one that is
> running will backup all the used log files anyway
> (except the current log file).
> Note: this is not 100% error proof as there still is a
>
> gap between checking for a running "ontape -a -d"
>
> process and starting a new one.
>
> b) Capture all ontape output (stdout and stderr) to a
> file and put in your alarm program some additional
> code to check this file for above dscribed errors.
> If such an error is detected, write a warning (to online.log
> or whatever) because now surely it is time to do a
> level-0 archive ...
> Note: this may also not bee 100% error proof. An error
>
> may still slip your attention. It highly depends on the
>
> implementation of the error checking/scanning ...
>
> c) The respective defect is: idsdb00165611 (for IDS 11.10).
> While in the upcoming 11.50.xC3 this is fixed already,
> you may contact IBM Informix Tech Support (check
> your contract) to get a fix of this problem for
> your IDS 11.10 ...
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>

Messages In This Thread

• Logical Log files MISSING
  VIKAS HIVARKAR -- Friday, 10 October 2008, at 1:01 p.m.
  • RE: Logical Log files MISSING
    Clifton Bean -- Friday, 10 October 2008, at 1:17 p.m.
    • Re: RE: Logical Log files MISSING
      VIKAS HIVARKAR -- Friday, 10 October 2008, at 1:33 p.m.
      • Re: RE: Logical Log files MISSING
        VIKAS HIVARKAR -- Friday, 10 October 2008, at 1:48 p.m.
        • Re: RE: Logical Log files MISSING
          MIKE MAGIE -- Friday, 10 October 2008, at 2:45 p.m.
          • Re: RE: Logical Log files MISSING
            MIKE MAGIE -- Friday, 10 October 2008, at 2:54 p.m.
  • Re: Logical Log files MISSING
    Davis Kwong -- Friday, 10 October 2008, at 3:55 p.m.
    • Re: Logical Log files MISSING
      Martin Fuerderer -- Monday, 13 October 2008, at 5:05 a.m.
      • Re: Logical Log files MISSING
        VIKAS HIVARKAR -- Tuesday, 14 October 2008, at 7:42 a.m.
        • Re: Logical Log files MISSING
          Martin Fuerderer -- Tuesday, 14 October 2008, at 10:22 a.m.
        • Re2: Logical Log files MISSING
          Martin Fuerderer -- Tuesday, 14 October 2008, at 10:40 a.m.
      • Re: Logical Log files MISSING
        VIKAS HIVARKAR -- Wednesday, 15 October 2008, at 1:56 a.m.
        • Re: Logical Log files MISSING
          Martin Fuerderer -- Thursday, 16 October 2008, at 9:28 a.m.
  • Re: Logical Log files MISSING
    Art Kagel -- Friday, 10 October 2008, at 5:14 p.m.
    • Re: Logical Log files MISSING
      ROBERT CLOW -- Friday, 10 October 2008, at 8:10 p.m.

IDS Forum is maintained by Administrator with WebBBS 5.12.