Conference orarep::nomahs::dec_data_distributor

Ann,

You no doubt read about the transfer method REPLICATION WITH NO DELETE, and I
think the terminology here must have confused you.  Such a transfer we informal-
ly call an archive (or history, if you like) transfer.  Updates made to the
source (production) database are replicated to the target (history) database.
Inserts result in inserts at the target.  Updates result in inserts, updates or
deletes in the target, and deletes result in deletes in the target.  However,
deletes are treated is a special way.  The rows are not actually deleted from
the history database.  They are marked as having been deleted from the source.
Therefore, rows which have been deleted from the production database remain in
the history database.

Does the preceding paragraph clear things up for you?  I am confused by what
you expect when you say that normally you want the transfer to run 'with
delete', but not after the weekly delete procedure.  It sounds as if you are
saying that you want some deletes to really delete rows from the history data-
base and other deletes to not do so.  I doubt that is what you intended to say.

You ask how one deals with a transfer that fails or a reinitialization.  I'll
discuss different types of failure.

Let's say that an update transfer is underway and that the network goes down.
The updates are applied in multiple transactions, one transaction per original
source transaction (at least that's true up through ROR 7.0-0).  When the
system comes back up and the update transfer is rerun, it picks up from where
it left off, repeating only the partially completed transaction and continuing
from there.  Failure recovery can be built into the transfer schedule.

Infrequently, data corruption errors occur have been reported in the source
(production) RDB$CHANGES table.  Such errors force you to restart the transfer
from the beginning.  That means you must REINITIALIZE the transfer and then
rerun it.  When you do so, ROR erases all live data from the target tables.
Live data is data not marked as having been deleted.  Then a fresh snapshot of
the source data is copied to the target after which the incremental update
cycle resumes as normal.

The preceding has one drawback.  Any delete operations which were recorded in
the RDB$CHANGES table but which were not processed because they came after the
point of RDB$CHANGES corruption are not reflected in the target database.  This
problem is on our list of things to correct, but realistically a correction is
likely to be a long way off.

Claude

Hallo Claude,


>I am confused by what you expect when you say that normally you want the 
>transfer to run 'with delete', but not after the weekly delete procedure.  
>It sounds as if you are saying that you want some deletes to really delete 
>rows from the history database and other deletes to not do so.  I doubt 
>that is what you intended to say.

Yes, I can understand you were confused! So was I and you didn't 
misunderstand me! I was hoping there was a workaround with dummy transfers, 
purging/deleting rows in rdb$changes or whatever to what I wanted to achieve! 
But instead, we are looking into the possibility of doing both a logical 
and physical deletion of the rows that are deleted because an order was 
cancelled etc. The other rows deleted in the source database because of age, 
will not have this flag/field set. So with a replication transfer with 
NO DELETE, we'll end up with all rows in the target (history) database, but
can use this field/flag as a selection criteria when finding processed orders.


>Infrequently, data corruption errors occur have been reported in the source
>(production) RDB$CHANGES table.  Such errors force you to restart the transfer
>from the beginning.  That means you must REINITIALIZE the transfer and then
>rerun it.  When you do so, ROR erases all live data from the target tables.
>Live data is data not marked as having been deleted.  Then a fresh snapshot of
>the source data is copied to the target after which the incremental update
>cycle resumes as normal.

Is it correct that it should be a replication transfer with the TO EXISTING 
clause? I assume an existing database can be a multi-file database?


>Any delete operations which were recorded in
>the RDB$CHANGES table but which were not processed because they came after the
>point of RDB$CHANGES corruption are not reflected in the target database.  This
>problem is on our list of things to correct, but realistically a correction is
>likely to be a long way off.

So there's nothing at all that can be done to update the target database, or?


One last thing! Could you expand a little on what will happen in the target
database when altering or adding table definitions in the source database, 
using a replication transfer with the NO DELETE and TO EXISTING clauses.

Thanks a lot for your help!

/Ann

>Is it correct that it should be a replication transfer with the TO EXISTING 
>clause? I assume an existing database can be a multi-file database?

Correct.

>So there's nothing at all that can be done to update the target database, or?

You were asking about the problem with lost deletes in the case when a transfer
is forced to be reinitialized.  There is no automated way of figuring out what
rows were deleted.  I can think of a way that will help and that will likely
solve the problem in most cases, but it is still not 100% foolproof.  We can
discuss it if you'ld like.

>One last thing! Could you expand a little on what will happen in the target
>database when altering or adding table definitions in the source database, 
>using a replication transfer with the NO DELETE and TO EXISTING clauses.

If you add a table to the source database, that tale will not be replicated to
the target database.  That is, once you define a replicaiton transfer, the
list of tables the transfer will handle does not change.  To change it you
have to drop the transfer definition and recreate it including the new table.
Apart from that being an operational headache, that will work.

You are not permitted to alter a table definition in the source database while
there is a replication transfer defined on that table.  Rdb won't let you do
it.  You'ld have to drop the transfer definition, make your alteration, re-
create the transfer definition, and reexecute the transfer.  That will result
in the target table definition being changed.  Rows already in the target table
will not be modified.

Claude

    Hi,
    
    Thanks for all your help. I will get back to you if we get any more
    questions if and when we install ROR!
    
    Regards,
    Ann