Dual System DeawMGT Design and Implementation Notes

Contents

  1. Dual System DeawMGT Design and Implementation Notes
  2. Data Transfer
    1. Theory of Operation
    2. Overview Example Record Transfer
      1. Internal System to Shared System
      2. Shared System to Internal System
  3. Databases and Staging Area
    1. Databases and Configuration
    2. Database Triggers
    3. Staging Database
  4. Main Classes
  5. Rest Requests
    1. Mirroring Transactions
    2. Authorization of REST transactions
  6. Configuration Issues
    1. Software Packaged
  7. Testing Issues
    1. Show Next Record PHP Script

Data Transfer

Theory of Operation

  1. The internal system actively contacts the shared system using a REST request:
    • Polls for updates on shared system, stored in the staging area on the shared system
    • Sends updates to the shared system, stored in the staging area on the internal system
  2. The tables that are transferred are defined as sub-site settings in the sub-site settings file (See below)
    • The list of tables to transfer is different for each system
  3. Database inserts, updates and deletes cause triggers to:
    • Create a record in the local staging area containing the complete record (serialized) just inserted, updated or inserted.
  4. Anything else?

Overview Example Record Transfer

Internal System to Shared System

  1. The internal system insert (or updates) a User record in the local database linthal114dev1

  2. An insert (or update) trigger ins_User (or upd_Users) is activated, which creates a new record in the local staging area in table linthal114stg1.MirrorRecords (see record content below)

  3. The event dispatcher on the internal system, started from crontab every minutes, starts the event handler MirrorOutEH caused by a rescheduling record in table Events.

  4. The event handler MirrorOutEH reads the local staging area (linthal114stg1.MirrorRecords) using Mirroring::getNextRecord() and sends the record serialized to the shared system using the REST function Mirroring::restProcessMirrorRecord. 5 The REST function Mirroring::restProcessMirrorRecord calls Mirroring::processMirrorRecord and inserts (or updates) the appropriate Users record on the shared system.

  5. The event handler MirrorOutEH calls the function Mirroring::setProcessed to update the field 'processStatus" in the linthal114stg1.MirrorRecords record to processed (2) or error (-1).

Shared System to Internal System

  1. Shared system insert (or updates) a Comments record in the local database linthal114dev2

  2. An insert (or update) trigger ins_Comments (or upd_Comments) is activated, which creates a new record in the local staging area in table linthal114stg2.MirrorRecords (see record content below)

  3. The event dispatcher on the internal system, started from crontab every minutes, starts the event handler MirrorInEH caused by a rescheduling record in table Events.

  4. The event handler MirrorInEH reads the next serialized record from the remote staging area (linthal114stg2.MirrorRecords) calling the REST function Mirroring::restGetNextRecord() and calls the function Mirroring::processMirrorRecord. 5 The function Mirroring::processMirrorRecord inserts (or updates) the appropriate Comments record on the internal system.

  5. The event handler MirrorOutEH calls the REST function Mirroring::restSetProcessed to update the field 'processStatus" in the linthal114stg2.MirrorRecords record to processed (2) or error (-1).

Databases and Staging Area

Databases and Configuration

Each system has two databases:

The list of tables that are transferred are defined in the SiteSettings:

The mirrorig feature is switched on in InstanceSettings on both systems:

The sub-site settings are set in InstanceSettings on both systems, to 1 on the internal:

and to 2 on the shared system:

The MirrorIn and MirrorOut events must be enabled, and the url to the shared system must be defined in the InstanceSettings of the internal system.

Database Triggers

Staging Database

The staging area has a single table MirrorRecords: 

+--------------------+---------------------+------+-----+---------+----------------+
| Field              | Type                | Null | Key | Default | Extra          |
+--------------------+---------------------+------+-----+---------+----------------+
| id                 | bigint(20) unsigned | NO   | PRI | NULL    | auto_increment |
| createTimestamp    | datetime            | NO   |     | NULL    |                |
| processStatus      | int(11)             | NO   | MUL | NULL    |                |
| statusTimestamp    | datetime            | NO   |     | NULL    |                |
| tableName          | varchar(64)         | NO   |     | NULL    |                |
| operation          | char(1)             | NO   |     | NULL    |                |
| keyWhere           | varchar(255)        | NO   |     | NULL    |                |
| phpSerializeRecord | text                | YES  |     | NULL    |                |
+--------------------+---------------------+------+-----+---------+----------------+

Main Classes

Set the debug flag in these classes to see what is happening:

Rest Requests

Mirroring Transactions

The following is a list of the REST URLs and what they are used for:

Authorization of REST transactions

Configuration Issues

Software Packaged

The following packages are required (in addition to normal DrawMGT)?

Testing Issues

Show Next Record PHP Script

For displaying the next record to be processed. Calls the getnext REST transaction. Can be started on any system, only the url must be correct, but the main goal is to simulate the normal operation, so start it on the internal system with the url to the shared system.

Sets the processStatus field in the staging database to read, but this doesn' matter (causes a warning only, to state, that the record was already read but the processing was not completed). If you see the associated array of the originally inserted, updated database record 

<?php
  $r = new HttpRequest('http://narvi.softxs.ch/dev/linthal114dev2/mirror/getnext', HttpRequest::METH_GET);
  // $r->addQueryData(array('category' => 3));
  try {
    $r->send();
    if ($r->getResponseCode() == 200) {
      $content = $r->getResponseBody();
      print "content:$content\n";
      $record = unserialize($content);
      var_export( $record );
      $dbrecord = unserialize($record['phpSerializeRecord']);
      var_export( $dbrecord );
    } else {
      print( "response code: " . $r->getResponseCode() . "\n" );
    }
  } catch (HttpException $ex) {
    echo $ex;
  }
?>
Copyright 2008-2014, SoftXS GmbH, Switzerland