Dual System DrawMGT Design and Implementation Notes

Contents

  1. Dual System DrawMGT Design and Implementation Notes
  2. Data Transfer
    1. Theory of Operation
    2. Overview Example Record Transfer
      1. Internal System to Shared System
      2. Shared System to Internal System
  3. Databases and Staging Area
    1. Databases and Configuration
    2. Database Triggers
    3. Staging Database
  4. Main Classes
  5. Other Issues
    1. Uniqueness of Drawing Codes
    2. Uniqueness of Serial Codes
    3. Placeholder Document
  6. Rest Requests
    1. Mirroring Transactions
    2. Authorization of REST transactions
  7. Configuration Issues
    1. Software Packages
  8. Testing Issues
    1. Show Next Record PHP Script

Data Transfer

Theory of Operation

  1. The internal system actively contacts the shared system using a REST request:
    • Polls for updates on shared system, stored in the staging area on the shared system
    • Sends updates to the shared system, stored in the staging area on the internal system
  2. The tables that are transferred are defined as sub-site settings in the sub-site settings file (See below)
    • The list of tables to transfer is different for each system
    • All reference data are transferred from the internal to the shared system
    • Drawings, Comments created on the shared system are transferred to the internal system but NOT from internal to the shared system

    • Table Users handled specially: created and updated records are transferred from the internal to the shared system and password related fields are transferred from the shared system to the internal system to have unified login on the two systems.

  3. Database inserts, updates and deletes cause triggers to:
    • Create a record in the local staging area containing the complete record (serialized) just inserted, updated or inserted.
  4. Placeholder Drawings records are created on the external system for all drawingCode of internal Drawings records
    • Allow unique Drawings.drawingCode values
    • Allow to get next serial numbers
  5. Placeholder Drawings records are created on the external system for all serial code of internal Drawings records
    • Allow unique Drawings.serialCodeTypeId, Drawings.serialCodeNo, Drawings.serialCodePartvalues
    • Allow to get next serial code numbers

Overview Example Record Transfer

Internal System to Shared System

The operation is explained on an example: a Users record insert or (update) on the internal system

  1. The internal system insert (or updates) a Users record in the local database linthal114dev1

  2. An insert (or update) trigger ins_Users (or upd_Users) is activated, which creates a new record in the local staging area in table linthal114stg1.MirrorRecords (see record content below)

  3. The event dispatcher on the internal system, started from crontab every minutes, starts the event handler MirrorOutEH caused by a rescheduling record in table Events.

  4. The event handler MirrorOutEH reads the local staging area (linthal114stg1.MirrorRecords) using Mirroring::getNextRecord() and sends the record serialized to the shared system using the REST function Mirroring::restProcessMirrorRecord. 5 The REST function Mirroring::restProcessMirrorRecord calls Mirroring::processMirrorRecord and inserts (or updates) the appropriate Users record on the shared system. Note, that mirroring is disabled by using the global SQL variable @mirroringDisabled during this operation to avoid mirroring back to the internal system.

  5. The event handler MirrorOutEH calls the function Mirroring::setProcessed to update the field 'processStatus" in the linthal114stg1.MirrorRecords record to processed (2) or error (-1).

Shared System to Internal System

The operation is explained on an example: a Comments record insert or (update) on the shared system

  1. Shared system inserts (or updates) a Comments record in the local database linthal114dev2

  2. An insert (or update) trigger ins_Comments (or upd_Comments) is activated, which creates a new record in the local staging area in table linthal114stg2.MirrorRecords (see record content below)

  3. The event dispatcher on the internal system, started from crontab every minutes, starts the event handler MirrorInEH caused by a rescheduling record in table Events.

  4. The event handler MirrorInEH reads the next serialized record from the remote staging area (linthal114stg2.MirrorRecords) calling the REST function Mirroring::restGetNextRecord() and calls the function Mirroring::processMirrorRecord. 5 The function Mirroring::processMirrorRecord inserts (or updates) the appropriate Comments record on the internal system. Note, that mirroring is disabled by using the global SQL variable @mirroringDisabled during this operation to avoid mirroring back to the shared system.

  5. The event handler MirrorOutEH calls the REST function Mirroring::restSetProcessed to update the field 'processStatus" in the linthal114stg2.MirrorRecords record to processed (2) or error (-1).

Databases and Staging Area

Databases and Configuration

Each system has two databases:

The list of tables that are transferred are defined in the SiteSettings:

The mirrorig feature is switched on in InstanceSettings on both systems:

The sub-site settings are set in InstanceSettings on both systems, to 1 on the internal:

and to 2 on the shared system:

The MirrorIn and MirrorOut events must be enabled, and the url to the shared system must be defined in the InstanceSettings of the internal system.

Database Triggers

Staging Database

The staging area has a single table MirrorRecords: 

+--------------------+---------------------+------+-----+---------+----------------+
| Field              | Type                | Null | Key | Default | Extra          |
+--------------------+---------------------+------+-----+---------+----------------+
| id                 | bigint(20) unsigned | NO   | PRI | NULL    | auto_increment |
| createTimestamp    | datetime            | NO   |     | NULL    |                |
| processStatus      | int(11)             | NO   | MUL | NULL    |                |
| statusTimestamp    | datetime            | NO   |     | NULL    |                |
| tableName          | varchar(64)         | NO   |     | NULL    |                |
| operation          | char(1)             | NO   |     | NULL    |                |
| keyWhere           | varchar(255)        | NO   |     | NULL    |                |
| phpSerializeRecord | text                | YES  |     | NULL    |                |
+--------------------+---------------------+------+-----+---------+----------------+

Main Classes

Set the debug flag in these classes to see what is happening:

Other Issues

Uniqueness of Drawing Codes

It should be checked, that the value of Drawings.drawingCode is system-wide unique on new or updated Drawings record.

Uniqueness of Serial Codes

It should be checked, that the value set of Drawings.serialCodeTypeId, Drawings.serialCodeNo, Drawings.serialCodePartvalues is system-wide unique on new or updated Drawings record.

Placeholder Document

The placeholder documents, mentioned above have only the following fields set:

Rest Requests

Mirroring Transactions

The following is a list of the REST URLs and what they are used for:

Authorization of REST transactions

Configuration Issues

See Setting Up a Dual DrawMGT System.

Software Packages

The following packages are required (in addition to normal DrawMGT)?

Testing Issues

Show Next Record PHP Script

For displaying the next record to be processed. Calls the getnext REST transaction. Can be started on any system, only the url must be correct, but the main goal is to simulate the normal operation, so start it on the internal system with the url to the shared system.

Sets the processStatus field in the staging database to read, but this doesn' matter (causes a warning only, to state, that the record was already read but the processing was not completed). If you see the associated array of the originally inserted, updated database record 

<?php
  $r = new HttpRequest('http://narvi.softxs.ch/dev/linthal114dev2/mirror/getnext', HttpRequest::METH_GET);
  // $r->addQueryData(array('category' => 3));
  try {
    $r->send();
    if ($r->getResponseCode() == 200) {
      $content = $r->getResponseBody();
      print "content:$content\n";
      $record = unserialize($content);
      var_export( $record );
      $dbrecord = unserialize($record['phpSerializeRecord']);
      var_export( $dbrecord );
    } else {
      print( "response code: " . $r->getResponseCode() . "\n" );
    }
  } catch (HttpException $ex) {
    echo $ex;
  }
?>

DualSystemDesignNotes (last edited 2011-03-18 10:55:15 by 195)

Copyright 2008-2014, SoftXS GmbH, Switzerland