Managing V2 Backups and Application Data

V2Master | V2HighLevelDesign

Introduction

This sections describes:

  1. How application data (e.g. all application data and state) for individual V2 applications are managed
  2. How backups for individual V2 applications are managed
  3. How backups for VMs are managed

Definitions

  1. Application - An application that manages its state in a database and a set of data files

  2. VM - A virtual machine, in which supporting services and application are run

    1. One or more Applications run on a VM, limited by the capacity of the VM

  3. Host System - A hardware/software platform supporting a set of services and application VMs

    1. One or more VMs run on a Host System, limited by the capacity of the Host System

  4. Service - A host or network service that supports Applications and their underlying infrastructure (see Services)

The entire infrastructure can be scaled by adding additional VMs and Host Systems

Services

Each Host System must have:

  1. DNS server in a VM that resolves the internal names and IP address of the VMs

  2. forwarding-only DNS server that forward DNS requests to the DNS server VM

  3. rinetd daemon that forwards incoming TCP traffic to the appropriate VM based on the port number. Typically

    1. All web traffic (ports 80 and 443) is forwarded to the Reverse Proxy server

    2. Note that DNS traffic, which uses both TCP and UDP protocols, is managed by the Host System's forwarding-only DNS server
    3. If the Host System has a VM with a Puppet server then port 8140 is forwarded to it

  4. Reverse Proxy, an Apache module in a web server VM, that forward web requests to the appropriate VM based on the hostname in the HTTP request

Additional services that might be found on a Host System:

  1. Outward facing DNS
  2. Puppet master server

Requirements

Application Data Management Requirements

  1. To ensure good performance given the host environment, consisting of:
    1. A limited amount of RAM
    2. A limited amount of high-speed SSD-based disk space
    3. A large amount of lower-speed RAID1 hard-disk space

Recommendation:

  1. Store the base operating system and application support service (web-server, database server, etc.) on SSD
  2. Store the minimum application footprint, e.g. the actively accessed part of the application (code, configuration, database) on SSD
  3. Store the applications revision and attachment files on the RAID hard disk

  4. Issue where to store thumbnail images?

Backup Requirements

  1. To guarantee that the data required to completely restore or duplicate an instance of a application is always available. At a minimum the data to be backed up includes:
    1. The application's files
    2. The application's database
  2. Backups should be made automatically on a nightly basis
  3. The backup data should be stored:
    1. On one or more backup servers
    2. On one or more off-site storage devices (typically USB drives)
  4. There should not a lot of effort required to re-configure backups if an application is moved from one VM or Host Machine to another

Application Data Management

Application Directory structure

Backup Directory Structures

Application Backup Directories

VM Backup Directories

Backup Server Backup Directories

Summary of Backup System

Copyright 2008-2014, SoftXS GmbH, Switzerland