V2 Deployment

V2Master

Contents

Introduction

This page describes:

The server architecture for deploying publicly accessible MAPS and V2 systems
The software infrastructure that supports MAPS and V2 deployment
How to install publicly accessible beta instances of V2 and MAPS

Server Architecture

Overview

Access to one or more physical servers, accessible via a limited number of public IP addresses
1. Use virtual hosts (DNS aliases) for addressing services and application instances
2. Use a reverse proxy server to dispatch to the appropriate server, virtual machine
The physical servers will collectively support multiple services:
1. Rails-based MAPS and V2 systems
2. PHP-based DrawMGT systems
3. Wiki systems
4. Web servers
5. DNS servers
6. CVS and Git servers
Services hosted in virtual machines:
1. Easy to migrate VMs to alternate physical servers
2. Host OS configuration does not need to be changed based on changing application requirements
3. Scaling to additional physical or could-based machines is possible
Automated deployment of V2 systems:
1. Creation of an instance of a largely pre-configured virtual machine
2. Provisioning of the virtual machine (VM) with all site and instance specific configuration
3. Deployment of the V2 application instance, including the site an instance specific configuration
  1. Initially each instance of a V2 application will reside in its own VM
Automated monitoring:
1. MAPS and V2 instances
2. Virtual machines
3. Physical servers
4. Supporting infrastructure (DNS servers, reverse-proxy servers, etc.)

Supporting Technologies

Server operating system	CentOS 6.4
Virtual Machines	Vagrant, which runs on top of VirtualBox
VM provisioning	Puppet, a Ruby-based system configuration tool
Application deployment	Capistrano, a Ruby Gem for deploying Rails applications
Reverse proxy server	mod_proxy, an Apache module
Monitoring	Nagios and RRDTool

Selected CentOS because:
1. Acceptable to corporate and enterprise clients
2. Supports Vagrant and VirtualBox (FreeBSD is currently unable to host
The purpose of the reverse proxy server is to route incoming web requests to the appropriate server and virtual machine
Selected Apache mod_proxy because:
1. We have experience configuring and using Apache
2. We do not have high performance requirements and therefore don't need to be particularly concerned about choosing the best performing reverse proxy server.
  1. Our highest traffic DrawMGT sites, which have 500-800 users, generate at most 25,000 requests per week, meaning (given a five day week and a 10 hour day) an average only 500 requests per hour. E.g. significantly less than one per second
3. Apache and mod_proxy can be swapped out and replaced with something else, without affecting the rest of the infrastructure

Server Deployment

General Recommendations

Have spare hardware capacity
1. Do not have idle backup servers, but use extra servers in production roles
2. Run multiple identical servers with load split between them
Regularly migrate services to different servers to ensure:
1. It is possible and that there are no hidden problems
2. We can rapidly and reliably restore services in the event of a failure

Hardware

64-bit Intel architecture
Multi-core, fast CPU(s)
Lots of RAM
SSD (non-mirrored) disk for host operating system and the active part of the guest VMs (e.g. the guest OS, Rails, web and DB servers)
Mirror/RAID hard disk for application data and backup staging

Note that Virtualization must be enabled in the underlying PC BIOS in order for Vagrant and VirtualBox VMs to function

Infrastructure Services

Puppet Based Server Configuration

All servers should be managed using Puppet
The configuration information, for our entire infrastructure should be stored in a single Git repository
All configuration changes should be done on the Puppet Server and propagated to the target server

Provisioning of Applications Created by MAPS

Applications sold by MAPS will be provisioned and managed automatically:

DNS entries
Reverse proxy configuration
VM provisioning - Network configuration and hostname
Application deployment - Site and instance configuration
Backup - Configuration and execution
Monitoring

New Servers

The new servers should be implemented in VMs on new servers.

MAPS Server - Master Application and Payment System server, including public web site and sales font-end
Puppet Server - Automated server configuration via Puppet
Reverse Proxy Server - Routing incoming web requests to the back-end VMs and applications responsible for serving them

Existing Servers

The existing servers are currently running on dedicated servers and should be migrated to separate VMs.

DNS Servers - Public DNS servers. Currently residing in Zug (zg-1.softxs.ch) and Budapest (bp-1.softxs.hu)
Mail Server - Incoming and outgoing email and IMAP based email access. Handling mail aliases for DrawMGT customer systems
Git and CVS Servers - Source control systems
Web Server - Static web pages
Blog Server - DrawMGT customer MoinMoin based Wikis and WordPress based blog system
DrawMGT Server - Host for DrawMGT PHP based applications

Virtual Machines

VMs created and managed by Vagrant
Initially, deploy one VM per application instance. E.g. each application instance has its own VM
1. For paid applications, the application's VM will be located on an SSD
2. For free (or low-cost) applications, the application's VM will be located on a normal hard disk
File sharing from the VM to the host system:
1. Puppet configuration files - Not sure if this needed, given a Puppet server
2. Backup staging area - For application documents, database backups and log files
3. Application documents

VM Provisioning

VM boxes, which are templates for creating new VMs, are stored in a directory: /home/vagrant/boxes
1. Standard boxes VM boxes can be downloaded from: http://www.vagrantbox.es
We will deploy a CentOS 6.4 box, with Rails infrastructure, Apache, Passenger and MySQL already installed (but not completely configured)
VM instances, which can be either running, suspended or halted, are stored in a directory: /home/vagrant/VirtualBox VMs
VMs are created on the host system:
1. In one directory per VM: Suggest /home/vm/{name}
2. The VM directory contains Vagrantfile that defines the VMs basic properties:
  1. The base box upon which it is based
  2. Network and hostname configuration
  3. Directories shared with the host OS
  4. Configuration options: memory limits, etc.
  5. Note that Vagrant allows multiple VMs to be defined in a single Vagrant file. We will not use this feature
3. The host's vagrant user is used to access the VM from the command line
Create and save an SSH key for accessing the machines?

Tips

Vagrant uses VirtualBox, by Oracle, to manage VMs
1. Use the VBoxManage command to monitor VMs. Must be run as user vagrant (running displays an empty list!)
  1. VBoxManage list [-l] vms - Lists all VM instances that have been defined
  2. VBoxManage list [-l] runningvms - Lists all VM instances that are running
Always shutdown a VM before attempting to destroy it. Destroying a running VM will usually hang

TODO Issues to document

Network configuration

VM Provisioning

To be completed - describe:
- DNS config (and naming convention)
- Revery proxy config
- Network config
- Rails setup
  - Ruby version 1.9.x
  - Rails and associated Gems
- Apache setup and config
  - Phusion Passenger apache module
- Puppet

Application Deployment

Includes MAPS and V2 deployment
To be completed - describe:
- Application issues
  - Gemfile.lock
- Apache config
- Site/instance Git repository
  - Structure of repository
  - Deployment files
    - database.yml
- Capistrano

Loki DMZ Test System

This section describes the configuration on the Rails deployment environment on the loki.softxs.ch server, located in the DMZ in the AH server room.

Services Provided

Current Services and Servers

All these services should be migrated to VM based servers.

idun.softxs.ch - email and DNS server
1. Incoming and outgoing SMPT mail transfer
2. IMAPS access to email
3. Storage of email
4. DNS server for DMZ, not used publicly
honir.softxs.ch - web server
1. Private web pages
  1. Mackay family tree
  2. Robert L. Mackay Diaries
  3. Innovation web site
  4. Salandra
  5. Sho Takahashi
  6. Alan
  7. Venture internal pages
2. Business web pages
  1. Demo systems
    1. Hydro-2007
    2. ITA-Demo
    3. LTF-Demo
    4. SpecMGT-Demo
3. Web based access to email, via SquirrelMail
4. Wikis, via MoinMoin
  1. SoftXS public web pages
  2. DrawMGT documentation in English and German
  3. Internal Wiki
  4. Customer Wikis
    1. IDP
    2. Cardenillo
    3. MET
    4. NDD
    5. Sisimiut
    6. HCMC2
5. AHYTRA - hydrodynamic modeling software and associated web-based plotting and display
  1. Available via http://lu.softxs.ch/venture/ahytra
ymir.softxs.ch - Source control and blog server
1. CVS repositories
2. Git repositories
  1. Git repository web access, via gitweb
3. Blogs, via WordPress
  1. Salandra - AH private blog. http://blog.salandra.ch
  2. SoftXS - Unused. http://blog.softxs.ch
4. DrawMGT test systems (can be discarded)

VM Based Services and Servers

DHCP Server
DNS Server

VM Server #1

Basic server configuration:

Hostname

loki.softxs.ch

IP Address, external

81.221.23.36

IP Addrees, DMZ

192.168.2.4

Network configuration:

ADSL router configuration

Hardware: Cisco 826

NAT Forwarding

  Pro Inside global         Inside local          Outside local         Outside global
  tcp 81.221.23.36:22       192.168.1.36:22       ---                   ---  # SSH
  tcp 81.221.23.36:25       192.168.1.36:25       ---                   ---  # SMTP
  tcp 81.221.23.36:53       192.168.1.36:53       ---                   ---  # DNS TCP
  udp 81.221.23.36:53       192.168.1.36:53       ---                   ---  # DNS UDP
  tcp 81.221.23.36:80       192.168.1.36:80       ---                   ---  # HTTP
  tcp 81.221.23.36:443      192.168.1.36:443      ---                   ---  # HTTPS
  tcp 81.221.23.36:465      192.168.1.36:465      ---                   ---  # SMTPS
  tcp 81.221.23.36:993      192.168.1.36:993      ---                   ---  # IMAPS

Firewall configuration

Hardware: Soekris net4501
Firewall software: mon0wall

NAT forwarding configuration. Includes

  IF    Proto    Src  Dest IP     Source IP            Dst   Description
  WAN   TCP       22  192.168.2.4 (ext. 192.128.1.36)   22   Incoming SSH to loki
  WAN   TCP/UDP   53  192.168.2.4 (ext. 192.128.1.36)   53   Incoming DNS to loki
  WAN   TCP       80  192.168.2.4 (ext. 192.128.1.36)   80   Incoming HTTP to loki
  WAN   TCP      443  192.168.2.4 (ext. 192.128.1.36)  443   Incoming HTTPS to loki

ZG3 Beta Systems

This sections describes the configuration of the Rails environment on the zg-3.softxs.ch server, located in the Datawire datacenter in Cham.

Environment

The V2 server is a virtual host on the zg-3.softxs.ch server.

The server has following software systems:

FreeBSD 9.0-RELEASE FreeBSD 9.0-RELEASE #0
Apache Server version: Apache/2.2.23 (FreeBSD)
Phusion Passenger apache module
Ruby: ruby 1.9.3p327 (2012-11-10 revision 37606) [amd64-freebsd9]
Rails 3.2.11
mysql Ver 14.14 Distrib 5.5.28, for FreeBSD9.0 (amd64) using 5.2
And many Gems

The installation location for Rails applications:

/v01/local/www/rails

For each application two items are required:

In the rails directory the following is required:
1. A directory with the path {app}-app where the git repository is cloned
2. A symbolic link {app} which points to the {app}-app/public directory
In /usr/local/etc/apache22/httpd.conf there must be a RackBaseUri defined. See below

Example directory structure for the V2pp and MAPS installations:

Note that the V2p0 and MAPS applications are currently configured for the development environment

$ cd /v01/local/www/rails
$ ls -l
lrwxr-xr-x   1 alan  www    16 Jan 24 13:00 maps -> maps-app/public
drwxrwxr-x  13 root  www    21 Jan 24 13:00 maps-app
lrwxr-xr-x   1 root  www    15 Jan 17 16:20 v2p0 -> v2p0-app/public
drwxrwxrwx  15 alan  www    23 Jan 24 17:14 v2p0-app

Apache Configuration

The following is configured in usr/local/etc/apache22/httpd.conf

## ====== Rails ======

LoadModule passenger_module /usr/local/lib/ruby/gems/1.9/gems/passenger-3.0.17/ext/apache2/mod_passenger.so
PassengerRoot /usr/local/lib/ruby/gems/1.9/gems/passenger-3.0.17
PassengerRuby /usr/local/bin/ruby19

<VirtualHost *:80>
  DocumentRoot  /v01/local/www/rails
  ServerName    v2.softxs.ch
  <Directory /v01/local/www/rails>
    Allow from all
  </Directory>

  # -- v2p0 app

  RackBaseURI   /v2p0
  <Directory /v01/local/www/rails/v2p0>
    RailsEnv    development
    Options     -MultiViews
  </Directory>

  # -- maps app

  RackBaseURI   /maps
  <Directory /v01/local/www/rails/maps>
    RailsEnv    development
    Options     -MultiViews
  </Directory>

  # -- test0 app

  RackBaseURI   /test0
  <Directory /v01/local/www/rails/test0>
    RailsEnv    development
    Options     -MultiViews
  </Directory>

  # -- test1 app

  RackBaseURI   /test1
  <Directory /v01/local/www/rails/test1>
    RailsEnv    test
    Options     -MultiViews
  </Directory>
</VirtualHost>

Procedure

Log into zg-3.softxs.ch
1. You must be a member of group www
cd to the {app}-app dircetory
Git pull/fetch. Typically:
- ```
git pull origin master
```
Bundle install
- ```
bundle install
```
  Use 'sudu gem install' as necessary to install any missing Gems. Note that it appears that bundle install detects the presence of sudo and asks for a password for the bundle install if new gems must be installed. If any Gems are installed then you need to restart Apache:
  - ```
  cd /usr/local/etc/rc.d
  sudo ./apache22 restart
```
Then check in /var/log/httpd-error.log to make sure there were no errors when Apache restarted.

Run rake tasks as necessary. The typical list of rake tasks is:

rake db:drop
rake db:create
rake db:migrate
rake db:seed_fu
rake db:populate

If you need to perform any tweaks in the database, use the following user/password to access the DB server:
- ```
mysql -uroot -psqladmin
```
Test the result. Links:
1. MAPS http://v2.softxs.ch/maps
2. V2p0 http://v2.softxs.ch/v2p0

Hostname	loki.softxs.ch
IP Address, external	81.221.23.36
IP Addrees, DMZ	192.168.2.4