= V2 Deployment Checklist = [[V2Master]] | [[V2Deployment]] <> = Introduction = = DNS Name Configuration = 1. DNS configuration is performed on the Puppet server mgt.vh01.softxs.ch {{{ ssh -p 10104 root@loki cd /etc/puppet }}} 1. Configure new DNS names. As ''root'' on the Puppet server {{{ cd /etc/puppet/modules/bind9/files/master # -- Edit the following two files: vi zones/all/{domain_name}. # Update the 'Serial' with today's date and the sequence number (normally '01') :x vi generic/all/{domain_name}. # Add CNAME record for the sub-domain pointing to the host server :x }}} 1. Test DNS Names. Make sure the new DNS name is defined on the all following DNS servers: 1. '''zg-1.softxs.ch''' - Public server 1. '''zg-3.softxs.ch''' - Public server 1. '''loki.softxs.ch''' - LU DMZ 1. '''odin.softxs.ch''' - LU DMZ 1. '''modi.softxs.ch''' - LU Internal network = Setup Virtual Machine & Virtual Host = Requires the following steps: 1. Define VM Vagrantfile generation 1. Define VM configuration 1. Define Virtual Host configuration 1. Start new VM 1. Log into VM and perform the initial OS update and first ''puppet agent'' run 1. Setup SSH login certificate == Configuration for Vagrantfile Generation == 1. Configure the Vagrantfile generation in the Puppet server mgt.vh01.softxs.ch. As user ''root'' on the Puppet server {{{ ssh -p 10104 root@lu-4.softxs.ch # loki.softxs.ch cd /etc/puppet/modules/manifests vi nodes.pp # Find the node definition for the host server (zg-1, loki, odin, etc.) # Add a ''v2_server::vm'' definition, similar to the following: node "zg-3.softxs.ch" { # host VM ... create_resources( v2_server::vm, { # VM definition ... 'v0402' => { vm_hostname => 'v0402', vm_fqdn => 'v0402.vh03.softxs.ch', vm_box => 'precise64_rails', vm_memory => 3072, vm_ip => '172.16.4.2', vm_netmask => '255.240.0.0', vm_ssh_port => 20402, vm_synched_folders => { 'vagrant' => { vm_dir => "/vagrant", host_dir => "." }, 'data' => { vm_dir => "/data", host_dir => "/v01/data/v0402", owner => "www-data", group => "v2", mount_options => ["dmode=775", "fmode=764"], }, 'backup' => { vm_dir => "/backup", host_dir => "/v01/backup/local/v0402", owner => "www-data", group => "v2", mount_options => ["dmode=775", "fmode=764"], }, }, }, ... }}} 1. The critical parameters are: (See [[V2VirtualServers]] for example configurations and the setup conventions) 1. Hostname 1. Fully qualified domain name (fqdn) 1. Virtual machine ''box'' 1. Memory 1. IP address 1. Port number 1. Paths for the /data & /backup directories == Define Reverse Proxy == 1. Define a ''reverse proxy'' entry on the host machine, to forward http requests to the VM. As user ''root'' on the Puppet server {{{ cd /etc/puppet vi manifests/nodes.pp node "zg-3.softxs.ch" { # Find the node of the host machine ... create_resources( rproxy::vhost, { ... # Add an entry similar to the following: 'v0402' => { host => 'v030402.softxs.ch', ip => '172.16.4.2' }, ... } ... } }}} 1. Where: 1. ''v0402'' is the hostname of the VM 1. ''host'' is the external (public) DNS name 1. ''ip'' is the internal IP address of the VM == Define Application Backup & Data Directories == 1. Define Backup & Data directories for the VM. As user ''root'' on the Puppet server {{{ cd /etc/puppet vi manifests/nodes.pp node "zg-3.softxs.ch" { # Find the node of the host machine ... create_resources( v2_server::mkdir, { ... # Add entries similar to the following "v0402-backup" => { target => "/v01/backup/local/v0402" }, "v0402-data" => { target => "/v01/data/v0402" }, ... } ... } }}} 1. Where: 1. ''v0402'' is the hostname of the VM 1. ''/v01/backup/local'' is the top-level directory of host machine's local backup tree 1. ''/v01/data'' is the top-level directory of host machine's application directory tree == Run Puppet Agent on the Host Server == 1. Wait for ''puppet agent'' to run on the host machine to implement the updates 1. Or run it by hand: {{{ puppet agent --onetime --no-daemonize --verbose }}} 1. This will also create a new VM directory on the host machine: * /home/vagrant/vms/{name} Where ''name'' is the VM new host name == Document New Virtual Machine on Wiki == 1. Document the new VM in the Wiki page: [[V2VirtualServers]] == Define a Puppet Node for the New Virtual Machine == 1. Define a new ''puppet node'' for the vm, named with the VM's fully qualified domain name: {{{ node "v0402.vh03.softxs.ch" { # VM FQDN class { 'puppet_agent': period => 10 } class { 'postfix_satellite': node_name => $name, mailname => 'softxs.ch', relayhost => 'smtp.softxs.ch' } include apache include mysql include passenger class { 'rails_user': user => 'v2' } # ..place for vhost definition (see next section).. } }}} == Define a Virtual Host on the VM == 1. Add an ''apache::vhost'' definition to the VM node: {{{ node "v0402.vh03.softxs.ch" { # VM node ... apache::vhost {'trial.softxs.ch': template => 'apache/vhost-v2.conf.erb', docroot => '/home/v2/rails/trial.softxs.ch', priority => 25, servername => 'trial.softxs.ch', options => '-Indexes', subvhosts => { 'trial1' => { }, 'trial2' => { }, }, ... } }}} 1. The ''subvhosts'' are used to define the relative paths for the individual applications == Start Virtual Machine == 1. As user ''vagrant'' on the host machine start the VM: {{{ cd /home/vagrant/vms/{name} # Where 'name' is the new VM host name vagrant up # Starts the VM }}} 1. Log into the VM {{{ vagrant ssh }}} 1. Perform an initial software update on the VM {{{ su - apt-get update }}} 1. Perform an initial ''puppet agent'' run on the VM {{{ puppet agent --onetime --no-daemonize --verbose }}} 1. This should create a vhost, including: 1. Apache configuration files, located at: 1. /etc/apache2/sites-available 1. /etc/apache2/sites-enabled 1. Base directories for the application(s): 1. /home/v2/{domain}/{app(s)} == Setup V2 User SSH Certificate == 1. In order to allow automatic login for application deployment using Capistrano, you must install a SSH key on the VM for the v2 user: {{{ ssh -p {port} v2@{host} }}} 1. Where * ''port'' is the SSH port number defined in the VM's Vagrantfile * ''host'' is the base servers domain name 1. Create a .ssh directory. As user ''v2'' on the VM: {{{ cd ~ mkdir .ssh # If the directory doesn't already exist chmod 0700 .ssh cd .ssh vi authorized_keys # Add your SSH public key, typically from your local machine, in the file ''~/.ssh/identity.pub'' or ''~/.ssh/id_dsa.pub'' :x chmod 0600 authorized_keys }}} 1. Test the key. As your local user on your development (or other) system, from which you plan to do application deployment, make sure the following command works without having to enter a password: {{{ ssh -p {port} v2@host }}} 1. Where: * ''port'' is the SSH port number defined in the VM's Vagrantfile * ''host'' is the base servers domain name = Setup V2 Deployment Instance = 1. Work Organiser application is done from the git repository: '''git.softxs.ch:/home/git/gitroot/v2deploy.git''' == Define Instance == 1. Go to the Works Organiser instance directory: At the top of ''v2deploy'' repository: {{{ cd app/v2p0/site/proto/instance }}} 1. Create an new instance directory. Typically it's named {dns_name}-{path}, where 1. ''dns_name'' is the DNS name used to access the application host (a VM) 1. ''path'' is the relative path 1. For example: '''demo.softxs.ch-hydro''', which would be access with the url: [[http://demo.softxs.ch/hydro]] 1. Create the following directory & files in the instance directory: 1. ''deploy.rb'' - Capistrano deployment paramenters, including server, directories, etc 1. ''files'' - directory 1. ''config'' - directory 1. ''database.yml'' - Define the application's database 1. ''email_secret.txt'' - Key for sending email 1. ''settings.local.yml'' - Define the site/instance application specific settings 1. The settings for these files are explained in the following sections == Define deploy.rb == 1. The ''deploy.rb'' file defines the Capistrano deployment parameters. 1. Example ''deploy.rb'' file: {{{ # deploy.rb # app: v2p0, site: proto, instance: demo.softxs.ch-tasks Capistrano::Configuration.instance.load do # -- Server parameters role :web, "demo.softxs.ch" role :app, "demo.softxs.ch" role :db, "demo.softxs.ch", :primary => true set :vm_ssh_port, 20108 set :basepath, "/home/v2/rails" set :servpath, "demo.softxs.ch" set :relpath, "tasks" set :deploy_to, "#{basepath}/#{servpath}/#{relpath}-app" # -- Relative path on server (disable if no relative path) # WARNING: RAILS_RELATIVE_URL_ROOT must = '/' + Settings.SXS.Application.Url.Path set :asset_env, "RAILS_RELATIVE_URL_ROOT='/#{relpath}'" # -- Owner/group of deployed code set :user, "v2" # set :group, "www-data" # # -- Git parameters set :branch, "master" # Git branch to deploy set :scm_user, ENV['CAP_USER'] || ENV['USER'] # -- Installation set :use_sudo, false end }}} 1. The key settings are: 1. ''roles'' - The server(s) where the application will be defined. Normally they are all set to the same machine 1. '':web'' - The web server 1. '':app'' - The application server 1. '':db'' - The SQL server 1. '':vm_ssh_port'' - The SSH port to use to log into the server(s) 1. '':basepath'' - The absolute path where applications are installed 1. '':servpath'' - The relative path to the vhost 1. '':relpath'' - The relative path to the application 1. '':branch'' - The git branch to install == settings.local.yml == == database.yml == == Email Secret Key == == Home Page Setup == = Deploy Initial V2 System = == Capistrano Deployment == == Test Local Login == == Test Invitation & MAPS Login == = Start Delayed Job Task = = Register Application in MAPS = = Test Outgoing Email = = Configure & Test Incoming Email = = Install V2 Configuration = = Import Configuration = = Import Data & Documents = = End-to-End Test = = Setup Backups = Application backups run on a nightly basis. There are three parts: 1. A database backup to the application's ''../shared/backup'' directory 1. A backup of the application's ''-app'' directory tree (which includes ''shared/backup'') to a sub-directory in the MV's ''/backup'' (NFS mount-ed from the host server) 1. A backup server ''rsync''-ing the host machine's ''backup'' area (TODO: described elsewhere) Backups are based on the following cronjobs: 1. On the application VM, setup cronjobs as user ''v2'': 1. Backup the application database 1. Backup the application's ''-app'' directory tree == Application Database Backup == 1. On the application VM, as user ''v2'' Setup a cronjob like the following: {{{ 23 03 * * * /home/v2/rails/{domain}/{relpath}-app/current/script/ruby_cron.sh \ /home/v2/rails/{domain}/{relpath}-app/current/script/backup_db \ -o /home/v2/rails/{domain}/{relpath)-app/shared/backup \ -u{usr} -p{pwd} -z }}} 1. The cronjob should run every 24 hours and its time should be ''before'' the root ''rsync'' cronjob shown below 1. Where: 1. ''{domain}'' is the application's domain name. E.g. ''poyry.works-organiser.com'' 1. ''{relpath}'' is the relative URL path to the application. E.g. ''stadelhofen'' 1. ''{usr}'' is the MySQL administrative user's name. E.g. ''root'' 1. ''{pwd}'' is the MySQL administrative user's name. E.g. ''secret'' 1. ''-z'' is the option to compress the output with ''bzip2'' == Application Directory Tree Backup == '''''Warning''''': These cronjobs currently run as root, as there are permissions errors running as user v2 1. On the application VM, as user ''v2'' Setup a cronjob like the following: {{{ /usr/bin/rsync -qa /home/v2/rails/{domain}/{relpath}-app/ /backup/{domain}/{relpath}/ }}} 1. The cronjob should run every 24 hours and its time should be ''before'' the backup server's collection of the host machine's nightly ''backup'' data 1. Where: 1. ''{domain}'' is the application's domain name. E.g. ''poyry.works-organiser.com'' 1. ''{relpath}'' is the relative URL path to the application. E.g. ''stadelhofen'' = Setup Monitoring =