V2 Server Setup

V2Master

• Contents

  1. V2 Server Setup
  2. Introduction
     1. Basic Platform
  3. Operating System Installation
     1. Ubuntu Installation
     2. Initial Configuration
  4. Setup Mirrored Disks
     1. Device Names
     2. Procedure
  5. Additional System Configuration
     1. NTP Configuration
     2. Puppet Installation and Configuration
        1. Puppet Installation
        2. Puppet Configuration
     3. Puppet Connection to Puppetmaster

Introduction

1. A Linux server capable of hosting multiple virtual machines (VMs)
2. Basic setup performed by hand:
   1. OS installation
   2. Install development tools
   3. Setup RAID disk mirroring

3. Further provisioning automated by use of Puppet:

   1. VirtualBox and Vagrant installation

   2. Rinetd setup on host system
   3. Reverse proxy gateway in VM
   4. Internal DNS server in VM

Basic Platform

1. Hardware:
   1. Multi-core Intel architecture with 64-bit processor(s)
   2. 1 x SSD drive for the root, swap and main applications
   3. 2 x identical hard disks for backups
2. Software:

   1. VirtualBox - Virtualization software

   2. Vagrant - Ruby-based command line front-end for VirtualBox

   3. Other standard Linux packages (Bind, Apache, MySQL, Rails, etc.) as needed

Operating System Installation

1. Download Ubuntu 12.04.3 LTS Server

   1. Available from: http://www.ubuntu.com/download/server

   2. Select: 64-bit (recommended) - This is called amd64, with is appropriate for Interl and AMD 64-bit processors

2. Prepare a bootable memory stick
   1. See instructions at:

      1. On an Ubuntu system: http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-ubuntu

Ubuntu Installation

1. Boot system from memory stick containing Ubuntu 12.04.3 LTS server

2. Select language: English

3. Selection option: Install Ubuntu Server

4. Select system language: English

5. Select location: other --> Europe --> Switzerland

6. Select locale: en_US.UTF-8

7. Configure network:

   1. DHCP configuration will start, enter cancel

   2. Select primary network interface - This question does not appear for systems with a single ethernet port

   3. Select: Configure manually

   4. IP address: enter systems's IP address'

   5. Gate IP address: enter system's gateway address

   6. Nameserver address(es): enter system's primary DNS IP address(es)'

   7. Hostname: enter system's hostname

8. Create initial user
   1. Full name:
   2. User name:

   3. Enter & re-enter password

   4. Encrypt home directory: no

9. Time zone: Accept Europe/Zurich

10. Partition disks:
    1. Select entry for SSD drive

    2. Select Automatically partition and use LVM

    3. Select entire disk
    4. Accept proposed partition and accept writing partition table to disk

11. Installs base operating system

12. Enter proxy information: Enter return (e.g. no proxy required)

13. Software selection: Select SSHD server, and nothing else

14. Installs and configures more software

15. Install GRUB boot loader: Yes

16. Installation complete. Remove memory stick a reboot

Initial Configuration

1. Login as the user defined during the installation
2. Set the root password:

   • $ sudo bash
     # passwd

3. Update system:

   • # apt-get update
     # apt-get -y dist-upgrade

4. Install development tools, required for VirtualBox installation

   • # apt-get -y install build-essential autoconf libtool pkg-config

Setup Mirrored Disks

Setup RAID-1 based mirrored disks

Device Names

In a typical installation, the SSD and two hard disks have device names assigned as follows:

1. SSD: /dev/sda

2. Hard disk 1: /dev/sdb

3. Hard disk 2: /dev/sdc

Procedure

1. Partition and format hard disks with ext4 filesystems

   1. Partition: Perform this procedure once for heach hard dick, typically for /dev/sdb and /dev/sdc. Enter the following commands:

      • # fdisk /dev/sdb
          p  # Print partition table
          d  # Delete all existing partitions. May need to use this command multiple times
          n  # Create a new partition, accept defaults, which are for a primary partition using all availa disk space
          w  # Write partition table to disk and exit

   2. Example:

      • Command (m for help): p
          ..prints partition table (which will be empty for a new disk)..
        Command (m for help): n
        Partition type:
          p   primary (0 primary, 0 extended, 4 free)
          e   extended
        Select (default p):
        Partition number (1-4, default 1):
        Using default value 1
        
        Command (m for help): w
        The partition table has been altered!
        Calling ioctl() to re-read partition table.

2. Setup RAID-1 set:
   1. Create mount point:

      • # mkdir /v01

   2. Create multi-disk array for RAID set:

      • # mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1

3. Show information about the RAID set:

   • # mdadm --detail /dev/md0
     /dev/md0:
             Version : 1.2
       Creation Time : Tue Nov 19 16:12:03 2013
          Raid Level : raid1
          Array Size : 1953381248 (1862.89 GiB 2000.26 GB)
       Used Dev Size : 1953381248 (1862.89 GiB 2000.26 GB)
        Raid Devices : 2
       Total Devices : 2
         Persistence : Superblock is persistent
     
         Update Time : Thu Nov 21 16:48:44 2013
               State : active
      Active Devices : 2
     Working Devices : 2
      Failed Devices : 0
       Spare Devices : 0
     
                Name : odin:0  (local to host odin)
                UUID : 7289ef81:334df27d:389e9383:c225b4f5
              Events : 160
     
         Number   Major   Minor   RaidDevice State
            0       8       17        0      active sync   /dev/sdb1
            1       8       33        1      active sync   /dev/sdc1
     
     # mdadm --detail --scan
     ARRAY /dev/md/0 metadata=1.2 name=odin:0 UUID=7289ef81:334df27d:389e9383:c225b4f5
     
     # blkid /dev/md0
     /dev/md0: UUID="21f4f1a5-7f60-4f89-986d-84cfd6de49b4" TYPE="ext4"

4. Format the RAID set:

   • # mkfs -t ext4 /dev/md0

5. Update /etc/mdadm/mdadm.conf and add the following: (Note use of the UUID, obtained from the mdadm --detail --scan command):

   • # 2 x 2TB mirrored hard drives:
     ARRAY /dev/md0 level=raid1 num-devices=2 metadata=1.2 UUID=7289ef81:334df27d:389e9383:c225b4f5

6. Update /etc/fstab and add the following: (Note use of the UUID, obtained from the blkid /dev/md0 command):

   • # 2 x 2TB mirrod disks
     UUID=21f4f1a5-7f60-4f89-986d-84cfd6de49b4 /v01 ext4 defaults 0 2

7. Update /etc/initramfs-tools/conf.d/mdadm to contain: (This allow the server to reboot when the RAID set is degraded. If this is not set, when the RAID set is degraded, which is the apparently case while the RAID is initially being built, the system will not boot, except through the rescue menu entry of the system's boot menu).

   • BOOT_DEGRADED=true

8. Test mount and umount the RAID set:

   • # mount /v01
     # df -h
     ...should list /dev/md0 with the expected disk capacity
     # umount /v01

9. Reboot system to make sure it boots and that the /v01 file system is present:

   • # reboot now

Additional System Configuration

NTP Configuration

1. Install NTP daemon

   • # apt-get -y install ntp

2. Edit /etc/ntp.conf to contain the following:

   • server ntp1.softxs.ch
     server ntp2.softxs.ch
     server ntp3.softxs.ch
     server ntp4.softxs.ch

3. Start NTP daemon:

   • # service ntp restart

4. Check damon running and able to connect with time servers: (You should see output like the following):

   • # ntpq -p
          remote           refid      st t when poll reach   delay   offset  jitter
     ==============================================================================
      caledonia.dataw 129.69.1.153     2 u    7   64    1   19.662   54.345   0.000
      ntp0.as34288.ne .MRS.            1 u    6   64    1   27.857   14.460   0.000
      arthur.testserv 162.23.41.56     2 u    5   64    1   23.029   12.674   0.000
      ms21.snowflakeh 81.94.123.17     2 u    4   64    1   35.802   19.073   0.000
      europium.canoni 193.79.237.14    2 u    3   64    1   40.978    9.453   0.000

Puppet Installation and Configuration

Puppet Installation

1. Setup backup area:

   • # mkdir -p /v01/home/backup

2. Download and install recent version of Puppet (the Ubuntu package is generally out of date)

   • # cd /v01/home/backup
     # wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb
     
     # dpkg -i puppetlabs-release-precise.deb
     # apt-get update
     # apt-get -y install puppet

3. Check version of puppet and facter (Puppet's support tool for getting OS specific information):

   • # puppet --version
     3.3.2
     
     # facter --version
     1.7.3

     Make sure the puppet version is 3.3.x or later.

Puppet Configuration

1. Setup Puppet main directory:

   • # mkdir -p /etc/puppet
     # cd /etc/puppet

2. Edit file /etc/conf/puppet.conf to contain the following:

   • [agent]
       server      = puppet
       report      = true
       pluginsync  = false

3. Edit file /etc/conf/auth.conf to contain the following:

   • # This is an example auth.conf file, which implements the
     # defaults used by the puppet master.
     
     ### Authenticated paths - these apply only when the client
     ### has a valid certificate and is thus authenticated
     
     # allow nodes to retrieve their own catalog
     path ~ ^/catalog/([^/]+)$
     method find
     allow $1
     
     # allow nodes to retrieve their own node definition
     path ~ ^/node/([^/]+)$
     method find
     allow $1
     #allow *
     #allow thor.softxs.ch
     
     # allow all nodes to access the certificates services
     path /certificate_revocation_list/ca
     method find
     allow *
     
     # allow all nodes to store their reports
     path /report
     method save
     allow *
     
     # unconditionally allow access to all file services
     # which means in practice that fileserver.conf will
     # still be used
     path /file
     allow *
     
     ### Unauthenticated ACL, for clients for which the current master doesn't
     ### have a valid certificate; we allow authenticated users, too, because
     ### there isn't a great harm in letting that request through.
     
     # allow access to the master CA
     path /certificate/ca
     auth any
     method find
     allow *
     
     path /certificate/
     auth any
     method find
     allow *
     
     path /certificate_request
     auth any
     method find, save
     allow *
     
     # this one is not stricly necessary, but it has the merit
     # of showing the default policy, which is deny everything else
     path /
     auth any

Puppet Connection to Puppetmaster

1. On the new system as root: Make a test connection to the puppetmaster (puppet.softxs.ch):

   • # cd /etc/puppet
     puppet agent --test --verbose
     Info: Creating a new SSL key for odin.softxs.ch
     Notice: Using less secure serialization of reports and query parameters for compatibility
     Notice: with older puppet master. To remove this notice, please upgrade your master(s) 
     Notice: to Puppet 3.3 or newer.
     Notice: See http://links.puppetlabs.com/deprecate_yaml_on_network for more information.
     Info: Caching certificate for ca
     Info: Creating a new SSL certificate request for odin.softxs.ch
     Info: Certificate Request fingerprint (SHA256): B3:F3:30:C0:AD:C3:48:2E:31:34:EA:36:74:DD:24:75:4B:E9:82:45:F7:93:A1:9B:F1:A8:A7:B8:54:8F:5B:FA
     Exiting; no certificate found and waitforcert is disabled

2. On the puppetmaster, puppet.softxs.ch, view and sign the cetificate:

   • # puppet cert --list
       "odin.softxs.ch" (SHA256) B3:F3:30:C0:AD:C3:48:2E:31:34:EA:36:74:DD:24:75:4B:E9:82:45:F7:93:A1:9B:F1:A8:A7:B8:54:8F:5B:FA
     
     # puppet cert --sign odin.softxs.ch
     Notice: Signed certificate request for odin.softxs.ch
     Notice: Removing file Puppet::SSL::CertificateRequest odin.softxs.ch at '/etc/puppet/ssl/ca/requests/odin.softxs.ch.pem'

3. Make puppet test tun again to verify the certificate works (on new system as root):

   • # puppet agent --verbose --no-daemonize  --onetime
     Notice: Using less secure serialization of reports and query parameters for compatibility
     Notice: with older puppet master. To remove this notice, please upgrade your master(s) 
     Notice: to Puppet 3.3 or newer.
     Notice: See http://links.puppetlabs.com/deprecate_yaml_on_network for more information.
     Info: Caching certificate for odin.softxs.ch
     Info: Caching certificate_revocation_list for ca
     Info: Retrieving plugin
     Notice: /File[/var/lib/puppet/lib/puppet]/ensure: created
     Notice: /File[/var/lib/puppet/lib/puppet/face]/ensure: created
     ...and many more messages about files in /var/lib/puppet...
     Info: Caching catalog for odin.softxs.ch
     Info: Applying configuration version '1385393941'
     Info: Creating state file /var/lib/puppet/state/state.yaml
     Notice: Finished catalog run in 0.03 seconds