• V2MigrationTools

Migration External Data into V2

V2Master

• Contents

  1. Migration External Data into V2
  2. Introduction
  3. Import Tools
     1. V2Import Rails Modul
        1. Module Classes
        2. Module Usage
        3. DSL File Format
        4. CSV File Format
     2. V2Import Rake Task

Introduction

Import Tools

V2Import Rails Modul

Module Classes

• A new module V2Import implemented in lib/v2_import.rb. The module has the following main classes:

  • Helper - methods used in other classes of the module
  • Reader::DSLReader - class to read and check the import DSL configuration file
  • Reader::CSVReader - read the input CSV file into a hash
  • Import - loads data into the database based on the result of two previous class

Module Usage

• The module should be initialized first passing the the filenames (CSV file, import configuration DSL file) and CSV options:

  • import = V2Import::Import.new csv_file, config_file, csv_options

  csv_options are option of the standard Ruby CSV.new method. They can be used to change the default separators (col_sep: ',', row_sep: 'auto', quote_char: '"'). The module initialization exits with exception on the first configuration error found. The initialzed module can be queried to show the internal data formats:

  • y import.config_hash # show the internal format of the import configuration, read from the DSL file
    y import.csv_hash    # show the internal format of the input CSV file

  The load method of the initialized module should be called to load the data into the database:

  • import.load(load_options)

  load_options:

  • insert_only: default = true, all records are new. Currently not implemented, probably goes into the DSL.
  • rollback: default = false, rollback at the end, good for testing the input.

  The load method doesn't exits on errors, but rollbacks every changes if data error occurred. The 2 calls can be chained of course:

  • V2Import::Import.new(csv_file, config_file, csv_options).load(load_options)

DSL File Format

CSV File Format

V2Import Rake Task