• V2MailProcessing

V2 Email Processing

V2Master | V2HighLevelDesign

Requirements

Basic Requirements

1. Outgoing Emails

   1. Users can implicitly generate notifications, which may cause emails to be send to selected recipients, by creating and updating notes attached to objects
      1. All notification recipients must be registered in the system, as either internal or external users

   2. Editing or updating a note causes a notification event to be triggered:

      1. The recipients of the notification are taken from a subscriber list

      2. The subscriber list is associated with the object to which the note is attached
      3. The requirements for managing of the subscriber list are described below
      4. When a notification is triggered, emails are sent to users, depending on the settings of their profile flags:

         1. Profile mail flag To: Controls the sending email when the user is set in a note's 'To' field

         2. Profile mail flag CC: Controls the sending email when the user is a subscriber to the object to which the note is attached

      5. The email contains
         1. The note text and
         2. A link for downloading a zip file of all the files attached to the note
            1. The key in the download link is individual for each recipient and note
            2. The download link expires in a specific interval defined in system settings
            3. The system must record each use of a download key in the systems audit trail

2. Incoming Emails

   1. Incoming emails are routed, by an external mail server, to a V2 system
      1. The mail server's routing configuration is completely separate from the V2 configuration
   2. Incoming emails are saved as "incoming emails" which is a read-only object and converted also into unattached notes:
      1. Field mappings

         1. Note Text: email body

         2. Note Subject: subject

         3. Note Date: email date

         4. Note To: blank, unless the email has only one To recipient, who can be mapped to the email address of a user

      2. Email attachments are converted to note files and attached to the note
   3. An unattached note, representing an email, is filed in a folder based on:
      1. The email's delivery address
      2. And the V2 system configuration information, which maps delivery addresses to folders
      3. Or filed in a default folder, which is defined always as the last configuration, with email address as '.*'.
      4. The incoming email configuration will have:
         1. Email address, can contain simple wildcards ('*','?')
         2. Folder
         3. Issuer
   4. The notes must later be manually attached to revisions, tasks, transmittals, users, etc.
      1. When the note is attached, a notification is triggered

      2. When the note is attached, the user can also (optionally) set the note's To field

         1. Which will cause the user to be added as a subscriber
         2. A notification to be triggered to for the user in the to field
   5. Email notes (e.g. where the original source of the note is via incoming email)
      1. Can be edited, but when an email note edited it is marked as "edited"
         1. The editing is if subject, mail text or date changed

         2. Issue: What happens to HTML emails which no corresponding text part?

      2. The display of email notes is slightly different than normal notes
         1. Different background color

         2. The notation Email appears in the note block header

         3. the notation Edited appears in the note block header, if the email has been edited

   6. Notes cannot be cloned

3. Audit Trail

   1. Complete audit trail of all incoming and outgoing emails, including:
      1. Email header
      2. Message body
      3. All attachments
   2. View all sent and received email:
      1. Including their complete headers, in their original form
      2. Must record whether each recipient was sent an email, based on their email flags in their profile
   3. Make lists of all or selected emails including (perhaps in the note list with a different view):
      1. Subject line
      2. Date and time
      3. Sender
      4. Recipient(s)
      5. Link to view the original email, its header and attachments
   4. Make a list of all emails sent or received to a particular (perhaps in the note list with a different view):
      1. Internal or external user, including all their email addresses (even old addresses?)
      2. Email address
      3. Domain name

4. Notifications

   1. Notifications to subscribers are triggered when:
      1. A revision/task/etc. note is added/attached/updated
      2. A file is attached to a revision/task note
      3. No emails are triggered from unattached notes, including unfiled, unattached emails

   2. Email notifications are set to all subscribers that have the mail CC flag set in their user profile

   3. If a note is directed to a specific user (e.g. by setting the note's To field), an email notification is generated, provided the recipient's To email flag is set in their user profile

   4. Issue: How to handle creating a note and then editing or attaching files, but only send one email?

      1. Have a timeout, defined in the settings, causing the system to wait for a specified amount of time before acting on the notification trigger

   5. Ideas for future implementations:

      1. Folders subscriber lists. E.g. notifications when objects are added to Folders or edited, as part of the workflow implementation

      2. New object notifications

Details

1. Note To Field

   1. If an external user is set as to recipient who has no to profile flag set, a validation error will appear

2. Subscriber Lists

   1. In an object detail screen:
      1. Users can subscribe/de-subscribe themselves to the object (read permission is required on the object)
      2. Users with update permission on the object can subscribe/de-subscribe themselves and other users to the object
      3. There is a block containing the list of subscribers
         1. The entries indicating if the recipients is a restricted viewer

         2. Issue Do restricted viewers see it?

      4. Users who can not view the object will be marked with gray background

      5. User's to and/or cc profile flags will be displayes as hover or simple in parenthesis

      6. Users are automatically added as subscribers to an object when:
         1. They issue a note

         2. They are indicated as the to recipient of a note

   2. In a new/edit object screen:
      1. If internal users will be saved on the subscriber list without view permission, a warning flash message will appear
      2. If folder is changed a warning pop-up should appear that the subscriber list should be checked.
   3. Never trigger notifications or send email to disabled users
   4. The system must respect a note's restricted_flag an send emails appropriately.
      1. Do not trigger notifications for restricted notes to users who not able to view them
      2. Users who would not see a note will not get emails or be able to view the note in the dashboard

3. Email Folders

   1. Mark email folders in folder list, so that administrators:
      1. Understand why the folders cannot be deleted
         1. Instead of the delete button ('X') an info ('?') button will appear, which displays the usage of the folder by object types (doc, task, email_route, etc.)
      2. Correct configuration solves permissions issues associated with user being able to view incoming emails before they are filed

4. Dashboard

   1. Lists recent attached notes, reverse ordered by date

   2. The Notification dashboard might include two parts (or have a column that indicates for each notification):

      1. Notifications where I am a CC recipient

      2. Notifications directed to me, e.g. where I am addressed in the note's To filed

   3. The same item should not be listed more than once
   4. Dashboard should have options to determine exactly what is displayed (filter for the last n days, last n items)
   5. The dashboard might have per-user default settings (last n days, last n items, etc.)

5. Users

   1. Users can elect to be sent emails, or not, by setting/unsetting the email To and CC flags in their user profile

      1. The system must record whether emails were actually sent to individual users

      2. For external users, both email flags should be set to false by default

   2. The note text is sent
      1. Including a download link for all the files attached to the note

         1. Issue: Based on a system wide configuration flag?

         2. In the case of an HTML email a warning will be appended to the note text.

Design Issues

1. There is no relation between an email note and a transmittal. E.g. transmittals are unchanged
2. Can external users be subscribers to objects?

   1. Yes, and they will be sent emails subject to the settings of their user profile email To and CC flags

3. Can we implement drag & drop object improved linking? See the following examples:

   1. Drag & Drop Demo

   2. Another Drag & Drop Demo

4. What to do with email headers? Currently the raw email is saved in a file asset but hidden
5. What to do with emails that contain long copies of previous emails? Suggest storing them as attachment files? We can not split incoming emails.
6. Do recipient lists get copied when you clone an object? No

   1. Idea for future: Implement a copy subscribers between objects

   2. Implementing Workflows will affect how users become subscribers to objects. It might include Folder subscriber lists

7. We have currently an audit trail to hold the URL, IP, user, elapsed time, etc. Is this sufficient?

Implementation

Data Model Changes

1. New email routing table
2. New object subscribers table
3. Notes: Add a locked_flag field
4. Notes: Add a respond_to field, which links to another note (id) already in the system
5. Audit Trail: add new fields the register all relevant information

Interface to Email System

Outgoing email is sent using Active``Mailer

Incoming email is accepted via an HTTP POST transaction

1. The email server must be configured to initiate the transaction
2. The email server must do spam filtering, perhaps admin could delete incoming emails (or move to a spam folder)

POSTFIX configuration to forward incoming emails to the V2 system

Add a similar line to all email addresses to be forwarded to the V2 system:

• v2mail1:        |"/var/www/html/dev/rails/v2p0/script/email_handler.sh incoming_email http://localhost:3010/"

Replace

• v2mail1 with the email address without domain

• /var/www/html/dev/rails/v2p0/ with the path to script/email_handler.sh, normally the main rails directory

• http://localhost:3010/ with the root URL of the V2 system from the mail server

Note that the script email_handler.sh is using a secret key from config/email_secret.txt, it must be set in the capistrano deployment.

User Interface Changes

1. Email Filing Interface

   1. Should be able to re-link/attach starting from the email instead of from the revision/task
      1. Possibly using a modal dialog with a search pane

      2. Possibly using drag & drop

      3. Possibly using a 'clipboard' holding only one object per user

2. Detail Screens

   1. New block listing 'Recipients' in objects containing notes
      1. Includes add self (and others, based on privileges)
      2. Includes remove self (and others, based on privileges)
   2. Ability to copy/promote a note file into a revision file

      1. Issue: When you do this do you create an additional copy of the file?

      2. You may want to change the file name when you do this. It is possible to edit it after move/copy

3. Detail/Edit Screens

   1. Implement new always send me emails profile flag

   2. Implement new send me emails when address to me profile flag

   3. In user update screen: Disable email flag checkbox (and set to true) if user is an external user

4. Dashboard Pane

   1. to be defined