This is a proposal to take email handling to the next level in Tiki21 and Tiki22, enhancing our Cypht integration, and taking advantage of the new JMAP standard. Calendar Invitations by email, CalDAV and CardDAV were added to Tiki21.

Introduction

Email handling (reading, replying and archiving) should be part of a larger collaborative workflow. Instead of one gigantic mail store, we should have a number of smaller ones that make sense to one's workflow (ex.: around projects, tasks, clients, etc.) and that can easily be shared and prioritized.

This will improve Tiki for use cases such as:

Example

I am working on project X. In one view, I should see all related

  • tasks
  • time sheet entries
  • team members and roles
  • emails sent and received
  • wiki pages and files


Tiki already does a great job with all of these except emails. Emails about a task, client or whatever should "live" near the other info, so team members get an overall view, facilitating:

  • work prioritization: team members can handle all email related to a high-priority task before moving on to a lower priority task
  • time tracking: handling email is an important task that can lead to creating other tasks, or adding updates on existing tasks. And thus, instead of investing 2 hours to handle general email (time which then can't be attributed to a project), email handling becomes part of the tasks
  • team work: relevant emails now become part of a task and it provides better context to all involved
  • archiving: for auditing, it provides a more complete picture

UX flow / sequence

  1. Zillions of emails come in via Groupmail or Webmail, and they are in a mail store (usually IMAP accessible), accessible via other mail clients (ex.: Outlook or Thunderbird)
  2. Curation step to relate emails and threads to one or many tracker items or wiki pages
    • This moves them out of IMAP
      • There is no point in duplicating the data
    • If an email or thread is tagged to many tracker items, it should be visible to the user (so user answering this email knows it can impact more than one task)
  3. Within tracker item or wiki page, there are folders or tags, including "inbox" , "sent" and "archives"
    • In some tasks/projects, we'll want other workflows. Ex.: invoices received via email could be "inbox" , "data entered in accounting system" , etc.
  4. Cypht should be used to handle the emails (view email/thread and respond). By default, the reply mail and the rest of the thread should continue in the context of this tracker item or wiki page, so somehow, there should be an option for system to route future related messages to this section. Some may prefer not to have it automatic so manual curation continues.

Storage

  • Started: http://sourceforge.net/p/tikiwiki/code/75563
  • Stored in Tiki as native / immutable object for auditing
    • Should be part of Tiki backup and not depend on IMAP server, and thus, the storage options are
      • File Galleries (default)
      • Should be possible to use alternate storage in future versions: NoSQL, MailDir, MySQL Table, etc.
    • In Unified Index for quick search, and automation
    • Available in Tiki via a to-be-coded "email folders tracker field type"
      • Perhaps as a extension of the files field?
      • Should permits to do smart searching within. Use Cypht to parse the emails. Ex.: from / to / bcc / date / body / attachments (A bit the same idea for CalDAV)
      • Should be possible to use multiple fields in the same trackers
      • Should be possible to drag and drop emails to the right folder
      • Should be possible to use filters (ex: show me all emails sent to and by a specific contact)
      • Should be settings to where to store
        • IMAP folder
        • Tiki
        • Other options?
    • Should know which emails are related to each other (ex.: a thread)
    • Attachments
      • Identify within (ex. fwd of a thread with some attachments)
      • Will be searchable like other files in Tiki
      • Detach attachment option (save disk space)

How data gets in

  • Manual upload (ex.: .eml)
  • Forward or Redirect to mail-in to Trackers
    • Manual or automatic redirect / forward
    • Add an email address to any tracker item (assuming more interop with mail server)
      • Some will BCC when emailing out from main mail client
    • Option to only accept mail from known emails (including aliases)
  • Webmail / Groupmail which reads IMAP
    • Via rules or manual operations, mails are moved

Analogies with an office

  • IMAP server
    • Mailroom (triage to determine where it should go)
    • Cafeteria and hallway discussions
    • Alerts about a content in Tiki (no need to keep long term as everything is in Tiki)
  • Tiki
    • Office (individual work)
    • Meeting rooms (team work)


The IMAP server should not contain any information that will be useful in more than one month. The traditional mail server is like a mailroom. From the moment it's clear that this email / thread could be useful in the organization in more than a month, it's moved to Tiki, where it can be prioritized.

  • And thus, if a team member moves on from an organization, they just need to check their recent emails to see if any should be moved to Tiki. And their mailbox can be archived with the confidence that nothing important will be missed.

Mail server deployment use cases

WikiSuite mail server as the main mail server

  • We can create email addresses of the fly
  • Full control of workflow (ex.: email patterns)

Keep current mail servers

  • Most organizations will enjoy other parts of WikiSuite, but not change their mail server so we'll interop via IMAP, and later Microsoft Exchange, and of course JMAP
    • We don't need to worry about painful parts (ex.: managing spam filters)
    • We can start progressively just with a small team, without having to deal with a larger change management project
    • We can rely on IMAP email hosting such as what is offered by Gandi.net
    • But we can't create email addresses on the fly, or email patterns

WikiSuite mail server as a sub-domain

This brings us the best of both worlds. And filters can be used to selectively redirect mails from the main domain.
legalsupport@example.com will be redirected to support@legal.example.com

Aggregation

  • In the real World, team members receive relevant emails in various mailboxes. Cypht's aggregator approach will be very useful to move emails from wherever to a useful place in Tiki.

Example workflows

Using IMAP as a gateway

  • IMAP inbox
    • Archive message (still in IMAP but hidden from immediate view, but searchable if needed)
    • Move to Tiki
      • Perhaps there could be an IMAP folder for this
  • Tiki mail store
    • "Moved from IMAP": from which it's moved (in reality tagged to appropriate task or wiki page or invoice or anything...)

All in Tiki

In some cases, it may be simpler to skip the curation part described above and handle everything in Tiki

Forwarding and BCC

Some may prefer to duplicate mail and opt for Tiki mail-in to trackers. So they forward/redirect mail to Tiki mail-in and when they send out email, the add the Tiki mail-in address in BCC

Tiki and Cyrus IMAP interop