TRIM

Phase 1

• Disable site during upgrade (via .htaccess) Done
• Update DB (1.x to 1.y sql script) Done
• Handle both 1.9 and 1.10
• Cron job for automatic check and report on file integrity (what about cache files?) Done
• Something to indicate TRIM is working (sometimes, it is busy for many minutes and we are just left to wait, not knowing if it's stalled) ... may not be possible
  • Ok. Then, please indicate (in general) what TRIM is doing. Something like "TRIM will now transfer all files from SourceForge to TRIM (master) and then upload them to your instance. As long as you are not getting an error message, TRIM is hard at work! This operation could take 10-20 minutes or even a few hours, depending on server performance and network speed." Done
• run fixperms.sh Done
• Be more robust to various paths for php5 (ex.: using "which php" to find it) Done (using variables in Makefile so modification only required once)

Phase 2

• For TRIM to handle backups Done
• Make instance should suggest some values (ask for SSH host name and SSH user first). If you just type "Enter", it picks this value. These defaults should be editable somewhere in a config file. It also serves as a good example for the proper format. The default format which works for Fantastico hosting:
  • Web root : /home/$sshuser/public_html
  • Web URL : http://$sshhost
  • Working directory : /home/$sshuser/trim_working (TRIM should create with appropriate rights) Done
• Admin interface to edit and delete instances (maybe a ssl protected web interface?) or here
• To create database, database user and permissions from this user when installing Tiki. Could be in tiki-install.php as well
  • TRIM will need to be indicated the database login with sufficient rights to do this
  • Use this new info to populate db/local.php
  • Maybe pick profile from TRIM? (equivalent to tiki-install.php?) Done
• Nicer error message (from .htaccess when site is being updated) Done
• Backup
  • Cron job
  • Option to backup things outside of Tiki (ex.: mail directory, access logs, etc)
  • To be able to handle symlinks
    • /home/user/www is a symlink of /home/user/public_html and if I indicate /home/user/www as the web root of the instance, TRIM backup will not backup files) bug or feature enhancement request?
• bundled _htaccess
  • Rename the bundled _htaccess to .htaccess for instances running on Apache
    • Would be better to set an alias from .htaccess to _htaccess This is what the Tiki installer tried to do by default
• major upgrades
  • Handle transition from CVS to SVN (Sylvie has an unreleased script)
  • 1.9.x to 1.10.x, which requires 2 .sql operations
• Easier to manage in shell interface
  • document so people know to type things like C 0-52
  • "Next page" when too much info in one screen (sometimes hundreds of lines of scrollback), but maybe this is a bad idea because it'll prevent unattended operations. hmmm
  • Add a note before entering a passphrase in addinstance.php : "If you enter a passphrase, you will need to enter it every time you run TRIM, and thus, automatic, unattended operations (like backups, file integrity checks, etc.) will not be possible." done
  • If mysqladmin fails (ex.: bad login, MySQL down), it should offer to re-enter value or to abort database creation. In this last case, the system should indicate something like : "TRIM was unable to create your database with the information you provided. You should create it via your admin panel (cPanel, etc) and provide the information when you point your browser to tiki-install.php" done
• Indicate which warnings and error messages not to worry about (could be a link to a wiki page)
  • time stamp 2008-06-02 22:33:48 is 3938 s in the future
  • errors on setup.sh (when insufficient perms)
  • "warning: locate: warning: database /var/lib/slocate/slocate.db' is more than 8 days old" Nothing to do with TRIM
  • ssh: profiles.tiki.org: Name or service not known Seems solved--
  • Make convert should handle 2.2 to 3.0 This is done by make upgrade
• Perform database setup... -> use stars for 2 password fields fixed
• Make it possible to have unattended operations. This is useful for a WikiFarm.
• Plan for Tiki Suite.

Phase 3

• TRIM can sometimes use 100% of the CPU (especially when compressing files): should we add the "nice" command? done
• Should be a way to set a different folder/disk for backups (since this will become huge!)
  • Should also be a way to set permissions, so other software can access. Ex.: http://wikisuite.org/TRIM-and-Syncthing-for-automated-offsite-backups
• TRIM should add db/lock file
• TRIM should populate {$lastup}, which appears in templates/tiki-bot_bar.tpl. However, lastup should be (de)activatable and filterable by group. This is now well handled in Tiki. SVN revision number is from the code, and display is a module, which can be assigned to certain groups
• Set password at install (or anytime!)should be in tiki-install.php as well 😊. recreate admin user if deleted?
• Investigate DB creation with ClearOS (perhaps cPanel, ISPConfig, etc.) TRIM creates database and user with MySQL root
• Once everything is nice and stable with SSH, add SFTP/FTP support (done) so we could use on hosts like Mosso Cloud sites
• In addition to check and backup, having an automatic update script option (useful to always keep test sites like demo.tiki.org up to date. Unattended updates won't be able to deal with SVN conflicts. Maybe just send a warning email?
• Check used/available disk space. (% of used space) where TRIM is installed. TRIM will not be able to make proper backups if it runs out of space. Checking disk space on each Tiki is the job of the Monitoring process.

Phase 4

• Alerts for failure of an unattended operation. Ex: backup fails because SSH connection is refused.
• Backup
  • Have backups generate a "live" version of Tiki.
    • So a web agency could have a locally running backup of all their Tikis with yesterday's data.
      • Insures an easy way to test the automated backup system
      • Staff can continue to work if server or Internet connection is down.
      • Data could be in a USB Tiki format so Tiki admins could copy/sync to a mobile drive to have a backup in their pocket at all times.
    • Could also be to setup read-only mirrors
      • Tiki would need a read-only mode. So when users log in, they are informed that any changes here will be overwritten at the next sync.
      • These mirrors could be used for DNS3, DNS4, DNS5, etc.
  • Move an instance from one server to another (backup & restore in one step?)
    • This could include (This is the manual procedure I do) closing the old site ($site_closed eq 'y'), setting $site_closed_msg="This site has moved to: http://..." or "This site has moved to a new server. If you are seeing this message, it's because the DNS has not yet propagated for you. Please try again in a few hours or use this temporary address: http://..." before the backup and opening the site on the new server ($site_closed eq 'n').
  • Save to Amazon S3 and similar online storage services
  • Backup of the database should handle multitiki like doc/devtools/backupdb.sh
  • Investigate if scripts like automysqlbackup, rsync-incr or rsnapshot have any concepts, features or code that would be useful for us.
• Profiles
  • Investigate if/how TRIM could help maintaining: OpenSourceCMS type demo to test/develop and show off profiles
  • Hourly build of each profile: http://profiles.tiki.org/profile_name
  • Investigate any synergy with the Profiles to apply database change to one, many or all instances.
    • Turn off unsafe, buggy or unused features
    • Add watch all wiki pages or watch all trackers to multiple Tikis
    • Use existing Tiki functions for mass operations. Ex.: add a Tiki user on dozens of Tikis, etc (In this case, it's important to use existing Tiki functions rather than alter the database directly)
• Management
  • Add branch & lastup when listing instance.
    • Sort by branches & date, so as to see who is most out of date, who needs a security release, etc.
• Install
  • At install, a maintenance.html (or install.html should be shown instead of files/folders)

Phase 5

• Trigger Server Monitoring for check for domain name expiry, uptime / performance / content testing /etc.
• Attempt a Recovery for common issues (ex.: MySQL crashed table)
• When suspicious files are detected, in addition to deleting them, it should be possible to send them to a non Web accessible "quarantine" for further analysis or even optionally send to the security squad.
• import mysql data a bit like BigDump
• Possibly automated running of testing feature.