byte order mark

The problem

• When a byte order mark (BOM) sneaks into PHP files (and even other file types), it creates all kinds of weird havoc. It's hard to detect because the BOM is not readily visible when you open a in text editors.

Symptoms

• Some of your .docx / .doc / .xlsx / .xls files are corrupt on download but weirdly .pdf files are fine
• "Headers already sent"
• Weird output, white pages
• An issue that no one else has ever reported in Tiki...

Solution

• A check from Tiki (php console.php dev:fixbom)
  • It should scans all Tiki files because the BOM is not necessarily in a file distributed by Tiki. (ex.: language/fr/custom.php)
• A check as part of the release process
  • Composer should check for and remove BOMs
  • You will find js files in external libs that have BOMs, these can be ignored as they are not an issue in js files.
    • We should change upstream the fix

In the meantime you can test in a shell with this command:

grep -rl $'\xEF\xBB\xBF' .

Related links

• http://en.wikipedia.org/wiki/Byte_order_mark
• https://github.com/smarty-php/smarty/commit/c589134cedc6a968a4e83028f08fcfc16c2f6c1f
• https://bugs.php.net/bug.php?id=22108
• http://stackoverflow.com/questions/8028957/headers-already-sent-by-php
• http://stackoverflow.com/questions/10290849/how-to-remove-multiple-utf-8-bom-sequences-before-doctype
• http://stackoverflow.com/questions/3255993/how-do-i-remove-i-from-the-beginning-of-a-file
• http://stackoverflow.com/questions/2558172/utf-8-bom-signature-in-php-files
• http://stackoverflow.com/questions/204765/elegant-way-to-search-for-utf-8-files-with-bom

Related tools

• http://emrahgunduz.com/categories/development/php/take-2-on-utf8-bom-remove-bom-with-php/
• http://svn.fckeditor.net/FCKeditor/trunk/_dev/php_bom_finder.php
• http://code.google.com/p/dompdf/source/diff?spec=svn434&r=434&format=side&path=/trunk/dompdf/include/dompdf.cls.php
• http://www.bryntyounce.com/filebomdetector.htm

• BOM