Category: Indexing

{img src="tiki-download_item_attachment.php?attId=491"}

{img fileId="605"}

If the user is admin, it should offer a link

{CODE()}Click here to rebuild index{CODE}

If user is not admin:

{CODE()}Contact the site administrator. The index needs rebuilding.{CODE}



Please do in trunk, and we can backport later (after 12.0, but before 12.1)

{syntax type="tiki" editor="plain"}
When you create a wikipage its title is available instantly in for example a tracker page selector.
Would it be possible to add/remove a Mathematical Calculation Tracker Field to this system?

After this edit:
https://dev.tiki.org/tiki-pagehistory.php?page=Revert+a+commit&history_offset=1&diff_style=sidediff&diff_style=sidediff&show_all_versions=y&compare=Compare&newver=0&oldver=6&tra_lang=sq&paginate=on&history_pagesize=60

I got:
{CODE()}Indexing failed while processing "Revert a commit" (type wiki page) with the error "RemoteTransportException[[search2][inet[/107.6.4.154:9300]][indices/mapping/put]]; nested: MergeMappingException[Merge failed with failures {[mapper [modification_date] of different type, current_type [string], merged_type [date]]}]; "{CODE}

On Tiki19
Last update from SVN (19.0svn): Thursday September 27, 2018 12:59:03 IDT - REV 67702 (InnoDB)

I create a first tracker after enabling the feature.

It show the following error:
Field tracker_id does not exist in the current index. If this is a tracker field, the proper syntax is tracker_field_tracker_id.

It shouldn’t show and error.
Eventually an info about re-indexing but would be better to do the job (first tracker is likely to be done by a new guy).

Seen on dev.tiki.org {sign user="marclaporte" datetime="2013-11-26T13:18:26+00:00"}

On creation of revision 66 of http://dev.tiki.org/Update, I got:
{img fileId="657"}

{CODE(caption="Error(s)")}Indexing failed while processing "Update" (type wiki page) with the error "RemoteTransportException[[search3][inet[/76.74.203.89:9300]][indices/mapping/put]]; nested: MergeMappingException[Merge failed with failures {[mapper [comment_count] of different type, current_type [long], merged_type [string], mapper [contributors] has different index values, mapper [contributors] has different tokenize values, mapper [contributors] has different index_analyzer, mapper [object_id] has different index values, mapper [object_id] has different tokenize values, mapper [object_id] has different index_analyzer, mapper [url] has different index values, mapper [url] has different tokenize values, mapper [url] has different index_analyzer, mapper [object_type] has different index values, mapper [object_type] has different tokenize values, mapper [object_type] has different index_analyzer, mapper [view_permission] has different index values, mapper [view_permission] has different tokenize values, mapper [view_permission] has different index_analyzer, mapper [title_initial] has different index values, mapper [title_initial] has different tokenize values, mapper [title_initial] has different index_analyzer, mapper [relations] has different index values, mapper [relations] has different tokenize values, mapper [relations] has different index_analyzer, mapper [deep_categories] of different type, current_type [long], merged_type [string], mapper [relation_types] has different index values, mapper [relation_types] has different tokenize values, mapper [relation_types] has different index_analyzer, mapper [categories] has different index values, mapper [categories] has different tokenize values, mapper [categories] has different index_analyzer, mapper [language] has different index values, mapper [language] has different tokenize values, mapper [language] has different index_analyzer, mapper [geo_located] has different index values, mapper [geo_located] has different tokenize values, mapper [geo_located] has different index_analyzer, mapper [allowed_groups] has different index values, mapper [allowed_groups] has different tokenize values, mapper [allowed_groups] has different index_analyzer]}]; "{CODE}

Hello,

Every time I save a wiki content, I got the following error:

"Search index could not be updated. The site is misconfigured. Contact an administrator.
Utf8 compatible lower case filter needs mbstring extension to be enabled."

I checked phpinfo at http://grip.umich.edu/tikiwiki/tiki-phpinfo.php, and the mbstring extension was enabled.

{syntax type="tiki" editor="plain"}
__unified_incremental_update doesn't check for feature_search__

Unified Search with Lucene is awesome and is "the future of Tiki".

Now, on certain server configurations, it doesn't work well. An easy workaround is to simply revert to the legacy MySQL Full Text Search.

However, even when using MySQL Full Text Search, unified_incremental_update is defaulted to yes, and tried to update Lucene index, and generates an error message

Workaround is to deactivate unified_incremental_update, or even better, upgrade to 12.x and use Unified Index, with MySQL as storage.
---
Update: this issue has been reproduced in 12.x svn (and untested in 14.x so far, afaik) also with Mysql Full Text Search as unified search engine (on a multitiki install). See: [item5673] . {sign user="xavi" datetime="2015-05-19T09:10:57+00:00"}

To reproduce.

As your Unified Index:
* Use Elasticsearch for a few days
* Switch to MySQL

Your setting will stay say that date of last re-index is the one from Elastic

In Tiki 12, I could see who I am logged as inside the login pop-up (in "popup" mode, which is on every default install).
Now it only displays "My account". This makes life uselessly hard to people who have many logins, or people who share a computer and want a quick remainder of who is logged in.

In tiki 12, it was provided with ~np~{user|userlink}~/np~
I suppose this was causing issues with side effects of something else (avatars?) so I suggest a simpler fix using ~np~{$user|username|escape:"html"}~/np~

{syntax type="tiki" editor="plain"}
Full thread is here:
http://tiki.org/forumthread47305

"So should we have a new pref "unified_index" and add it as dependency for these features (PluginList, PluginListExecute, etc.), and thus people whose servers don't work with it have the option to turn them off? (effectively acting like a Tiki6)"

There are issues at: ((Unified Index with MySQL Full Text Search))

Also, there should be warnings when site hits limitations of MySQL, that the admin needs to change to Zend_Lucene or ElasticSearch. Ideally, these warnings would come before hitting the limitations but this may not be easy to do.

Nelson (note to self): To make this bug actionable: these are the following actions. Many other things like excluding plugins by default are already done.

1) Make the logging use a different log file when triggered from the console vs the web (this will avoid any file perm problem).

2) Provide remarks box about including plugins in the admin panel.

3) Provide remarks box explaining that MySQL Full text search will only search first x characters of content (might affect large files in the file gallery for e.g.) - this is handled in item4641 already

__Unified Index:__

Works both in Elasticsearch and MariaDB
This works: https://dev.tiki.org/Wishlist-Search-Page

But:
https://dev.tiki.org/Tiki19#Open_or_Pending_blockers_and_regression works with Elasticsearch but not MariaDB

Code copied below:

{CODE()}
{LIST()}
{filter content="5" field="tracker_id"}
{filter field="tracker_status" content="o OR p"}
{filter field="tracker_field_version" content="331"}
{filter field="tracker_field_category" content="265 OR 172"}
{filter field="tracker_field_resolution_status" content="New OR Accepted OR Confirmed"}
{* sort mode="modification_date_ndesc" *}
{sort mode="tracker_field_priority_ndesc"}
{OUTPUT(template="table")}
{column field="mod_date" label="LastModif" mode="raw"}
{column field="tracker_status" label="" mode="raw"}
{column sort="tracker_field_resolution_status" field="tracker_field_resolution_status" label="Resolution Status" mode="raw"}
{column sort="tracker_field_submitted_by" field="tracker_field_submitted_by" label="Submitted By"}
{column field="tracker_field_assigned_to" label="Volunteered" mode="raw"}
{column field="tracker_field_subject" label="Title" mode="raw"}
{column label="Category" field="category" mode="raw"}
{column sort="tracker_field_version" field="tracker_field_version" label="Version" mode="raw"}
{column field="tracker_field_feature" label="Feature" mode="raw"}
{column sort="tracker_field_priority" field="tracker_field_priority" label="Priority" mode="raw"}
{tablesorter server="n" sortable="y"}
{OUTPUT}
{FORMAT(name="mod_date")}{display name="modification_date" format="date"}{FORMAT}
{FORMAT(name="tracker_status")}{display name="tracker_status" format="trackerrender"}{FORMAT}
{FORMAT(name="tracker_field_resolution_status")}{display name="tracker_field_resolution_status" default="" format="trackerrender" editable=inline}{FORMAT}
{FORMAT(name="tracker_field_submitted_by")}{display name="tracker_field_submitted_by" format="trackerrender"}{FORMAT}
{FORMAT(name="tracker_field_assigned_to")}{display name="tracker_field_assigned_to" format="trackerrender" editable=inline default=""}{FORMAT}
{FORMAT(name="tracker_field_subject")}{display name="tracker_field_subject" format="trackerrender"}{FORMAT}
{FORMAT(name="category")}{display name="tracker_field_category" format="trackerrender"}{FORMAT}
{FORMAT(name="tracker_field_version")}{display name="tracker_field_version" format="trackerrender"}{FORMAT}
{FORMAT(name="tracker_field_feature")}{display name="tracker_field_feature" format="trackerrender"}{FORMAT}
{FORMAT(name="tracker_field_priority")}{display name="tracker_field_priority" format="trackerrender" default="" }{FORMAT}
{LIST}
{CODE}

Set-up monitoring for all our pre-dogfood servers for all the checks we have, but especially last successful re-index. If last successful re-index is more than x (ex.:3) days old, an email alert should be sent out.

Here is a real-world example of why having checks on all pre-dogfood servers would have made everything smoother. An unidentified commit in trunk is causing indexing to crash:
https://dev.tiki.org/item4609

If we had been warned by a pre-dogfood sever, we could have found the commit and fixed the problem a few days after it appearing.

Note that this type of crash can also happen with some data not yet supported by the indexer, and thus, it can happen at any time, not just on code changes. And with a broken index, a Tiki site can be in big trouble because it doesn't just skip over the problem, it just dies. We are fortunate to have tons of data on all the pre-dogfood servers.

Because of issue 4609, we are stuck with respect to testing Unified Search with MySQL in trunk. Once this is solved, maybe something else will be discovered, so we to know ASAP so it's stable enough for Tiki12. This is crucial as per ((Search Dilemma))

As reported on the dev mailing list....

http://thread.gmane.org/gmane.comp.cms.tiki.devel/25682

Nelson: see comment for fix.

{syntax type="tiki" editor="plain"}
Not possible to delete tracker 16 here:
https://next.tiki.org/trackers

I can't reproduce on show, so it must be something with the data from tiki.org

This is blocking progress on ((Unified Search with MySQL Full Text Search))

This is related to http://dev.tiki.org/item4609

__Update 2014-07-29 For Tiki13__
{flash type="url" movie="display862" width="983" height="480"}

I've run a script on all profiles from all branches ( https://github.com/xavidp/TikiProfilesTester ) and I see that all profiles since 15.x throw this error message (which might be related to the fact that unified search index may be currently enabled by default)?
{CODE()}
PHP Fatal error: Uncaught Search_MySql_LimitReachedException: Could not perform index modification: Incorrect datetime value: '0000-00-00 00:00:00' for column 'creation_date' at row 1 in /var/www/html/15.x/lib/core/Search/MySql/QueryBuffer.php:55
Stack trace:
#0 /var/www/html/15.x/lib/core/Search/MySql/QueryBuffer.php(37): Search_MySql_QueryBuffer->realFlush()
#1 /var/www/html/15.x/lib/core/Search/MySql/Table.php(195): Search_MySql_QueryBuffer->flush()
#2 /var/www/html/15.x/lib/core/Search/MySql/Table.php(28): Search_MySql_Table->flush()
#3 [internal function]: Search_MySql_Table->__destruct()
#4 {main}
thrown in /var/www/html/15.x/lib/core/Search/MySql/QueryBuffer.php on line 55
{CODE}

Using php 5.6, and mysql (server) 5.7.12 (mysql client: libmysql - mysqlnd 5.0.11-dev - 20120503).

Profiles are applied through the command line with "php console.php p:a ProfileName"

Rebuilding index now reports:
{CODE()}ssh-w00ec615@dd29038:/www/htdocs/w00ec615/12x$ php console.php index:rebuild
Started rebuilding index...
Indexation
wiki page: 2
file: 0
trackeritem: 2
sheet: 1
comment: 0
Rebuilding index done
{CODE}

This was always in the web interface and recently added to the command line in http://sourceforge.net/p/tikiwiki/code/48172/
Great!

What would be even more useful is to have the same stats that are provided by the "Server Load" module from tiki-admin_modules.php, which provides Execution time, Memory usage and number of Queries

So then, we can get an idea when it's getting too long or using too much RAM.

We'd want this both in web & command line. This could be added to the ((Monitoring)) system and we could monitor system health.

Thanks!

When using MySQL Full text Search as the index. CustomSeach plugin doesn't seem to find all items as compared to a PHP Lucene index

See show instance

Show username: admin
Show password: 12345

Can be seen here:
* https://dev.tiki.org/Unprioritized+Open+Wishes
* https://dev.tiki.org/Move+from+30+minute+fixes+to+ease+filter

{syntax type="tiki" editor="plain"}
From the tracker plugin; autosavefields="243" autosavevalues="category(285)"
Category field should be saved with "Indexing" for value.

{syntax type="tiki" editor="plain"}
Date and time set to 3/11/2024 - 9:00 with inline editing after item has been created

{img src=show_image.php?id=10 }

{img fileId="396"}

For some misterious reason, since a few weeks (2 maybe {sign user="xavi" datetime="2015-05-19T08:56:45+00:00"} ) incremental search index fails (using either lucene or mysql full text search as the unified search engines)

Errors are related to permission problems (in an example with Lucene PHP as unified search engine, and at tracker item insertion):
{CODE()}
Search index could not be updated. The site is missconfigured. Contact an administrator.
fopen(temp/site1.example.com////////unified-index/_u.sti): failed to open stream: Permission denied.
{CODE}

{img fileId="1000" thumb="y" rel="box[g]"}


For a forum post, in another tiki 12 site, another user reported this error message:
{CODE()}
Search index could not be updated. The site is misconfigured. Contact an administrator.
fopen(temp/site2.example.com////unified-index/_e.sti): failed to open stream: Permission denied
Search index could not be updated. The site is misconfigured. Contact an administrator.
fopen(temp/site2.example.com////////////////////////////unified-index/_e.sti): failed to open stream: Permission denied
{CODE}

Content is added successfully (wiki page edition, tracker index, forum post, ...), but the error message is shown to the standard non-admin user as "Error", even if the setting "show php errors" is set to "Show to Admins only".



This is afecting all our tiki 12.x that we have in production (4) running on the same ((doc:multitiki)) instance, on an ((doc:ISPConfig))-powered server (in case this matters) . If unix permissions are related, then you need to know in advanced that ISPConfig requires permissions set as www-data for the owner, and clientN for the group, being "N" a number corresponding to the client number in the ISPConfig configuration. See ((doc:ISPConfig))
---
Related bug (I can't seem to select them through the field "Related-to" since they are not shown in the list, which is capped before reaching those ones starting with "Search index could not be updated. The site is misconfigured. Contact an administrator.":

"Search index could not be updated. The site is misconfigured. Contact an administrator."
http://dev.tiki.org/item4115