Category: Search

On a local setup and on Tiki25 I used hundreds of times the console with this command:

{CODE()}php console.php i:r -p --log{CODE}

Since a few days (I updated to the last bread) I have an error.
I believe extra-checks have been added or something has changed in the console process (because my configuration didn't changed) and it is causing all the process to fail if the log file cannot be written:

^
php console.php i:r -p --log
sh: tesseract: command not found
[2023-04-30 02:19] Started rebuilding index...
Logging to file(s):

* ../tmp/Search_Indexer_mysql_xxxxx_tiki25_console.log

Unified search
--------------

Engine: MySQL, version 5.7.39
< 1 sec/< 1 sec [>---------------------------] -- Rebuilding...sh: tesseract: command not found
1 sec/1 sec [============================] -- Rebuilding preferences indexerror: The search index could not be rebuilt.
__"../tmp/Search_Indexer_mysql_xxxx_tiki25_console.log" cannot be opened with mode "w"__


The search index could not be rebuilt. "../tmp/Search_Indexer_mysql_xxxx_tiki25_console.log" cannot be opened with mode "w"
^

I have re-re-run the setup.sh permissions setup.

Removing the "--log" allow the index:rebuilt to complete.

This is may be totally legit and true however, it should not stop the process. "index:rebuild" is critical today to keep freshness of data and correct Tiki behavior.
I suggest a warning to be displayed, but the process to keep on.

Editing https://tiki.org/Consultants and https://next.tiki.org/Consultants or visiting pages like https://tiki.org/tiki-admin_tracker_fields.php?trackerId=22 made the server crash half a dozen of times. When t.o crash I had also crash on next.t.o doing something else.

From my understanding both are not on the same server.
Is it possible something still ties them together or could it be (extraordinary) random server crash occurring at the same time ?

{img fileId="1219" thumb="box"}
{img fileId="1218" thumb="box"}
{img fileId="1217" thumb="box"}
{img fileId="1216" thumb="box"}
{img fileId="1215" thumb="box"}

Note: It happened before I rebuilt the index using the web interface. At the end of the rebuilding they were multiples errors (much diverse than the screenshot show).

{img fileId="1220" thumb="box"}

We (geoff and xavi) were considering the benefits of creating a Profile out of a use case of trackers, plugin list and custom search. To achieve such a goal, a pre-liminary step is to implement a tiki site in a show.t.o instance here in dev.t.o, which is similar to a custom setup that Geoff has for some project. And then, convert that setup into a profile in profiles.t.o

More to come. And feel free to join us if you have some ideas or spare time to contribute to this project! :-)
(you will learn how to create a profile out of a tiki site)

show instance (based on Tiki12 LTS):
http://xavi-9794-5603.show.tikiwiki.org/
u: admin
p: 12345

I can't make the basic example work.
https://doc.tiki.org/PluginCustomSearch#Basic_Usage

I'll reproduce it in show.t.o to make it easy for others to to review where the issue might be.

My goal is to leave one basic working example documented in doc.t.o

Admin password in that instance is:
u: admin
p: 12345

Go to Homepage, log in as admin, and try to search for any word found in the three items in that tracker. No results are found.
Index has been rebuilt many times.

I wanted to search content posted on dev.t.o. I searched through the search box at the header, as usual, for a keyword, and nothing is shown in the results
https://dev.tiki.org/tiki-searchindex.php?filter%7Econtent=alchemist&filter%7Edate_range=

Not even any message that no results are found (if that is the case)

Broken somehow?

I see weird and inconsistent search results on the Wishlist search form when I remove any tracker_status filter at https://dev.tiki.org/Wishlist-Search-Page . I see more than 2 "empty" tickets.

{img fileId="1750" thumb="box"}

I can't see the same at https://dev.tiki.org/tracker5
However if I try to sort by subject I have also weird results: https://dev.tiki.org/tracker5?status=&sort_mode=f_26_asc
The sorting is wrong and I have only 2 "empty" tickets.

{img fileId="1749" thumb="box"}

dev.t.o: form to search for related-to bugs became recently broken

I could use it yesterday with no problems, while reporting this bug:
https://dev.tiki.org/item7133-CSRF-False-positives

which is linked to a couple other items.
If I edit that item now, I can't search for other items agains CSRF, for instance (while previously linked items are still linked there).

New bug reports do not allow to search for related items when you edit the bug report.
Something has changed in the last 24h? (some svn up? some config changes somewhere?)

After this edit:
https://dev.tiki.org/tiki-pagehistory.php?page=Revert+a+commit&history_offset=1&diff_style=sidediff&diff_style=sidediff&show_all_versions=y&compare=Compare&newver=0&oldver=6&tra_lang=sq&paginate=on&history_pagesize=60

I got:
{CODE()}Indexing failed while processing "Revert a commit" (type wiki page) with the error "RemoteTransportException[[search2][inet[/107.6.4.154:9300]][indices/mapping/put]]; nested: MergeMappingException[Merge failed with failures {[mapper [modification_date] of different type, current_type [string], merged_type [date]]}]; "{CODE}

{CODE(wrap=>1)}
(18:30:35) xavi: to reproduce: I did search for "Report" at the top bar search
(18:30:42) xavi: showing this url: http://dev.tikiwiki.org/tiki-searchresults.php?highlight=Report&search=Go
(18:30:47) xavi: with this error:
(18:31:14) xavi: [off] You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '? and ttif.`value`=? and ttifOut.`fieldId`=?' at line 2 The query was: select ttifOut.`value` from `tiki_tracker_item_fields` ttifOut, `tiki_tracker_item_fields` ttif where ttifOut.`itemId`= ttif.`itemId`and ttif.`fieldId`=? and ttif.`value`=? and ttifOut.`fieldId`=?
{CODE}

---
Update on March 31st 2010, using Last update from SVN (5.0alpha): Friday 26 March, 2010 09:12:40 CET - REV 26326 :

{img src=http://dev.tikiwiki.org/tiki-download_item_attachment.php?attId=236 desc="http://dev.tikiwiki.org/tiki-download_item_attachment.php?attId=236"}

{syntax type="tiki" editor="plain"}
Tested on info.tw.o with FiveAlive theme...

Disabling the __Search box__ option on the Admin: Search page, Search Results tab (tiki-admin.php?page=search) will remove the Search bar from the menu.

This is incorrect behavior.

Tiki already shows the category and/or category path when listing wiki pages. I would like to also display the category information in search results.

This would help end-users determine the relevancy of the search result.

I would like Tiki to display each item's freetags when display the search results. This would help end-users determine the relevancy of each item.

I would like Tiki to display each item's freetags when display the search results. This would help end-users determine the relevancy of each item.

{syntax type="tiki" editor="plain"}
Hello, if I search an "entire site" I expect a search in all sections and as result all objects where I have the correct object permission. But this search is implemented with an inconsistent double permission check.

The "entire site" search fist checks the global view permission in each section. If this is true, the search is sent to the search engine of the section, which gives back results depending on the object permission.

As the default search is "entire site" (can be changed via template), all users without the global view permission in the sections get back an empty result-set. A click on the section search button reveals the searchresult depending on the object permission of the section.

This is somehow between a bug and a feature.

As far as I can tell, the solution to show "entire site" search results according to object permissions is very easy.

You can see here:
http://dev.tikiwiki.org/tiki-searchresults.php?highlight=mouseover&where=pages&search=Go

If I want to look up all users which are waiting for the admin to validate them, I can do this LIST plugin:
{CODE(theme="default")}{LIST()}
{filter field="object_type" exact="user" }
{filter field="waiting" exact="a" }
{pagination max="9999"}
{LIST}{CODE}

This works fine using unified search with option ''mysql fulltext search''.

Using option ''Elastic search'' nothing is returned because "a" is in the stopwords list. Removing "a" from the stopwords list fixes the issue.

I can understand why stopwords are stripped from searches such as content="something". That's the idea of stopwords.
__But on exact searches stopwords should not be stripped__. Because, by the definition of "exact search", one knows exactly what one is searching for.

Also, we should not get different results depending on the unified search option.

It seems some internal code in tiki to use elastic search needs to be updated to work with ElasticSearch 5.x:

Installed a blank new tiki, changed unified search engine to ElasticSearch (using v5.0.2 in Ubuntu 16.04 with their elasticsearch repository to have updated packages), and I applied Collaborative_Community_12 profile. I saw these errors just above the instructions page:

{CODE()}
Error(s)

Indexing failed while processing "HomePage" (type wiki page) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "Instructions" (type wiki page) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "Community" (type wiki page) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "Wiki Help" (type wiki page) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "1" (type forum) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "1" (type tracker) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "1" (type trackerfield) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "2" (type trackerfield) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "3" (type trackerfield) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "4" (type trackerfield) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "5" (type trackerfield) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "6" (type trackerfield) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "1" (type trackeritem) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "2" (type trackeritem) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "3" (type trackeritem) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "4" (type trackeritem) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "5" (type trackeritem) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "6" (type trackeritem) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "7" (type trackeritem) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "8" (type trackeritem) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "Community Members HomePage" (type wiki page) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "Anonymous" (type group) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "Registered" (type group) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "Editors" (type group) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "SubAdmins" (type group) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
Indexing failed while processing "Admins" (type group) with the error "The [string] type is removed in 5.0 and automatic upgrade failed because parameters [term_vector] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [title]"
{CODE}

And when I reindex, I get these other error messages:
{CODE()}
Error

all shards failed
×
Error

Indexing failed while processing "Community" (type wiki page) with the error "no such index"
×
Error

Indexing failed while processing "1" (type forum) with the error "no such index"
×
Error

Indexing failed while processing "1" (type file gallery) with the error "no such index"
×
Error

Indexing failed while processing "1" (type trackeritem) with the error "no such index"
×
Error

Indexing failed while processing "1" (type tracker) with the error "no such index"
×
Error

Indexing failed while processing "1" (type trackerfield) with the error "no such index"
×
Error

Indexing failed while processing "admin" (type user) with the error "no such index"
×
Error

Indexing failed while processing "Admins" (type group) with the error "no such index"
Rebuild Index
Your index was last fully rebuilt on Wednesday November 30, 2016 09:55:03 UTC.
×
Indexed

wiki page: 5
forum post: 0
forum: 1
file: 0
file gallery: 3
trackeritem: 8
tracker: 1
trackerfield: 6
comment: 0
user: 1
group: 5
{CODE}

https://github.com/Drive4ik/simple-tab-groups/issues/794

We should consider this for Tiki.

Step #1: Make a list of all features and contexts that have this search as you type feature.

On a Tiki24 using ElasticSearch(7.x), using the console I recreate the index.
It fails after several minutes with the following error:
{CODE()}
php console.php i:r -p
[2023-04-04 08:39] Started rebuilding index...

Unified search
--------------

Engine: Elastic
1 min/1 min [====================>-------] -- Processing activity documentsA error was encountered while running a command
Argument 1 passed to Search_Type_Json::__construct() must be of the type array, string given, called in /Users/bernardsfez/Documents/Shocksite/www/htdocs/wiki-to-yes.org_tiki24/lib/core/Search/Elastic/TypeFactory.php on line 81 on line 13 of /Users/xxx/xxx/xxx/www/htdocs/xxx_tiki24/lib/core/Search/Type/Json.php
{CODE}
''The error may be or may be not related to the fact that EalsticSearch is not running... that's not the point of the ticket''.


When I login as admin I can see that ElasticSearch is ''visibly'' not running. ?
(this error too could be improved, connection is not "just" refused, Elasticsearch is not running at all)

{CODE()}
Search index failure
Unable to connect to localhost:9200 . Error #0: stream_socket_client(): unable to connect to localhost:9200 (Connection refused)
{CODE}

After starting Elasticsearch service on the server everything is back to normal.

Prior to indexing (or any operation that involve the search engine) there should be a check it is running and if not an appropriate message should be displayed. (with or without fallback set).

The syntax:

-+ ~np~{filter field="tracker_field_permName" content="NOT "}~/np~+-

No longer works in plugin list in 23.x (and possibly 22.x) with elasticsearch.

It still works fine with the mysql engine, but it definitely fails (and finds everything, empty or not) in es7, 6 and 5.

More... finally confirmed the last version it worked in was 18.x, 21.x is broken too (also that i could only get 18.x to work properly with es5, 6 and 7 refused to index successfully) {sign user="jonnybradley" datetime="2021-08-09T17:41:43+00:00"}

[http://tikiwiki.org/tiki-searchresults.php?highlight=rss&where=forums&search=Search+in+content|Link to error]

On the main forum page of tw.o I enter the text "rss" then search button and get the following:



The following error message was returned:

You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1

The query was:
SELECT `rssId`, `url`, `actions` FROM `tiki_rss_modules` WHERE (`lastUpdated` < ? - `refresh`) AND `rssId` IN( )

Values:

1. 1270869772

The built query was likely:
SELECT `rssId`, `url`, `actions` FROM `tiki_rss_modules` WHERE (`lastUpdated` < '1270869772' - `refresh`) AND `rssId` IN( )

This would permit cleaner URLs and easier searching.

__Where?__

Wiki page names
Usernames
tags?
etc.

__What?__
Spaces & underscore to be equivalents
a and A are equivalent. This makes perfect sense for page names, user names, etc Accented characters (äâ) should be as well

For wiki page names (see how Wikipedia does it) but it could be that ~np~((User_Profile)), ((User+Profile)) and ((User Profile))~/np~ are equivalent.


Should hyphen (-) be an equivalent character as well?

How about (,) and (.)

. doesn't play nice with rewrite rules




Moved discussion to: ((Character substitutions))

{syntax type="tiki" editor="plain"}
Hello,

I just submit a new search, the SQL request crashes with "system error" as reported in item 3104.

This seems to be solved a few days ago, but it was not a full solution.
Here beyond the beginning of the request :

SELECT tti.*, ttif.`value`, ttf.`type`, sttif.`value` as `sortvalue` FROM ( `tiki_tracker_items` tti INNER JOIN `tiki_tracker_item_fields` ttif ON tti.`itemId` = ttif.`itemId` INNER JOIN `tiki_tracker_fields` ttf ON ttf.`fieldId` = ttif.`fieldId`) LEFT JOIN (`tiki_tracker_item_fields` sttif) ON (tti.`itemId` = sttif.`itemId` AND sttif.`fieldId` = 41) INNER JOIN `tiki_tracker_item_fields` ttif0 ON (ttif0.`itemId` = ttif.`itemId`) INNER JOIN `tiki_objects` tob43 ON (tob43.`itemId` = tti.`itemId`) INNER JOIN `tiki_category_objects` tco43 ON (tob43.`objectId` = tco43.`catObjectId`) INNER JOIN `tiki_tracker_item_fields` ttif1 ON (ttif1.`itemId` = ttif0.`itemId`) INNER JOIN `tiki_objects` tob47 ON (tob47.`itemId` = tti.`itemId`) INNER JOIN `tiki_category_objects` tco47 ON (tob47.`objectId` = tco47.`catObjectId`) WHERE tti.`trackerId` = ? and (`status`=? or `status`=?) AND ttif0.`fieldId`=? AND tob43.`type` = 'trackeritem' AND tco43.`categId` IN ( ) AND ttif1.`fieldId`=? AND tob47.`type` = 'trackeritem' AND tco47.`categId` IN ( ) GROUP BY tti.`itemId` ORDER BY `sortvalue` desc...

You can notice that the syntax uses in "IN ()" clause which is not accepted by MYSQL.

!!Solution, first steps
Check all SQL queries for "IN()" clause and find a bypass anywhere in tikiwiki.

Trebly

With feature_search enabled errors produced from error.tpl etc end up with the header, modules etc being repeated inside the page under the error message.

This can be seen here: [http://demo.tiki.org/12x/made+up+page] {sign user="jonnybradley" datetime="2013-10-13T16:13:38+00:00"}

This is a side-effect of the change to using Smarty Inheritance in the tempting system and due to tiki-searchindex.tpl now using "{extends 'layout_view.tpl'}".


Should also happen with feature_search_fulltext but the logic's borked currently... will fix that once this is fixed.

Weird issue, maybe too specific of the tiki 12 configuration for a local chapter of the World Social Forum in Catalonia.
But this is serious, and since might happen to other tikis, I report it before I forget what happened, and which manual workaround allowed us to keep going.

We have set up a Tiki site to gather information on workshops in a tracker. The Tracker (through a Wiki page with PluginTracker) also shows the registration fields, in order to get a user registered with the information of the workshop they want to propose.

That wiki page is shown through an iframe in from a wordpress (yes, this is real life; main site was created first with Wordpress, and they only want to use Tiki for the tracker system, so far).

http://www.forumsocialcatala.cat/fscat2014/inscripcio-tallers-fscat2014/

{GROUP(groups="Registered")}This is using this tiki url inside:
http://2014clean.forumsocialcatala.cat/Inscripcions?cookietab=2

inside plugin group since I don't want robots to crawl the tiki directly
{GROUP}

Once the form was submitted, Tiki displayed a page that an error occurred ("did you check your db is up & running", etc)
Apache logs don't show anything rellevant.
Tiki logs indicate that the database table

__dbname.index_52d52c9cad158__

was missing. Checking the mysql db directly, I saw that another similar db table name was there:

__dbname.index_5305ecc55cd29__

I did clear the Tiki caches many times, rebuild the serch index many times, changed from mysqlfts to lucene (clear caches and rebuild index each time) and back, a few times. Even disabled the unified search index. (clearing tiki caches also), and no way, each time the new item was attempted to get added to the Tiki site:
* that error page was shown
* the tiki logs showed the message that __dbname.index_52d52c9cad158__ was missing
* the user got added, but not the tracker item with the workshop information.

I only know that the system worked well until Feb 14th (the date of the last valid item in the tracker). After that, I svn updated to recent revisions yesterday (maybe to r49950ish or similar, I don't know for sure), and today svn-updated again, to r49974. And that mysql table either dissapeared, or was rebuilt to a different name, or the code decided to look for a different name of table at the database.

---
Update: I admit, that this is too of an edge case.
I have not been able to reproduce in a clone of that db in my localhost, so I close this bug report. I hope this issue doesn't happen to anyone else. {sign user="xavi" datetime="2014-03-05T12:56:35+00:00"}