Category: Import-Export

Uploading a Mediawiki-XML-Dump created by Mediwikis dumpBackup.php leads to an empty page and an apache segfault on debian etch 64bit.

From /var/log/messages

{CODE()}apache2[3658]: segfault at 7fff3f74cef8 ip 7f3837b9d2f7 sp 7fff3f74ce00 error 6 in libpcre.so.3.12.0[7f3837b8e000+25000]{CODE}

File is beeing uploaded correctly in /tmp, upload to File Galleries works correctly.
changed the following values in

{CODE()}
/etc/php5/apache2/php.ini

post_max_size=128M
upload_max_filesize = 128M
max_execution_time = 600
max_input_time = 600
memory_limit = 32M (also tried 64M,128M,256M)
{CODE}

{CODE()}
/etc/apache2/apache2.conf
Timeout 600
{CODE}

Running Debian Etch amd64 all packages actual version of the distri.

Ways to reproduce:
1.) create a XML-Dump from Mediawiki via {CODE()}php dumpBackup.php --full --contents > dump.xml{CODE}
2.) use tikiwiki-importer via ->admin->tikiwiki-importer
3.) Upload in 1. created xml-dump

Neither changing export settings of ~np~ dumpBackup.php --full or --current with or without --content ~/np~ nor changing import options in tikiwiki-importer changes outcome.


---
Addition:
-Upgraded to tikiwiki4, upgraded to debian stable (lenny) as suggested
Problem still exists:

{CODE()}apache2[2231]: segfault at 7fff2184eec0 ip 7f9019c1bf6b sp 7fff2184edc0 error 6 in libpcre.so.3.12.1[7f9019c0b000+28000]{CODE}

Update of database to 7.0 doesn't work properly so I was need to export wiki articles to tiki 7.0 version so I tried to use this feature (Tiki -> TikiPages -> XML Zip. I have extended version of PHP so it shouldn't be PHP Problem.

When our users attempt run the Calendar export (either via tiki-calendar.php or tiki-calendar_params_ical.php) a 500 internal server error occurs.

From apache access.log:
{CODE(caption="access.log")}10.100.44.149 - - [30/Apr/2010:07:33:27 -0400] "POST /eiadwiki/tiki-calendar_export_ical.php HTTP/1.1" 500 - "http://eiadwiki/eiadwiki/tiki-calendar_params_ical.php" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3"{CODE}

From PHP error log:
No error reported log.

HTTP Request:
{CODE(caption="HTTP Request")}POST /eiadwiki/tiki-calendar_export_ical.php HTTP/1.1
Host: eiadwiki
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Referer: http://eiadwiki/eiadwiki/tiki-calendar.php
Cookie: javascript_enabled=y; menu=@menu42__50%3Ac@menu42__600%3Ac@menu42__1050%3Ao@menu42__200%3Ac@menu42__350%3Ac@filtercal%3Ao@exportcal%3Ao; tab=1; activeTabseiadwiki%2Ftiki-adminusers.php=tab1; local_tz=AM; PHPSESSID=mkiu7d2r4jlj74f7hlt2brvvj4
Content-Type: application/x-www-form-urlencoded
Content-Length: 41

export=y&calendarIds%5B%5D=1&valid=Export{CODE}

HTTP Response:
{CODE(caption="HTTP Response")}HTTP/1.0 500 Internal Server Error
Date: Fri, 30 Apr 2010 11:37:38 GMT
Server: Apache
X-Powered-By: PHP/5.3.2-dev
Expires: 0
Cache-Control: private
Pragma: dummy=bogus
Content-disposition: inline; filename=calendar.ics
Content-Length: 0
Content-Transfer-Encoding: quoted-printable
Connection: close
Content-Type: text/calendar; method=REQUEST; charset=iso-8859-15{CODE}

We are running in a multi tiki environment on Windows Server 2003, tikwiki 3.2 and php 5.3.

Can not ((XML Import-Export)).

It looks as though there is a Calendar import feature using CSV (tiki-calendar_import.php) however there are no options to export as CSV in tikiwiki 3.2. The only Calendar export option that I see is exporting to iCal (via tiki-calendar_export_ical.php) but that is not working for our group ([http://dev.tikiwiki.org/tiki-view_tracker_item.php?itemId=3152]).

Can Calendar export to CSV be possible?

We are running in a multi tiki environment on Windows Server 2003, tikwiki 3.2 and php 5.3.

When exporting tracker items, the generated csv file contains a blank line after every 100 items.

...
100 "180";"48";"13-06-2005";""
101 "133";"1";"30-04-2005";"5"
102
103 "179";"47";"31-01-2005";"0"
104 "178";"46";"31-01-2005";""

An imported Mediawiki page that has

{CODE(wrap="0",ishtml="0",ln="0",wiki="0",rtl="0",cpy="0")}* a bullet
** sub bullet{CODE}

gets translated to

{CODE(wrap="0",ishtml="0",ln="0",wiki="0",rtl="0",cpy="0")}* a bullet
{DL()}
sub bullet
{DL}{CODE}

which results in the sub bullet and text not displaying on the page at all, except being represented by a small icon.

Yet if the importer had just left it as

{CODE(wrap="0",ishtml="0",ln="0",wiki="0",rtl="0",cpy="0")}** sub bullet{CODE}

it would have worked.

__Update:__ Sometimes it does get the nested bullets. But there are other places where Mediawiki stuff ends up in the DL tags where it shouldn't - with the effect that it disappears from the visible page.

Tested in version 3.3 and 7:
Importing tracker items through CSV does not properly process the date field.

My exported CSV contains e.g. "24/12/2010 19:55"
The imported data only contains "24/12/2010" The time has been left out.

{CODE(caption="incorrect code 3.3" colors="php")}# Version 3.3
Line 1697
} elseif ($field['type'] == 'f' || $field['type'] == 'j') {
if ($dateFormat == 'mm/dd/yyyy') {
list($m, $d, $y) = split('/', $data[$i]);
$data[$i] = $tikilib->make_time(0, 0, 0, $m, $d, $y);
} elseif ($dateFormat == 'dd/mm/yyyy') {
list($d, $m, $y) = split('/', $data[$i]);
$data[$i] = $tikilib->make_time(0, 0, 0, $m, $d, $y);
}
}{CODE}

Replaced by following code :
{CODE(caption="correct code 3.3" colors="php") } elseif ($field['type'] == 'f' || $field['type'] == 'j') {
$l = strlen($data[$i]);
switch ($l) {
case ($l == 10):
# Field does not contain the time
if ($dateFormat == 'mm/dd/yyyy') {
list($m, $d, $y) = explode('/', $data[$i]);
$data[$i] = $tikilib->make_time(0, 0, 0, $m, $d, $y);
} elseif ($dateFormat == 'dd/mm/yyyy') {
list($d, $m, $y) = explode('/', $data[$i]);
$data[$i] = $tikilib->make_time(0, 0, 0, $m, $d, $y);
}
break;
case ($l == 16):
# Field contains HH:MM
list($fd, $ft) = explode(' ', $data[$i]);
if ($dateFormat == 'mm/dd/yyyy') {
list($m, $d, $y) = explode('/', $fd);
list($hh, $mm) = explode(':', $ft);
$data[$i] = $tikilib->make_time($hh, $mm, 0, $m, $d, $y);
} elseif ($dateFormat == 'dd/mm/yyyy') {
list($d, $m, $y) = explode('/', $fd);
list($hh, $mm) = explode(':', $ft);
$data[$i] = $tikilib->make_time($hh, $mm, 0, $m, $d, $y);
}
break;
case ($l == 19):
# Field contains HH:MM:SS
list($fd, $ft) = explode(' ', $data[$i]);
if ($dateFormat == 'mm/dd/yyyy') {
list($m, $d, $y) = explode('/', $fd);
list($hh, $mm) = explode(':', $ft);
$data[$i] = $tikilib->make_time($hh, $mm, $ss, $m, $d, $y);
} elseif ($dateFormat == 'dd/mm/yyyy') {
list($d, $m, $y) = explode('/', $fd);
list($hh, $mm) = explode(':', $ft);
$data[$i] = $tikilib->make_time($hh, $mm, $ss, $m, $d, $y);
}
break;
}
}{CODE}

The same for the latest version 8.3

{CODE(caption="Incorrect code 8.3" wrap="0" colors="php")} case 'f':
case 'j':
if ($dateFormat == 'mm/dd/yyyy') {
list($m, $d, $y) = preg_split('#/#', $data[$i]);
$data[$i] = $tikilib->make_time(0, 0, 0, $m, $d, $y);
} elseif ($dateFormat == 'dd/mm/yyyy') {
list($d, $m, $y) = preg_split('#/#', $data[$i]);
$data[$i] = $tikilib->make_time(0, 0, 0, $m, $d, $y);
} elseif ($dateFormat == 'yyyy-mm-dd') {
list($y, $m, $d) = preg_split('#-#', $data[$i]);
$data[$i] = $tikilib->make_time(0, 0, 0, $m, $d, $y);
}
break;
}{CODE}


{CODE(caption="Correct code v8.3" wrap="0" colors="php")}# Line 1730
case 'f':
case 'j':
$l = strlen($data[$i]);
switch ($l) {
case ($l == 10):
# Field does not contain the time
if ($dateFormat == 'mm/dd/yyyy') {
list($m, $d, $y) = explode('/', $data[$i]);
$data[$i] = $tikilib->make_time(0, 0, 0, $m, $d, $y);
} elseif ($dateFormat == 'dd/mm/yyyy') {
list($d, $m, $y) = explode('/', $data[$i]);
$data[$i] = $tikilib->make_time(0, 0, 0, $m, $d, $y);
} elseif ($dateFormat == 'yyyy-mm-dd') {
list($y, $m, $d) = explode('-', $data[$i]);
$data[$i] = $tikilib->make_time(0, 0, 0, $m, $d, $y);
}
break;
case ($l == 16):
# Field contains HH:MM
list($fd, $ft) = explode(' ', $data[$i]);
list($hh, $mm) = explode(':', $ft);
if ($dateFormat == 'mm/dd/yyyy') {
list($m, $d, $y) = explode('/', $fd);
$data[$i] = $tikilib->make_time($hh, $mm, 0, $m, $d, $y);
} elseif ($dateFormat == 'dd/mm/yyyy') {
list($d, $m, $y) = explode('/', $fd);
$data[$i] = $tikilib->make_time($hh, $mm, 0, $m, $d, $y);
} elseif ($dateFormat == 'yyyy-mm-dd') {
list($y, $m, $d) = explode('-', $fd);
$data[$i] = $tikilib->make_time($hh, $mm, 0, $m, $d, $y);
}
break;
case ($l == 19):
# Field contains HH:MM:SS
list($fd, $ft) = explode(' ', $data[$i]);
list($hh, $mm, $ss) = explode(':', $ft);
if ($dateFormat == 'mm/dd/yyyy') {
list($m, $d, $y) = explode('/', $fd);
$data[$i] = $tikilib->make_time($hh, $mm, $ss, $m, $d, $y);
} elseif ($dateFormat == 'dd/mm/yyyy') {
list($d, $m, $y) = explode('/', $fd);
$data[$i] = $tikilib->make_time($hh, $mm, $ss, $m, $d, $y);
} elseif ($dateFormat == 'yyyy-mm-dd') {
list($y, $m, $d) = explode('-', $fd);
$data[$i] = $tikilib->make_time($hh, $mm, $ss, $m, $d, $y);
}
break;
}
break;
{CODE}

Importing tracker items through CSV only sets the created and lastModif value properly if the dates from the CSV file are in the numerical php date format.

When you import a csv file where the created and lastModif dates are written as proper dates, the imported tracker items have a created and lastModif date/timestamp of the date & time of the import action.

example csv :

"itemId","status","created","lastModif","VRID -- 265"
"849","o","07/12/2009 13:24","07/12/2009 13:24","4"

Whenever I export a csv file, it contains the dates as shown above, so you can't just export data and import data on the fly without modifying all the values...

c.f. [https://dev.tiki.org/item3500|bug 3500]

Hi,

i am pretty new to tikiwiki (please point my to any guides if i am missing something here) and currently testing the 8.3 release.
As i do have several mediawiki installations it is a must for me to be able to import data from mediawiki to tiki.

I did export an .xml file via dumpBackup.php from one of my mediawikis which resulted in a 85mb file - which opens without errors in a random selected webbrowser.

As a next step i tried to use the Tiki Importer.
- Choosed source type (mediawiki) & pressed ok & got forwarded to next page
- used the default selections (uncheck, uncheck, 1, option 3) & selected the xml local file -> pressed the 'import' button
- warning id displayed that a script will be started and may take some time & that i should have a backup -> i do select 'OK' here

-> 3 sec later i am back at the start page of the tiki importer.


Problem:
- no error is displayed
- no import is done


question:
- is that function supposed to work?
- could it be related to the size of the xml file? even then i would consider an error.
- can i offer any additional informations?


best regards in advance
fidel


EDIT:
#####
seems to be related with the size of the .xml file.

case 1: small file - around 1 MB -> worked (i do see a progress window)
case 2: big file - around 85 MB -> failed - see above for a detailed description)

Main problem i do see here is that in case 2 the user/admin gets no feedback at all.

Say I have a desktop spreadsheet file. I can copy-paste this to Tiki spreadsheet and it makes it fun & fast to move data to Tiki. (It's also easy to export to .csv to re-use in the desktop spreadsheet).

But what if I want my info in a tracker?

((doc:PluginTrackerList)) has displaysheet=y which displays in spreadsheet view. This is nice but we'd need to be able to edit existing records, and easily add records.

For future versions 11 ? 12 ?

Hi,

There are several main marked up languages that we know well.

I think that it will be useful to be able to exchange (import, export) documents with other selected marked up languages.

!!I particularly think to two categories :

!!!1- The task and documents managers used by development tools : Eclipse-Mylin Aptana environment and Bugzilla

Remark :
About this category, I redact documents using textile to describe the developments integrated into projects and bugzilla.
This because I have near fifty elementary project developed since 4.0, stabilized at 8.3 and that I am preparing for 10.x. (They remain for most of them available and someones must be adapted - galleries particularly)

!!!2- The wikies
The problem is to exchange data with users of main wiki.

The need is to use the large capabilities of tikiwiki to hold projects management and documentation. But as it is not specialized it cannot be directly used.

!!There are several ways, this is completely opened :
1- Be compatible i/O with textile which is very closed from basic tiki marked up language. This for basic articles.
2- Develop a plugin using tikiwiki as marked up language for Mylin
3- Look farther with a full XSL XSL-Fo (note that with eclipse all marked up supported languages can be exported as XSL-

!!On the way, proposal
1- First : list of major marked up languages for wiki and task managers
2- Links import export existing from one to another
3- Choose the optimum
!!
Aim :
- Prepare use of tikiwiki in project management
- Be compatible with environments as Eclipse to with other project management tools

!!Reason(s)
- Tiki provides a lot of coherent and very useful tools to manage the content of projects (documents and detailed task management, studies) and groupware life and management which is the core of project management (the classical tools for planning are only in my mind a little part). __But it cannot be connected with specialized existing tools and information systems in various businesses. The problem and aim is to prune efficiently tikiwiki into these organizations.
__
Trebly

/lib/pear/Text/Wiki/Render/Tiki/Redirect.php has an error in line 38

{CODE(wrap="0",ishtml="0",ln="0",wiki="0",rtl="0",cpy="0")}return '{REDIRECT(pageName="';{CODE}

should be

{CODE(wrap="0",ishtml="0",ln="0",wiki="0",rtl="0",cpy="0")}return '{REDIRECT(page="';{CODE}

since Tiki's redirect takes "page", not "pageName".

I tried to import a MediaWiki XML file and got the following error mesage:

Loading and validating the XML file

Fatal error: Class 'DOMDocument' not found in /wiki/tikiwiki-4.1/lib/importer/tikiimporter_wiki_mediawiki.php on line 69

I am not a PHP programmer but this looks like not all the modules are linked together.


Any help would be appreciated.

Thanks
John

The Mediawiki Importer fails with:

[Sun Nov 28 22:17:33 2010] [error] ... PHP Notice: Undefined offset: 1 in /var/www/tiki/lib/pear/Text/Wiki/Parse/Mediawiki/Wikilink.php on line 263, referer: .../tiki/tiki-importer.php
[Sun Nov 28 22:17:38 2010] [notice] child pid 13376 exit signal Segmentation fault (11)

This is on Ubuntu Lucid with a 2.6.32-25-server kernel. It happens while the Importer is in the initial "Loading and validating the XML file" phase, and the Apache child instance running it of course just dies.

Update: It happens ever after I comment out the lines in Wikilink.php that threw the error - that is, there's still a segfault. This is with Apache 2.2.14 and PHP 5.3.2.

This may be the same bug as http://dev.tiki.org/tiki-view_tracker_item.php?itemId=2853&trackerId=5&show=view&reloff=1&cant=2&status=o&trackerId=5&sort_mode=f_41_desc&filterfield=26&filtervalue[26]=Importer - but that shows as fixed, so ....

This may also be related to this preg_replace_callback() bug: https://bugs.launchpad.net/ubuntu/+source/php5/+bug/369737 - although that's unresolved. That function is used in Table.php and Url.php in lib/pear/Text/Wiki/Parse/Mediawiki.

Update 2: Installed a fresh Fedora 14, put Tiki on it, and it gets the same segfault, in the same place in each of the 2 different XML files I'm attempting this with. Now, in both cases these are 64-bit, in case that has anything to do with it (but who runs a 64-bit server today?). In each case these are relatively large XML files - but our requirement is to convert a fair-sized Mediawiki if we're to use Tiki at all.

Various tools are able to built a structured documentation form internet exploration.
A good example is ZOTERO.
The product can get urls, html pages content, manage a hierarchical structure, set marks etc...
The product is able to export XML files with various basic patterns :
1. Zotero RDF
2. MODS
3. RIS
4. Refer/BibIX
5. Unqualified Dublin Core RDF
6. WikiPedia Citation templates
7. BibTeX

These structured informations can be imported to a wiki structure.
he interest of Zotero RDF (but I had not the time to the Unqualified Dublin Core RDF have the advantage of containing the structure definition the link to external url and if the html page has been captured the component html page with the elementary component has images.
This model his the closest of wiki structures.

Too the Wikipedia Citation templates is close but from the url refs;

I have not found where are the text and synthesis objectives for "import" and data shared (XML files as RDF Zotero structures), and multiwiki, in 4.x and 5.x

I am interested in participating to these developments.

Best regards to everybody

Implement something like the special page [http://en.wikipedia.org/wiki/Special:Book|Wikipedia Book] to generate some revenue from community assets