History: Tiki-End-to-End-Testing
Source of version: 2 (current)
Copy to clipboard
{REMARKSBOX(type="warning" title="Proposed feature, not merged yet")}
This page documents the Playwright end-to-end test suite proposed in [https://gitlab.com/tikiwiki/tiki/-/merge_requests/9887|merge request !9887], targeting __Tiki 31__. It is __not merged yet__, so -+tests/e2e/+- does not exist in a standard checkout: you need the branch from that merge request to follow this page. Details may still change while the merge request is under review. Once it is merged, remove this box.
{REMARKSBOX}
{maketoc}
! End-to-End Tests
Tiki ships a [https://playwright.dev|Playwright] end-to-end test suite in -+tests/e2e/+-. It drives a real browser against a running Tiki instance and covers functional flows that unit tests cannot reach: logging in, clicking through tabs and modals, submitting forms, and verifying what the user actually sees.
This page describes how to run the suite, how it is structured, and how to add tests to it.
!! Prerequisites
* A running Tiki instance, local or CI. Self-signed HTTPS certificates are accepted.
** ''Or none at all'': see [#Self-Contained_Mode|Self-Contained Mode], which provisions a disposable instance with a dedicated test database served by PHP's built-in web server. This is what CI uses.
* Node.js 20 or newer
* -+@playwright/test+- ^1.58.2, installed by -+npm install+- at the Tiki root
* The Tiki database seeded with test users. This is automatic, see below.
!! First-time setup
The fastest path is one command. It creates -+.env+- from the template if missing, installs the Playwright dependency and Chromium, then runs the smoke suite to confirm the environment works.
{CODE(colors="bash")}
tests/e2e/bin/e2e-setup.sh # setup + smoke run
tests/e2e/bin/e2e-setup.sh --no-run # setup only
{CODE}
By hand, from the Tiki root:
{CODE(colors="bash")}
npm install # @playwright/test is a root devDependency
npx playwright install chromium
{CODE}
!! Environment variables
Copy -+tests/e2e/.env.template+- to -+tests/e2e/.env+- and adjust it for your setup:
{CODE(colors="bash")}
cp tests/e2e/.env.template tests/e2e/.env
{CODE}
||__Variable__|__Default__|__Purpose__
-+TIKI_BASE_URL+-| -+http://localhost+-|Base URL of your Tiki instance
-+TIKI_FIXTURE_CMD+-|auto-detected|PHP invocation prefix for fixture scripts
-+TIKI_ADMIN_USER+-| -+admin+-|Admin username for tests requiring admin access
-+TIKI_ADMIN_PASS+-| -+admin1234+-|Admin password
-+TIKI_TEST_USER+-| -+Test1+-|Registered non-admin username for permission tests
-+TIKI_TEST_PASS+-| -+test1pass+-|Registered user password
-+TIKI_E2E_WEBSERVER+-|unset|Set to -+1+- for self-contained mode: serve this checkout via -+php -S+-
-+TIKI_E2E_PORT+-| -+8686+-|Port for the built-in web server in self-contained mode
-+TIKI_E2E_DB_*+-| -+tiki_e2e+- on -+127.0.0.1+-|Dedicated test DB host, name, user and password (self-contained mode)||
Override -+TIKI_FIXTURE_CMD+- to match your environment. Examples: -+php+- when running plain PHP from the Tiki root, -+ddev exec php+- for a DDEV setup, or -+docker exec my-container php /var/www/html+- for Docker.
The config auto-loads -+tests/e2e/.env+- if it exists. Variables already present in the environment are never overridden, so CI pipelines that inject variables directly are unaffected.
!! Test users
Most suites require two registered users plus an admin with known passwords:
||__User__|__Password__|__Role__
Test1| -+test1pass+-|Registered, primary tester
Test2| -+test2pass+-|Registered, secondary
admin| -+admin1234+-|Administrator||
Test1 and Test2 are created automatically by the global setup. To create or reset them manually, run from the Tiki root:
{CODE(colors="bash")}
php tests/e2e/fixtures/create_test_users.php
{CODE}
Adapt the PHP invocation to your environment: -+ddev exec php+-, -+docker exec+-, or plain -+php+-.
!! Automatic environment seeding
Before the whole session, -+common/global-setup.ts+-, wired through -+globalSetup+- in the Playwright config, runs two fixtures once so no manual admin steps are needed:
# -+create_test_users.php+- ensures Test1 and Test2 exist with known passwords.
# -+enable_encryption.php+- persists the __User Encryption__ security preference -+feature_user_encryption='y'+- in the database.
This is best-effort. If -+TIKI_FIXTURE_CMD+- is misconfigured, or you run the harness-only -+smoke/tc00+- spec with no live Tiki, the setup prints a clear warning and continues rather than aborting the run. Suites that genuinely need the seeded state then fail with their own descriptive errors. Run -+npx playwright test suites/smoke/tc01-environment.spec.ts+- to pinpoint what is missing.
!! Running the tests
Playwright and its config live in the repo-root -+package.json+-, so the simplest way to run the suite is via the root npm scripts, from the __Tiki root__:
{CODE(colors="bash")}
npm run test:e2e # all suites
npm run test:e2e:smoke # smoke suites only (framework + environment checks)
npm run test:e2e:report # open the last HTML report
{CODE}
Both reporters run simultaneously: -+list+- in the terminal and -+html+- written to -+tests/e2e/playwright-report/+-.
For filtered runs, call Playwright directly from -+tests/e2e/+-, where -+playwright.config.ts+- lives:
{CODE(colors="bash")}
cd tests/e2e
npx playwright test # all suites
npx playwright test shared-secrets/ # a single domain suite
npx playwright test wiki-pages/
npx playwright test shared-secrets/tc02-create-key.spec.ts # a single spec file
# Point at a non-default instance
TIKI_BASE_URL=https://tiki.example.com npx playwright test
{CODE}
!!! Interactive and debug modes
{CODE(colors="bash")}
# Open Playwright UI: browse, filter, and re-run tests interactively
TIKI_BASE_URL=https://tiki.example.com npx playwright test --ui
# Step through a failing test with Playwright Inspector
TIKI_BASE_URL=https://tiki.example.com npx playwright test --debug
{CODE}
!!! Viewing the HTML report
{CODE(colors="bash")}
cd tests/e2e
npx playwright show-report
{CODE}
This opens an interactive report at -+http://localhost:9323+- with pass/fail per test, plus traces, screenshots and error details.
!!! Overriding the reporter
To use only one reporter, overriding the config defaults:
{CODE(colors="bash")}
npx playwright test --reporter=list # terminal only
npx playwright test --reporter=html # HTML only
{CODE}
!! Directory structure
{CODE()}
tests/e2e/
├── bin/
│ ├── e2e-setup.sh Onboarding helper: .env + deps + smoke run
│ └── e2e-provision.sh Self-contained mode: dedicated test DB + clean install
├── common/ Shared utilities for all domain suites
│ ├── auth.ts login, logout, ADMIN, TEST_USER constants
│ ├── fixtures.ts FIXTURE_CMD, useSuiteFixtures (convention-based setup/teardown)
│ ├── global-setup.ts Runs once: seeds users + enables encryption
│ ├── test.ts Custom test fixture (BASE_PATH subpath prefixing)
│ └── ui.ts activateBootstrapTab (async Bootstrap tab activation)
├── fixtures/ PHP setup/teardown scripts for test data
│ ├── helpers.php Shared PHP helpers (bootstrap, key/tracker ops, PDO)
│ ├── create_test_users.php Global: ensures Test1/Test2 exist
│ ├── enable_encryption.php Global: persists feature_user_encryption='y'
│ ├── clean_baseline.php Maintenance: wipes all TC-prefixed keys/trackers
│ ├── provision_admin.php Provisioning: sets admin password + pass_confirm
│ ├── provision_defaults.php Provisioning: test-friendly prefs on fresh installs
│ └── shared-secrets/ Domain-scoped fixture scripts
│ ├── tc04_{setup,teardown}.php
│ ├── tc06_{setup,teardown}.php
│ ├── tc10_{setup,teardown}.php
│ └── tc11_{setup,teardown}.php
├── suites/ All runnable test domains (testDir scope)
│ ├── shared-secrets/ Domain: Shared Secrets / Tracker Encryption
│ │ ├── helpers.ts Domain-specific helpers (re-exports common/auth)
│ │ └── tc*.spec.ts
│ ├── wiki-pages/ Domain: Wiki page management
│ │ ├── helpers.ts Domain-specific helpers (re-exports common/auth)
│ │ └── tc*.spec.ts
│ └── smoke/ Domain: Framework smoke tests (subpath routing)
│ └── tc*.spec.ts
└── playwright.config.ts testDir "./suites": scoped to runnable suites only
{CODE}
!! Adding a new test domain
# Create a subdirectory: -+tests/e2e/suites/<domain-name>/+-
# Add -+helpers.ts+-, importing shared auth from -+../../common/auth+- and adding domain-specific helpers.
# Add -+tc*.spec.ts+- spec files.
# If PHP fixtures are needed, add a -+fixtures/<domain-name>/+- subfolder.
# To give the domain its own CI play button, add its name to the -+E2E_SUITE+- matrix list in -+.gitlab-ci.yml+-. That is the only CI change required.
!! Fixtures
PHP scripts in -+tests/e2e/fixtures/+- create and clean up the database objects a given test case needs.
Per-testcase fixtures are wired __by naming convention__. A spec never writes its own -+beforeAll+- / -+afterAll+- plumbing; it adds one line at the top:
{CODE(colors="typescript")}
import { useSuiteFixtures } from "../../common/fixtures";
const fx = useSuiteFixtures(test);
{CODE}
For a spec at -+suites/<suite>/tcNN-*.spec.ts+-, the runner looks for -+fixtures/<suite>/tcNN_setup.php+- and -+fixtures/<suite>/tcNN_teardown.php+- and runs whichever exist. The call is a no-op for specs without fixture scripts.
* __setup__ runs in -+beforeAll+- and must print a single JSON object on stdout, available to the spec as -+fx.data+- (for example -+fx.data.trackerId+-). If setup fails, the whole spec file is skipped with a warning pointing at -+TIKI_FIXTURE_CMD+-.
* __teardown__ runs in -+afterAll+- and receives the setup's JSON output __base64-encoded as its first CLI argument__, which is shell-safe through -+ddev exec+-, Docker or plain PHP, so it can read back the IDs setup created.
Most scripts -+require_once+- -+tiki-setup.php+- and must be invoked with the Tiki web root as the working directory. The exception is -+create_test_users.php+-, which connects to the database through PDO directly using -+db/local.php+-. It needs no Tiki bootstrap and works with plain PHP CLI.
||__Script__|__Used by__|__Purpose__
-+create_test_users.php+-|Global setup|Ensure Test1/Test2 exist with known passwords
-+enable_encryption.php+-|Global setup|Persist -+feature_user_encryption='y'+- in the DB, idempotent
-+clean_baseline.php+-|Maintenance|Wipe all TC-prefixed keys and trackers from the DB, for recovery after a killed run in existing-instance mode
-+provision_admin.php+-|Provisioning|Set admin password and -+pass_confirm+- on a fresh install, called by -+e2e-provision.sh+-
-+provision_defaults.php+-|Provisioning|Test-friendly prefs on a fresh install, for example no admin wizard, called by -+e2e-provision.sh+-
-+shared-secrets/tc04_setup.php+-|shared-secrets|Create tracker, encrypted field and key
-+shared-secrets/tc04_teardown.php+-|shared-secrets|Remove TC-04 tracker and key
-+shared-secrets/tc06_setup.php+-|shared-secrets|Create tracker and key, return Test1 share string
-+shared-secrets/tc06_teardown.php+-|shared-secrets|Remove TC-06 tracker and key
-+shared-secrets/tc10_setup.php+-|shared-secrets|Create tracker and key with share string
-+shared-secrets/tc10_teardown.php+-|shared-secrets|Remove TC-10 tracker and key
-+shared-secrets/tc11_setup.php+-|shared-secrets|Create key for the regeneration test
-+shared-secrets/tc11_teardown.php+-|shared-secrets|Remove TC-11 key||
Suites -+tc04+-, -+tc06+-, -+tc10+- and -+tc11+- are the ones that run PHP fixture scripts to seed the database.
!! Test suites
!!! -+shared-secrets/+-: Shared Secrets and Tracker Encryption
||__File__|__Coverage__
-+tc01-encryption-tab.spec.ts+-|Encryption tab visible and accessible
-+tc02-create-key.spec.ts+-|Key creation with one shared user
-+tc03-create-key-with-users.spec.ts+-|Key creation with shared users
-+tc04-tracker-field-encryption.spec.ts+-|Write and read an encrypted tracker field
-+tc05-edit-key.spec.ts+-|Edit key metadata: rename, description, duplicate guard
-+tc06-unlock-flow.spec.ts+-|Encrypted field unlock flow: share entry, session unlock
-+tc07-delete-key.spec.ts+-|Key deletion flow
-+tc08-security.spec.ts+-|Unauthorized access is blocked
-+tc09-algorithm.spec.ts+-|Algorithm dropdown, absent on 27.x and future-proofed
-+tc10-manual-key-entry.spec.ts+-|Manual share entry via the UI modal
-+tc11-key-regeneration.spec.ts+-|Key regeneration flow||
!!! -+wiki-pages/+-: Wiki Page Management
||__File__|__Coverage__
-+tc01-page-lifecycle.spec.ts+-|Create, view content and delete a wiki page
-+tc02-page-access.spec.ts+-|Page listing accessible, nonexistent page prompts creation||
!!! -+smoke/+-: Framework and Environment Smoke Tests
Fast checks run first to catch harness or environment problems before a domain suite does.
||__File__|__Needs live Tiki__|__Coverage__
-+tc00-subpath-routing.spec.ts+-|no| -+page.goto+- subpath prefixing: prepends -+BASE_PATH+- to leading-slash URLs, leaves absolute and relative paths untouched
-+tc01-environment.spec.ts+-|yes|Environment readiness: instance reachable, admin credentials log in, User Encryption feature enabled||
!! Self-Contained Mode
''Dedicated test DB plus built-in web server.''
By default the suite targets an existing Tiki instance through -+TIKI_BASE_URL+-. Self-contained mode instead provisions a __disposable Tiki in this checkout__: dedicated test database, scripted clean install, PHP's built-in web server. No pre-existing instance or web server is needed, and a development database can never be touched by a test run.
{CODE(colors="bash")}
# 1. Provision once (writes db/local.php, installs the schema, sets the admin password).
# Refuses to overwrite an existing db/local.php: use a fresh checkout or worktree.
tests/e2e/bin/e2e-provision.sh
# 2. Run. Playwright starts and stops the php -S server automatically:
cd tests/e2e && TIKI_E2E_WEBSERVER=1 npx playwright test
{CODE}
Configuration comes from the environment or -+tests/e2e/.env+-: -+TIKI_E2E_DB_HOST+-, -+TIKI_E2E_DB_NAME+-, -+TIKI_E2E_DB_USER+- and -+TIKI_E2E_DB_PASS+- select the test database, and -+TIKI_E2E_PORT+- the server port, default 8686. In this mode -+TIKI_BASE_URL+- and -+TIKI_FIXTURE_CMD+- are overridden: the spawned server and this checkout's host PHP are, by definition, the system under test.
Every run starts from a schema-fresh database. Re-run -+tests/e2e/bin/e2e-provision.sh --force+- to reset it. Tests therefore cannot leak state into, or depend on state from, a developer instance.
!! CI integration
The dedicated -+e2e-tests+- stage in -+.gitlab-ci.yml+- runs the suite fully self-contained, with no pre-existing Tiki instance involved. It contains one manual job per scope, all sharing the -+.e2e-base+- template:
||__Job__|__Runs__
-+e2e-all+-|Every suite under -+suites/+-
-+e2e-suite: [[smoke]+-|Framework and environment smoke checks
-+e2e-suite: [[shared-secrets]+-|The Shared Secrets domain
-+e2e-suite: [[wiki-pages]+-|The wiki-pages domain||
The per-suite jobs come from a -+parallel:matrix+- on the -+E2E_SUITE+- variable. After -+mkdir suites/<domain>/+-, adding the domain name to that matrix list is the only CI change needed to get a dedicated play button for it.
What each job does:
# __Image and services.__ The official Playwright image, Node plus browsers with the version pinned to the -+@playwright/test+- devDependency, plus a MariaDB service container aliased -+mysql+-.
# __Dependencies via -+needs+-.__ The job downloads artifacts from three build-stage jobs: -+composer+- for -+vendor_bundled/+-, -+node_modules+-, and -+node_build+- for the -+public/generated/+- JS and CSS bundles plus theme CSS. The -+node_build+- artifacts are essential: without them every page loads without Bootstrap JS and all tab and modal interactions time out.
# __PHP CLI.__ Installed via apt inside the job: -+php-cli+-, the DB, XML and intl extensions, and -+php-bcmath+-, which the Shamir secret-sharing library behind shared-secrets key creation requires.
# __Provision.__ -+bin/e2e-provision.sh+- writes -+db/local.php+- against the MariaDB service, installs the Tiki schema and sets the admin password. This is self-contained mode, with -+TIKI_E2E_WEBSERVER=1+-.
# __Run.__ -+npx playwright test+- serves the checkout with -+php -S+- through the Playwright -+webServer+- hook and runs the job's scope: everything for -+e2e-all+-, or the -+suites/${E2E_SUITE}/+- subtree for a matrix job. The PHP server's stderr, which is per-request log lines plus any PHP errors, is redirected to -+tests/e2e/php-server.log+-. Piped into the job output it exceeds GitLab's 4MB log limit and truncates the test results. The file is uploaded as an artifact, and an -+after_script+- greps it for PHP fatals so they still show in the job log.
All jobs are -+when: manual+- and -+allow_failure: true+- while runtimes are validated on the shared runners. Promote them to automatic once stable. To launch one, open the pipeline on the merge request and press the play button on the job in the -+e2e-tests+- stage.
!!! Accessing the HTML report artifact
Every job uploads -+tests/e2e/playwright-report/+- and -+tests/e2e/php-server.log+- as artifacts, on pass or fail, kept for 2 days. To view the report:
# Open the finished job page in GitLab.
# In the right sidebar under __Job artifacts__, click __Browse__.
# Navigate to -+tests/e2e/playwright-report/+- and click -+index.html+-. GitLab renders it directly, giving the interactive Playwright report with pass/fail per test, screenshots, videos and traces for failures.
Direct URL pattern:
{CODE()}
https://gitlab.com/tikiwiki/tiki/-/jobs/<job-id>/artifacts/file/tests/e2e/playwright-report/index.html
{CODE}
Alternatively, __Download__ the artifact zip and open -+tests/e2e/playwright-report/index.html+- locally, or unzip it and run -+npx playwright show-trace <path-to>/trace.zip+- to step through a failure.
!! Troubleshooting
||__Symptom__|__Likely cause and fix__
Global setup warns that fixtures could not run| -+TIKI_FIXTURE_CMD+- does not match your environment. Set it to -+php+-, -+ddev exec php+-, or your Docker equivalent.
A spec file is skipped with a fixture warning|Its -+tcNN_setup.php+- failed. Run the script manually from the Tiki root to see the PHP error.
Tabs or modals time out|The frontend bundles are missing. Run -+npm run build+- so -+public/generated/+- exists, otherwise Bootstrap JS never loads.
Key creation fails with a math error| -+bcmath+- is not installed. The Shamir secret-sharing library behind shared secrets requires it.
Leftover TC-prefixed keys or trackers after a killed run|Run -+php tests/e2e/fixtures/clean_baseline.php+- from the Tiki root, in existing-instance mode only.
-+e2e-provision.sh+- refuses to run|A -+db/local.php+- already exists. Use a fresh checkout or worktree, or pass -+--force+- to reset.||
!! Related pages
* ((Testing))
* ((Tiki-Unit-Testing))
* ((Tiki-Testing-with-Selenium))
* [https://playwright.dev/docs/intro|Playwright documentation]