.. _meson-test: Meson Test Template =================== The ``meson-test.yml`` template provides a standardized workflow for testing projects built with the `Meson build system `_ in a GitLab CI environment. It supports running tests, collecting coverage reports, while handling multiplatform and multi-toolchain scenarios. Overview -------- This template enables you to: - Run tests on Meson-built projects for various target platforms and toolchains using the multiplatform :ref:`oci-images`. - Generate test artifacts and JUnit reports for use in CI dashboards and summary stages. - Collect and publish coverage reports using `Gcovr `_ for supported environments using :ref:`gcovr-summary`. Usage in CI ----------- This template is included in CI pipelines (see :ref:`meson-test-examples`) to define jobs for each supported platform and toolchain. Each job uses the template to set up the test environment, run Meson tests, and optionally generate coverage reports. The template is designed to work with build artifacts produced by :ref:`meson-build`. .. _ci-matrix-limitations-meson-test: Parallel matrix limitations ^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. note:: Similar to :ref:`the build stage `, full parallel matrix support is limited due to outstanding issues in GitLab CI. Because of these limitations, the test stage for each target will wait until all toolchain builds are finished before starting. For example, if you build with both GNU and LLVM toolchains enabled, GitLab will wait for both builds to complete before running the tests for each toolchain separately. Inputs ------ The template accepts the following inputs: Target ^^^^^^ - ``target``: Build target in the form ``OS-TYPE-ARCH`` (e.g., ``linux-native-amd64``). This is similar to an OCI platform string but uses a hyphen instead of a slash, and includes a ``TYPE`` (either ``native`` or ``cross``). See :ref:`platform-naming` for details. - ``parallel_matrix``: Parallel matrix definition with toolchain, QEMU CPU configuration, and optionally other parameters. Used to create a job matrix for testing multiple configurations. Each entry should specify the toolchain(s) and QEMU CPU(s) to use for the target. Default: ``[{ TOOLCHAIN: [gnu, llvm], QEMU_CPU: [""] }]``. Meson ^^^^^ - ``meson_test_timeout_multiplier``: Test timeout multiplier flag used for Meson test execution. May need to be increased for slow targets. Default: ``20``. - ``meson_test_args``: Extra arguments passed directly to ``meson test``. Default: ``""``. - ``meson_testthreads``: Sets ``MESON_TESTTHREADS`` environment variable. For some platforms, tests should be executed one by one and without multithreading to prevent Gcovr errors. Default: ``0``. Gcovr ^^^^^ - ``gcovr_flags``: Additional flags passed to the Gcovr tool for coverage report preprocessing. Default: ``""``. CI Job ^^^^^^ - ``allow_failure``: Set the ``allow_failure`` flag for jobs expected to fail. Set ``retry`` input to 0 to prevent unnecessary retries. Default: ``false``. - ``build_job_name_suffix``: Suffix for the build job name (set in ``job_name_suffix`` for the build job). Used to create job dependency (note: :ref:`ci-matrix-limitations-meson-test`). Default: ``""``. - ``extends``: Array of jobs to extend this job template. Must include an OCI image template (see :ref:`top-level-reqs`) and can include target activation pattern. Default: ``[.ci-multiplatform-base]``. - ``job_name_prefix``: Prefix for the job name. Useful for disabling or customizing jobs. Default: ``""``. - ``job_name_suffix``: Suffix for the job name. Can be used to prevent job duplication for jobs for the same target. Default: ``""``. - ``runner_tag``: GitLab runner tag for this job. Default: ``""``. - ``stage``: CI stage name for the job. Default: ``test``. - ``timeout``: GitLab job timeout property. May need to be increased for slow targets. Default: ``1h``. Job Structure ------------- Each job created by the template: - Depends on the corresponding build job (see :ref:`meson-build`). - Sets up the test environment and variables, including build and coverage directories. - Runs ``meson test`` with configurable timeout and arguments. - Optionally generates coverage reports using Gcovr if coverage was enabled during the build stage. - Publishes test logs, coverage reports, and JUnit XML reports as CI artifacts. Coverage Reporting ------------------ If coverage was enabled in the build stage (see ``enable_gnu_coverage`` in :ref:`meson-build`), the test job will generate coverage reports using Gcovr. Reports are stored in the ``coverage`` directory and include JSON and HTML formats, as well as a summary printed to the job log. .. _meson-test-examples: Examples -------- Below are example job definitions using the template syntax: .. code-block:: yaml spec: inputs: ci_path: description: Path to the ci-multiplatform meson-test component with branch name. type: string default: gitlab.com/riseproject/ci/ci-multiplatform/meson-test@main runner_tag_qemu: description: Regular x86 runner with QEMU support. type: string default: "saas-linux-medium-amd64" --- include: # Native x86_64 target (no QEMU) with custom env variable. - component: $[[ inputs.ci_path ]] inputs: target: linux-native-amd64 runner_tag: $[[ inputs.runner_tag_qemu ]] parallel_matrix: - TOOLCHAIN: [gnu, llvm] PIXMAN_DISABLE: ["", "fast", "wholeops"] # PowerPC 64-bit target with QEMU_CPU. - component: $[[ inputs.ci_path ]] inputs: target: linux-native-ppc64le runner_tag: $[[ inputs.runner_tag_qemu ]] parallel_matrix: - TOOLCHAIN: [gnu, llvm] QEMU_CPU: [power10] # RISC-V 64-bit target with advanced QEMU_CPU RVV VLEN settings. - component: $[[ inputs.ci_path ]] inputs: target: linux-native-riscv64 runner_tag: $[[ inputs.runner_tag_qemu ]] parallel_matrix: - TOOLCHAIN: [gnu, llvm] QEMU_CPU: # Test on target without RVV (verify no autovectorization). - rv64,v=false # Test correctness for different VLENs. - rv64,v=true,vext_spec=v1.0,vlen=128,elen=64 - rv64,v=true,vext_spec=v1.0,vlen=256,elen=64 - rv64,v=true,vext_spec=v1.0,vlen=512,elen=64 - rv64,v=true,vext_spec=v1.0,vlen=1024,elen=64 Basic target with a custom variable ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This use case demonstrates testing a native x86_64 target without QEMU emulation. The job runs tests for both GNU and LLVM toolchains and allows customization of environment variables, such as ``PIXMAN_DISABLE``. The custom variable can be used by the test suite to enable or disable specific features, control test behavior, or pass configuration options. For example, setting ``PIXMAN_DISABLE=fast`` disables certain optimizations during testing for Pixman. This flexibility allows you to tailor the test environment to match your project's requirements or to reproduce specific scenarios. Specifying ``QEMU_CPU`` ^^^^^^^^^^^^^^^^^^^^^^^ For platforms like PowerPC 64-bit (ppc64le), QEMU emulation is used to run tests. The job specifies the ``QEMU_CPU`` parameter (e.g., ``power10``) to control the CPU model used during emulation. This enables testing on architectures that are not natively available on CI runners. Advanced ``QEMU_CPU`` usage ^^^^^^^^^^^^^^^^^^^^^^^^^^^ This example targets RISC-V 64-bit platforms and leverages advanced ``QEMU_CPU`` settings to test different RISC-V Vector (RVV) vector register length (VLEN) configurations. By specifying various ``VLEN`` values, the job verifies correctness and behavior across multiple vector lengths. This is useful for projects that need to validate functionality on a wide range of emulated hardware features. Full Example ^^^^^^^^^^^^ For a full example, see :ref:`examples-meson`. Pixman ^^^^^^ Another example is in the Pixman project in :pixman_url:`.gitlab-ci.d/03-test.yml <-/blob/ci-external/.gitlab-ci.d/03-test.yml>` file.