Meson Test Template

The meson-test.yml template provides a standardized workflow for testing projects built with the Meson build system in a GitLab CI environment. It supports running tests, collecting coverage reports, while handling multiplatform and multi-toolchain scenarios.

Overview

This template enables you to:

  • Run tests on Meson-built projects for various target platforms and toolchains using the multiplatform OCI Images.

  • Generate test artifacts and JUnit reports for use in CI dashboards and summary stages.

  • Collect and publish coverage reports using Gcovr for supported environments using Gcovr Summary Template.

Usage in CI

This template is included in CI pipelines (see Examples) to define jobs for each supported platform and toolchain. Each job uses the template to set up the test environment, run Meson tests, and optionally generate coverage reports. The template is designed to work with build artifacts produced by Meson Build Template.

Parallel matrix limitations

Note

Similar to the build stage, full parallel matrix support is limited due to outstanding issues in GitLab CI.

Because of these limitations, the test stage for each target will wait until all toolchain builds are finished before starting. For example, if you build with both GNU and LLVM toolchains enabled, GitLab will wait for both builds to complete before running the tests for each toolchain separately.

Inputs

The template accepts the following inputs:

Target

  • target: Build target in the form OS-TYPE-ARCH (e.g., linux-native-amd64). This is similar to an OCI platform string but uses a hyphen instead of a slash, and includes a TYPE (either native or cross). See Platform Naming for details.

  • parallel_matrix: Parallel matrix definition with toolchain, QEMU CPU configuration, and optionally other parameters. Used to create a job matrix for testing multiple configurations. Each entry should specify the toolchain(s) and QEMU CPU(s) to use for the target. Default: [{ TOOLCHAIN: [gnu, llvm], QEMU_CPU: [""] }].

Meson

  • meson_test_timeout_multiplier: Test timeout multiplier flag used for Meson test execution. May need to be increased for slow targets. Default: 20.

  • meson_test_args: Extra arguments passed directly to meson test. Default: "".

  • meson_testthreads: Sets MESON_TESTTHREADS environment variable. For some platforms, tests should be executed one by one and without multithreading to prevent Gcovr errors. Default: 0.

Gcovr

  • gcovr_flags: Additional flags passed to the Gcovr tool for coverage report preprocessing. Default: "".

CI Job

  • allow_failure: Set the allow_failure flag for jobs expected to fail. Set retry input to 0 to prevent unnecessary retries. Default: false.

  • build_job_name_suffix: Suffix for the build job name (set in job_name_suffix for the build job). Used to create job dependency (note: Parallel matrix limitations). Default: "".

  • extends: Array of jobs to extend this job template. Must include an OCI image template (see Top-level Requirements) and can include target activation pattern. Default: [.ci-multiplatform-base].

  • job_name_prefix: Prefix for the job name. Useful for disabling or customizing jobs. Default: "".

  • job_name_suffix: Suffix for the job name. Can be used to prevent job duplication for jobs for the same target. Default: "".

  • runner_tag: GitLab runner tag for this job. Default: "".

  • stage: CI stage name for the job. Default: test.

  • timeout: GitLab job timeout property. May need to be increased for slow targets. Default: 1h.

Job Structure

Each job created by the template:

  • Depends on the corresponding build job (see Meson Build Template).

  • Sets up the test environment and variables, including build and coverage directories.

  • Runs meson test with configurable timeout and arguments.

  • Optionally generates coverage reports using Gcovr if coverage was enabled during the build stage.

  • Publishes test logs, coverage reports, and JUnit XML reports as CI artifacts.

Coverage Reporting

If coverage was enabled in the build stage (see enable_gnu_coverage in Meson Build Template), the test job will generate coverage reports using Gcovr. Reports are stored in the coverage directory and include JSON and HTML formats, as well as a summary printed to the job log.

Examples

Below are example job definitions using the template syntax:

spec:
  inputs:
    ci_path:
      description:
        Path to the ci-multiplatform meson-test component with branch name.
      type: string
      default: gitlab.com/riseproject/ci/ci-multiplatform/meson-test@main
    runner_tag_qemu:
      description: Regular x86 runner with QEMU support.
      type: string
      default: "saas-linux-medium-amd64"
---

include:
  # Native x86_64 target (no QEMU) with custom env variable.
  - component: $[[ inputs.ci_path ]]
    inputs:
      target: linux-native-amd64
      runner_tag: $[[ inputs.runner_tag_qemu ]]
      parallel_matrix:
        - TOOLCHAIN: [gnu, llvm]
          PIXMAN_DISABLE: ["", "fast", "wholeops"]

# PowerPC 64-bit target with QEMU_CPU.
- component: $[[ inputs.ci_path ]]
  inputs:
    target: linux-native-ppc64le
    runner_tag: $[[ inputs.runner_tag_qemu ]]
    parallel_matrix:
      - TOOLCHAIN: [gnu, llvm]
        QEMU_CPU: [power10]

# RISC-V 64-bit target with advanced QEMU_CPU RVV VLEN settings.
- component: $[[ inputs.ci_path ]]
  inputs:
    target: linux-native-riscv64
    runner_tag: $[[ inputs.runner_tag_qemu ]]
    parallel_matrix:
      - TOOLCHAIN: [gnu, llvm]
        QEMU_CPU:
          # Test on target without RVV (verify no autovectorization).
          - rv64,v=false
          # Test correctness for different VLENs.
          - rv64,v=true,vext_spec=v1.0,vlen=128,elen=64
          - rv64,v=true,vext_spec=v1.0,vlen=256,elen=64
          - rv64,v=true,vext_spec=v1.0,vlen=512,elen=64
          - rv64,v=true,vext_spec=v1.0,vlen=1024,elen=64

Basic target with a custom variable

This use case demonstrates testing a native x86_64 target without QEMU emulation. The job runs tests for both GNU and LLVM toolchains and allows customization of environment variables, such as PIXMAN_DISABLE.

The custom variable can be used by the test suite to enable or disable specific features, control test behavior, or pass configuration options. For example, setting PIXMAN_DISABLE=fast disables certain optimizations during testing for Pixman. This flexibility allows you to tailor the test environment to match your project’s requirements or to reproduce specific scenarios.

Specifying QEMU_CPU

For platforms like PowerPC 64-bit (ppc64le), QEMU emulation is used to run tests. The job specifies the QEMU_CPU parameter (e.g., power10) to control the CPU model used during emulation. This enables testing on architectures that are not natively available on CI runners.

Advanced QEMU_CPU usage

This example targets RISC-V 64-bit platforms and leverages advanced QEMU_CPU settings to test different RISC-V Vector (RVV) vector register length (VLEN) configurations. By specifying various VLEN values, the job verifies correctness and behavior across multiple vector lengths. This is useful for projects that need to validate functionality on a wide range of emulated hardware features.

Full Example

For a full example, see Meson Project Example.

Pixman

Another example is in the Pixman project in .gitlab-ci.d/03-test.yml file.