Problem: test_codestyle.vim takes too much time and takes more than
90s, this contributes to the overall time for all runners and
Solution: Create a dedicated 'make codestyle' target in the testdir
Makefiles. Remove test_codestyle from the main test list.
Update GitHub CI to run this check as a separate step
in the 'normal' features build.
Increase CI timeout to 45 minutes.
fixes: #19740
Signed-off-by: Christian Brabandt <cb@256bit.org>
The input files with syntax test cases are never compiled or
interpreted on behalf of test runners, just read, and their
parts are rendered in accordance with syntax definitions for
associated languages, to be compared with previously vetted
renderings. Whether their arbitrary contents will be valid
programs, benign programs, etc., is left for test authors to
decide and verify in their own environments. As executable
and non-executable files equally qualify for testing and yet
executability is never exercised, and since maintaining
executable files turns out to be a short-lived exception
than common practice, let us persist in keeping syntax files
non-executable and enforce it with a CI check.
closes: #19433
Co-authored-by: Christian Brabandt <cb@256bit.org>
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: When running tests on a source tree checked out with git for
Windows, the old tests fail.
The Git for Windows installer installs git with core.autocrlf=true by
default. If you check out, build, and run tests using such a git, the
old test .ok files will likely fail because they use CRLF line endings.
Tests on Windows assume that .ok files use LF line endings, and
appropriately convert the line endings of related files. This
assumption breaks down when .ok files use CRLF.
Solution: Force LF line endings for old test .ok files in the
.gitattributes file. Related to that, we've stopped explicitly
specifying line endings when checking out in CI, since this is no longer
necessary.
While at it, also fix a typo in the comment of the gitattributes file.
closes: #19086
Signed-off-by: Muraoka Taro <koron.kaoriya@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Current CI Windows testing does not test Vim using ConPTY.
Solution: Added a configuration to enable ConPTY to the Windows matrix.
This configuration suppresses the installation of winpty,
allowing testing with ConPTY.
Since the fixes made at the end of last year, all tests now pass when
using ConPTY, just like when using winpty. ConPTY itself is stable on
recent versions of Windows, so there are no longer any reasons not to
test it.
closes: #19066
Signed-off-by: Muraoka Taro <koron.kaoriya@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Generating prototype files does not work on all platforms
Solution: Rework prototypes generation using python instead of cproto,
enable it in CI to test it for each PR (Hirohito Higashi).
closes: #18045
Signed-off-by: Hirohito Higashi <h.east.727@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
> The macOS 13 runner image will be retired by December 4th, 2025.
Update to the macos-15-intel runner.
It seems that runners ending with "large" require an enterprise plan, so
macos-15-intel is the only other available macOS Intel runner.
Also both macOS 15 runners now use Xcode 16.4 by default, so there is no
need to install it manually.
closes: #18471
Signed-off-by: zeertzjq <zeertzjq@outlook.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: CI is not run with clang 21
Solution: Update CI to v21, fix a few warnings that are uncovered when
using clang 21 (Philip H)
closes: #18142
Signed-off-by: Philip H. <47042125+pheiduck@users.noreply.github.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
It causes CI to fail because the netbeans package has been moved
related: tunisiano187/Chocolatey-packages#3916
related: #17631
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: missing Wayland clipboard support
Solution: make it work (Foxe Chen)
fixes: #5157closes: #17097
Signed-off-by: Foxe Chen <chen.foxe@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: CI Tests favor GTK2 over GTK3
Solution: Install GTK3 dependencies and debug packages for CI workflows,
update ASAN suppression list, update required dependency
checks for the tests (Drew Vogel)
closes: #17253
Signed-off-by: Drew Vogel <dvogel@github>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: MS-Windows: Not possible to pass additional flags to Make_mvc
Solution: Introduce $CI_FLAGS and use it to pass additional flags for
the Github CI in order to treat size conversion warnings
(C4267) as errors (Yegappan Lakshmanan)
closes: #17028
Signed-off-by: Yegappan Lakshmanan <yegappan@yahoo.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Vim tests for features such as python3 relies on checking the feature
flag exists by doing `has('python3')`. However, if the feature itself is
broken and the flag returns 0, the relevant tests will simply silently
get ignored and CI will passed erroneously. As a preventive measure, as
basic checks to make sure certain feature flags are correct as a basic
smoke test.
Currently only checking two types of feature flags:
1. Features that depend on system packages being installed properly
(e.g. sodium) and could be erroneously dropped if the CI environment
changed or a bug exists in the configure script.
2. Scripting languages. When in dynamic mode, these feature flags (e.g.
"ruby", "python3") will return 0 when the lib cannot be found or the
code has an initialization bug. This happened in #16964 where CI
still passed despite Python 3 being broken.
closes: #16998
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
When adding new highlight groups, one needs to make sure to also add a
"default link NewHlGroup ExistingHlGroup" in highlight.c code, so that
when resetting a color scheme the old color won't be left behind.
So add a Makefile in the 'ci' directory that verifies that all
documented '*hl-<groupname>' from the documentation are either reflected
in the source code, or belong to a list of 'known to be ignored'
highlight groups and let that check run as part of the CI test suite.
related: #16676closes: #16678
Signed-off-by: Christ van Willegen <cvwillegen@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: CI: uses Ubuntu 22.04 runners
Solution: Switch to Ubuntu 24.04 runners, make a few adjustments for
different $TMPDIR (Drew Vogel)
closes: #16442
Signed-off-by: Drew Vogel <dvogel@github>
Signed-off-by: Christian Brabandt <cb@256bit.org>
This patch adds jobs to run tests on arm64 GitHub Actions hosted runner
for tiny and huge features.
closes: #16477
Signed-off-by: rhysd <lin90162@yahoo.co.jp>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Apparently even when loading the snd-dummy kernel module, the
CI tests for the sounds feature in test_sound.vim are already skipped.
So let's just remove all of this (even so we may loose a bit of coverage
information here).
closes: #16391
Signed-off-by: Philip H. <47042125+pheiduck@users.noreply.github.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Make sure to use the latest Xcode (16.2) which contains fixes in the
compiler, as GitHub Actions defaults to using 16.0 still. See #15764.
closes: #16239
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: CI: tests can be improved
Solution: collect failed indent tests, raise timeout for search()
functions when using ASAN/Valgrind (Aliaksei Budavei)
ASan-instrumented Vim builds tend to run slower (x2) than
non-instrumented Vim builds and occasionally make indent
tests fail when "search*()" functions time out and give up
further execution.
Reference:
https://github.com/google/sanitizers/wiki/AddressSanitizercloses: #15974
Co-authored-by: Christian Brabandt <cb@256bit.org>
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: tests: Mac OS tests are too flaky
Solution: Increase max test timeout to 25 minutes,
allow up to 10 retries on Mac OS runners,
refactor runtest.vim (Milly).
closes: #15940
Co-authored-by: K.Takata <kentkt@csc.jp>
Signed-off-by: Milly <milly.ca@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: tests: not clear what tests cause asan failures
Solution: append testname to $ASAN_OPTIONS
Mention what test causes ASAN failures by appending the testname
to log_path in $ASAN_OPTIONS/$UBSAN_OPTIONS. This assumes 'log_path' is
always the last sub-option in $ASAN_OPTIONS.
While at it, also make the CI run with `-O0` instead of `-O1` when ASAN
is enable since this causes line numbers to disappear.
closes: #15927
Signed-off-by: Christian Brabandt <cb@256bit.org>
