What tests do I run to package in distro?

[Ashvith10]

I am trying to package the next-gen version of QuickJS in Guix - which may replace the older, unmaintained version or exist as it's own new package - that, I'll have to ask the Guix maintainers.

Usually the packages build out of the box without any issues just by setting the build system (in this case, cmake-build-system), but given how CTest wasn't used (hope that I am not wrong here), I wish to understand - as a package maintainer, what relevant tests should I run to ensure that the package expression ensures reproducible builds?

Is make test enough? Because in my system, excluding test, all the other test rule in Makefile fails (test262, test262-check, test262-fast and test262-update).

Also relevant to the earlier point - is it okay if I do not pull submodules (test262) when trying to build?

[saghul]

If you want to be thorough, the test262 suite should be run. There is a fast variant, but even the "slow" one is fast enough now, thanks to the multi-threading enhancements Ben made.

[Ashvith10]

Yes @saghul . From what I see, even the CI for ubuntu-latest (Release) has failures similar to the above log (which unfortunately, got buried in the logs above, but I'll add it below for comparision).

Result: 83/78324 errors, 3772 excluded, 5387 skipped, 83 new

It seems to be passing because the workflow runs time make test262 and not make test262 and the correct return type is not captured.

Do I still add test262? Or should tests simply suffice, along with the standalone and api test? I also noticed that microbench is missing from the test?

[saghul]

They test different things. What you want to test for is kinda up to you. In general your results shouldn't differ from ours. Barring some very weird environment problems.

Good catch on time, I'll take a look!

[Ashvith10]

@saghul Since the upstream does not check for tests, I guess I can enable test262 for the next version when it is fixed, or else the package won't be accessible, right?

Another issue (or maybe not) that I came across was at line 211 of test_std.js - which assumes /bin/sh, but for the sake of non-FHS compatibility (ahem, Guix, NixOS, Gobo), as well as not hardcoding it to /bin/sh, /usr/bin/env sh may be used. In my case, I have patched it with (which "sh"), which is a part of Guile/Guix.

I am also not sure if this is a issue or not, but when -DBUILD_SHARED_LIBS is set to TRUE, -DQJS_BUILD_LIBC:BOOL=TRUE fails - I can only build it as a static library. For reference, here's all the flags I am using:

(list
  ...
 #:configure-flags
      #~(list "-DBUILD_SHARED_LIBS:BOOL=TRUE"
              "-DQJS_BUILD_EXAMPLES:BOOL=TRUE"
              "-DQJS_BUILD_CLI_WITH_MIMALLOC:BOOL=TRUE"
              "-DQJS_ENABLE_ASAN:BOOL=TRUE"
              "-DQJS_ENABLE_UBSAN:BOOL=TRUE"
              ;; MSAN and TSAN is perhaps not supported on gcc at the moment?
              "-DQJS_ENABLE_MSAN:BOOL=FALSE"
              "-DQJS_ENABLE_TSAN:BOOL=FALSE"
              ;; False because of linker issues
              "-DQJS_BUILD_LIBC:BOOL=FALSE")
...
)

Here's the log for the same:

ld: CMakeFiles/function_source.dir/gen/function_source.c.o: in function `main':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/function_source.c:66:(.text.startup+0x20): undefined reference to `js_std_set_worker_new_context_func'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/function_source.c:67:(.text.startup+0x28): undefined reference to `js_std_init_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/function_source.c:68:(.text.startup+0x2f): undefined reference to `js_module_loader'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/function_source.c:70:(.text.startup+0x52): undefined reference to `js_std_add_helpers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/function_source.c:71:(.text.startup+0x66): undefined reference to `js_std_eval_binary'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/function_source.c:72:(.text.startup+0x6e): undefined reference to `js_std_loop'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/function_source.c:76:(.text.startup+0x7d): undefined reference to `js_std_free_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/function_source.c:74:(.text.startup+0xa2): undefined reference to `js_std_dump_error'
ld: CMakeFiles/hello.dir/gen/hello.c.o: in function `main':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello.c:38:(.text.startup+0x20): undefined reference to `js_std_set_worker_new_context_func'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello.c:39:(.text.startup+0x28): undefined reference to `js_std_init_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello.c:40:(.text.startup+0x2f): undefined reference to `js_module_loader'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello.c:42:(.text.startup+0x52): undefined reference to `js_std_add_helpers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello.c:43:(.text.startup+0x66): undefined reference to `js_std_eval_binary'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello.c:44:(.text.startup+0x6e): undefined reference to `js_std_loop'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello.c:48:(.text.startup+0x7d): undefined reference to `js_std_free_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello.c:46:(.text.startup+0xa2): undefined reference to `js_std_dump_error'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/function_source.dir/build.make:101: function_source] Error 1
make[2]: Leaving directory '/tmp/guix-build-quickjs-0.9.0.drv-0/build'
make[1]: *** [CMakeFiles/Makefile2:263: CMakeFiles/function_source.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/hello.dir/build.make:101: hello] Error 1
make[2]: Leaving directory '/tmp/guix-build-quickjs-0.9.0.drv-0/build'
make[1]: *** [CMakeFiles/Makefile2:289: CMakeFiles/hello.dir/all] Error 2
ld: CMakeFiles/hello_module.dir/gen/hello_module.c.o: in function `JS_NewCustomContext':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello_module.c:81:(.text+0x2a): undefined reference to `js_std_eval_binary'
ld: CMakeFiles/hello_module.dir/gen/hello_module.c.o: in function `main':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello_module.c:92:(.text.startup+0x20): undefined reference to `js_std_set_worker_new_context_func'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello_module.c:93:(.text.startup+0x28): undefined reference to `js_std_init_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello_module.c:94:(.text.startup+0x2f): undefined reference to `js_module_loader'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello_module.c:96:(.text.startup+0x52): undefined reference to `js_std_add_helpers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello_module.c:97:(.text.startup+0x66): undefined reference to `js_std_eval_binary'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello_module.c:98:(.text.startup+0x6e): undefined reference to `js_std_loop'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello_module.c:102:(.text.startup+0x7d): undefined reference to `js_std_free_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/hello_module.c:100:(.text.startup+0xa2): undefined reference to `js_std_dump_error'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/hello_module.dir/build.make:101: hello_module] Error 1
make[2]: Leaving directory '/tmp/guix-build-quickjs-0.9.0.drv-0/build'
make[1]: *** [CMakeFiles/Makefile2:315: CMakeFiles/hello_module.dir/all] Error 2
[ 86%] Linking C executable test_fib
/gnu/store/2lxfijiqqljxdxr2ppqqrn5czkh4r1rq-cmake-minimal-3.24.2/bin/cmake -E cmake_link_script CMakeFiles/test_fib.dir/link.txt --verbose=1
/gnu/store/86fc8bi3mciljxz7c79jx8zr4wsx7xw8-gcc-11.4.0/bin/gcc -O2 -g -DNDEBUG -fsanitize=address -fno-sanitize-recover=all -fno-omit-frame-pointer -fsanitize=undefined CMakeFiles/test_fib.dir/examples/fib.c.o CMakeFiles/test_fib.dir/gen/test_fib.c.o -o test_fib  -Wl,-rpath,/tmp/guix-build-quickjs-0.9.0.drv-0/build libqjs.so.0.9.0 -ldl -lm 
[ 90%] Linking C executable api-test
/gnu/store/2lxfijiqqljxdxr2ppqqrn5czkh4r1rq-cmake-minimal-3.24.2/bin/cmake -E cmake_link_script CMakeFiles/api-test.dir/link.txt --verbose=1
/gnu/store/86fc8bi3mciljxz7c79jx8zr4wsx7xw8-gcc-11.4.0/bin/gcc -O2 -g -DNDEBUG -fsanitize=address -fno-sanitize-recover=all -fno-omit-frame-pointer -fsanitize=undefined "CMakeFiles/api-test.dir/api-test.c.o" -o api-test  -Wl,-rpath,/tmp/guix-build-quickjs-0.9.0.drv-0/build libqjs.so.0.9.0 -ldl -lm 
ld: CMakeFiles/test_fib.dir/gen/test_fib.c.o: in function `JS_NewCustomContext':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/test_fib.c:54:(.text+0x20): undefined reference to `js_init_module_os'
ld: CMakeFiles/test_fib.dir/gen/test_fib.c.o: in function `main':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/test_fib.c:66:(.text.startup+0x20): undefined reference to `js_std_set_worker_new_context_func'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/test_fib.c:67:(.text.startup+0x28): undefined reference to `js_std_init_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/test_fib.c:68:(.text.startup+0x2f): undefined reference to `js_module_loader'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/test_fib.c:70:(.text.startup+0x52): undefined reference to `js_std_add_helpers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/test_fib.c:71:(.text.startup+0x66): undefined reference to `js_std_eval_binary'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/test_fib.c:72:(.text.startup+0x6e): undefined reference to `js_std_loop'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/test_fib.c:76:(.text.startup+0x7d): undefined reference to `js_std_free_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/gen/test_fib.c:74:(.text.startup+0xa2): undefined reference to `js_std_dump_error'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/test_fib.dir/build.make:117: test_fib] Error 1
make[2]: Leaving directory '/tmp/guix-build-quickjs-0.9.0.drv-0/build'
make[1]: *** [CMakeFiles/Makefile2:393: CMakeFiles/test_fib.dir/all] Error 2
make[2]: Leaving directory '/tmp/guix-build-quickjs-0.9.0.drv-0/build'
[ 90%] Built target api-test
[ 93%] Linking C executable qjsc
/gnu/store/2lxfijiqqljxdxr2ppqqrn5czkh4r1rq-cmake-minimal-3.24.2/bin/cmake -E cmake_link_script CMakeFiles/qjsc.dir/link.txt --verbose=1
/gnu/store/86fc8bi3mciljxz7c79jx8zr4wsx7xw8-gcc-11.4.0/bin/gcc -O2 -g -DNDEBUG -fsanitize=address -fno-sanitize-recover=all -fno-omit-frame-pointer -fsanitize=undefined CMakeFiles/qjsc.dir/qjsc.c.o -o qjsc  -Wl,-rpath,/tmp/guix-build-quickjs-0.9.0.drv-0/build:::::::::::::::::::: libqjs.so.0.9.0 -ldl -lm 
ld: CMakeFiles/qjsc.dir/qjsc.c.o: in function `output_object_code':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/qjsc.c:175:(.text+0x850): undefined reference to `js_std_dump_error'
ld: CMakeFiles/qjsc.dir/qjsc.c.o: in function `jsc_module_loader':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/qjsc.c:247:(.text+0xd51): undefined reference to `js_load_file'
ld: CMakeFiles/qjsc.dir/qjsc.c.o: in function `compile_file':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/qjsc.c:285:(.text.startup+0x60b): undefined reference to `js_load_file'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjsc.c:301:(.text.startup+0xf60): undefined reference to `js_std_dump_error'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/qjsc.dir/build.make:101: qjsc] Error 1
make[2]: Leaving directory '/tmp/guix-build-quickjs-0.9.0.drv-0/build'
make[1]: *** [CMakeFiles/Makefile2:133: CMakeFiles/qjsc.dir/all] Error 2
[ 96%] Linking C executable qjs
/gnu/store/2lxfijiqqljxdxr2ppqqrn5czkh4r1rq-cmake-minimal-3.24.2/bin/cmake -E cmake_link_script CMakeFiles/qjs_exe.dir/link.txt --verbose=1
/gnu/store/86fc8bi3mciljxz7c79jx8zr4wsx7xw8-gcc-11.4.0/bin/gcc -O2 -g -DNDEBUG -Wl,--export-dynamic -rdynamic -fsanitize=address -fno-sanitize-recover=all -fno-omit-frame-pointer -fsanitize=undefined CMakeFiles/qjs_exe.dir/gen/repl.c.o CMakeFiles/qjs_exe.dir/gen/standalone.c.o CMakeFiles/qjs_exe.dir/qjs.c.o -o qjs  -Wl,-rpath,/tmp/guix-build-quickjs-0.9.0.drv-0/build:::::::::::::::::::: libqjs.so.0.9.0 /gnu/store/9ypfhkz6f1bgjq95817m4989qfd68j84-mimalloc-2.1.2/lib/libmimalloc.so.2.1 -ldl -lm 
ld: CMakeFiles/qjs_exe.dir/qjs.c.o: in function `load_standalone_module':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:90:(.text+0xae): undefined reference to `js_module_set_import_meta'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:95:(.text+0xd6): undefined reference to `js_std_await'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:100:(.text+0x12a): undefined reference to `js_std_dump_error'
ld: CMakeFiles/qjs_exe.dir/qjs.c.o: in function `eval_buf':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:126:(.text+0xc4d): undefined reference to `js_module_set_import_meta'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:133:(.text+0xc73): undefined reference to `js_std_await'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:138:(.text+0xca2): undefined reference to `js_std_dump_error'
ld: CMakeFiles/qjs_exe.dir/qjs.c.o: in function `eval_file':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:154:(.text+0xd3d): undefined reference to `js_load_file'
ld: CMakeFiles/qjs_exe.dir/qjs.c.o: in function `JS_NewCustomContext':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:233:(.text+0xfeb): undefined reference to `js_init_module_std'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:234:(.text+0xff8): undefined reference to `js_init_module_os'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:235:(.text+0x1005): undefined reference to `js_init_module_bjson'
ld: CMakeFiles/qjs_exe.dir/qjs.c.o: in function `main':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:614:(.text.startup+0x840): undefined reference to `js_std_set_worker_new_context_func'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:615:(.text.startup+0x848): undefined reference to `js_std_init_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:623:(.text.startup+0x863): undefined reference to `js_module_loader'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:626:(.text.startup+0x874): undefined reference to `js_std_promise_rejection_tracker'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:713:(.text.startup+0x8a4): undefined reference to `js_std_free_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:713:(.text.startup+0xda2): undefined reference to `js_std_free_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:629:(.text.startup+0x120f): undefined reference to `js_std_add_helpers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:743:(.text.startup+0x1361): undefined reference to `js_std_free_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:690:(.text.startup+0x17ec): undefined reference to `js_std_eval_binary'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:697:(.text.startup+0x1817): undefined reference to `js_std_loop'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:713:(.text.startup+0x1907): undefined reference to `js_std_free_handlers'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:690:(.text.startup+0x1b23): undefined reference to `js_std_eval_binary'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:700:(.text.startup+0x1b2b): undefined reference to `js_std_loop'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/qjs.c:703:(.text.startup+0x1b3b): undefined reference to `js_std_dump_error'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/qjs_exe.dir/build.make:134: qjs] Error 1
make[2]: Leaving directory '/tmp/guix-build-quickjs-0.9.0.drv-0/build'
make[1]: *** [CMakeFiles/Makefile2:159: CMakeFiles/qjs_exe.dir/all] Error 2
[100%] Linking C executable run-test262
/gnu/store/2lxfijiqqljxdxr2ppqqrn5czkh4r1rq-cmake-minimal-3.24.2/bin/cmake -E cmake_link_script CMakeFiles/run-test262.dir/link.txt --verbose=1
/gnu/store/86fc8bi3mciljxz7c79jx8zr4wsx7xw8-gcc-11.4.0/bin/gcc -O2 -g -DNDEBUG -fsanitize=address -fno-sanitize-recover=all -fno-omit-frame-pointer -fsanitize=undefined "CMakeFiles/run-test262.dir/run-test262.c.o" -o run-test262  -Wl,-rpath,/tmp/guix-build-quickjs-0.9.0.drv-0/build libqjs.so.0.9.0 -ldl -lm 
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `JS_NewCustomContext':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:1747:(.text+0x49): undefined reference to `js_init_module_std'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:1748:(.text+0x56): undefined reference to `js_init_module_os'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:1749:(.text+0x63): undefined reference to `js_init_module_bjson'
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `js_module_loader_test':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:1004:(.text+0x5bc): undefined reference to `js_load_file'
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `agent_start':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:653:(.text+0x2a94): undefined reference to `js_std_dump_error'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:680:(.text+0x2b2c): undefined reference to `js_std_dump_error'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:646:(.text+0x2b4e): undefined reference to `js_std_dump_error'
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `eval_buf':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:1569:(.text+0xab9c): undefined reference to `js_std_loop'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:1571:(.text+0xabaf): undefined reference to `js_std_dump_error'
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `run_test_buf':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:1768:(.text+0xd43f): undefined reference to `js_std_init_handlers'
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `load_file':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:975:(.text+0xd5e5): undefined reference to `js_load_file'
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `run_test_buf':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:1807:(.text+0xd978): undefined reference to `js_std_free_handlers'
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `JS_NewCustomContext':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:1747:(.text+0xd9dc): undefined reference to `js_init_module_std'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:1748:(.text+0xd9e9): undefined reference to `js_init_module_os'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:1749:(.text+0xd9f6): undefined reference to `js_init_module_bjson'
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `load_file':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:975:(.text+0xdcbe): undefined reference to `js_load_file'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:975:(.text+0xed29): undefined reference to `js_load_file'
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `run_test262_harness_test':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:2051:(.text+0xee35): undefined reference to `js_std_dump_error'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:2062:(.text+0xef64): undefined reference to `js_std_dump_error'
ld: /tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:2038:(.text+0xef84): undefined reference to `js_std_dump_error'
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `main':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:2169:(.text.startup+0xb2): undefined reference to `js_std_set_worker_new_context_func'
ld: CMakeFiles/run-test262.dir/run-test262.c.o: in function `load_file':
/tmp/guix-build-quickjs-0.9.0.drv-0/source/run-test262.c:975:(.text.startup+0x4a9): undefined reference to `js_load_file'
collect2: error: ld returned 1 exit status

[saghul]

Since the upstream does not check for tests, I guess I can enable test262 for the next version when it is fixed, or else the package won't be accessible, right?

Sure.

Another issue (or maybe not) that I came across was at line 211 of test_std.js - which assumes /bin/sh, but for the sake of non-FHS compatibility (ahem, Guix, NixOS, Gobo), as well as not hardcoding it to /bin/sh, /usr/bin/env sh may be used. In my case, I have patched it with (which "sh"), which is a part of Guile/Guix.

I'd say a PR changing that would be welcome.

I am also not sure if this is a issue or not, but when -DBUILD_SHARED_LIBS is set to TRUE, -DQJS_BUILD_LIBC:BOOL=TRUE fails - I can only build it as a static library. For reference, here's all the flags I am using:

What type of build do you want to make? An end user build?

[Ashvith10]

What type of build do you want to make? An end user build?

In my case, I am trying to write a Guix package expression so that quickjs can be built in a reproducible manner in an isolated environment. So basically, a package that can be used by other packages. Apart from packages already dependent on quickjs, I also have plans to pack txiki.js as well sometimes later. I also have plans to switch from nodejs-bootstrap (v10.24.1) to quickjs to build llhttp, but I don't have a lot of knowledge with regards to that.

[saghul]

I tested make test262 locally and it passes:

make test262
cmake --build build -j 10
-- Building in Debug mode
-- Building with AppleClang 16.0.0.16000026 on Darwin-23.6.0
-- Performing Test COMPILER_SUPPORTS_Wformat=2
-- Performing Test COMPILER_SUPPORTS_Wformat=2 - Success
-- Performing Test COMPILER_SUPPORTS_/Od
-- Performing Test COMPILER_SUPPORTS_/Od - Failed
-- Performing Test COMPILER_SUPPORTS_O0
-- Performing Test COMPILER_SUPPORTS_O0 - Success
--   BUILD_SHARED_LIBS: OFF
--   QJS_BUILD_EXAMPLES: OFF
--   QJS_BUILD_CLI_STATIC: OFF
--   QJS_BUILD_CLI_WITH_MIMALLOC: OFF
--   QJS_BUILD_CLI_WITH_STATIC_MIMALLOC: OFF
--   QJS_ENABLE_ASAN: OFF
--   QJS_ENABLE_MSAN: OFF
--   QJS_ENABLE_TSAN: OFF
--   QJS_ENABLE_UBSAN: OFF
--   QJS_BUILD_LIBC: OFF
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/saghul/src/quickjs/build
[  9%] Building C object CMakeFiles/qjs.dir/cutils.c.o
[  9%] Building C object CMakeFiles/qjs.dir/libbf.c.o
[ 13%] Building C object CMakeFiles/qjs.dir/libregexp.c.o
[ 18%] Building C object CMakeFiles/qjs.dir/quickjs.c.o
[ 22%] Building C object CMakeFiles/qjs.dir/libunicode.c.o
[ 27%] Linking C static library libqjs.a
[ 27%] Built target qjs
[ 36%] Building C object CMakeFiles/api-test.dir/api-test.c.o
[ 36%] Building C object CMakeFiles/function_source.dir/gen/function_source.c.o
[ 40%] Building C object CMakeFiles/run-test262.dir/run-test262.c.o
[ 45%] Building C object CMakeFiles/qjsc.dir/qjsc.c.o
[ 54%] Building C object CMakeFiles/qjs_exe.dir/gen/standalone.c.o
[ 63%] Building C object CMakeFiles/qjs_exe.dir/qjs.c.o
[ 63%] Building C object CMakeFiles/qjsc.dir/quickjs-libc.c.o
[ 63%] Building C object CMakeFiles/function_source.dir/quickjs-libc.c.o
[ 72%] Building C object CMakeFiles/run-test262.dir/quickjs-libc.c.o
[ 72%] Building C object CMakeFiles/qjs_exe.dir/gen/repl.c.o
[ 77%] Building C object CMakeFiles/qjs_exe.dir/quickjs-libc.c.o
[ 81%] Linking C executable api-test
[ 81%] Built target api-test
[ 86%] Linking C executable function_source
[ 90%] Linking C executable run-test262
[ 95%] Linking C executable qjsc
[100%] Linking C executable qjs
[100%] Built target function_source
[100%] Built target run-test262
[100%] Built target qjsc
[100%] Built target qjs_exe
build/run-test262 -m -c test262.conf -a
82/79087/5386
Average memory statistics for 79396 tests:

QuickJS-ng memory usage -- 0.9.0 version, 64-bit, Little Endian, malloc limit: 0

NAME                    COUNT     SIZE
memory allocated         1135   156802  (138.2 per block)
memory used              1146   148223  (0 overhead, 7.5 average slack)
atoms                     607    38797  (63.9 per atom)
objects                   199    14354  (72.1 per object)
  properties             1011    17837  (5.1 per object)
  shapes                   71    15330  (215.9 per shape)
bytecode functions         14     7094
  bytecode                 14     1097  (78.4 per function)
  pc2line                  13      240  (18.5 per function)
C functions               118
arrays                      1
  fast arrays               1
  elements                  2       32  (2.0 per fast array)

Result: 83/79396 errors, 3236 excluded, 5387 skipped

The exit code is 0 regardless of the last line, since we have some expected test failures. I don't think the time call is relevant but I'll remove it.

[Ashvith10]

What if the command was run directly, without using make? Does that make a difference? In my case, it says that status 256 was returned:

error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "./run-test262" arguments: ("-m" "-c" "../source/test262.conf" "-c" "../source/test262-fast.conf" "-a") exit-status: 1 term-signal: #f stop-signal: #f> 
phase `check' failed after 195.4 seconds
command "./run-test262" "-m" "-c" "../source/test262.conf" "-c" "../source/test262-fast.conf" "-a" failed with status 1
build process 10 exited with status 256
note: keeping build directory `/tmp/guix-build-quickjs-0.9.0.drv-49'

[saghul]

build/run-test262 -m -c test262.conf -a
82/79290/5387
Average memory statistics for 79396 tests:

QuickJS-ng memory usage -- 0.9.0 version, 64-bit, Little Endian, malloc limit: 0

NAME                    COUNT     SIZE
memory allocated         1135   156802  (138.2 per block)
memory used              1146   148223  (0 overhead, 7.5 average slack)
atoms                     607    38797  (63.9 per atom)
objects                   199    14354  (72.1 per object)
  properties             1011    17837  (5.1 per object)
  shapes                   71    15330  (215.9 per shape)
bytecode functions         14     7094
  bytecode                 14     1097  (78.4 per function)
  pc2line                  13      240  (18.5 per function)
C functions               118
arrays                      1
  fast arrays               1
  elements                  2       32  (2.0 per fast array)

Result: 83/79396 errors, 3236 excluded, 5387 skipped

saghul  …/quickjs   ci-no-time  
❯ echo $?
0

I suspect something may not be friendly with the paths. I'd suggest you run the test suite while on the quickjs root directory.

[Ashvith10]

I think I figured it out - this may have to do with:

# name of the error file for known errors
errorfile=test262_errors.txt

# exclude tests enumerated in this file (see also [exclude] section)
#excludefile=test262_exclude.txt

Since now they're in the store, the new locations may have to be patched, which I'll know after substituting. However, one thing I've noticed is that there's no such file as test262_exclude.txt in the quickjs repo?

[saghul]

We have them in the main file, but having a separate file is supported.

[coreyapowell]

I noticed you guys don't provide the test262.patch. The original fork shows that test262.patch was updated 2 weeks ago, but non262.js on the test262 site was updated last week.
So I'd like to know what you recommend, does this need to stay current every time this is executed? To package in distro?

[saghul]

I'd recommend running the suite on the same commit that is pinned here.

[bnoordhuis]

And re: test262.patch, it's not needed with quickjs-ng.