Categories: OpenJDK

Implementing SpinPause on Windows AArch64

I reported a gtest failure in [JDK-8374735] Implement SpinPause on Windows AArch64 – Java Bug System a few weeks ago and took a stab (ha) at implementing SpinPause() on openjdk/jdk at 27dbdec297fc8030812f7290a7601b6a99defb46. I could see no reason why the Linux AArch64 implementation couldn’t be used on the Windows AArch64 platform. Here is one caller of SpinPause on Windows-AArch64:

jvm.dll!SpinPause() Line 296	C++
jvm.dll!ObjectMonitor::short_fixed_spin(JavaThread * current, int spin_count, bool adapt) Line 2379	C++
jvm.dll!ObjectMonitor::try_spin(JavaThread * current) Line 2401	C++
jvm.dll!ObjectMonitor::spin_enter(JavaThread * current) Line 493	C++
jvm.dll!ObjectMonitor::enter(JavaThread * current) Line 506	C++
jvm.dll!ObjectSynchronizer::inflate_and_enter(oopDesc * object, BasicLock * lock, ObjectSynchronizer::InflateCause cause, JavaThread * locking_thread, JavaThread * current) Line 2149	C++
jvm.dll!ObjectSynchronizer::enter(Handle obj, BasicLock * lock, JavaThread * current) Line 1846	C++
jvm.dll!ObjectLocker::ObjectLocker(Handle obj, JavaThread * __the_thread__) Line 477	C++
jvm.dll!InstanceKlass::link_class_impl(JavaThread * __the_thread__) Line 1018	C++
jvm.dll!InstanceKlass::link_class(JavaThread * __the_thread__) Line 933	C++
jvm.dll!InstanceKlass::initialize_impl(JavaThread * __the_thread__) Line 1233	C++
jvm.dll!InstanceKlass::initialize_preemptable(JavaThread * __the_thread__) Line 838	C++
jvm.dll!InterpreterRuntime::_new(JavaThread * current, ConstantPool * pool, int index) Line 222	C++
0000021f7b427cc0()	Unknown

Another is the MacroAssembler:

jvm.dll!MacroAssembler::spin_wait() Line 6649	C++
jvm.dll!StubGenerator::generate_spin_wait() Line 10270	C++
jvm.dll!StubGenerator::generate_final_stubs() Line 11788	C++
jvm.dll!StubGenerator::StubGenerator(CodeBuffer * code, BlobId blob_id) Line 11980	C++
jvm.dll!StubGenerator_generate(CodeBuffer * code, BlobId blob_id) Line 11991	C++
jvm.dll!initialize_stubs(BlobId blob_id, int code_size, int max_aligned_stubs, const char * timer_msg, const char * buffer_name, const char * assert_msg) Line 190	C++
jvm.dll!StubRoutines::initialize_final_stubs() Line 233	C++
jvm.dll!final_stubs_init() Line 243	C++
jvm.dll!init_globals2() Line 205	C++
jvm.dll!Threads::create_vm(JavaVMInitArgs * args, bool * canTryAgain) Line 622	C++
jvm.dll!JNI_CreateJavaVM_inner(JavaVM_ * * vm, void * * penv, void * args) Line 3621	C++
jvm.dll!JNI_CreateJavaVM(JavaVM_ * * vm, void * * penv, void * args) Line 3712	C++
jli.dll!InitializeJVM(const JNIInvokeInterface_ * * * pvm, const JNINativeInterface_ * * * penv, InvocationFunctions * ifn) Line 1506	C
jli.dll!JavaMain(void * _args) Line 494	C
jli.dll!ThreadJavaMain(void * args) Line 632	C
ucrtbase.dll!00007ffba234b028()	Unknown

This is another spot where the MacroAssembler::spin_wait() method is called:

jvm.dll!MacroAssembler::spin_wait() Line 6649	C++
jvm.dll!onspinwaitNode::emit(C2_MacroAssembler * masm, PhaseRegAlloc * ra_) Line 22285	C++
jvm.dll!PhaseOutput::scratch_emit_size(const Node * n) Line 3150	C++
jvm.dll!MachNode::emit_size(PhaseRegAlloc * ra_) Line 157	C++
jvm.dll!MachNode::size(PhaseRegAlloc * ra_) Line 149	C++
jvm.dll!PhaseOutput::shorten_branches(unsigned int * blk_starts) Line 528	C++
jvm.dll!PhaseOutput::Output() Line 330	C++
jvm.dll!Compile::Code_Gen() Line 3137	C++
jvm.dll!Compile::Compile(ciEnv * ci_env, ciMethod * target, int osr_bci, Options options, DirectiveSet * directive) Line 896	C++
jvm.dll!C2Compiler::compile_method(ciEnv * env, ciMethod * target, int entry_bci, bool install_code, DirectiveSet * directive) Line 147	C++
jvm.dll!CompileBroker::invoke_compiler_on_method(CompileTask * task) Line 2348	C++
jvm.dll!CompileBroker::compiler_thread_loop() Line 1990	C++
jvm.dll!CompilerThread::thread_entry(JavaThread * thread, JavaThread * __the_thread__) Line 69	C++
jvm.dll!JavaThread::thread_main_inner() Line 777	C++
jvm.dll!JavaThread::run() Line 761	C++
jvm.dll!Thread::call_run() Line 242	C++
jvm.dll!thread_native_entry(void * t) Line 565	C++
ucrtbase.dll!00007ffba234b028()	Unknown

Below are some example commands showing how to which assembly instruction to use in the SpinWait() call and how many of them should be used. On my Surface Pro X, the SB instruction is not supported. A good post to read about various barrier instructions is The AArch64 processor (aka arm64), part 14: Barriers – The Old New Thing.

$ $JDKTOTEST/bin/java -Xcomp -XX:-TieredCompilation -XX:+UnlockDiagnosticVMOptions -XX:OnSpinWaitInst=sb ProducerConsumerLoops
Error occurred during initialization of VM
OnSpinWaitInst is SB but current CPU does not support SB instruction

$ $JDKTOTEST/bin/java -Xcomp -XX:-TieredCompilation -XX:+UnlockDiagnosticVMOptions -XX:OnSpinWaitInst=nop -XX:OnSpinWaitInstCount=5 ProducerConsumerLoops

The snippet below shows the 4 instructions in the spin_wait stub if the isb instruction is selected with a count of 3 (after copying the Linux SpinPause implementation). Whether or not this is a good idea is not the point, this is about showing what the flags do.

000001800B9D0700  isb         sy  
000001800B9D0704  isb         sy  
000001800B9D0708  isb         sy  
000001800B9D070C  ret 

extern "C" {
  int SpinPause() {
00007FFAB97A5CF8  stp         fp,lr,[sp,#-0x20]!  
00007FFAB97A5CFC  mov         fp,sp  
    using spin_wait_func_ptr_t = void (*)();
    spin_wait_func_ptr_t func = CAST_TO_FN_PTR(spin_wait_func_ptr_t, StubRoutines::aarch64::spin_wait());
00007FFAB97A5D00  bl          StubRoutines::aarch64::spin_wait (07FFAB97A63B8h)+#0xFFFF8005DA859DF6  
00007FFAB97A5D04  mov         x8,x0  
00007FFAB97A5D08  str         x8,[sp,#0x10]  
    assert(func != nullptr, "StubRoutines::aarch64::spin_wait must not be null.");
00007FFAB97A5D0C  mov         w8,#0  
00007FFAB97A5D10  cmp         w8,#0  
00007FFAB97A5D14  bne         SpinPause+34h (07FFAB97A5D2Ch)  
00007FFAB97A5D18  bl          DebuggingContext::is_enabled (07FFAB8619D18h)+#0xFFFF8005DF5832E8  
00007FFAB97A5D1C  uxtb        w8,w0  
00007FFAB97A5D20  mov         w8,w8  
00007FFAB97A5D24  cmp         w8,#0  
00007FFAB97A5D28  bne         SpinPause+74h (07FFAB97A5D6Ch)  
00007FFAB97A5D2C  ldr         x8,[sp,#0x10]  
00007FFAB97A5D30  cmp         x8,#0  
00007FFAB97A5D34  bne         SpinPause+74h (07FFAB97A5D6Ch)  
00007FFAB97A5D38  adrp        x8,g_assert_poison (07FFABAA80F88h)+#0xFFFF800635588740  
00007FFAB97A5D3C  ldr         x9,[x8,g_assert_poison (07FFABAA80F88h)+#0xFFFF80063E9FB581]  
00007FFAB97A5D40  mov         w8,#0x58  
00007FFAB97A5D44  strb        w8,[x9]  
00007FFAB97A5D48  adrp        x8,siglabels+690h (07FFABA5C5000h)  
00007FFAB97A5D4C  add         x3,x8,#0x450  
00007FFAB97A5D50  adrp        x8,siglabels+690h (07FFABA5C5000h)  
00007FFAB97A5D54  add         x2,x8,#0x488  
00007FFAB97A5D58  mov         w1,#0x128  
00007FFAB97A5D5C  adrp        x8,siglabels+690h (07FFABA5C5000h)  
00007FFAB97A5D60  add         x0,x8,#0x4B0  
00007FFAB97A5D64  bl          report_vm_error (07FFAB8D15210h)+#0xFFFF8005DF046B1B  
00007FFAB97A5D68  nop  
00007FFAB97A5D6C  mov         w8,#0  
00007FFAB97A5D70  cmp         w8,#0  
00007FFAB97A5D74  bne         SpinPause+14h (07FFAB97A5D0Ch)  
    (*func)();
00007FFAB97A5D78  ldr         x8,[sp,#0x10]  
00007FFAB97A5D7C  blr         x8  
    // If StubRoutines::aarch64::spin_wait consists of only a RET,
    // SpinPause can be considered implemented. There will be a sequence
    // of instructions for:
    // - call of SpinPause
    // - load of StubRoutines::aarch64::spin_wait stub pointer
    // - indirect call of the stub
    // - return from the stub
    // - return from SpinPause
    // So '1' always is returned.
    return 1;
00007FFAB97A5D80  mov         w0,#1  
00007FFAB97A5D84  ldp         fp,lr,[sp],#0x20  
00007FFAB97A5D88  ret  
00007FFAB97A5D8C  ?? ?????? 
} 

SpinPause is also used by the G1 collector as shown in the callstack below:

jvm.dll!SpinPause() Line 307	C++
jvm.dll!TaskTerminator::DelayContext::do_step() Line 61	C++
jvm.dll!TaskTerminator::offer_termination(TerminatorTerminator * terminator) Line 166	C++
jvm.dll!TaskTerminator::offer_termination() Line 105	C++
jvm.dll!G1ParEvacuateFollowersClosure::offer_termination() Line 601	C++
jvm.dll!G1ParEvacuateFollowersClosure::do_void() Line 626	C++
jvm.dll!G1EvacuateRegionsBaseTask::evacuate_live_objects(G1ParScanThreadState * pss, unsigned int worker_id, G1GCPhaseTimes::GCParPhases objcopy_phase, G1GCPhaseTimes::GCParPhases termination_phase) Line 673	C++
jvm.dll!G1EvacuateRegionsTask::evacuate_live_objects(G1ParScanThreadState * pss, unsigned int worker_id) Line 762	C++
jvm.dll!G1EvacuateRegionsBaseTask::work(unsigned int worker_id) Line 730	C++
jvm.dll!WorkerTaskDispatcher::worker_run_task() Line 73	C++
jvm.dll!WorkerThread::run() Line 200	C++
jvm.dll!Thread::call_run() Line 242	C++
jvm.dll!thread_native_entry(void * t) Line 565	C++
ucrtbase.dll!00007ffba234b028()	Unknown

Outstanding Challenges

How do I get full stacks in Visual Studio? We need to integrate jstack’s functionality into the VS debugger.


Categories: OpenJDK

Building OpenJDK 18 for Windows AArch64

I am investigating a (Windows AArch64) test that passes on jdk17u but fails on jdk21u. I need to bisect to the first commit with a release build that fails this test. Unfortunately, Latest Releases | Adoptium doesn’t have Windows AArch64 builds between jdk17u and jdk21u so I had to build them myself. To find the commits for the intermediate releases, I used this command:

git show-ref --tags

The output was of this form:

...
26bac24088faab1c3c82223c27f45dcea1462f87 refs/tags/jdk-17-ga
cba5cf5c2f83de5352ccf3fd3b247f0381daae43 refs/tags/jdk-18+0
cf839a043ff13762a5d1671e42132e5407a9a7bf refs/tags/jdk-18+1
...

This let me identify the tag I needed on GitHub: openjdk/jdk at jdk-18-ga. I set up the jdk18u build option in my personal build script: Add config options for more jdk versions · swesonga/scratchpad@98ab982 and set up google/googletest at v1.14.0. Unfortunately, I got compilation errors stating that ‘FLAGS_gtest_internal_run_death_test’: is not a member of ‘testing::internal’. There were also errors about an identifier that could not be found in winnt.h. Here’s how I addressed them.

Identifier not found in winnt.h

There were errors in “C:\Program Files (x86)\Windows Kits\10\Include\10.0.26100.0\um\winnt.h” stating that the _CountOneBits64 identifier could not be found:

ERROR: Build failed for target 'images' in configuration 'windows-aarch64-server-release' (exit code 2) 
Stopping sjavac server

=== Output from failing command(s) repeated here ===
* For target hotspot_variant-server_libjvm_gtest_launcher-objs_gtestLauncher.obj:
gtestLauncher.cpp
c:\progra~2\wi3cf2~1\10\include\100261~1.0\um\winnt.h(6343): error C3861: '_CountOneBits64': identifier not found
   ... (rest of output omitted)
* For target hotspot_variant-server_libjvm_gtest_objs_BUILD_GTEST_LIBJVM_pch.obj:
BUILD_GTEST_LIBJVM_pch.cpp
c:\progra~2\wi3cf2~1\10\include\100261~1.0\um\winnt.h(6343): error C3861: '_CountOneBits64': identifier not found
   ... (rest of output omitted)
* For target hotspot_variant-server_libjvm_libgtest_objs_gtest-all.obj:
gtest-all.cc
c:\progra~2\wi3cf2~1\10\include\100261~1.0\um\winnt.h(6343): error C3861: '_CountOneBits64': identifier not found
   ... (rest of output omitted)

Based on “identifier not found” while compiling for ARM64 with Visual Studio 2019 – Stack Overflow and MSVC ARM build error with winnt.h · Issue #11684 · actions/runner-images, the likely problem was that the latest SDK was being used instead of one of the older SDKs. These are the SDKs that were on my machine:

user@machine MINGW64 /c/Program Files (x86)/Windows Kits/10/Include
$ ls -1
10.0.19041.0/
10.0.22621.0/
10.0.26100.0/

The Visual Studio 2019 installer showed only the Windows 10 SDK installed:

I set off on a journey of discovery, seeking to learn how the SDK is selected for the build.

VS_ENV_CMD is set in the TOOLCHAIN_CHECK_POSSIBLE_VISUAL_STUDIO_ROOT macro in toolchain_microsoft.m4 to the 64-bit path in the TOOLCHAIN_CHECK_POSSIBLE_VISUAL_STUDIO_ROOT macro. This path on my machine is “C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsamd64_arm64.bat“. It contains this line: @call "%~dp0vcvarsall.bat" x64_arm64 %*. This command calls the vcvarsall.bat file in the same directory. “build\windows-aarch64-server-release\configure-support\config.log” doesn’t seem to have any SDK-related output. vcvarsall.bat in turn calls “C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\VsDevCmd.bat“. I don’t think it passes the -winsdk argument. VsDevCmd.bat then calls “C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\vsdevcmd\core\winsdk.bat“, which has a GetWin10SdkDir function that is called if the VSCMD_ARG_WINSDK environment variable has not been set. GetWin10SdkDirHelper queries these locations:

  1. HKLM\SOFTWARE\Wow6432Node
  2. HKCU\SOFTWARE\Wow6432Node
  3. HKLM\SOFTWARE
  4. HKCU\SOFTWARE

More specifically, it searches in the Microsoft\Microsoft SDKs\Windows\v10.0 subkey of each of them for the InstallationFolder value. The WindowsSdkDir environment variable is set to the value found here. It would be set to “C:\Program Files (x86)\Windows Kits\10\” on my machine (shown below).

I tried setting the environment variable before configuring the build but that didn’t work:

export VSCMD_ARG_WINSDK=10.0.22621.0

The configure output from this section of the TOOLCHAIN_EXTRACT_VISUAL_STUDIO_ENV macro is shown below:

  AC_MSG_NOTICE([Trying to extract Visual Studio environment variables for $TARGET_CPU])
  AC_MSG_NOTICE([using $VS_ENV_CMD $VS_ENV_ARGS])
configure: Using default toolchain microsoft (Microsoft Visual Studio)
configure: Found Visual Studio installation at /cygdrive/c/progra~2/micros~3/2019/Enterprise using well-known name
configure: Found Microsoft Visual Studio 2019
configure: Trying to extract Visual Studio environment variables for aarch64
configure: using /cygdrive/c/progra~2/micros~3/2019/Enterprise/vc/auxiliary/build/vcvarsamd64_arm64.bat
configure: Setting extracted environment variables for aarch64

This shows that no arguments are passed to vcvarsamd64_arm64.bat (and therefore to vcvarsall.bat as well). Since vcvarsall.bat has logic that parses 10.* strings into the __VCVARSALL_WINSDK variable (to pass on to VsDevCmd.bat), I realized that I could just specify the SDK version when calling vcvarsamd64_arm64.bat. I used this diff (on commit 0f2113cee79):

diff --git a/make/autoconf/toolchain_microsoft.m4 b/make/autoconf/toolchain_microsoft.m4
index 2600b431cfb..a7d6aaae250 100644
--- a/make/autoconf/toolchain_microsoft.m4
+++ b/make/autoconf/toolchain_microsoft.m4
@@ -349,7 +349,7 @@ AC_DEFUN([TOOLCHAIN_EXTRACT_VISUAL_STUDIO_ENV],
   # We can't pass -vcvars_ver=$VCVARS_VER here because cmd.exe eats all '='
   # in bat file arguments. :-(
   $FIXPATH $CMD /c "$TOPDIR/make/scripts/extract-vs-env.cmd" "$VS_ENV_CMD" \
-      "$VS_ENV_TMP_DIR/set-vs-env.sh" $VCVARS_VER $VS_ENV_ARGS \
+      "$VS_ENV_TMP_DIR/set-vs-env.sh" $VCVARS_VER $VS_ENV_ARGS 10.0.22621.0 \
       > $VS_ENV_TMP_DIR/extract-vs-env.log | $CAT 2>&1
   PATH="$OLDPATH"

This enabled the build to use the SDK version I specified.

gtest undeclared identifier Error

The remaining build failures were related to gtests:

ERROR: Build failed for target 'images' in configuration 'windows-aarch64-server-release' (exit code 2) 

=== Output from failing command(s) repeated here ===
* For target buildjdk_hotspot_variant-server_libjvm_gtest_objs_gtestMain.obj:
gtestMain.cpp
d:\java\forks\openjdk\jdk\test\hotspot\gtest\gtestMain.cpp(233): error C2039: 'FLAGS_gtest_internal_run_death_test': is not a member of 'testing::internal'
d:\repos\googletest\googlemock\include\gmock/gmock-nice-strict.h(80): note: see declaration of 'testing::internal'
d:\java\forks\openjdk\jdk\test\hotspot\gtest\gtestMain.cpp(233): error C2065: 'FLAGS_gtest_internal_run_death_test': undeclared identifier
   ... (rest of output omitted)
* For target hotspot_variant-server_libjvm_gtest_objs_gtestMain.obj:
gtestMain.cpp
d:\java\forks\openjdk\jdk\test\hotspot\gtest\gtestMain.cpp(233): error C2039: 'FLAGS_gtest_internal_run_death_test': is not a member of 'testing::internal'
d:\repos\googletest\googlemock\include\gmock/gmock-nice-strict.h(80): note: see declaration of 'testing::internal'
d:\java\forks\openjdk\jdk\test\hotspot\gtest\gtestMain.cpp(233): error C2065: 'FLAGS_gtest_internal_run_death_test': undeclared identifier
   ... (rest of output omitted)

* All command lines available in /cygdrive/d/java/forks/openjdk/jdk/build/windows-aarch64-server-release/make-support/failure-logs.
=== End of repeated output ===

I was using google/googletest at v1.13.0 but the openjdk/jdk repo moved to it in 8300806: Update googletest to v1.13.0 · openjdk/jdk@ccf2f58, which is after jdk18u (the failing source code is in jdk18u/test/hotspot/gtest/gtestMain.cpp at master · openjdk/jdk18u). Per [JDK-8300806] Update googletest to v1.13.0 – Java Bug System, this change was backported to jdk17u, which explains why I might have been confused about this. So I really should have been using the google/googletest at release-1.8.1 tag to build jdk-18-ga. This switch finally enabled me to successfully cross-compile the jdk-18-ga sources to get a Windows AArch64 release build.


Categories: Assembly, OpenJDK

OpenJDK SafeFetch Implementation on AArch64

I changed Windows AArch64 exception handling in 8348862: runtime/ErrorHandling/CreateCoredumpOnCrash fails on Windows aarch64 by swesonga · Pull Request #27074 · openjdk/jdk. In the process, I discovered that the safefetch implementation on Windows AArch64 was broken. SafeFetchXX is a function that takes a pointer and an error value and tries to read the value at the pointer. If it is read successfully, that value is returned, otherwise, the error value is returned.

Before my changes, safefetch.hpp included safefetch_windows.hpp, which uses structured exception handling. The read is done in a __try { } __except block. However, the Windows AArch64 port uses vectored exception handling. This is therefore not the right approach. I added the !defined(_M_ARM64) check to ensure that safefetch_static.hpp is included instead. This requires us to implement SafeFetch32_impl and SafeFetchN_impl, the same way the Linux and macosx AArch64 implementation do. These functions are declared as extern C because they will be implemented in assembly, specifically in safefetch_windows_aarch64.S. Here’s the implementation of SafeFetchN_impl (copied to match the other 2 AArch64 platforms):

    ; Support for intptr_t SafeFetchN(intptr_t* address, intptr_t defaultval);
    ;
    ;  x0 : address
    ;  x1 : defaultval

    ALIGN  4
    EXPORT _SafeFetchN_fault
    EXPORT _SafeFetchN_continuation
    EXPORT SafeFetchN_impl

SafeFetchN_impl
_SafeFetchN_fault
    ldr      x0, [x0]
    ret

_SafeFetchN_continuation
    mov      x0, x1
    ret

    END

Notice that it is a 4 assembly instructions function. The ldr instruction tries to dereference the pointer in x0. If the memory access succeeds, the function returns the loaded value successfully. Otherwise, the exception handler will be invoked. The exception handling logic checks whether the exception being handled was caused by the safefetch load. This is where the _SafeFetchN_fault label comes into play. If the exception is an EXCEPTION_ACCESS_VIOLATION, we can check whether the PC was at the _SafeFetchN_fault (the ldr) instruction. If so, the exception handler sets the PC in the OS CONTEXT structure to the _SafeFetchN_continuation instruction. The exception handler then returns EXCEPTION_CONTINUE_EXECUTION to allow execution to resume successfully at the mov instruction, which simply loads x0 with the error value that was passed in x1. The 32-bit safefetch function has an identical structure.


Categories: Assembly, OpenJDK

Selecting the Proper OpenJDK Windows AArch64 Assembler

A few months ago, I was investigating some exception handling OpenJDK bugs on Windows AArch64. One of the bugs was in the safefetch implementation. I needed to switch part of the implementation to assembly language (similar to the Linux and macosx aarch64 safefetch implementations). Compilation failed after I added the new safefetch_windows_aarch64.S assembly source file. The failing command line was in the .cmdline file when the build terminated:

From build\windows-aarch64-server-slowdebug\make-support\failure-logs\hotspot_variant-server_libjvm_objs_safefetch_windows_aarch64.obj.cmdline

/cygdrive/c/java/forks/dups2/openjdk/jdk/build/windows-aarch64-server-slowdebug/fixpath exec /cygdrive/c/progra~1/micros~1/2022/enterprise/vc/tools/msvc/14.44.35207/bin/hostarm64/arm64/ml64.exe -nologo -c -I/cygdrive/c/java/forks/dups2/openjdk/jdk/src/hotspot/os_cpu/windows_aarch64 -I/cygdrive/c/java/forks/dups2/openjdk/jdk/src/hotspot/os/windows -I/cygdrive/c/java/forks/dups2/openjdk/jdk/src/hotspot/os/windows -Fo/cygdrive/c/java/forks/dups2/openjdk/jdk/build/windows-aarch64-server-slowdebug/hotspot/variant-server/libjvm/objs/safefetch_windows_aarch64.obj -Ta /cygdrive/c/java/forks/dups2/openjdk/jdk/src/hotspot/os_cpu/windows_aarch64/safefetch_windows_aarch64.S

I could manually reproduce the failure by running the command in a developer command prompt:

C:\dev\temp> ml64 -nologo -Ta safefetch_windows_aarch64.S
 Assembling: safefetch_windows_aarch64.S
safefetch_windows_aarch64.S(31) : error A2008:syntax error : .
safefetch_windows_aarch64.S(32) : error A2008:syntax error : .
safefetch_windows_aarch64.S(33) : error A2008:syntax error : .type
safefetch_windows_aarch64.S(34) : error A2034:must be in segment block
safefetch_windows_aarch64.S(36) : error A2034:must be in segment block
safefetch_windows_aarch64.S(37) : error A2034:must be in segment block
safefetch_windows_aarch64.S(38) : error A2008:syntax error : ldr
safefetch_windows_aarch64.S(39) : error A2034:must be in segment block
safefetch_windows_aarch64.S(41) : error A2008:syntax error : .
safefetch_windows_aarch64.S(42) : error A2008:syntax error : .
safefetch_windows_aarch64.S(43) : error A2008:syntax error : .type
safefetch_windows_aarch64.S(44) : error A2034:must be in segment block
safefetch_windows_aarch64.S(45) : error A2034:must be in segment block
safefetch_windows_aarch64.S(46) : error A2034:must be in segment block
safefetch_windows_aarch64.S(53) : error A2008:syntax error : .
safefetch_windows_aarch64.S(54) : error A2008:syntax error : .
safefetch_windows_aarch64.S(55) : error A2008:syntax error : .type
safefetch_windows_aarch64.S(56) : error A2034:must be in segment block
safefetch_windows_aarch64.S(58) : error A2008:syntax error : .
safefetch_windows_aarch64.S(59) : error A2008:syntax error : .
safefetch_windows_aarch64.S(60) : error A2008:syntax error : .type
safefetch_windows_aarch64.S(61) : error A2034:must be in segment block
safefetch_windows_aarch64.S(62) : error A2034:must be in segment block
safefetch_windows_aarch64.S(63) : error A2008:syntax error : ldr
safefetch_windows_aarch64.S(64) : error A2034:must be in segment block
safefetch_windows_aarch64.S(66) : error A2008:syntax error : .
safefetch_windows_aarch64.S(67) : error A2008:syntax error : .
safefetch_windows_aarch64.S(68) : error A2008:syntax error : .type
safefetch_windows_aarch64.S(69) : error A2034:must be in segment block
safefetch_windows_aarch64.S(70) : error A2034:must be in segment block
safefetch_windows_aarch64.S(71) : error A2034:must be in segment block
safefetch_windows_aarch64.S(71) : error A2088:END directive required at end of file

As it turns out, ml64.exe is the x64 assembler – see ML and ML64 command-line reference | Microsoft Learn. I switched to armasm64.exe (see ARM Assembler command-line reference | Microsoft Learn) but it failed with identical arguments.

"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\bin\Hostx64\arm64\armasm64.exe" -nologo -c -Ta safefetch_windows_aarch64.S
error A2029: unknown command-line argument or argument value -c

 Usage:      armasm [<options>] sourcefile objectfile
             armasm [<options>] -o objectfile sourcefile
             armasm -h              for help

Removing only -c gives:

error A2029: unknown command-line argument or argument value -Ta

/Ta filenameAssembles source file whose name doesn’t end with the .asm extension.

ML and ML64 command-line reference | Microsoft Learn

/cAssembles only. Does no linking.

ML and ML64 command-line reference | Microsoft Learn

The -Ta flag in the command line comes from jdk/make/common/native/CompileFile.gmk and the -c flag causing an error is from jdk/make/autoconf/flags-other.m4. I first tried moving the -Ta flag but that broke the build:

Command from build\windows-x86_64-server-slowdebug\make-support\failure-logs\support_native_jdk.incubator.vector_libjsvml_jsvml_d_acos_windows_x86.obj.cmdline

/cygdrive/d/java/ms/dups/openjdk-jdk/build/windows-x86_64-server-slowdebug/fixpath exec /cygdrive/c/progra~1/mib055~1/2022/enterprise/vc/tools/msvc/14.44.35207/bin/hostx64/x64/ml64.exe -nologo -c -Ta -Fo/cygdrive/d/java/ms/dups/openjdk-jdk/build/windows-x86_64-server-slowdebug/support/native/jdk.incubator.vector/libjsvml/jsvml_d_acos_windows_x86.obj /cygdrive/d/java/ms/dups/openjdk-jdk/src/jdk.incubator.vector/windows/native/libjsvml/jsvml_d_acos_windows_x86.S

From build\windows-x86_64-server-slowdebug\make-support\failure-logs\support_native_jdk.incubator.vector_libjsvml_jsvml_d_acos_windows_x86.obj.log

 Assembling: -Fod:\java\ms\dups\openjdk-jdk\build\windows-x86_64-server-slowdebug\support\native\jdk.incubator.vector\libjsvml\jsvml_d_acos_windows_x86.obj
MASM : fatal error A1000:cannot open file : -Fod:\java\ms\dups\openjdk-jdk\build\windows-x86_64-server-slowdebug\support\native\jdk.incubator.vector\libjsvml\jsvml_d_acos_windows_x86.obj

I just needed to have a separate else branch to handle setting up armasm64.exe to avoid passing ml64.exe flags to armasm64.exe. This successfully assembled my AArch64 assembly source file. However, the JVM would terminate with an access violation, which clearly isn’t supposed to happen because the fetch is supposed to be safe, by definition! I asked copilot: when would the program counter pointing at this aarch64 instruction result in an access violation? mov x0, x1. One scenario:

The Program Counter (PC) is pointing to an invalid address

  • If the PC is pointing to a location that is not mapped in the process’s address space (e.g., due to corruption, jumping to unmapped memory, or executing data as code), then fetching the instruction itself could trigger an access violation.
  • Example: If the PC points to a region of memory that has been freed or is protected (e.g., read-only or non-executable), the CPU will raise a fault when trying to fetch or decode the instruction.

This gave me a hint that my assembly instructions were probably not in an executable page! I found the AREA directive details at ARM Compiler armasm Reference Guide Version 6.01. It was tricky that the first AREA argument is a name and could therefore be anything. If I recall correctly, I think the access violation was because I didn’t have the CODE attribute on the AREA. With that fixed, I was able to successfully execute the compiled JVM.

I opened 8365579: ml64.exe is not the right assembler for Windows aarch64 by swesonga · Pull Request #26791 · openjdk/jdk to fix this issue. I branched off from openjdk/jdk at b0f98df75aee1e94a8c4b3eb8d0b1f4e715011ae for my changes. I initially removed the -Ta argument because it appeared to be a no-op. Code review feedback indicated that I need the -Ta flag unless I could document that it is a no-op. I discovered that it is essential e.g. if your assembly sources are in a .obj file!

Here are some of my fix attempts. I started with -Tasdf to ensure that I got an error indicating that this argument was being picked up by the build.

diff --git a/make/common/native/CompileFile.gmk b/make/common/native/CompileFile.gmk
index 26472da6d02..4d8aaef4445 100644
--- a/make/common/native/CompileFile.gmk
+++ b/make/common/native/CompileFile.gmk
@@ -234,9 +234,14 @@ define CreateCompiledNativeFileBody
            $(SED) $(DEPENDENCY_TARGET_SED_PATTERN) $$($1_DEPS_FILE) > $$($1_DEPS_TARGETS_FILE)
           else
             # For assembler calls just create empty dependency lists
+            ifeq ($(OPENJDK_TARGET_CPU), aarch64)
+              $1_NON_ASM_EXTENSION_FLAG :=
+            else
+              $1_NON_ASM_EXTENSION_FLAG := "-Tasdf "
+            endif
            $$(call ExecuteWithLog, $$@, $$(call MakeCommandRelative, \
                $$($1_COMPILER) $$($1_FLAGS) \
-               $(CC_OUT_OPTION)$$($1_OBJ) $$($1_SRC_FILE))) \
+               $(CC_OUT_OPTION)$$($1_OBJ) $$($1_NON_ASM_EXTENSION_FLAG) $$($1_SRC_FILE))) \
                | $(TR) -d '\r' | $(GREP) -v -e "Assembling:" || test "$$$$?" = "1" ; \
            $(ECHO) > $$($1_DEPS_FILE) ; \
            $(ECHO) > $$($1_DEPS_TARGETS_FILE)

Attempt 2:

diff --git a/make/common/native/CompileFile.gmk b/make/common/native/CompileFile.gmk
index 26472da6d02..697adbb6e26 100644
--- a/make/common/native/CompileFile.gmk
+++ b/make/common/native/CompileFile.gmk
@@ -155,6 +155,12 @@ define CreateCompiledNativeFileBody
         endif
         $1_FLAGS := $$($1_FLAGS) -DASSEMBLY_SRC_FILE='"$$($1_REL_ASM_SRC)"' \
             -include $(TOPDIR)/make/data/autoheaders/assemblyprefix.h
+      else ifeq ($(TOOLCHAIN_TYPE), microsoft)
+        ifeq ($(OPENJDK_TARGET_CPU), aarch64)
+          $1_NON_ASM_EXTENSION_FLAG :=
+        else
+          $1_NON_ASM_EXTENSION_FLAG := "-Tasdf "
+        endif
       endif
     else ifneq ($$(filter %.cpp %.cc %.mm, $$($1_FILENAME)), )
       # Compile as a C++ or Objective-C++ file
@@ -236,7 +242,7 @@ define CreateCompiledNativeFileBody
             # For assembler calls just create empty dependency lists
            $$(call ExecuteWithLog, $$@, $$(call MakeCommandRelative, \
                $$($1_COMPILER) $$($1_FLAGS) \
-               $(CC_OUT_OPTION)$$($1_OBJ) $$($1_SRC_FILE))) \
+               $(CC_OUT_OPTION)$$($1_OBJ) $$($1_NON_ASM_EXTENSION_FLAG) $$($1_SRC_FILE))) \
                | $(TR) -d '\r' | $(GREP) -v -e "Assembling:" || test "$$$$?" = "1" ; \
            $(ECHO) > $$($1_DEPS_FILE) ; \
            $(ECHO) > $$($1_DEPS_TARGETS_FILE)

This resulted in this error, which confirmed that it was a valid place to set the flag:

=== Output from failing command(s) repeated here ===
* For target support_native_jdk.incubator.vector_libjsvml_BUILD_LIBJSVML_run_ld:
LINK : fatal error LNK1181: cannot open input file 'd:\java\forks\dups12\openjdk\jdk\build\windows-x86_64-server-slowdebug\support\native\jdk.incubator.vector\libjsvml\jsvml_d_acos_windows_x86.obj'
* For target support_native_jdk.incubator.vector_libjsvml_jsvml_d_acos_windows_x86.obj:
 Assembling: sdf 
MASM : fatal error A1000:cannot open file : sdf 
* For target support_native_jdk.incubator.vector_libjsvml_jsvml_d_asin_windows_x86.obj:
 Assembling: sdf 
MASM : fatal error A1000:cannot open file : sdf 

After Magnus’s feedback on 8/23, I reverted that change and tried this instead:

diff --git a/make/autoconf/flags.m4 b/make/autoconf/flags.m4
index d50538108a4..8ba1a313cb2 100644
--- a/make/autoconf/flags.m4
+++ b/make/autoconf/flags.m4
@@ -320,6 +320,11 @@ AC_DEFUN([FLAGS_SETUP_TOOLCHAIN_CONTROL],
 [
   if test "x$TOOLCHAIN_TYPE" = xmicrosoft; then
     CC_OUT_OPTION=-Fo
+    if test "x$OPENJDK_TARGET_CPU" = xaarch64; then
+      AS_NON_ASM_EXTENSION_FLAG=
+    else
+      AS_NON_ASM_EXTENSION_FLAG=-Tazzz
+    endif
   else
     # The option used to specify the target .o,.a or .so file.
     # When compiling, how to specify the to be created object file.
diff --git a/make/common/native/CompileFile.gmk b/make/common/native/CompileFile.gmk
index 26472da6d02..7f8e8ffeddc 100644
--- a/make/common/native/CompileFile.gmk
+++ b/make/common/native/CompileFile.gmk
@@ -236,7 +236,7 @@ define CreateCompiledNativeFileBody
             # For assembler calls just create empty dependency lists
            $$(call ExecuteWithLog, $$@, $$(call MakeCommandRelative, \
                $$($1_COMPILER) $$($1_FLAGS) \
-               $(CC_OUT_OPTION)$$($1_OBJ) $$($1_SRC_FILE))) \
+               $(CC_OUT_OPTION)$$($1_OBJ) $(AS_NON_ASM_EXTENSION_FLAG) $$($1_SRC_FILE))) \
                | $(TR) -d '\r' | $(GREP) -v -e "Assembling:" || test "$$$$?" = "1" ; \
            $(ECHO) > $$($1_DEPS_FILE) ; \
            $(ECHO) > $$($1_DEPS_TARGETS_FILE)

The configure script failed:

Runnable configure script is not present
Generating runnable configure script at /cygdrive/d/java/forks/dups12/openjdk/jdk/build/.configure-support/generated-configure.sh
Using autoconf at /usr/bin/autoconf [autoconf (GNU Autoconf) 2.72]
-:166141: error: possibly undefined macro: AS_NON_ASM_EXTENSION_FLAG
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure: Configuration created at Sat Aug 23 15:11:36 MDT 2025.

On the prompt " recipe commences before first target" Copilot says:

The error message “recipe commences before first target” in GNU Make typically means that there’s a line in your Makefile that starts with a tab (indicating a recipe), but it appears before any target has been defined. In Makefiles, recipes (commands to execute) must follow a target and its dependencies.

I moved the check further up, which fixed the build. After the PR was closed, I got a comment about the quotes I introduced! They shouldn’t be there :(.

/cygdrive/d/java/forks/dups15/openjdk/jdk/build/windows-x86_64-server-slowdebug/fixpath exec /cygdrive/c/progra~1/mib055~1/2022/enterprise/vc/tools/msvc/14.44.35207/bin/hostx64/x64/ml64.exe -nologo -c -Fo/cygdrive/d/java/forks/dups15/openjdk/jdk/build/windows-x86_64-server-slowdebug/support/native/jdk.incubator.vector/libjsvml/jsvml_d_atan2_windows_x86.obj "-Ta" /cygdrive/d/java/forks/dups15/openjdk/jdk/src/jdk.incubator.vector/windows/native/libjsvml/jsvml_d_atan2_windows_x86.S

Fortunately, the quotes don’t break assembling the source code as demonstrated by the 2 commands below:

copy test_x86_assembler.S test_x86_assembler.lib

"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\bin\Hostx64\x86\ml.exe" -nologo -c -Fo test_x86_assembler.obj "-Ta" test_x86_assembler.lib
 Assembling: test_x86_assembler.lib

"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\bin\Hostx64\x86\ml.exe" -nologo -c -Fo test_x86_assembler.obj test_x86_assembler.lib
MASM : fatal error A1017:missing source filename

I cleaned this up in 8366195: Remove unnecessary quotes around -Ta ml64 assembler argument by swesonga · Pull Request #27021 · openjdk/jdk, also moving the declaration of the argument to match -Fo as suggested in #26791.


Categories: OpenJDK

Native Stack Printing on Windows AArch64

Another test failure I investigated last month was in the runtime/jni/nativeStack/TestNativeStack.java test. This is the setup I used to run the test in my MSYS environment.

export JDKARCH=aarch64
export DEBUGLEVEL=slowdebug
export JDKSRCPATH=/c/java/forks/openjdk/jdk
export JDKBUILDPATH="${JDKSRCPATH}/build/windows-${JDKARCH}-server-${DEBUGLEVEL}"
export JDKTOTEST="${JDKBUILDPATH}/images/jdk"
export JTREGNATIVEPATH1="${JDKBUILDPATH}/support/test/hotspot/jtreg/native/lib"
export JTREGNATIVEPATH2="${JDKBUILDPATH}/support/test/jdk/jtreg/native/lib"
export JTREGNATIVEPATH3="${JDKBUILDPATH}/support/test/lib/native/lib"
export GTESTPATH="${JDKBUILDPATH}/images/test/hotspot/gtest/server"
export JTREGBINPATH=/c/java/binaries/jtreg/jtreg-8+2

export TESTTORUN=test/hotspot/jtreg/runtime/jni/nativeStack/TestNativeStack.java

./run-jtreg-test.sh $JDKSRCPATH $JDKTOTEST $JTREGBINPATH/lib/jtreg.jar $TESTTORUN -nativepath:$JTREGNATIVEPATH1

The test fails with the output below. The key line is Native frames: <unavailable>.

STDOUT:
Command line: [C:\java\forks\openjdk\jdk\build\windows-aarch64-server-slowdebug\images\jdk\bin\java.exe -cp C:\java\forks\openjdk\jdk\JTwork\classes\runtime\jni\nativeStack\TestNativeStack.d;C:\java\forks\openjdk\jdk\test\hotspot\jtreg\runtime\jni\nativeStack;C:\java\forks\openjdk\jdk\JTwork\classes\test\lib;C:\java\binaries\jtreg\jtreg-7.5.2\lib\javatest.jar;C:\java\binaries\jtreg\jtreg-7.5.2\lib\jtreg.jar;C:\java\binaries\jtreg\jtreg-7.5.2\lib\junit-platform-console-standalone-1.11.0.jar;C:\java\binaries\jtreg\jtreg-7.5.2\lib\testng-7.3.0.jar;C:\java\binaries\jtreg\jtreg-7.5.2\lib\guice-5.1.0.jar;C:\java\binaries\jtreg\jtreg-7.5.2\lib\jcommander-1.82.jar -Xmx512m -Xcheck:jni -Djava.library.path=C:\java\forks\openjdk\jdk\build\windows-aarch64-server-slowdebug\support\test\hotspot\jtreg\native\lib TestNativeStack$Main ]
[2025-09-04T15:54:18.312130500Z] Gathering output for process 22380
[2025-09-04T15:54:18.753851700Z] Waiting for completion for process 22380
[2025-09-04T15:54:18.756861300Z] Waiting for completion finished for process 22380
Output and diagnostic info for process 22380 was saved into 'pid-22380-output.log'
STDERR:
 stdout: [Triggering a JNI warning
WARNING in native method: JNI call made without checking exceptions when required to from CallStaticObjectMethod
Native frames: <unavailable>
];
 stderr: [WARNING: A restricted method in java.lang.System has been called
WARNING: java.lang.System::loadLibrary has been called by TestNativeStack in an unnamed module (file:/C:/java/forks/openjdk/jdk/JTwork/classes/runtime/jni/nativeStack/TestNativeStack.d/)
WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module
WARNING: Restricted methods will be blocked in a future release unless native access is enabled

]
 exitValue = -2147483645

java.lang.RuntimeException: Expected to get exit value of [0], exit value is: [-2147483645]
        at jdk.test.lib.process.OutputAnalyzer.shouldHaveExitValue(OutputAnalyzer.java:522)
        at TestNativeStack.main(TestNativeStack.java:57)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:565)
        at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:335)
        at java.base/java.lang.Thread.run(Thread.java:1474)

I added a DebugBreak call to the line outputing that message to see how we got there:

KernelBase.dll!...DebugBreak() Line 2582	C++
jvm.dll!NativeStackPrinter::print_stack_from_frame(outputStream * st, frame fr, char * buf, int buf_size, bool print_source_info, int max_frames) Line 80	C++
jvm.dll!NativeStackPrinter::print_stack_from_frame(outputStream * st, char * buf, int buf_size, bool print_source_info, int max_frames) Line 104	C++
jvm.dll!NativeStackPrinter::print_stack(outputStream * st, char * buf, int buf_size, unsigned char * & lastpc, bool print_source_info, int max_frames) Line 40	C++
jvm.dll!JavaThread::print_jni_stack() Line 1775	C++
jvm.dll!check_pending_exception(JavaThread * thr) Line 192	C++
jvm.dll!functionEnter(JavaThread * thr) Line 218	C++
jvm.dll!checked_jni_CallStaticObjectMethod(JNIEnv_ * env, _jclass * clazz, _jmethodID * methodID, ...) Line 1321	C++
nativeStack.dll!generateWarning(const JNINativeInterface_ * * env) Line 66	C
nativeStack.dll!thread_start(void * unused) Line 92	C
ucrtbase.dll!00007ffb0970b028()	Unknown
kernel32.dll!00007ffb0bbd8740()	Unknown
ntdll.dll!RtlUserThreadStart(long(*)(void *) StartAddress, void * Argument) Line 1184	C

The failure path is as follows. HAVE_PLATFORM_PRINT_NATIVE_STACK is not defined on Windows AArch64. Consequently, Windows AArch64 uses the implementation of os::platform_print_native_stack that simply returns false. This results in NativeStackPrinter::print_stack having to call NativeStackPrinter::print_stack_from_frame instead. However, the context is null. As a result, the frame used for printing the stack is obtained from os::current_frame(), which returns an empty frame. The frame’s pc() method returns nullptr and the “Native frames: <unavailable>” message is printed.

The fix for this issue is to define the os::platform_print_native_stack method for Windows AArch64 and share the implementation of the Windows x64 os::win32::platform_print_native_stack method with Windows AArch64. I opened [JDK-8369322] Implement native stack printing for Windows-AArch64 – Java Bug System and the associated PR 8369322: Implement native stack printing for Windows-AArch64 by swesonga · Pull Request #27680 · openjdk/jdk fixing this failure. With this change, the native frames are now printed as shown below:

Triggering a JNI warning
WARNING: A restricted method in java.lang.System has been called
WARNING: java.lang.System::loadLibrary has been called by TestNativeStack in an unnamed module (file:/C:/java/forks/openjdk/jdk/JTwork/classes/runtime/jni/nativeStack/TestNativeStack.d/)
WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module
WARNING: Restricted methods will be blocked in a future release unless native access is enabled

Native thread is running and attaching as daemon ...
About to trigger JNI Warning
WARNING in native method: JNI call made without checking exceptions when required to from CallStaticObjectMethod
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [jvm.dll+0x10e8aa8]  os::win32::platform_print_native_stack+0x58  (os_windows_aarch64.cpp:143)
V  [jvm.dll+0x10598ac]  os::platform_print_native_stack+0x34  (os_windows_aarch64.inline.hpp:38)
V  [jvm.dll+0x1059588]  NativeStackPrinter::print_stack+0x48  (nativeStackPrinter.cpp:35)
V  [jvm.dll+0xba74a0]  JavaThread::print_jni_stack+0x120  (javaThread.cpp:1775)
V  [jvm.dll+0xcb0e84]  check_pending_exception+0x84  (jniCheck.cpp:192)
V  [jvm.dll+0xcb0f24]  functionEnter+0x4c  (jniCheck.cpp:218)
V  [jvm.dll+0xcbbb70]  checked_jni_CallStaticObjectMethod+0xf0  (jniCheck.cpp:1321)
C  [nativeStack.dll+0x1264]  generateWarning+0x13c  (libnativeStack.c:66)
C  [nativeStack.dll+0x1364]  thread_start+0xa4  (libnativeStack.c:92)
C  [ucrtbase.dll+0x2b028]  (no source info available)
C  [KERNEL32.DLL+0x8740]  (no source info available)
C  [ntdll.dll+0xd47a4]  (no source info available)

Native thread terminating

C:\java\forks\openjdk\jdk\build\windows-aarch64-server-slowdebug\images\jdk\bin\java.exe (process 30308) exited with code 0 (0x0).
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

Categories: OpenJDK

Why did Windows gtest catch a std::exception-derived exception

Last month, I investigated OpenJDK gtest failures on Windows. The error message was that the gtests Caught std::exception-derived exception escaping the death test statement. I tracked the commit responsible for the failures to 8343756: CAN_SHOW_REGISTERS_ON_ASSERT for Windows · openjdk/jdk@0054bbe.

gtest Death Test Structure

One of the failing tests is jdk/test/hotspot/gtest/utilities/test_vmerror.cpp.

TEST_VM_ASSERT_MSG(vmErrorTest, assert1, "assert.str == nullptr. failed: expected null") {
  vmassert(str == nullptr, "expected null");
}

The TEST_VM_ASSERT_MSG macro is defined as follows

#define TEST_VM_ASSERT_MSG(category, name, msg)                     \
  static void test_  ## category ## _ ## name ## _();               \
                                                                    \
  static void child_ ## category ## _ ## name ## _() {              \
    ::testing::GTEST_FLAG(throw_on_failure) = true;                 \
    test_ ## category ## _ ## name ## _();                          \
    gtest_exit_from_child_vm(0);                                    \
  }                                                                 \
                                                                    \
  TEST(category, CONCAT(name, _vm_assert)) {                        \
    ASSERT_EXIT(child_ ## category ## _ ## name ## _(),             \
                ::testing::ExitedWithCode(1),                       \
                "assert failed: " msg);                             \
  }                                                                 \
                                                                    \
  void test_ ## category ## _ ## name ## _()

gtest_exit_from_child_vm(0) cleanly exits the JVM after calling test_vmErrorTest_assert1(). The ASSERT_EXIT macro expects the JVM to crash. The overall design of the death tests is documented in googletest/googletest/include/gtest/gtest-death-test.h at v1.14.0 · google/googletest. The key takeaway is that a child process is started, it executes the death test, and its exit code and stderr are compared with the expected code and message (the latter via regex matching).

// Asserts that a given `statement` causes the program to exit, with an
// integer exit status that satisfies `predicate`, and emitting error output
// that matches `matcher`.
#define ASSERT_EXIT(statement, predicate, matcher) \
  GTEST_DEATH_TEST_(statement, predicate, matcher, GTEST_FATAL_FAILURE_)

I was trying to ensure my understanding of the exit code being an exact match is correct. The line EXPECT_EXIT(_exit(1), testing::ExitedWithCode(1), ""); from googletest/googletest/test/googletest-death-test-test.cc at v1.14.0 · google/googletest supports this hypothesis. The EXPECT_EXIT macro comment (below) left me wondering how ASSERT_EXIT does not continue on to successive tests. The difference between these two macros is in the final parameter, which is GTEST_NONFATAL_FAILURE_ for the EXPECT_EXIT macro.

// Like `ASSERT_EXIT`, but continues on to successive tests in the
// test suite, if any:
#define EXPECT_EXIT(statement, predicate, matcher) \
  GTEST_DEATH_TEST_(statement, predicate, matcher, GTEST_NONFATAL_FAILURE_)

The GTEST_DEATH_TEST_ macro creates a DeathTest instance and executes the death test statement. The WindowsDeathTest::AssumeRole() method, which is key in this, is described as follows: it

creates a child process with the same executable as the current process to run the death test. The child process is given the –gtest_filter and –gtest_internal_run_death_test flags such that it knows to run the current death test only.

The GTEST_EXECUTE_DEATH_TEST_STATEMENT_ macro was the source of the error message!

#define GTEST_EXECUTE_DEATH_TEST_STATEMENT_(statement, death_test)           \
  try {                                                                      \
    GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement);               \
  } catch (const ::std::exception& gtest_exception) {                        \
    fprintf(                                                                 \
        stderr,                                                              \
        "\n%s: Caught std::exception-derived exception escaping the "        \
        "death test statement. Exception message: %s\n",                     \
        ::testing::internal::FormatFileLocation(__FILE__, __LINE__).c_str(), \
        gtest_exception.what());                                             \
    fflush(stderr);                                                          \
    death_test->Abort(::testing::internal::DeathTest::TEST_THREW_EXCEPTION); \
  } catch (...) {                                                            \
    death_test->Abort(::testing::internal::DeathTest::TEST_THREW_EXCEPTION); \
  }

Root Causing the std::exception

The question now became, why were we catching this std::exception? I asked copilot: how does a windows access violation turn into a std::exception? Part of its answer mentioned the /EHsc compiler flag so I decided to examine the flags used to compile the JVM binaries. I searched for the regex out:[^\s]+jvm.dll in the build logs and found this jvm.dll linker command. Note that 2 separate jvm.dll files get built, one for the product and another for the gtests. The /IMPLIB (Name Import Library) | Microsoft Learn flag was present, but didn’t look relevant.

I then searched for cl.exe .+jvm.lib to get the compiler command line but this gave the compiler commands for gtest-all.cc and gmock-all.cc. The -EHsc flag (/EH (Exception handling model) | Microsoft Learn) was present for these 2 files though! Next, I searched for “gtestLauncher” and found the compiler command generating gtestLauncher.obj. Notice it didn’t have -EHsc!

I also realized that I should have searched for jvm.obj! Here is the single occurence of the jvm\.obj regex. It doesn’t have -EHsc either! Hmm, strange: jdk/make/hotspot/lib/CompileGtest.gmk says gtests should have it! I then searched the make folder for the regex CFLAGS_[^\w] and the primary suspect (of the 3 results) is the SetupCompilerFlags target in jdk/make/common/native/Flags.gmk. That target was last modified in 8325877: Split up NativeCompilation.gmk · openjdk/jdk@0d51b76.

Next, I examined build\windows-x86_64-server-slowdebug\configure-support\config.log and found these lines:

OPENJDK_TARGET_OS='windows'
...
OPENJDK_TARGET_OS_TYPE='windows'
...
TOOLCHAIN_TYPE='microsoft'

I thought that this snippet from jdk/make/common/native/Flags.gmk should have picked them up!

define SetupCompilerFlags
  # Pickup extra OPENJDK_TARGET_OS_TYPE, OPENJDK_TARGET_OS, TOOLCHAIN_TYPE and
  # OPENJDK_TARGET_OS plus OPENJDK_TARGET_CPU pair dependent variables for CFLAGS.
  $1_EXTRA_CFLAGS := $$($1_CFLAGS_$(OPENJDK_TARGET_OS_TYPE)) $$($1_CFLAGS_$(OPENJDK_TARGET_OS)) \
      $$($1_CFLAGS_$(TOOLCHAIN_TYPE)) \
      $$($1_CFLAGS_$(OPENJDK_TARGET_OS)_$(OPENJDK_TARGET_CPU))

What was the leading $1_ prefix though? I wasn’t sure but I tried this change next:

diff --git a/make/hotspot/lib/CompileGtest.gmk b/make/hotspot/lib/CompileGtest.gmk
index d2cdc7685c9..3908e94b624 100644
--- a/make/hotspot/lib/CompileGtest.gmk
+++ b/make/hotspot/lib/CompileGtest.gmk
@@ -63,7 +63,7 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBGTEST, \
         unused-result zero-as-null-pointer-constant, \
     DISABLED_WARNINGS_clang := format-nonliteral undef unused-result, \
     DEFAULT_CFLAGS := false, \
-    CFLAGS := $(JVM_CFLAGS) \
+    CFLAGS := $(JVM_CFLAGS) -EHsc \
         -I$(GTEST_FRAMEWORK_SRC)/googletest \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock \
@@ -94,7 +94,7 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBJVM, \
     EXCLUDE_PATTERNS := $(JVM_EXCLUDE_PATTERNS), \
     EXTRA_OBJECT_FILES := $(BUILD_LIBJVM_ALL_OBJS), \
     DEFAULT_CFLAGS := false, \
-    CFLAGS := $(JVM_CFLAGS) \
+    CFLAGS := $(JVM_CFLAGS) -EHsc \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include \
         $(addprefix -I, $(GTEST_TEST_SRC)), \

The build failed with this error:

checking for apk... [not found]
checking for pandoc... [not found]
/cygdrive/d/java/forks/openjdk/jdk/build/.configure-support/generated-configure.sh: line 64028: syntax error: unexpected end of file
configure exiting with result code 2

That file appeared to be truncated??? VSCode was doing something related to building the Java projects in the repo. It is possible that something in VSCode could have interrupted this but I just removed the build folder then reexamined the change.

I decided to find out who uses SetupCompilerFlags and found jdk/make/common/NativeCompilation.gmk. It is in turn called by SetupJdkNativeCompilation (which I had been trying to change)! The actual compilation is kicked off by jdk/make/common/NativeCompilation.gmk and done by jdk/make/common/native/CompileFile.gmk. The latter calls SetupCompileFileFlags, which is defined in jdk/make/common/native/Flags.gmk. I noticed that it includes the extra CFLAGS and CXXFLAGS in jdk/make/common/native/Flags.gmk. The most important observation though was that jdk/make/common/native/CompileFile.gmk uses the CXXFLAGS for .cpp files! I tried this change but it didn’t pass the flags to the compiler either.

diff --git a/make/hotspot/lib/CompileGtest.gmk b/make/hotspot/lib/CompileGtest.gmk
index d2cdc7685c9..8241ad04cb9 100644
--- a/make/hotspot/lib/CompileGtest.gmk
+++ b/make/hotspot/lib/CompileGtest.gmk
@@ -69,6 +69,7 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBGTEST, \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include, \
     CFLAGS_windows := -EHsc, \
+    CXXFLAGS_windows := -EHsc, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     OPTIMIZATION := $(JVM_OPTIMIZATION), \
     COPY_DEBUG_SYMBOLS := $(GTEST_COPY_DEBUG_SYMBOLS), \
@@ -99,6 +100,7 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBJVM, \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include \
         $(addprefix -I, $(GTEST_TEST_SRC)), \
     CFLAGS_windows := -EHsc, \
+    CXXFLAGS_windows := -EHsc, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc) \
         undef stringop-overflow, \

Was the code I was changing even used? I tried this change to answer that:

diff --git a/make/hotspot/lib/CompileGtest.gmk b/make/hotspot/lib/CompileGtest.gmk
index d2cdc7685c9..4554b3c89f5 100644
--- a/make/hotspot/lib/CompileGtest.gmk
+++ b/make/hotspot/lib/CompileGtest.gmk
@@ -66,9 +66,11 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBGTEST, \
     CFLAGS := $(JVM_CFLAGS) \
         -I$(GTEST_FRAMEWORK_SRC)/googletest \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
+        -I$(GTEST_FRAMEWORK_SRC)/googletest/include/saintstest \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include, \
     CFLAGS_windows := -EHsc, \
+    CXXFLAGS_windows := -EHsc, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     OPTIMIZATION := $(JVM_OPTIMIZATION), \
     COPY_DEBUG_SYMBOLS := $(GTEST_COPY_DEBUG_SYMBOLS), \
@@ -96,9 +98,11 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBJVM, \
     DEFAULT_CFLAGS := false, \
     CFLAGS := $(JVM_CFLAGS) \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
+        -I$(GTEST_FRAMEWORK_SRC)/googletest/include/saintstest \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include \
         $(addprefix -I, $(GTEST_TEST_SRC)), \
     CFLAGS_windows := -EHsc, \
+    CXXFLAGS_windows := -EHsc, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc) \
         undef stringop-overflow, \

My include path didn’t appear in the include paths for gtestLauncher.obj! I searched the repo for googlemock and the only place that path could be coming from was CompileGtest.gmk. However, I then noticed that the gtest launcher has its own configuration section. Sheesh. Here is the diff that I used to definitively see how these includes work:

diff --git a/make/hotspot/lib/CompileGtest.gmk b/make/hotspot/lib/CompileGtest.gmk
index d2cdc7685c9..72161d6d5d2 100644
--- a/make/hotspot/lib/CompileGtest.gmk
+++ b/make/hotspot/lib/CompileGtest.gmk
@@ -66,9 +66,11 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBGTEST, \
     CFLAGS := $(JVM_CFLAGS) \
         -I$(GTEST_FRAMEWORK_SRC)/googletest \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
+        -I$(GTEST_FRAMEWORK_SRC)/googletest/include/saintstest1 \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include, \
     CFLAGS_windows := -EHsc, \
+    CXXFLAGS_windows := -EHsc, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     OPTIMIZATION := $(JVM_OPTIMIZATION), \
     COPY_DEBUG_SYMBOLS := $(GTEST_COPY_DEBUG_SYMBOLS), \
@@ -96,9 +98,11 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBJVM, \
     DEFAULT_CFLAGS := false, \
     CFLAGS := $(JVM_CFLAGS) \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
+        -I$(GTEST_FRAMEWORK_SRC)/googletest/include/saintstest2 \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include \
         $(addprefix -I, $(GTEST_TEST_SRC)), \
     CFLAGS_windows := -EHsc, \
+    CXXFLAGS_windows := -EHsc, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc) \
         undef stringop-overflow, \
@@ -150,8 +154,10 @@ $(eval $(call SetupJdkExecutable, BUILD_GTEST_LAUNCHER, \
     CFLAGS := $(JVM_CFLAGS) \
         -I$(GTEST_FRAMEWORK_SRC)/googletest \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
+        -I$(GTEST_FRAMEWORK_SRC)/googletest/include/saintstest3 \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include, \
+    CXXFLAGS_windows := -EHsc, \
     LD_SET_ORIGIN := false, \
     LDFLAGS_unix := $(call SET_SHARED_LIBRARY_ORIGIN), \
     JDK_LIBS := gtest:libjvm, \

gtestLauncher.exe was now being compiled with -EHsc but the gtests still failed. Since jvm.dll is compiled without -EHsc, I added it to see if the test behavior would change. I started by searching for libjvm in the codebase. This is the additional change I made:

diff --git a/make/hotspot/lib/CompileJvm.gmk b/make/hotspot/lib/CompileJvm.gmk
index 6b5edc85b23..f7ae373ff17 100644
--- a/make/hotspot/lib/CompileJvm.gmk
+++ b/make/hotspot/lib/CompileJvm.gmk
@@ -179,6 +179,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBJVM, \
     EXCLUDE_PATTERNS := $(JVM_EXCLUDE_PATTERNS), \
     DEFAULT_CFLAGS := false, \
     CFLAGS := $(JVM_CFLAGS), \
+    CXXFLAGS_windows := -EHsc, \
     abstract_vm_version.cpp_CXXFLAGS := $(CFLAGS_VM_VERSION), \
     arguments.cpp_CXXFLAGS := $(CFLAGS_VM_VERSION), \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc), \

At this point, I looked at the exception handler and it looked like what was happening was that returning EXCEPTION_CONTINUE_EXECUTION let the SEH handler in the gtests continue instead of the code down the report_vm_error path! I decided to create my own handler but needed to look up the syntax. I used Structured Exception Handling (C/C++) | Microsoft Learn.

diff --git a/src/hotspot/share/utilities/debug.hpp b/src/hotspot/share/utilities/debug.hpp
index 12724153659..e40c16c1c59 100644
--- a/src/hotspot/share/utilities/debug.hpp
+++ b/src/hotspot/share/utilities/debug.hpp
@@ -39,7 +39,21 @@ class oopDesc;
 #define CAN_SHOW_REGISTERS_ON_ASSERT
 extern char* g_assert_poison;
 extern const char* g_assert_poison_read_only;
+#if (defined(_WINDOWS))
+// We use structured exception handling when writing to the poison variable.
+// This allows us to continue execution and perform error reporting instead of
+// bailing out to other SEH handlers such as those in the googletest code.
+#include <excpt.h>
+#define TOUCH_ASSERT_POISON                                                  \
+do {                                                                         \
+  __try {                                                                    \
+    (*g_assert_poison) = 'X';                                                \
+  } __except (EXCEPTION_CONTINUE_EXECUTION) {                                \
+  }                                                                          \
+} while (0)
+#else
 #define TOUCH_ASSERT_POISON (*g_assert_poison) = 'X';
+#endif // _WINDOWS
 void initialize_assert_poison();
 void disarm_assert_poison();
 bool handle_assert_poison_fault(const void* ucVoid);

This change failed to build with the following errors, the most notable of which is Compiler Error C2712: cannot use __try in functions that require object unwinding.

...\jdk\src\hotspot\share\utilities/growableArray.hpp(81): error C2712: Cannot use __try in functions that require object unwinding
...\jdk\src\hotspot\share\classfile/vmClassID.hpp(41): error C3615: constexpr function 'EnumeratorRangeImpl::end_value' cannot result in a constant expression
...\jdk\src\hotspot\share\utilities/enumIterator.hpp(97): note: failure was caused by a statement or an expression that is not valid in a constexpr context
...\jdk\src\hotspot\share\utilities/unsigned5.hpp(190): error C3615: constexpr function 'UNSIGNED5::max_encoded_in_length' cannot result in a constant expression
...\jdk\src\hotspot\share\utilities/unsigned5.hpp(191): note: failure was caused by a statement or an expression that is not valid in a constexpr context
...\jdk\src\hotspot\cpu\x86\register_x86.hpp(61): error C3615: constexpr function 'Register::RegisterImpl::encoding' cannot result in a constant expression
...\jdk\src\hotspot\cpu\x86\register_x86.hpp(61): note: failure was caused by a statement or an expression that is not valid in a constexpr context
...\jdk\src\hotspot\cpu\x86\register_x86.hpp(233): error C3615: constexpr function 'XMMRegister::XMMRegisterImpl::encoding' cannot result in a constant expression
...\jdk\src\hotspot\cpu\x86\register_x86.hpp(233): note: failure was caused by a statement or an expression that is not valid in a constexpr context
...\jdk\src\hotspot\share\runtime/park.hpp(131): error C2712: Cannot use __try in functions that require object unwinding
...\jdk\src\hotspot\share\runtime/mutexLocker.hpp(235): error C2712: Cannot use __try in functions that require object unwinding
...\jdk\src\hotspot\share\runtime/mutexLocker.hpp(240): error C2712: Cannot use __try in functions that require object unwinding
...\jdk\src\hotspot\share\runtime/mutexLocker.hpp(250): error C2712: Cannot use __try in functions that require object unwinding
...\jdk\src\hotspot\share\runtime/mutexLocker.hpp(255): error C2712: Cannot use __try in functions that require object unwinding

At this point, I realized that I needed to disable SEH at the gtest level. I turned off GTEST_HAS_SEH with this change and finally got the gtests to pass!

diff --git a/make/hotspot/lib/CompileGtest.gmk b/make/hotspot/lib/CompileGtest.gmk
index d2cdc7685c9..d9e73fc3847 100644
--- a/make/hotspot/lib/CompileGtest.gmk
+++ b/make/hotspot/lib/CompileGtest.gmk
@@ -68,7 +68,7 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBGTEST, \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include, \
-    CFLAGS_windows := -EHsc, \
+    CFLAGS_windows := -EHsc -DGTEST_HAS_SEH=0, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     OPTIMIZATION := $(JVM_OPTIMIZATION), \
     COPY_DEBUG_SYMBOLS := $(GTEST_COPY_DEBUG_SYMBOLS), \
@@ -98,7 +98,7 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBJVM, \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include \
         $(addprefix -I, $(GTEST_TEST_SRC)), \
-    CFLAGS_windows := -EHsc, \
+    CFLAGS_windows := -EHsc -DGTEST_HAS_SEH=0, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc) \
         undef stringop-overflow, \
@@ -152,6 +152,7 @@ $(eval $(call SetupJdkExecutable, BUILD_GTEST_LAUNCHER, \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include, \
+    CFLAGS_windows := -EHsc -DGTEST_HAS_SEH=0, \
     LD_SET_ORIGIN := false, \
     LDFLAGS_unix := $(call SET_SHARED_LIBRARY_ORIGIN), \
     JDK_LIBS := gtest:libjvm, \
diff --git a/make/hotspot/lib/CompileJvm.gmk b/make/hotspot/lib/CompileJvm.gmk
index 6b5edc85b23..42f3969c775 100644
--- a/make/hotspot/lib/CompileJvm.gmk
+++ b/make/hotspot/lib/CompileJvm.gmk
@@ -179,6 +179,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBJVM, \
     EXCLUDE_PATTERNS := $(JVM_EXCLUDE_PATTERNS), \
     DEFAULT_CFLAGS := false, \
     CFLAGS := $(JVM_CFLAGS), \
+    CFLAGS_windows := -DGTEST_HAS_SEH=0, \
     abstract_vm_version.cpp_CXXFLAGS := $(CFLAGS_VM_VERSION), \
     arguments.cpp_CXXFLAGS := $(CFLAGS_VM_VERSION), \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc), \

What was not sure of was whether the JVM reporting code was running (vs the JVM just exiting) and whether there was a narrower way to pass the GTEST_HAS_SEH define – I noticed it in thousands of lines in the compilation log, which might also explain why I was getting error C2712: Cannot use __try in functions that require object unwinding in many more places than I expected when I added the -EHsc flag when compiling jvm.obj. Therefore, it was logical to try to find the minimal diff that would fix the gtests. Here’s one I tried:

diff --git a/make/hotspot/lib/CompileGtest.gmk b/make/hotspot/lib/CompileGtest.gmk
index d2cdc7685c9..95794ff0bbe 100644
--- a/make/hotspot/lib/CompileGtest.gmk
+++ b/make/hotspot/lib/CompileGtest.gmk
@@ -152,6 +152,7 @@ $(eval $(call SetupJdkExecutable, BUILD_GTEST_LAUNCHER, \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include, \
+    CFLAGS_windows := -DGTEST_HAS_SEH=0, \
     LD_SET_ORIGIN := false, \
     LDFLAGS_unix := $(call SET_SHARED_LIBRARY_ORIGIN), \
     JDK_LIBS := gtest:libjvm, \
diff --git a/make/hotspot/lib/CompileJvm.gmk b/make/hotspot/lib/CompileJvm.gmk
index 6b5edc85b23..42f3969c775 100644
--- a/make/hotspot/lib/CompileJvm.gmk
+++ b/make/hotspot/lib/CompileJvm.gmk
@@ -179,6 +179,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBJVM, \
     EXCLUDE_PATTERNS := $(JVM_EXCLUDE_PATTERNS), \
     DEFAULT_CFLAGS := false, \
     CFLAGS := $(JVM_CFLAGS), \
+    CFLAGS_windows := -DGTEST_HAS_SEH=0, \
     abstract_vm_version.cpp_CXXFLAGS := $(CFLAGS_VM_VERSION), \
     arguments.cpp_CXXFLAGS := $(CFLAGS_VM_VERSION), \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc), \

The gtests built from the diff below still failed:

diff --git a/make/hotspot/lib/CompileGtest.gmk b/make/hotspot/lib/CompileGtest.gmk
index d2cdc7685c9..403613c406c 100644
--- a/make/hotspot/lib/CompileGtest.gmk
+++ b/make/hotspot/lib/CompileGtest.gmk
@@ -98,7 +98,7 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBJVM, \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include \
         $(addprefix -I, $(GTEST_TEST_SRC)), \
-    CFLAGS_windows := -EHsc, \
+    CFLAGS_windows := -EHsc -DGTEST_HAS_SEH=0, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc) \
         undef stringop-overflow, \
diff --git a/make/hotspot/lib/CompileJvm.gmk b/make/hotspot/lib/CompileJvm.gmk
index 6b5edc85b23..42f3969c775 100644
--- a/make/hotspot/lib/CompileJvm.gmk
+++ b/make/hotspot/lib/CompileJvm.gmk
@@ -179,6 +179,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBJVM, \
     EXCLUDE_PATTERNS := $(JVM_EXCLUDE_PATTERNS), \
     DEFAULT_CFLAGS := false, \
     CFLAGS := $(JVM_CFLAGS), \
+    CFLAGS_windows := -DGTEST_HAS_SEH=0, \
     abstract_vm_version.cpp_CXXFLAGS := $(CFLAGS_VM_VERSION), \
     arguments.cpp_CXXFLAGS := $(CFLAGS_VM_VERSION), \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc), \

This manual approach of finding the minimal change needed was tedious so I decided to add my own defines to see which portions of the gmk files are used and for which compile/link commands:

diff --git a/make/hotspot/lib/CompileGtest.gmk b/make/hotspot/lib/CompileGtest.gmk
index d2cdc7685c9..acf0eae159a 100644
--- a/make/hotspot/lib/CompileGtest.gmk
+++ b/make/hotspot/lib/CompileGtest.gmk
@@ -68,7 +68,7 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBGTEST, \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include, \
-    CFLAGS_windows := -EHsc, \
+    CFLAGS_windows := -EHsc -DGTEST_HAS_SEH=0 -DMYTEST_LOCATION1, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     OPTIMIZATION := $(JVM_OPTIMIZATION), \
     COPY_DEBUG_SYMBOLS := $(GTEST_COPY_DEBUG_SYMBOLS), \
@@ -98,7 +98,7 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBJVM, \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include \
         $(addprefix -I, $(GTEST_TEST_SRC)), \
-    CFLAGS_windows := -EHsc, \
+    CFLAGS_windows := -EHsc -DGTEST_HAS_SEH=0 -DMYTEST_LOCATION2, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc) \
         undef stringop-overflow, \
@@ -152,6 +152,7 @@ $(eval $(call SetupJdkExecutable, BUILD_GTEST_LAUNCHER, \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include, \
+    CFLAGS_windows := -EHsc -DGTEST_HAS_SEH=0 -DMYTEST_LOCATION3, \
     LD_SET_ORIGIN := false, \
     LDFLAGS_unix := $(call SET_SHARED_LIBRARY_ORIGIN), \
     JDK_LIBS := gtest:libjvm, \
diff --git a/make/hotspot/lib/CompileJvm.gmk b/make/hotspot/lib/CompileJvm.gmk
index 6b5edc85b23..5994ffc6be1 100644
--- a/make/hotspot/lib/CompileJvm.gmk
+++ b/make/hotspot/lib/CompileJvm.gmk
@@ -179,6 +179,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBJVM, \
     EXCLUDE_PATTERNS := $(JVM_EXCLUDE_PATTERNS), \
     DEFAULT_CFLAGS := false, \
     CFLAGS := $(JVM_CFLAGS), \
+    CFLAGS_windows := -DGTEST_HAS_SEH=0 -DMYTEST_LOCATION0, \
     abstract_vm_version.cpp_CXXFLAGS := $(CFLAGS_VM_VERSION), \
     arguments.cpp_CXXFLAGS := $(CFLAGS_VM_VERSION), \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc), \

Location 1 only showed up for the 2 files below (matching the INCLUDE_FILES in make/hotspot/lib/CompileGtest.gmk), which made it clear that -DGTEST_HAS_SEH=0 was needed in this section.

  1. /cygdrive/c/repos/googletest/googlemock/src/gmock-all.cc
  2. /cygdrive/c/repos/googletest/googletest/src/gtest-all.cc

For location 2, there were 214 lines matching the regex DMYTEST_LOCATION2.+.cpp and 213 lines matching the regex DMYTEST_LOCATION2.+/test/hotspot/gtest/.+.cpp. The location 2 define was therefore correctly scoped to the gtests only. These 213 lines compiled files like test_blocktree.cpp and test_vmerror.cpp. The line that was different between the 2 regexes was the one compiling build/windows-x86_64-server-slowdebug/hotspot/variant-server/libjvm/gtest/objs/BUILD_GTEST_LIBJVM_pch.cpp. Location 3 was only used for compiling test/hotspot/gtest/gtestLauncher.cpp. The challenging case was location 0, which seemed to appear for way more files than it should. Was it really necessary? No it wasn’t! That made life much easier for me.

Inspecting gTest Code Coverage

In the course of this investigation, I considered using time travel debugging to see which code paths were executed. An alternative was to see whether the exception filter code was covered at the end of the gtest execution! The path to the code coverage tool is C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\Extensions\Microsoft\CodeCoverage.Console\Microsoft.CodeCoverage.Console.exe – it should be in the path by default in the Developer Command Prompt. I kicked it off with these commands:

cd build/windows-x86_64-server-slowdebug/images/test/hotspot/gtest/server
mkdir ../server-orig
cp * ../server-orig

Microsoft.CodeCoverage.Console.exe instrument gtestLauncher.exe
Microsoft.CodeCoverage.Console.exe instrument jvm.dll

Microsoft.CodeCoverage.Console.exe collect "gtestLauncher.exe -jdk:D:\java\forks\openjdk\jdk\build\windows-x86_64-server-slowdebug\images\jdk"

start output.coverage

Search for “exceptionfilter” in the Code Coverage Results pane to view the code coverage for the exception filter.

Verifying Execution Path

The first time I paused execution of the gtests in the debugger, stopped in jdk/src/hotspot/share/utilities/bitMap.cpp. I set a breakpoint there. I liked this location because I could move execution into the failure path (in the assembly view). This was how I saw the gtest structured exception handler kicking in without the JVM’s failure reporting code executing. With the tests now passing, I found the write to the poison location just going through without interruption. Did this mean the test was broken? Or did it mean that the exception filter ran and successfully said to continue execution? I think it has to be the latter but I’ll need time travel debugging to verify this. In the meantime, I sought to at least ensure there were multiple test processes involved.

Verifying Multiple Processes are Started

I started Process Monitor and added a filter for path containing “slowdebug”. Notice tons of PIDs for gtestlauncher.exe in the process monitor screenshot below (as expected).

I could successfully execute the error handling path by manually moving the program counter (RIP) after skipping into the failure path of BitMap::verify_range. Why didn’t the PID change in the debugger? Oh wait, was I was still stepping thereby causing recursion? This callstack did not support that hypothesis. Looks like it was just error reporting continuing to execute.

>	jvm.dll!BitMap::verify_range(unsigned __int64 beg, unsigned __int64 end) Line 212	C++
 	jvm.dll!BitMap::clear_range(unsigned __int64 beg, unsigned __int64 end) Line 280	C++
 	jvm.dll!JVMFlag::printFlags(outputStream * out, bool withComments, bool printRanges, bool skipDefaults) Line 706	C++
 	jvm.dll!VMError::report(outputStream * st, bool _verbose) Line 1260	C++
 	jvm.dll!VMError::report_and_die(int id, const char * message, const char * detail_fmt, char * detail_args, Thread * thread, unsigned char * pc, const void * siginfo, const void * context, const char * filename, int lineno, unsigned __int64 size) Line 1847	C++
 	jvm.dll!report_vm_error(const char * file, int line, const char * error_msg, const char * detail_fmt, ...) Line 195	C++
 	jvm.dll!CompressedKlassPointers::check_init<int>(int var) Line 154	C++
 	jvm.dll!CompressedKlassPointers::shift() Line 218	C++
 	jvm.dll!CompressedKlassPointers::print_mode(outputStream * st) Line 301	C++
 	jvm.dll!VMError::report(outputStream * st, bool _verbose) Line 1196	C++
 	jvm.dll!VMError::report_and_die(int id, const char * message, const char * detail_fmt, char * detail_args, Thread * thread, unsigned char * pc, const void * siginfo, const void * context, const char * filename, int lineno, unsigned __int64 size) Line 1847	C++
 	jvm.dll!report_vm_error(const char * file, int line, const char * error_msg, const char * detail_fmt, ...) Line 195	C++
 	jvm.dll!BitMap::verify_limit(unsigned __int64 bit) Line 206	C++
 	jvm.dll!BitMap::to_words_align_down(unsigned __int64 bit) Line 94	C++
 	jvm.dll!BitMap::word_addr(unsigned __int64 bit) Line 144	C++
 	jvm.dll!BitMap::set_bit(unsigned __int64 bit) Line 37	C++
 	jvm.dll!JfrEventVerifier::set_field_bit(unsigned __int64 field_idx) Line 41	C++
 	jvm.dll!JfrEvent<EventTenuringDistribution>::set_field_bit(unsigned __int64 field_idx) Line 267	C++
 	jvm.dll!EventObjectAllocationOutsideTLAB::set_objectClass(const Klass * new_value) Line 7304	C++
 	jvm.dll!trace_flag_changed<bool,EventBooleanFlagChanged>(JVMFlag * flag, const bool old_value, const bool new_value, const JVMFlagOrigin origin) Line 39	C++
 	jvm.dll!TypedFlagAccessImpl<bool,EventBooleanFlagChanged>::check_constraint_and_set(JVMFlag * flag, void * value_addr, JVMFlagOrigin origin, bool verbose) Line 78	C++
 	jvm.dll!FlagAccessImpl_bool::set_impl(JVMFlag * flag, void * value_addr, JVMFlagOrigin origin) Line 98	C++
 	jvm.dll!FlagAccessImpl::set(JVMFlag * flag, void * value, JVMFlagOrigin origin) Line 49	C++
 	jvm.dll!JVMFlagAccess::set_impl(JVMFlag * flag, void * value, JVMFlagOrigin origin) Line 307	C++
 	jvm.dll!JVMFlagAccess::set_or_assert(JVMFlagsEnum flag_enum, int type_enum, void * value, JVMFlagOrigin origin) Line 353	C++
 	jvm.dll!JVMFlagAccess::set<bool,0>(JVMFlagsEnum flag_enum, bool value, JVMFlagOrigin origin) Line 101	C++
 	jvm.dll!Flag_UseLargePagesIndividualAllocation_set(bool value, JVMFlagOrigin origin) Line 69	C++
 	jvm.dll!os::init() Line 4436	C++
 	jvm.dll!Threads::create_vm(JavaVMInitArgs * args, bool * canTryAgain) Line 463	C++
 	jvm.dll!JNI_CreateJavaVM_inner(JavaVM_ * * vm, void * * penv, void * args) Line 3589	C++
 	jvm.dll!JNI_CreateJavaVM(JavaVM_ * * vm, void * * penv, void * args) Line 3680	C++
 	jvm.dll!init_jvm(int argc, char * * argv, bool disable_error_handling, JavaVM_ * * jvm_ptr) Line 94	C++
 	jvm.dll!JVMInitializerListener::OnTestStart(const testing::TestInfo & test_info) Line 124	C++
 	jvm.dll!testing::internal::TestEventRepeater::OnTestStart(const testing::TestInfo & parameter) Line 3858	C++
 	jvm.dll!testing::TestInfo::Run() Line 2821	C++
 	jvm.dll!testing::TestSuite::Run() Line 3015	C++
 	jvm.dll!testing::internal::UnitTestImpl::RunAllTests() Line 5920	C++
 	jvm.dll!testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,bool>(testing::internal::UnitTestImpl * object, bool(testing::internal::UnitTestImpl::*)() method, const char * location) Line 2614	C++
 	jvm.dll!testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,bool>(testing::internal::UnitTestImpl * object, bool(testing::internal::UnitTestImpl::*)() method, const char * location) Line 2648	C++
 	jvm.dll!testing::UnitTest::Run() Line 5484	C++
 	jvm.dll!RUN_ALL_TESTS() Line 2317	C++
 	jvm.dll!runUnitTestsInner(int argc, char * * argv) Line 290	C++
 	jvm.dll!runUnitTests(int argc, char * * argv) Line 371	C++
 	gtestLauncher.exe!main(int argc, char * * argv) Line 40	C++
 	[Inline Frame] gtestLauncher.exe!invoke_main() Line 78	C++
 	gtestLauncher.exe!__scrt_common_main_seh() Line 288	C++
 	kernel32.dll!00007ffdcbdce8d7()	Unknown
 	ntdll.dll!00007ffdcc97c34c()	Unknown

One advantage of the stack above is that it showed how the os::init code gets executed (which I was curious about when wondering whether the exception filter was being set up). Disabling the breakpoint just before skipping into the failure path and resuming execution now led to the JVM dying:

[==========] Running 1197 tests from 205 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from AltHashingTest
[ RUN      ] AltHashingTest.halfsiphash_test_ByteArray
[       OK ] AltHashingTest.halfsiphash_test_ByteArray (0 ms)
[ RUN      ] AltHashingTest.halfsiphash_test_CharArray
[       OK ] AltHashingTest.halfsiphash_test_CharArray (0 ms)
[ RUN      ] AltHashingTest.halfsiphash_test_FromReference
[       OK ] AltHashingTest.halfsiphash_test_FromReference (0 ms)
[----------] 3 tests from AltHashingTest (2 ms total)

[----------] 1 test from ThreadsListHandle
[ RUN      ] ThreadsListHandle.sanity_vm
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (d:\java\forks\openjdk\jdk\src\hotspot\share\utilities\bitMap.cpp:208), pid=39132, tid=109872
#  assert(bit <= _size) failed: BitMap limit out of bounds: 0 > 64
#
# JRE version:  ((uninitialized)) (slowdebug build )
# Java VM: OpenJDK 64-Bit Server VM (slowdebug 26-internal-adhoc.saint.jdk, mixed mode, sharing, tiered, compressed class ptrs, unknown gc, windows-amd64)
# Core dump will be written. Default location: D:\java\forks\openjdk\jdk\build\windows-x86_64-server-slowdebug\images\test\hotspot\gtest\server\hs_err_pid39132.mdmp
#
# An error report file with more information is saved as:
# D:\java\forks\openjdk\jdk\build\windows-x86_64-server-slowdebug\images\test\hotspot\gtest\server\hs_err_pid39132.log
#
#

D:\java\forks\openjdk\jdk\build\windows-x86_64-server-slowdebug\images\test\hotspot\gtest\server\gtestLauncher.exe (process 39132) exited with code 1 (0x1).
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

Here’s the stack for when the breakpoint is hit:

jvm.dll!BitMap::verify_limit(unsigned __int64 bit) Line 206
jvm.dll!BitMap::to_words_align_down(unsigned __int64 bit) Line 94
jvm.dll!BitMap::word_addr(unsigned __int64 bit) Line 144
jvm.dll!BitMap::set_bit(unsigned __int64 bit) Line 37
jvm.dll!JfrEventVerifier::set_field_bit(unsigned __int64 field_idx) Line 41
jvm.dll!JfrEvent<EventTenuringDistribution>::set_field_bit(unsigned __int64 field_idx) Line 267
jvm.dll!EventObjectAllocationOutsideTLAB::set_objectClass(const Klass * new_value) Line 7304
jvm.dll!trace_flag_changed<bool,EventBooleanFlagChanged>(JVMFlag * flag, const bool old_value, const bool new_value, const JVMFlagOrigin origin) Line 39
jvm.dll!TypedFlagAccessImpl<bool,EventBooleanFlagChanged>::check_constraint_and_set(JVMFlag * flag, void * value_addr, JVMFlagOrigin origin, bool verbose) Line 78
jvm.dll!FlagAccessImpl_bool::set_impl(JVMFlag * flag, void * value_addr, JVMFlagOrigin origin) Line 98
jvm.dll!FlagAccessImpl::set(JVMFlag * flag, void * value, JVMFlagOrigin origin) Line 49
jvm.dll!JVMFlagAccess::set_impl(JVMFlag * flag, void * value, JVMFlagOrigin origin) Line 307
jvm.dll!JVMFlagAccess::set_or_assert(JVMFlagsEnum flag_enum, int type_enum, void * value, JVMFlagOrigin origin) Line 353
jvm.dll!JVMFlagAccess::set<bool,0>(JVMFlagsEnum flag_enum, bool value, JVMFlagOrigin origin) Line 101
jvm.dll!Flag_UseLargePagesIndividualAllocation_set(bool value, JVMFlagOrigin origin) Line 69
jvm.dll!os::init() Line 4436
jvm.dll!Threads::create_vm(JavaVMInitArgs * args, bool * canTryAgain) Line 463
jvm.dll!JNI_CreateJavaVM_inner(JavaVM_ * * vm, void * * penv, void * args) Line 3589
jvm.dll!JNI_CreateJavaVM(JavaVM_ * * vm, void * * penv, void * args) Line 3680
jvm.dll!init_jvm(int argc, char * * argv, bool disable_error_handling, JavaVM_ * * jvm_ptr) Line 94
jvm.dll!JVMInitializerListener::OnTestStart(const testing::TestInfo & test_info) Line 124
jvm.dll!testing::internal::TestEventRepeater::OnTestStart(const testing::TestInfo & parameter) Line 3858
jvm.dll!testing::TestInfo::Run() Line 2821
jvm.dll!testing::TestSuite::Run() Line 3015
jvm.dll!testing::internal::UnitTestImpl::RunAllTests() Line 5920
jvm.dll!testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,bool>(testing::internal::UnitTestImpl * object, bool(testing::internal::UnitTestImpl::*)() method, const char * location) Line 2614
jvm.dll!testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,bool>(testing::internal::UnitTestImpl * object, bool(testing::internal::UnitTestImpl::*)() method, const char * location) Line 2648
jvm.dll!testing::UnitTest::Run() Line 5484
jvm.dll!RUN_ALL_TESTS() Line 2317
jvm.dll!runUnitTestsInner(int argc, char * * argv) Line 290
jvm.dll!runUnitTests(int argc, char * * argv) Line 371
gtestLauncher.exe!main(int argc, char * * argv) Line 40
[Inline Frame] gtestLauncher.exe!invoke_main() Line 78

Pull Request Feedback

With all this information at hand, I opened 8364664: gtest death tests failing on Windows by swesonga · Pull Request #26661 · openjdk/jdk. One reviewer asked about removing the -EHsc flag from the gtests altogether. I tried it with the change below:

diff --git a/make/hotspot/lib/CompileGtest.gmk b/make/hotspot/lib/CompileGtest.gmk
index d2cdc7685c9..2c6b5f23516 100644
--- a/make/hotspot/lib/CompileGtest.gmk
+++ b/make/hotspot/lib/CompileGtest.gmk
@@ -68,7 +68,6 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBGTEST, \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include, \
-    CFLAGS_windows := -EHsc, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     OPTIMIZATION := $(JVM_OPTIMIZATION), \
     COPY_DEBUG_SYMBOLS := $(GTEST_COPY_DEBUG_SYMBOLS), \
@@ -98,7 +97,6 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBJVM, \
         -I$(GTEST_FRAMEWORK_SRC)/googletest/include \
         -I$(GTEST_FRAMEWORK_SRC)/googlemock/include \
         $(addprefix -I, $(GTEST_TEST_SRC)), \
-    CFLAGS_windows := -EHsc, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc) \
         undef stringop-overflow, \

The build failed with this error:

c:\progra~1\mib055~1\2022\enterprise\vc\tools\msvc\14.44.35207\include\__msvc_ostream.hpp(781): error C2220: the following warning is treated as an error
c:\progra~1\mib055~1\2022\enterprise\vc\tools\msvc\14.44.35207\include\__msvc_ostream.hpp(781): warning C4530: C++ exception handler used, but unwind semantics are not enabled. Specify /EHsc
c:\progra~1\mib055~1\2022\enterprise\vc\tools\msvc\14.44.35207\include\__msvc_ostream.hpp(781): note: the template instantiation context (the oldest one first) is
c:\repos\googletest\googletest\include\gtest/gtest-message.h(118): note: see reference to function template instantiation 'std::basic_ostream<char,std::char_traits<char>> &std::operator <<<std::char_traits<char>>(std::basic_ostream<char,std::char_traits<char>> &,const char *)' being compiled

This is expected since the googletest code is using C++ exception handling. The more significant revelation for me is that other groups are running gtests on Windows with --gtest_catch_exceptions=0 which disables the inbuilt exception handler. This is done using the GTestWrapper. This comment is helpful because it has links to the earlier issues around this space and explicitly clarifies that C++ exceptions are not used in libjvm code. While this PR did not result in a patch, it was an educational investigation for me!


Categories: OpenJDK

ShowRegistersOnAssertTest Failure on Windows AArch64

I recently investigated why the ShowRegistersOnAssertTest.java test was failing on Windows AArch64. This test was originally added in [JDK-8191101] Show register content in hs-err file on assert – Java Bug System to “retrieve the current context when an assert happens and make that part of the hs-err file.” I searched the codebase for the strings Registers:" and "RAX=" (used in the test) and verified that the hserr output the test is examining is generated by the os::print_context method (see the x86_64 Windows os::print_context, Linux os::print_context, and BSD os::print_context implementations).

The aarch64 Windows os::print_context, Linux os::print_context, and BSD os::print_context implementations use different register names. BSD’s os::print_context writes x0= to the error log whereas aarch64 Windows os::print_context writes X0 =. The BSD implementation should fail this test as well but the log in Update the copyright year · swesonga/jdk@3470f00 did not show this test running! It also didn’t appear in jdk/test/hotspot/jtreg/ProblemList.txt. Actually, a search for the string ShowRegistersOnAssertTest yielded results in that test file only. That led me to review it more closely and find that it didn’t run on macos because the test requires Linux or Windows. However, the fix for the test failure on Windows was obviously using the correct pattern for Windows:

diff --git a/test/hotspot/jtreg/runtime/ErrorHandling/ShowRegistersOnAssertTest.java b/test/hotspot/jtreg/runtime/ErrorHandling/ShowRegistersOnAssertTest.java
index 3b038ebd8a0..b0138625450 100644
--- a/test/hotspot/jtreg/runtime/ErrorHandling/ShowRegistersOnAssertTest.java
+++ b/test/hotspot/jtreg/runtime/ErrorHandling/ShowRegistersOnAssertTest.java
@@ -76,7 +76,11 @@ private static void do_test(boolean do_assert, // true - assert, false - guarant
             } else if (Platform.isX86()) {
                 pattern = new Pattern[] { Pattern.compile("Registers:"), Pattern.compile("EAX=.*")};
             } else if (Platform.isAArch64()) {
-                pattern = new Pattern[] { Pattern.compile("Registers:"), Pattern.compile("R0=.*")};
+                if (Platform.isLinux()) {
+                    pattern = new Pattern[] { Pattern.compile("Registers:"), Pattern.compile("R0=.*")};
+                } else if (Platform.isWindows()) {
+                    pattern = new Pattern[] { Pattern.compile("Registers:"), Pattern.compile("X0 =.*")};
+                }
             } else if (Platform.isS390x()) {
                 pattern = new Pattern[] { Pattern.compile("General Purpose Registers:"),
                                           Pattern.compile("^-{26}$"),

To verify that the data in the error log matched what the test expected, I used this change:

diff --git a/test/hotspot/jtreg/runtime/ErrorHandling/ShowRegistersOnAssertTest.java b/test/hotspot/jtreg/runtime/ErrorHandling/ShowRegistersOnAssertTest.java
index 3b038ebd8a0..76917f06a02 100644
--- a/test/hotspot/jtreg/runtime/ErrorHandling/ShowRegistersOnAssertTest.java
+++ b/test/hotspot/jtreg/runtime/ErrorHandling/ShowRegistersOnAssertTest.java
@@ -87,7 +87,9 @@ private static void do_test(boolean do_assert, // true - assert, false - guarant
                 pattern = new Pattern[] { Pattern.compile("Registers:") };
             }
             // Pattern match the hs_err_pid file.
-            HsErrFileUtils.checkHsErrFileContent(hs_err_file, pattern, false);
+            boolean is_verbose = false;
+            boolean printHserrOnError = true;
+            HsErrFileUtils.checkHsErrFileContent(hs_err_file, pattern, null, true, is_verbose, printHserrOnError);
         }
     }

I filed [JDK-8366483] ShowRegistersOnAssertTest uses wrong register pattern string for Windows on AArch64 – Java Bug System and fixed the test bug in 8366483: ShowRegistersOnAssertTest uses wrong register pattern string for Windows on AArch64 by swesonga · Pull Request #27022 · openjdk/jdk.


Script for Parsing jtreg GitHub Actions Results

I was reviewing jtreg test failures in some GitHub actions last week. Since I was only interested in the failures, I decided to write a script to extract the failure details from the log files I had downloaded from GitHub. This looked like another task for the VS Code agent so I wrote the algorithm for extracting the failure details into individual text files, which would be easier to review. Below is the prompt I used.

write a python script that processes all the text files in a user-specified folder. For each text file:
- split it into sections using "--------------------------------------------------" as a separator
- generate a filename for the section from the first line after the separator. To do so, use the text after "TEST: " and replace /, #, and . with underscores and append the ".txt" extension.
- if the section contains the string "test jdk:" then write the content of the entire section to a file with the generated name.

I thought I would need to do at least some debugging of the script but I did not! The script worked flawlessly. I didn’t even need to execute it myself because the flow of using the agent included running the script on my raw folder and then generating a README file! See the script and the README at Add scripts generated by Claude Sonnet 4 VS Code agent · swesonga/scratchpad@f5e8057. The agent was using the Claude Sonnet 4 model.


Categories: OpenJDK, Testing

Running jtreg shell tests on Windows

As part of the jdk11u release process, I needed to run some shell tests on my Windows desktop to determine whether they failed due to a product issue or an environment issue. I defaulted to using my Git Bash environment instead of Cygwin. This post shares some errors I ran into as a result of the different shell environment. The key takeaway was to run such tests in Cygwin whenever I encountered path errors.

rmic Tests

The rmic Tools Reference page describes it as follows:

You use the rmic compiler to generate stub and skeleton class files using the Java Remote Method Protocol (JRMP).

The first test failure I investigated was in jdk11u/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh at jdk-11.0.28+6 · openjdk/jdk11u. I used my run-jtreg-test.sh script to execute it:

./run-jtreg-test.sh /d/java/ms/openjdk-jdk11u /d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 /c/java/binaries/jtreg/jtreg-7.4+1/lib/jtreg.jar test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh -nativepath:/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6-test-image/hotspot/jtreg/native

This is the actual command that was executed:

/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java -Xmx512m -jar /c/java/binaries/jtreg/jtreg-7.4+1/lib/jtreg.jar -agentvm -ignore:quiet -automatic -xml -vmoption:-Xmx512m -timeoutFactor:4 -concurrency:1 -testjdk:/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 -verbose:fail,error,summary -nativepath:/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6-test-image/hotspot/jtreg/native test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh

The script failed on my local machine with the error below:

STDOUT:
STDERR:
+ defdir=./default_output
+ refdir=./reference_output
+ rm -rf ./default_output ./reference_output
+ mkdir ./default_output ./reference_output
+ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic -classpath D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/rmi/rmic/defaultStubVersion/run.d -keep -nowrite -d ./default_output G1Impl
/mnt/d/java/ms/openjdk-jdk11u/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh: 49: /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic: not found

I confirmed that rmic.exe exists in the bin directory of the jdk.

$ ls -1 /d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic*
/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic.exe*

Changing the test to directly refer to rmic.exe instead of rmic (as shown in the diff below) resulted in the test passing on my machine. I concluded that this specific issue must therefore be a test bug (i.e. the test should avoid this issue on Windows). See Shell Tests in jtreg for possible ways to fix this. This is one reason why shell scripts are being discouraged for OpenJDK testing.

diff --git a/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh b/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh
index 02f71d0c81..0b80015cf4 100644
--- a/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh
+++ b/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh
@@ -46,8 +46,8 @@ refdir=./reference_output
 rm -rf $defdir $refdir
 mkdir $defdir $refdir

-${TESTJAVA}/bin/rmic -classpath ${TESTCLASSES:-.} -keep -nowrite -d $defdir G1Impl
-${TESTJAVA}/bin/rmic -classpath ${TESTCLASSES:-.} -keep -nowrite -d $refdir -v1.2 G1Impl
+${TESTJAVA}/bin/rmic.exe -classpath ${TESTCLASSES:-.} -keep -nowrite -d $defdir G1Impl
+${TESTJAVA}/bin/rmic.exe -classpath ${TESTCLASSES:-.} -keep -nowrite -d $refdir -v1.2 G1Impl

 diff -r $defdir $refdir

Notice that the diff(1) – Linux manual page command has an exit code of 0 if the files are identical. This test failed in the test environment with this warning from jdk11u/src/jdk.rmic/share/classes/sun/rmi/rmic/resources/rmic.properties at jdk-11.0.28+6 · openjdk/jdk11u

STDOUT:
Warning: generation and use of skeletons and static stubs for JRMP	
is deprecated. Skeletons are unnecessary, and static stubs have	
been superseded by dynamically generated stubs. Users are	
encouraged to migrate away from using rmic to generate skeletons and static	
stubs. See the documentation for java.rmi.server.UnicastRemoteObject.

This warning is output by the sun.rmi.rmic.Main.compile method. The arguments to rmic.exe are parsed by Main.parseArgs. The -v1.2 argument specifies the non-default generator to use. My local machine must be using a different default generator (or so I thought).

Outstanding Questions at this Point

  1. Why couldn’t rmic be found on my machine?
  2. Why did this test fail in the test environment?

jrunscript Test

The next test failure I dug into was jdk11u/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh at jdk-11.0.28+6 · openjdk/jdk11u. I switched to MSYS2 for this one.

./run-jtreg-test.sh /d/java/ms/openjdk-jdk11u /d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 /c/java/binaries/jtreg/jtreg-7.4+1/lib/jtreg.jar test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh -nativepath:/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6-test-image/hotspot/jtreg/native

This failed with another path error.

STDOUT:
STDERR:
/mnt/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh: 33: .: cannot open D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh: No such file

My first thought was “where is the TESTSRC set“? It actually appeared in the jtreg output, confirming that it was correctly set:

TESTFILE=D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh \
TESTSRC=D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript \
TESTSRCPATH=D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript \

So what does the failing line do? I asked Copilot: what does this command do in a bash script? . ${TESTSRC-.}/common.sh

  1. . (dot command)
    This is shorthand for source, which executes the contents of a file in the current shell environment rather than in a subshell. This means any variables, functions, or changes made by the script will persist in the current shell.
  2. ${TESTSRC-.}
    This is a parameter expansion. It means:
    • Use the value of the TESTSRC environment variable if it is set.
    • If TESTSRC is not set, use . (the current directory) as the default.
  3. /common.sh
    This appends /common.sh to the path determined above.

This error was a Windows path issue again. Notepad++ could open D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh so the path itself was fine. I used this workaround:

diff --git a/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh b/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
index 6a3d4e76ff..94001967b0 100644
--- a/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
+++ b/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
@@ -30,10 +30,11 @@
 # @run shell jrunscript-eTest.sh
 # @summary Test that output of 'jrunscript -e' matches the dash-e.out file

-. ${TESTSRC-.}/common.sh
+comn=`/mnt/c/software/msys64/usr/bin/cygpath.exe ${TESTSRC-.}/common.sh`
+. "/mnt$comn"

 setup
-${JAVA} ${TESTVMOPTS} ${TESTJAVAOPTS} -cp ${TESTCLASSES} CheckEngine
+${JAVA}.exe ${TESTVMOPTS} ${TESTJAVAOPTS} -cp ${TESTCLASSES} CheckEngine
 if [ $? -eq 2 ]; then
     echo "No js engine found and engine not required; test vacuously passes."
     exit 0

This got me to the actual test error:

STDOUT:
Output of jrunscript -e differ from expected output. Failed.
STDERR:
Warning: Nashorn engine is planned to be removed from a future JDK release
diff: D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/dash-e.out: No such file or directory

I couldn’t see which command generated the output though, so I added set -ex to the top of the script (like run.sh in the previous test). This was the resulting output:

STDOUT:
STDERR:
+ /mnt/c/software/msys64/usr/bin/cygpath.exe D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ comn=/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ . /mnt/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ setup
+ [ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 =  ]
+ [ D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/tools/jrunscript/jrunscript-eTest.d =  ]
+ [ D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript =  ]
+ uname -s
+ OS=Linux
+ PS=:
+ FS=/
+ golden_diff=diff
+ JRUNSCRIPT=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/jrunscript
+ JAVAC=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/javac
+ JAVA=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java
+ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java.exe -Xmx512m -cp D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/tools/jrunscript/jrunscript-eTest.d CheckEngine
Warning: Nashorn engine is planned to be removed from a future JDK release
+ [ 0 -eq 2 ]
+ rm -f jrunscript-eTest.out
+ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/jrunscript -J-Dnashorn.args.prepend=--no-deprecation-warning -J-Djava.awt.headless=true -l nashorn -e println('hello')

Aha! Notice the root cause of the filename issues: OS=Linux! This also confirmed that I was using the same diff command. I didn’t get the error message at jdk11u/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh at jdk-11.0.28+5 · openjdk/jdk11u because because of set -ex (the e means exit immediately if any command fails, which does not match the behavior this test requires). After removing the e, I got this output:

STDOUT:
Output of jrunscript -e differ from expected output. Failed.
STDERR:
+ /mnt/c/software/msys64/usr/bin/cygpath.exe D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ comn=/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ . /mnt/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ setup
+ [ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 =  ]
+ [ D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/tools/jrunscript/jrunscript-eTest.d =  ]
+ [ D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript =  ]
+ uname -s
+ OS=Linux
+ PS=:
+ FS=/
+ golden_diff=diff
+ JRUNSCRIPT=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/jrunscript
+ JAVAC=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/javac
+ JAVA=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java
+ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java.exe -Xmx512m -cp D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/tools/jrunscript/jrunscript-eTest.d CheckEngine
Warning: Nashorn engine is planned to be removed from a future JDK release
+ [ 0 -eq 2 ]
+ rm -f jrunscript-eTest.out
+ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/jrunscript -J-Dnashorn.args.prepend=--no-deprecation-warning -J-Djava.awt.headless=true -l nashorn -e println('hello')
+ diff jrunscript-eTest.out D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/dash-e.out
diff: D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/dash-e.out: No such file or directory
+ [ 2 != 0 ]
+ echo Output of jrunscript -e differ from expected output. Failed.
+ rm -f jrunscript-eTest.out
+ exit 1

The ls command confirmed that the file exists:

$ ls -1 `cygpath D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/dash-e.out`
/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/dash-e.out

I patched the script as shown in the next diff:

diff --git a/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh b/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
index 6a3d4e76ff..4c7130857e 100644
--- a/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
+++ b/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
@@ -30,10 +30,13 @@
 # @run shell jrunscript-eTest.sh
 # @summary Test that output of 'jrunscript -e' matches the dash-e.out file

-. ${TESTSRC-.}/common.sh
+set -x
+
+comn=`/mnt/c/software/msys64/usr/bin/cygpath.exe ${TESTSRC-.}/common.sh`
+. "/mnt$comn"

 setup
-${JAVA} ${TESTVMOPTS} ${TESTJAVAOPTS} -cp ${TESTCLASSES} CheckEngine
+${JAVA}.exe ${TESTVMOPTS} ${TESTJAVAOPTS} -cp ${TESTCLASSES} CheckEngine
 if [ $? -eq 2 ]; then
     echo "No js engine found and engine not required; test vacuously passes."
     exit 0
@@ -44,7 +47,9 @@ fi
 rm -f jrunscript-eTest.out 2>/dev/null
 ${JRUNSCRIPT} -J-Dnashorn.args.prepend=--no-deprecation-warning -J-Djava.awt.headless=true -l nashorn -e "println('hello')" > jrunscript-eTest.out 2>&1

-$golden_diff jrunscript-eTest.out ${TESTSRC}/dash-e.out
+diffarg=`/mnt/c/software/msys64/usr/bin/cygpath.exe ${TESTSRC}/dash-e.out`
+
+$golden_diff jrunscript-eTest.out "/mnt$diffarg"
 if [ $? != 0 ]
 then
   echo "Output of jrunscript -e differ from expected output. Failed."

Avoiding Path Issues

This madness (in the diff above) that made me realize that I needed to fix the path issues and that perhaps Cygwin was the better environment for these tests. Sure enough, the test passed the first time I executed it in Cygwin:

./run-jtreg-test.sh /cygdrive/d/java/ms/dups/openjdk-jdk11u D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 C:/java/binaries/jtreg/jtreg-7.4+1/lib/jtreg.jar test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh -nativepath:D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6-test-image/hotspot/jtreg/native
Executing: D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java -Xmx512m -jar C:/java/binaries/jtreg/jtreg-7.4+1/lib/jtreg.jar -agentvm -ignore:quiet -automatic -xml -vmoption:-Xmx512m -timeoutFactor:4 -concurrency:1 -testjdk:D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 -verbose:fail,error,summary -nativepath:D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6-test-image/hotspot/jtreg/native test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
XML output  to D:\java\ms\dups\openjdk-jdk11u\JTwork
Passed: sun/tools/jrunscript/jrunscript-eTest.sh
Test results: passed: 1

Moral of the story: run OpenJDK shell tests on Windows in the Cygwin environment! At this point, the only outstanding question is about the difference between the local machine and the test environment the build was executed in. I thought that the fact that I didn’t get the rmic.jrmp.stubs.deprecated warning meant that a different default generator was used on my machine. However, I realized that the stdout messages were not being displayed! That warning was present in the output but I needed to open JTwork\sun\rmi\rmic\defaultStubVersion\run.jtr to see it!

----------System.out:(11/743)----------
Warning: generation and use of skeletons and static stubs for JRMP	
is deprecated. Skeletons are unnecessary, and static stubs have	
been superseded by dynamically generated stubs. Users are	
encouraged to migrate away from using rmic to generate skeletons and static	
stubs. See the documentation for java.rmi.server.UnicastRemoteObject.
Warning: generation and use of skeletons and static stubs for JRMP	
is deprecated. Skeletons are unnecessary, and static stubs have	
been superseded by dynamically generated stubs. Users are	
encouraged to migrate away from using rmic to generate skeletons and static	
stubs. See the documentation for java.rmi.server.UnicastRemoteObject.
TEST PASSED: default output identical to -v1.2 output
----------System.err:(8/706)----------
+ defdir=./default_output
+ refdir=./reference_output
+ rm -rf ./default_output ./reference_output
+ mkdir ./default_output ./reference_output
+ D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic -classpath D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/rmi/rmic/defaultStubVersion/run.d -keep -nowrite -d ./default_output G1Impl
+ D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic -classpath D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/rmi/rmic/defaultStubVersion/run.d -keep -nowrite -d ./reference_output -v1.2 G1Impl
+ diff -r ./default_output ./reference_output
+ echo 'TEST PASSED: default output identical to -v1.2 output'

This was sufficient for me to confirm that the build behaves as expected in this scenario.


Categories: OpenJDK

Building the OpenJDK Zero Variant on Windows

I was recently investigating the behavior of the OpenJDK interpreter on Windows. Demystifying the JVM: JVM Variants, Cppinterpreter and TemplateInterpreter describes two different interpreter implementations. I wanted to try reproducing the bug I was investigating using the simple BytecodeInterpreter. I was wondering how to enable it but the post led me to the realization that it’s not a configurable option. I needed to build the JVM specifically to include this interpreter. We need to get Julian Waters’ Windows/Zero branch with the changes I made to build using Visual C++: swesonga/jdk at swesonga/TheShermanTanker/experimental. The next sections cover the steps for building for x64 and ARM64.

Building Zero for Windows x64

Build libffi for Windows x64 (see Building libffi for Windows x64 with Visual C++) then set up the repo for use by OpenJDK as follows:

cd /c/repos/libffi
mkdir lib
cp x86_64-w64-mingw32/.libs/libffi-8.lib lib/libffi.lib
cp x86_64-w64-mingw32/include/ffi.h include/
cp x86_64-w64-mingw32/include/ffitarget.h include/

Configure the OpenJDK repo using these commands:

git checkout swesonga/TheShermanTanker/experimental

bash configure --with-jvm-variants=zero --with-libffi=/cygdrive/c/repos/libffi --disable-warnings-as-errors --with-debug-level=slowdebug --with-jtreg=/cygdrive/c/java/binaries/jtreg/jtreg-7.5.1+1 --with-gtest=/cygdrive/c/repos/googletest --with-extra-ldflags=-profile --with-boot-jdk=/cygdrive/c/java/binaries/jdk/x64/jdk-24+36

Build OpenJDK:

time /cygdrive/c/repos/scratchpad/scripts/java/cygwin/build-jdk.sh windows x86_64 slowdebug zero

The build will fail with Error: Failed to load D:\java\forks\dups11\openjdk\jdk\build\windows-x86_64-zero-slowdebug\jdk\bin\zero\jvm.dll. However, the build is still usable (all the binaries and symbols should be present). I am saving that investigation for another day. For now, verify that the build works by running this command:

$ build/windows-x86_64-zero-slowdebug/jdk/bin/java.exe -version
java version "25-internal" 2025-09-16
Java Runtime Environment (slowdebug build 25-internal-adhoc.USERsaint.jdk)
Java HotSpot 64-Bit Zero VM (slowdebug build 25-internal-adhoc.USERsaint.jdk, interpreted mode)

Building Zero for Windows AArch64

Once the steps in Building libffi for Windows ARM64 with Visual C++ in MSYS are complete, set up the repo for use by OpenJDK as follows:

cd /c/repos/libffi
mkdir lib
cp aarch64-w64-mingw32/.libs/libffi-8.lib lib/libffi.lib
cp aarch64-w64-mingw32/include/ffi.h include/
cp aarch64-w64-mingw32/include/ffitarget.h include/

Configure the OpenJDK repo using these commands:

git checkout swesonga/TheShermanTanker/experimental

bash configure --with-jvm-variants=zero --with-libffi=/cygdrive/d/repos/dups/libffi --disable-warnings-as-errors --with-debug-level=slowdebug --with-jtreg=/cygdrive/c/java/binaries/jtreg/jtreg-7.5.1+1 --with-gtest=/cygdrive/c/repos/googletest --with-extra-ldflags=-profile --with-boot-jdk=/cygdrive/c/java/binaries/jdk/x64/jdk-24+36

Build OpenJDK:

time /cygdrive/c/repos/scratchpad/scripts/java/cygwin/build-jdk.sh windows aarch64 slowdebug zero

Background Investigation

OpenJDK’s building.md says to use --with-jvm-variants configure argument to specify the “zero” variant:

bash configure --with-jvm-variants=zero --with-debug-level=slowdebug --with-jtreg=/cygdrive/c/java/binaries/jtreg/jtreg-7.5.1+1 --with-gtest=/cygdrive/c/repos/googletest --with-extra-ldflags=-profile --with-boot-jdk=/cygdrive/c/java/binaries/jdk/x64/jdk-24+36

The configure script failed with this error:

checking for --enable-hsdis-bundling... disabled, default
checking what hsdis backend to use... 'none', hsdis will not be built
checking if hsdis should be bundled... no
checking for --enable-libffi-bundling... disabled, default
checking for LIBFFI... checking for ffi.h... no
configure: error: Could not find libffi!
configure exiting with result code 1

I took at look at the source of the error message in jdk/make/autoconf/lib-ffi.m4 and realized that I need to have the repo.

git clone https://github.com/libffi/libffi.git
cd libffi
git checkout v3.4.8

I then added the --with-libffi argument to the configure script.

bash configure --with-jvm-variants=zero --with-debug-level=slowdebug --with-jtreg=/cygdrive/c/java/binaries/jtreg/jtreg-7.5.1+1 --with-gtest=/cygdrive/c/repos/googletest --with-extra-ldflags=-profile --with-boot-jdk=/cygdrive/c/java/binaries/jdk/x64/jdk-24+36 --with-libffi=/cygdrive/d/repos/libffi

This didn’t address the failure but closer inspection of the .m4 led me to realize that I need to build libffi. This was the genesis of the post on Building libffi for Windows x64 with Visual C++. Once I had built libffi and created the .lib file in the expected location of its repo, the configure script succeeded. I started the build with this command:

time /cygdrive/c/repos/scratchpad/scripts/java/cygwin/build-jdk.sh windows x86_64 slowdebug zero

The build failed!

* For target hotspot_variant-zero_libjvm_gtest_objs_BUILD_GTEST_LIBJVM_pch.obj:
BUILD_GTEST_LIBJVM_pch.cpp
d:\java\forks\dups11\openjdk\jdk\src\hotspot\share\runtime/globals.hpp(35): fatal error C1083: Cannot open include file: 'globals_windows_zero.hpp': No such file or directory
   ... (rest of output omitted)

This failure reminded me of the Windows/Zero mail where Julian Waters fixed build zero for windows (although his suggestion to fix this was getting shot down). I was wondering whether this was working at some point in a previous release so I tried openjdk/jdk11u-dev at c5407b6a8464fcc1eed31a2e9e30651e9011dbd3. I got the same error. The next step was to build Julian Waters’ Windows/Zero branch: Windows/Zero Port · TheShermanTanker/jdk@f504cd8. I imported it into my fork as follows:

git remote add TheShermanTanker https://github.com/TheShermanTanker/jdk
git fetch TheShermanTanker
git checkout experimental
git checkout -b swesonga/TheShermanTanker/experimental
git push --set-upstream origin swesonga/TheShermanTanker/experimental

Building this branch failed with this error:

* For target support_gensrc_java.base__SocketOptionRegistry.java:
/*
 * Copyright (c) 2008, 2020, Oracle and/or its affiliates. All rights reserved.
 *
...
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
 * version 2 for more details (a copy is included in the LICENSE file that
   ... (rest of output omitted)

* All command lines available in /cygdrive/d/java/forks/dups11/openjdk/jdk/build/windows-x86_64-zero-slowdebug/make-support/failure-logs.
=== End of repeated output ===

There was a .log file and a .cmd file in the failure-logs folder. build\windows-x86_64-zero-slowdebug\make-support\failure-logs\support_gensrc_java.base__SocketOptionRegistry.java.log contained this:

/*
 * Copyright (c) 2008, 2020, Oracle and/or its affiliates. All rights reserved.
 *
 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
 *
 * This code is free software; you can redistribute it and/or modify it
 * under the terms of the GNU General Public License version 2 only, as
 * published by the Free Software Foundation.  Oracle designates this
 * particular file as subject to the "Classpath" exception as provided
 * by Oracle in the LICENSE file that accompanied this code.
 *
 * This code is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
 * version 2 for more details (a copy is included in the LICENSE file that
 * accompanied this code).
 *
 * You should have received a copy of the GNU General Public License version
 * 2 along with this work; if not, write to the Free Software Foundation,
 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
 *
 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
 * or visit www.oracle.com if you need additional information or have any
 * questions.
 *
 */
/cygdrive/d/java/forks/dups11/openjdk/jdk/make/scripts/fixpath.sh: line 486: c:\progra~1\mib055~1\2022\enterp~1\vc\tools\msvc\1444~1.352\bin\hostx64\x64\cl.exe: command not found

build\windows-x86_64-zero-slowdebug\make-support\failure-logs\support_gensrc_java.base__SocketOptionRegistry.java.cmdline contained this:

( /usr/bin/gawk '/@@END_COPYRIGHT@@/{exit}1' /cygdrive/d/java/forks/dups11/openjdk/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template && /cygdrive/d/java/forks/dups11/openjdk/jdk/build/windows-x86_64-zero-slowdebug/fixpath exec /cygdrive/d/java/forks/dups11/openjdk/jdk/build/windows-x86_64-zero-slowdebug/fixpath exec /cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/tools/msvc/1444~1.352/bin/hostx64/x64/cl.exe -E -nologo -I/cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/tools/msvc/1444~1.352/include -I/cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/tools/msvc/1444~1.352/atlmfc/include -I/cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/auxili~1/vs/include -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100261~1.0/ucrt -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100261~1.0/um -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100261~1.0/shared -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100261~1.0/winrt -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100261~1.0/cppwinrt -I/cygdrive/c/progra~2/wi3cf2~1/netfxsdk/4.8/include/um -D_CRT_DECLARE_NONSTDC_NAMES -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_WARNINGS -nologo -MD -Zc:preprocessor -Zc:inline -Zc:throwingNew -permissive- -volatile:iso -utf-8 -Zc:wchar_t- -DLIBC=default -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0602 -DWIN32 -DIAL -DWINDOWS -DDEBUG -W3 -Z7 -experimental:deterministic -std:c11 -D_LITTLE_ENDIAN -DARCH='"amd64"' -Damd64 -D_LP64=1 -D_AMD64_ -Damd64 /cygdrive/d/java/forks/dups11/openjdk/jdk/src/java.base/share/classes/sun/nio/ch/SocketOptionRegistry.java.template 2> >(/usr/bin/grep -v '^SocketOptionRegistry.java.template$' >&2) | /usr/bin/gawk '/@@START_HERE@@/,0' | /usr/bin/sed -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED FILE - DO NOT EDIT/' -e 's/PREFIX_//' -e 's/^#.*//' ) > /cygdrive/d/java/forks/dups11/openjdk/jdk/build/windows-x86_64-zero-slowdebug/support/gensrc/java.base/sun/nio/ch/SocketOptionRegistry.java

This might be the first time I ever looked up AWK – Wikipedia since I needed to understand what was happening here. The hack I used to address this is Fix ‘command not found’ error · swesonga/jdk@6145ebb.

There were several warnings (e.g. warning C4267: ‘initializing’: conversion from ‘size_t’ to ‘int’, possible loss of data) and a build error that I fixed in Fix MSVC warnings and build errors · swesonga/jdk@bd519ea. I later realized that I could have used the --disable-warnings-as-errors configure argument but it was good for me to be aware of which issues I could run into later when using the zero interpreter. At this point, the build failed with this message:

* For target buildtools_create_symbols_javac__the.COMPILE_CREATE_SYMBOLS_batch:
Error: Failed to load D:\java\forks\dups11\openjdk\jdk\build\windows-x86_64-zero-slowdebug\jdk\bin\zero\jvm.dll

* All command lines available in /cygdrive/d/java/forks/dups11/openjdk/jdk/build/windows-x86_64-zero-slowdebug/make-support/failure-logs.

That error in the only line in the build\windows-x86_64-zero-slowdebug\make-support\failure-logs\buildtools_create_symbols_javac__the.COMPILE_CREATE_SYMBOLS_batch.log file. This was the .cmdline file (next to the .log file):

/cygdrive/d/java/forks/dups11/openjdk/jdk/build/windows-x86_64-zero-slowdebug/fixpath exec /cygdrive/d/java/forks/dups11/openjdk/jdk/build/windows-x86_64-zero-slowdebug/jdk/bin/javac -J-Djava.io.tmpdir=/cygdrive/d/java/forks/dups11/openjdk/jdk/build/windows-x86_64-zero-slowdebug/support/javatmp -g -Xlint:all -source 25 -target 25 -implicit:none -Xprefer:source -XDignore.symbol.file=true -encoding ascii --add-modules jdk.compiler,jdk.jdeps --add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-exports jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED --add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED --add-exports jdk.compiler/com.sun.tools.javac.jvm=ALL-UNNAMED -Werror -Xlint:-options -XDmodifiedInputs=/cygdrive/d/java/forks/dups11/openjdk/jdk/build/windows-x86_64-zero-slowdebug/buildtools/create_symbols_javac/_the.COMPILE_CREATE_SYMBOLS_batch.modfiles.fixed -d /cygdrive/d/java/forks/dups11/openjdk/jdk/build/windows-x86_64-zero-slowdebug/buildtools/create_symbols_javac @/cygdrive/d/java/forks/dups11/openjdk/jdk/build/windows-x86_64-zero-slowdebug/buildtools/create_symbols_javac/_the.COMPILE_CREATE_SYMBOLS_batch.filelist

However, the build process appeared to have actually created a usable JDK!

build/windows-x86_64-zero-slowdebug/jdk/bin/java.exe -version

That meant that I didn’t need to look any further into this error (at least not right away) so I moved on to building for the ARM64 platform. I initially tried cross compiling but this didn’t work. This is the configure command I tried for cross compiling.

bash configure --with-jvm-variants=zero --with-libffi=/cygdrive/d/repos/dups/libffi --openjdk-target=aarch64-unknown-cygwin --disable-warnings-as-errors --with-debug-level=slowdebug --with-jtreg=/cygdrive/c/java/binaries/jtreg/jtreg-7.5.1+1 --with-gtest=/cygdrive/c/repos/googletest --with-extra-ldflags=-profile --with-boot-jdk=/cygdrive/c/java/binaries/jdk/x64/jdk-24+36

The linker failed with a series of 19 unresolved symbols and a warning that the library machine type ‘ARM64’ conflicts with target machine type ‘x64’. Notice that the === Output from failing command(s) repeated here === section did not contain the linker warning LNK4272 since it omits the rest of the input after showing the first 14 lines. Scanning the build log is important in such cases where some of the output is omitted at the end.

...
fallbackLinker.o : error LNK2019: unresolved external symbol __imp_ffi_type_double referenced in function Java_jdk_internal_foreign_abi_fallback_LibFallback_ffi_1type_1double
fallbackLinker.o : error LNK2019: unresolved external symbol __imp_ffi_type_pointer referenced in function Java_jdk_internal_foreign_abi_fallback_LibFallback_ffi_1type_1pointer
d:\repos\dups\libffi\lib\libffi.lib : warning LNK4272: library machine type 'ARM64' conflicts with target machine type 'x64'
d:\java\forks\dups11\openjdk\jdk\build\windows-aarch64-zero-slowdebug\buildjdk\support\modules_libs\java.base\fallbackLinker.dll : fatal error LNK1120: 19 unresolved externals

At this point, I switched to my ARM64 machine (to avoid these mismatch issues) and copied the the libffi files onto it (after cloning the libffi repo). The --openjdk-target argument is no longer necessary for a native build on Windows ARM64. The build failed since the CONTEXT (x86 64-bit) struct (defined in C:\Program Files (x86)\Windows Kits\10\Include\10.0.26100.0\um\winnt.h is a platform specific struct. This fix was needed for ARM64: Fix build errors on Windows AArch64 · swesonga/jdk@d3ec3c7. OpenJDK now built successfully, well, until the same error about being unable to load jvm.dll.

I moved on to testing the build but this time it didn’t work! I added some debug output in Display GetLastError() on failure to load DLL · swesonga/jdk@ce7143e. GetLastError was 126, i.e. “the specified module could not be found” as per the System Error Codes (0-499) (WinError.h). I verified that the path to jvm.dll was valid then used Process Monitor to record events on the system while running java.exe. Turns out I needed to copy libffi-8.dll into the directory containing jvm.dll.