Saint's Log

Running jtreg shell tests on Windows

As part of the jdk11u release process, I needed to run some shell tests on my Windows desktop to determine whether they failed due to a product issue or an environment issue. I defaulted to using my Git Bash environment instead of Cygwin. This post shares some errors I ran into as a result of the different shell environment. The key takeaway was to run such tests in Cygwin whenever I encountered path errors.

rmic Tests

The rmic Tools Reference page describes it as follows:

You use the rmic compiler to generate stub and skeleton class files using the Java Remote Method Protocol (JRMP).

The first test failure I investigated was in jdk11u/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh at jdk-11.0.28+6 · openjdk/jdk11u. I used my run-jtreg-test.sh script to execute it:

./run-jtreg-test.sh /d/java/ms/openjdk-jdk11u /d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 /c/java/binaries/jtreg/jtreg-7.4+1/lib/jtreg.jar test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh -nativepath:/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6-test-image/hotspot/jtreg/native

This is the actual command that was executed:

/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java -Xmx512m -jar /c/java/binaries/jtreg/jtreg-7.4+1/lib/jtreg.jar -agentvm -ignore:quiet -automatic -xml -vmoption:-Xmx512m -timeoutFactor:4 -concurrency:1 -testjdk:/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 -verbose:fail,error,summary -nativepath:/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6-test-image/hotspot/jtreg/native test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh

The script failed on my local machine with the error below:

STDOUT:
STDERR:
+ defdir=./default_output
+ refdir=./reference_output
+ rm -rf ./default_output ./reference_output
+ mkdir ./default_output ./reference_output
+ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic -classpath D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/rmi/rmic/defaultStubVersion/run.d -keep -nowrite -d ./default_output G1Impl
/mnt/d/java/ms/openjdk-jdk11u/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh: 49: /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic: not found

I confirmed that rmic.exe exists in the bin directory of the jdk.

$ ls -1 /d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic*
/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic.exe*

Changing the test to directly refer to rmic.exe instead of rmic (as shown in the diff below) resulted in the test passing on my machine. I concluded that this specific issue must therefore be a test bug (i.e. the test should avoid this issue on Windows). See Shell Tests in jtreg for possible ways to fix this. This is one reason why shell scripts are being discouraged for OpenJDK testing.

diff --git a/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh b/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh
index 02f71d0c81..0b80015cf4 100644
--- a/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh
+++ b/test/jdk/sun/rmi/rmic/defaultStubVersion/run.sh
@@ -46,8 +46,8 @@ refdir=./reference_output
 rm -rf $defdir $refdir
 mkdir $defdir $refdir

-${TESTJAVA}/bin/rmic -classpath ${TESTCLASSES:-.} -keep -nowrite -d $defdir G1Impl
-${TESTJAVA}/bin/rmic -classpath ${TESTCLASSES:-.} -keep -nowrite -d $refdir -v1.2 G1Impl
+${TESTJAVA}/bin/rmic.exe -classpath ${TESTCLASSES:-.} -keep -nowrite -d $defdir G1Impl
+${TESTJAVA}/bin/rmic.exe -classpath ${TESTCLASSES:-.} -keep -nowrite -d $refdir -v1.2 G1Impl

 diff -r $defdir $refdir

Notice that the diff(1) – Linux manual page command has an exit code of 0 if the files are identical. This test failed in the test environment with this warning from jdk11u/src/jdk.rmic/share/classes/sun/rmi/rmic/resources/rmic.properties at jdk-11.0.28+6 · openjdk/jdk11u

STDOUT:
Warning: generation and use of skeletons and static stubs for JRMP	
is deprecated. Skeletons are unnecessary, and static stubs have	
been superseded by dynamically generated stubs. Users are	
encouraged to migrate away from using rmic to generate skeletons and static	
stubs. See the documentation for java.rmi.server.UnicastRemoteObject.

This warning is output by the sun.rmi.rmic.Main.compile method. The arguments to rmic.exe are parsed by Main.parseArgs. The -v1.2 argument specifies the non-default generator to use. My local machine must be using a different default generator (or so I thought).

Outstanding Questions at this Point

1. Why couldn’t rmic be found on my machine?
2. Why did this test fail in the test environment?

jrunscript Test

The next test failure I dug into was jdk11u/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh at jdk-11.0.28+6 · openjdk/jdk11u. I switched to MSYS2 for this one.

./run-jtreg-test.sh /d/java/ms/openjdk-jdk11u /d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 /c/java/binaries/jtreg/jtreg-7.4+1/lib/jtreg.jar test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh -nativepath:/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6-test-image/hotspot/jtreg/native

This failed with another path error.

STDOUT:
STDERR:
/mnt/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh: 33: .: cannot open D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh: No such file

My first thought was “where is the TESTSRC set“? It actually appeared in the jtreg output, confirming that it was correctly set:

TESTFILE=D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh \
TESTSRC=D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript \
TESTSRCPATH=D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript \

So what does the failing line do? I asked Copilot: what does this command do in a bash script? . ${TESTSRC-.}/common.sh

1. . (dot command)
   This is shorthand for source, which executes the contents of a file in the current shell environment rather than in a subshell. This means any variables, functions, or changes made by the script will persist in the current shell.
2. ${TESTSRC-.}
   This is a parameter expansion. It means:
   • Use the value of the TESTSRC environment variable if it is set.
   • If TESTSRC is not set, use . (the current directory) as the default.
3. /common.sh
   This appends /common.sh to the path determined above.

This error was a Windows path issue again. Notepad++ could open D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh so the path itself was fine. I used this workaround:

diff --git a/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh b/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
index 6a3d4e76ff..94001967b0 100644
--- a/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
+++ b/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
@@ -30,10 +30,11 @@
 # @run shell jrunscript-eTest.sh
 # @summary Test that output of 'jrunscript -e' matches the dash-e.out file

-. ${TESTSRC-.}/common.sh
+comn=`/mnt/c/software/msys64/usr/bin/cygpath.exe ${TESTSRC-.}/common.sh`
+. "/mnt$comn"

 setup
-${JAVA} ${TESTVMOPTS} ${TESTJAVAOPTS} -cp ${TESTCLASSES} CheckEngine
+${JAVA}.exe ${TESTVMOPTS} ${TESTJAVAOPTS} -cp ${TESTCLASSES} CheckEngine
 if [ $? -eq 2 ]; then
     echo "No js engine found and engine not required; test vacuously passes."
     exit 0

This got me to the actual test error:

STDOUT:
Output of jrunscript -e differ from expected output. Failed.
STDERR:
Warning: Nashorn engine is planned to be removed from a future JDK release
diff: D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/dash-e.out: No such file or directory

I couldn’t see which command generated the output though, so I added set -ex to the top of the script (like run.sh in the previous test). This was the resulting output:

STDOUT:
STDERR:
+ /mnt/c/software/msys64/usr/bin/cygpath.exe D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ comn=/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ . /mnt/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ setup
+ [ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 =  ]
+ [ D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/tools/jrunscript/jrunscript-eTest.d =  ]
+ [ D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript =  ]
+ uname -s
+ OS=Linux
+ PS=:
+ FS=/
+ golden_diff=diff
+ JRUNSCRIPT=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/jrunscript
+ JAVAC=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/javac
+ JAVA=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java
+ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java.exe -Xmx512m -cp D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/tools/jrunscript/jrunscript-eTest.d CheckEngine
Warning: Nashorn engine is planned to be removed from a future JDK release
+ [ 0 -eq 2 ]
+ rm -f jrunscript-eTest.out
+ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/jrunscript -J-Dnashorn.args.prepend=--no-deprecation-warning -J-Djava.awt.headless=true -l nashorn -e println('hello')

Aha! Notice the root cause of the filename issues: OS=Linux! This also confirmed that I was using the same diff command. I didn’t get the error message at jdk11u/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh at jdk-11.0.28+5 · openjdk/jdk11u because because of set -ex (the e means exit immediately if any command fails, which does not match the behavior this test requires). After removing the e, I got this output:

STDOUT:
Output of jrunscript -e differ from expected output. Failed.
STDERR:
+ /mnt/c/software/msys64/usr/bin/cygpath.exe D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ comn=/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ . /mnt/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/common.sh
+ setup
+ [ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 =  ]
+ [ D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/tools/jrunscript/jrunscript-eTest.d =  ]
+ [ D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript =  ]
+ uname -s
+ OS=Linux
+ PS=:
+ FS=/
+ golden_diff=diff
+ JRUNSCRIPT=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/jrunscript
+ JAVAC=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/javac
+ JAVA=/mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java
+ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java.exe -Xmx512m -cp D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/tools/jrunscript/jrunscript-eTest.d CheckEngine
Warning: Nashorn engine is planned to be removed from a future JDK release
+ [ 0 -eq 2 ]
+ rm -f jrunscript-eTest.out
+ /mnt/d/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/jrunscript -J-Dnashorn.args.prepend=--no-deprecation-warning -J-Djava.awt.headless=true -l nashorn -e println('hello')
+ diff jrunscript-eTest.out D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/dash-e.out
diff: D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/dash-e.out: No such file or directory
+ [ 2 != 0 ]
+ echo Output of jrunscript -e differ from expected output. Failed.
+ rm -f jrunscript-eTest.out
+ exit 1

The ls command confirmed that the file exists:

$ ls -1 `cygpath D:/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/dash-e.out`
/d/java/ms/openjdk-jdk11u/test/jdk/sun/tools/jrunscript/dash-e.out

I patched the script as shown in the next diff:

diff --git a/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh b/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
index 6a3d4e76ff..4c7130857e 100644
--- a/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
+++ b/test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
@@ -30,10 +30,13 @@
 # @run shell jrunscript-eTest.sh
 # @summary Test that output of 'jrunscript -e' matches the dash-e.out file

-. ${TESTSRC-.}/common.sh
+set -x
+
+comn=`/mnt/c/software/msys64/usr/bin/cygpath.exe ${TESTSRC-.}/common.sh`
+. "/mnt$comn"

 setup
-${JAVA} ${TESTVMOPTS} ${TESTJAVAOPTS} -cp ${TESTCLASSES} CheckEngine
+${JAVA}.exe ${TESTVMOPTS} ${TESTJAVAOPTS} -cp ${TESTCLASSES} CheckEngine
 if [ $? -eq 2 ]; then
     echo "No js engine found and engine not required; test vacuously passes."
     exit 0
@@ -44,7 +47,9 @@ fi
 rm -f jrunscript-eTest.out 2>/dev/null
 ${JRUNSCRIPT} -J-Dnashorn.args.prepend=--no-deprecation-warning -J-Djava.awt.headless=true -l nashorn -e "println('hello')" > jrunscript-eTest.out 2>&1

-$golden_diff jrunscript-eTest.out ${TESTSRC}/dash-e.out
+diffarg=`/mnt/c/software/msys64/usr/bin/cygpath.exe ${TESTSRC}/dash-e.out`
+
+$golden_diff jrunscript-eTest.out "/mnt$diffarg"
 if [ $? != 0 ]
 then
   echo "Output of jrunscript -e differ from expected output. Failed."

Avoiding Path Issues

This madness (in the diff above) that made me realize that I needed to fix the path issues and that perhaps Cygwin was the better environment for these tests. Sure enough, the test passed the first time I executed it in Cygwin:

./run-jtreg-test.sh /cygdrive/d/java/ms/dups/openjdk-jdk11u D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 C:/java/binaries/jtreg/jtreg-7.4+1/lib/jtreg.jar test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh -nativepath:D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6-test-image/hotspot/jtreg/native

Executing: D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/java -Xmx512m -jar C:/java/binaries/jtreg/jtreg-7.4+1/lib/jtreg.jar -agentvm -ignore:quiet -automatic -xml -vmoption:-Xmx512m -timeoutFactor:4 -concurrency:1 -testjdk:D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6 -verbose:fail,error,summary -nativepath:D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6-test-image/hotspot/jtreg/native test/jdk/sun/tools/jrunscript/jrunscript-eTest.sh
XML output  to D:\java\ms\dups\openjdk-jdk11u\JTwork
Passed: sun/tools/jrunscript/jrunscript-eTest.sh
Test results: passed: 1

Moral of the story: run OpenJDK shell tests on Windows in the Cygwin environment! At this point, the only outstanding question is about the difference between the local machine and the test environment the build was executed in. I thought that the fact that I didn’t get the rmic.jrmp.stubs.deprecated warning meant that a different default generator was used on my machine. However, I realized that the stdout messages were not being displayed! That warning was present in the output but I needed to open JTwork\sun\rmi\rmic\defaultStubVersion\run.jtr to see it!

----------System.out:(11/743)----------
Warning: generation and use of skeletons and static stubs for JRMP	
is deprecated. Skeletons are unnecessary, and static stubs have	
been superseded by dynamically generated stubs. Users are	
encouraged to migrate away from using rmic to generate skeletons and static	
stubs. See the documentation for java.rmi.server.UnicastRemoteObject.
Warning: generation and use of skeletons and static stubs for JRMP	
is deprecated. Skeletons are unnecessary, and static stubs have	
been superseded by dynamically generated stubs. Users are	
encouraged to migrate away from using rmic to generate skeletons and static	
stubs. See the documentation for java.rmi.server.UnicastRemoteObject.
TEST PASSED: default output identical to -v1.2 output
----------System.err:(8/706)----------
+ defdir=./default_output
+ refdir=./reference_output
+ rm -rf ./default_output ./reference_output
+ mkdir ./default_output ./reference_output
+ D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic -classpath D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/rmi/rmic/defaultStubVersion/run.d -keep -nowrite -d ./default_output G1Impl
+ D:/java/binaries/jdk/x64/2025-07/windows-jdk11u/jdk-11.0.28+6/bin/rmic -classpath D:/java/ms/openjdk-jdk11u/JTwork/classes/sun/rmi/rmic/defaultStubVersion/run.d -keep -nowrite -d ./reference_output -v1.2 G1Impl
+ diff -r ./default_output ./reference_output
+ echo 'TEST PASSED: default output identical to -v1.2 output'

This was sufficient for me to confirm that the build behaves as expected in this scenario.