When Experimenting with Async Profiler, I created a basic trial division factorization Java application. To run it, download the OpenJDK build if it isn’t already installed:
mkdir -p ~/java/binaries/jdk/x64
cd ~/java/binaries/jdk/x64
wget https://aka.ms/download-jdk/microsoft-jdk-17.0.7-linux-x64.tar.gz
tar xzf microsoft-jdk-17.0.7-linux-x64.tar.gz
Test the factorization application to verify that the Java build works.
export JAVA_HOME=~/java/binaries/jdk/x64/jdk-17.0.7+7
cd ~/repos/scratchpad/demos/java/FindPrimes
$JAVA_HOME/bin/javac Factorize.java
$JAVA_HOME/bin/java Factorize 123890571352112309857
# Use 4 threads to speed things up
$JAVA_HOME/bin/java Factorize 123890571352112309857 CUSTOM_THREAD_COUNT_VIA_THREAD_CLASS 4
Using hsdis
hsdis is a HotSpot plugin for disassembling dynamically generated code. Chriswhocodes was kind enough to build hsdis for various platforms and share the binaries on his website – hsdis HotSpot Disassembly Plugin Downloads (chriswhocodes.com). Download the appropriate hsdis binary and move it to the OpenJDK build’s lib directory, e.g.
wget https://chriswhocodes.com/hsdis/hsdis-amd64.so
export JAVA_HOME=~/java/binaries/jdk/x64/jdk-17.0.7+7
mv hsdis-amd64.so $JAVA_HOME/lib/
ls -l $JAVA_HOME/bin/hsdis*
We will need the PrintAssembly option to disassemble the code generated by the compiler when running a Java program. This option requires diagnostic VM options to be unlocked. This is the full command line for generating the disassembly from the application’s execution. The output is redirected to a code.asm file since it can be voluminous.
$JAVA_HOME/bin/java -XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly Factorize 123890571352112309857 CUSTOM_THREAD_COUNT_VIA_THREAD_CLASS 4 > code.asm
Here is a snippet of the disassembly in code.asm:
============================= C1-compiled nmethod ==============================
----------------------------------- Assembly -----------------------------------
Compiled method (c1) 2052 266 2 java.math.BigInteger::implMulAdd (81 bytes)
total in heap [0x00007f2e5943ca90,0x00007f2e5943d038] = 1448
relocation [0x00007f2e5943cbf0,0x00007f2e5943cc28] = 56
main code [0x00007f2e5943cc40,0x00007f2e5943ce00] = 448
stub code [0x00007f2e5943ce00,0x00007f2e5943ce30] = 48
metadata [0x00007f2e5943ce30,0x00007f2e5943ce38] = 8
scopes data [0x00007f2e5943ce38,0x00007f2e5943cee0] = 168
scopes pcs [0x00007f2e5943cee0,0x00007f2e5943d010] = 304
dependencies [0x00007f2e5943d010,0x00007f2e5943d018] = 8
nul chk table [0x00007f2e5943d018,0x00007f2e5943d038] = 32
--------------------------------------------------------------------------------
[Constant Pool (empty)]
--------------------------------------------------------------------------------
[Verified Entry Point]
# {method} {0x00000008000a47c0} 'implMulAdd' '([I[IIII)I' in 'java/math/BigInteger'
# parm0: rsi:rsi = '[I'
# parm1: rdx:rdx = '[I'
# parm2: rcx = int
# parm3: r8 = int
# parm4: r9 = int
# [sp+0x50] (sp of caller)
0x00007f2e5943cc40: mov %eax,-0x14000(%rsp)
0x00007f2e5943cc47: push %rbp
0x00007f2e5943cc48: sub $0x40,%rsp
0x00007f2e5943cc4c: movabs $0x7f2e38075370,%rax
0x00007f2e5943cc56: mov 0x8(%rax),%edi
0x00007f2e5943cc59: add $0x2,%edi
0x00007f2e5943cc5c: mov %edi,0x8(%rax)
0x00007f2e5943cc5f: and $0xffe,%edi
0x00007f2e5943cc65: cmp $0x0,%edi
0x00007f2e5943cc68: je 0x00007f2e5943cd52 ;*iload {reexecute=0 rethrow=0 return_oop=0}
; - java.math.BigInteger::implMulAdd@0 (line 3197)
0x00007f2e5943cc6e: movslq %r9d,%r9
0x00007f2e5943cc71: movabs $0xffffffff,%rax
0x00007f2e5943cc7b: and %rax,%r9
...
Finding the Java Installation Path
In the above example, I have used a Java build in a custom path. If you are using a Java build that is already installed, then a few extra steps might be needed to determine where the JAVA_HOME path, e.g.
saint@ubuntuvm:~$ which java
/usr/bin/java
saint@ubuntuvm:~$ ls -l `which java`
saint@ubuntuvm:~$ ls -l /etc/alternatives/java
8253757: Add LLVM-based backend for hsdis by magicus · Pull Request #7531 makes it possible to easily use LLVM as the hsdis backend. An LLVM installation is required for this. The official LLVM builds for the Windows platform do not work for building hsdis because they do not have all the prerequisite LLVM include files. See Building LLVM for Windows ARM64 – Saint’s Log (swesonga.org) for instructions on how to build LLVM for ARM64 Windows (on an x64 Windows host). To configure OpenJDK for LLVM as an hsdis backend on Windows ARM64, use this command:
bash configure --openjdk-target=aarch64-unknown-cygwin \
--with-hsdis=llvm \
--with-llvm=/cygdrive/d/dev/software/llvm-aarch64/
The JDK and hsdis can then be built as usual with these commands:
make images
make build-hsdis
make install-hsdis
cp /cygdrive/d/dev/software/llvm-aarch64/bin/LLVM-C.dll build/windows-aarch64-server-slowdebug/jdk/bin/
The generated JDK can then be deployed to an ARM64 machine like the Surface Pro X. To test LLVM’s disassembly, use the -XX:CompileCommand flag on the ARM64 machine:
/java -XX:CompileCommand="print java.lang.String::checkIndex" -version
Behind the Scenes
Missing Include File that Exists?
The path given to --with-llvm needs to be a Cygwin path if building in Cygwin. Otherwise, the build-hsdis target will fail with this error: c:\...\jdk\src\utils\hsdis\llvm\hsdis-llvm.cpp(58): fatal error C1083: Cannot open include file: 'llvm-c/Disassembler.h': No such file or directory. I caught this by inspecting build\windows-aarch64-server-release\make-support\failure-logs\support_hsdis_hsdis-llvm.obj.cmdline after the build failed. This was the only include that didn’t have Cygwin paths: -IC:/dev/repos/llvm-project/build_llvm_AArch64/install_local/include
Investigating Missing Disassembly
My first disassembly attempt did not work – only abstract disassembly was displayed:
...
# {method} {0x000002ca9940f2e8} 'checkIndex' '(II)V' in 'java/lang/String'
# parm0: c_rarg1 = int
# parm1: c_rarg2 = int
# [sp+0x30] (sp of caller)
0x000002ca87ad3940: 1f20 03d5 | e953 40d1 | 3f01 00f9 | ffc3 00d1 | fd7b 02a9 | a201 f837 | 3f00 026b | e200 0054
0x000002ca87ad3960: fd7b 42a9 | ffc3 0091
...
I verified that hsdis-aarch64.dll was present in the JDK’s bin folder. That was the only issue I had seen before that caused this behavior so I dug around to find the code that loads the hsdis DLL. A search for the “hsdis-” DLL prefix in the sources reveals the hsdis_library_name string used in the Disassembler::dll_load method. Notice that there is a Verbose flag that can display what is happening when loading the hsdis DLL!
void* Disassembler::dll_load(char* buf, int buflen, int offset, char* ebuf, int ebuflen, outputStream* st) {
int sz = buflen - offset;
int written = jio_snprintf(&buf[offset], sz, "%s%s", hsdis_library_name, os::dll_file_extension());
if (written < sz) { // written successfully, not truncated.
if (Verbose) st->print_cr("Trying to load: %s", buf);
return os::dll_load(buf, ebuf, ebuflen);
} else if (Verbose) {
st->print_cr("Try to load hsdis library failed: the length of path is beyond the OS limit");
}
return NULL;
}
This turns out to be a JVM flag! I try passing it to java.exe but -Verbose doesn’t do anything. Learn from HotSpot Command-Line Flags Overhaul – Design Doc – OpenJDK Wiki (java.net) that it’s a -XX: flag. Trying to use it causes JVM to complain that it is a develop-only flag.
Error: VM option 'Verbose' is develop and is available only in debug version of VM.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
The –enable-debug flag documented at Building OpenJDK (java.net) is required to generate a debug VM.
bash configure --openjdk-target=aarch64-unknown-cygwin \
--with-hsdis=llvm \
--with-llvm=/cygdrive/d/dev/software/llvm-aarch64/ \
--enable-debug
Running the debug JVM with the verbose flag now displays some diagnostic information:
.\java.exe -XX:CompileCommand="print java.lang.String::checkIndex" -XX:+Verbose -XX:+PrintMiscellaneous
CompileCommand: print java/lang/String.checkIndex bool print = true
============================= C1-compiled nmethod ==============================
----------------------------------- Assembly -----------------------------------
Trying to load: C:\dev\software\jdk-aarch64\jdk\bin\server\hsdis-aarch64.dll
Trying to load: C:\dev\software\jdk-aarch64\jdk\bin\server\hsdis-aarch64.dll
Trying to load: C:\dev\software\jdk-aarch64\jdk\bin\hsdis-aarch64.dll
Trying to load: hsdis-aarch64.dll
Could not load hsdis-aarch64.dll; Can't find dependent libraries; PrintAssembly defaults to abstract disassembly.
...
The error message substring “find dependent libraries” appears only once in the hotspot source code – in os::dll_load (which is called by Disassembler::dll_load). This error is displayed because LoadLibrary returns ERROR_MOD_NOT_FOUND.
The case of the DLL that refuses to load – The Old New Thing (microsoft.com) mentions loader snaps. The loader snaps are an option for the gflags tool found in the Windows Kits folder. The docs explain that GFlags is included in the Debugging Tools for Windows 10 (WinDbg) so a search for “Debugging Tools for Windows arm64” leads to Debugging ARM64 – Windows drivers. This says to install the Windows SDK, after which I now have the gflags binary (in the x86 folder)!
C:\Program Files (x86)\Windows Kits\10\Debuggers\arm64\gflags.exe
I still wasn’t sure how to see the snaps output. Show Loader Snaps in GFlags.exe, fails to capture any output in WinDbg – Stack Overflow implies that I should be able to use WinDbg to see what is failing to load.
Turns out the loader snaps aren’t really necessary. There is some critical info in the WinDbg diagnostic output:
2698:2e5c @ 03908953 - LdrpResolveDllName - ENTER: DLL name: .\LLVM-C.dll
2698:2e5c @ 03908953 - LdrpResolveDllName - RETURN: Status: 0xc0000135
...
2698:2e5c @ 03908968 - LdrpResolveDllName - ENTER: DLL name: C:\WINDOWS\LLVM-C.dll
2698:2e5c @ 03908968 - LdrpResolveDllName - RETURN: Status: 0xc0000135
...
2698:2e5c @ 03908968 - LdrpSearchPath - RETURN: Status: 0xc0000135
2698:2e5c @ 03908968 - LdrpProcessWork - ERROR: Unable to load DLL: "LLVM-C.dll", Parent Module: "C:\dev\software\jdk-aarch64\jdk\bin\hsdis-aarch64.dll", Status: 0xc0000135
hsdis-aarch64.dll is not being loaded because LLVM-C.dll cannot be found! Still learning the need for reading the full instructions to avoid unnecessary pain.
The post about Exploring the hsdis LLVM Support PR mentioned link errors when building hsdis using an LLVM backend on Windows (x86-64 host building JDK for the x86-64 platform). Before we look at why linking fails, we can get a simple repro for the error from the Cygwin logs. To get the command line used to invoke the linker, run make LOG=debug build-hsdis. Search the output for link.exe to find the failing command or open build\windows-x86_64-server-release\support\hsdis\BUILD_HSDIS_link.cmdline. Change the path from Cygwin to Windows style so that the command can be run in the x64 Native Tools Command Prompt.
cd C:\dev\repos\java\forks\jdk\build\windows-x86_64-server-release\support\hsdis\
c:\progra~2\micros~3\2019\enterp~1\vc\tools\msvc\1429~1.301\bin\hostx86\x64\link.exe -nologo -libpath:c:\dev\repos\llvm-project\build_llvm\install_local\\lib -dll -debug "-pdb:c:\dev\repos\java\forks\jdk\build\windows-x86_64-server-release\support\hsdis\hsdis.pdb" "-map:c:\dev\repos\java\forks\jdk\build\windows-x86_64-server-release\support\hsdis\hsdis.map" "-implib:c:\dev\repos\java\forks\jdk\build\windows-x86_64-server-release\support\hsdis\hsdis.lib" -libpath:c:\progra~2\micros~3\2019\enterp~1\vc\tools\msvc\1429~1.301\atlmfc\lib\x64 -libpath:c:\progra~2\micros~3\2019\enterp~1\vc\tools\msvc\1429~1.301\lib\x64 -libpath:c:\progra~2\wi3cf2~1\netfxsdk\4.8\lib\um\x64 -libpath:c:\progra~2\wi3cf2~1\10\lib\100190~1.0\ucrt\x64 -libpath:c:\progra~2\wi3cf2~1\10\lib\100190~1.0\um\x64 -out:c:\dev\repos\java\forks\jdk\build\windows-x86_64-server-release\support\hsdis\hsdis.dll c:\dev\repos\java\forks\jdk\build\windows-x86_64-server-release\support\hsdis\hsdis-llvm.obj c:\dev\repos\java\forks\jdk\build\windows-x86_64-server-release\support\hsdis\hsdis.dll.res
These are the resulting link errors mentioned in Exploring the hsdis LLVM Support PR.
Creating library c:\dev\repos\java\forks\jdk\build\windows-x86_64-server-release\support\hsdis\hsdis.lib and object c:\dev\repos\java\forks\jdk\build\windows-x86_64-server-release\support\hsdis\hsdis.exp
hsdis-llvm.obj : error LNK2019: unresolved external symbol LLVMCreateDisasm referenced in function "public: __cdecl hsdis_backend::hsdis_backend(unsigned __int64,unsigned __int64,unsigned char *,unsigned __int64,void * (__cdecl*)(void *,char const *,void *),void *,int (__cdecl*)(void *,char const *,...),void *,char const *,int)" (??0hsdis_backend@@QEAA@_K0PEAE0P6APEAXPEAXPEBD2@Z2P6AH23ZZ23H@Z)
hsdis-llvm.obj : error LNK2019: unresolved external symbol LLVMSetDisasmOptions referenced in function "public: __cdecl hsdis_backend::hsdis_backend(unsigned __int64,unsigned __int64,unsigned char *,unsigned __int64,void * (__cdecl*)(void *,char const *,void *),void *,int (__cdecl*)(void *,char const *,...),void *,char const *,int)" (??0hsdis_backend@@QEAA@_K0PEAE0P6APEAXPEAXPEBD2@Z2P6AH23ZZ23H@Z)
hsdis-llvm.obj : error LNK2019: unresolved external symbol LLVMDisasmDispose referenced in function "public: __cdecl hsdis_backend::~hsdis_backend(void)" (??1hsdis_backend@@QEAA@XZ)
hsdis-llvm.obj : error LNK2019: unresolved external symbol LLVMDisasmInstruction referenced in function "protected: virtual unsigned __int64 __cdecl hsdis_backend::decode_instruction(unsigned __int64,unsigned __int64,unsigned __int64)" (?decode_instruction@hsdis_backend@@MEAA_K_K00@Z)
hsdis-llvm.obj : error LNK2019: unresolved external symbol LLVMInitializeX86TargetInfo referenced in function LLVMInitializeNativeTarget
hsdis-llvm.obj : error LNK2019: unresolved external symbol LLVMInitializeX86Target referenced in function LLVMInitializeNativeTarget
hsdis-llvm.obj : error LNK2019: unresolved external symbol LLVMInitializeX86TargetMC referenced in function LLVMInitializeNativeTarget
hsdis-llvm.obj : error LNK2019: unresolved external symbol LLVMInitializeX86AsmPrinter referenced in function LLVMInitializeNativeAsmPrinter
hsdis-llvm.obj : error LNK2019: unresolved external symbol LLVMInitializeX86Disassembler referenced in function LLVMInitializeNativeDisassembler
c:\dev\repos\java\forks\jdk\build\windows-x86_64-server-release\support\hsdis\hsdis.dll : fatal error LNK1120: 9 unresolved externals
The hsdis_backend class uses functions in the LLVM libraries that cannot be resolved:
The X86 specific symbols are referenced by the calls to LLVMInitializeNativeTarget, LLVMInitializeNativeAsmPrinter, and LLVMInitializeNativeDisassembler.
Tracking Down the Linker Issues
We can use the DUMPBIN tool to inspect the LLVM libraries.
cd c:\dev\repos\llvm-project\build_llvm\install_local\lib
dumpbin LLVMX86Disassembler.lib
dumpbin /symbols /out:LLVMX86Disassembler.txt LLVMX86Disassembler.lib
The forfiles command is useful for dumping the symbols from all the libraries (forfiles was suggested at How to do something to each file in a directory with a batch script). I thought forfiles would work without the “cmd /c” prefix but that only resulted in dumpbin /summary output!
cd c:\dev\repos\llvm-project\build_llvm\install_local\lib
forfiles /m *.lib /c "cmd /c dumpbin /symbols /out:@fname.txt @file"
Now we can easily search for the symbols of interest, e.g.
> findstr /sipnc:"LLVMInitializeX86Disassembler" *.txt
LLVMX86Disassembler.txt:151:090 00000000 SECT2C notype () External | LLVMInitializeX86Disassembler
LLVMX86Disassembler.txt:926:397 00000000 SECT6B notype Static | $unwind$LLVMInitializeX86Disassembler
LLVMX86Disassembler.txt:929:39A 00000000 SECT6C notype Static | $pdata$LLVMInitializeX86Disassembler
So there really is no such symbol in this lib folder! I’m guessing I need to add another lib folder to the path. A quick search for LLVMInitializeX86Disassembler leads to this post on Using the LLVM MC Disassembly API. It mentions using llvm-config to set the linker flags. Shouldn’t running the bash configure command take care of this? Let’s see what’s in the configure output:
...
checking what hsdis backend to use... 'llvm'
checking for LLVM_CONFIG... C:/dev/repos/llvm-project/build_llvm/install_local/bin [user supplied]
/cygdrive/c/dev/repos/java/forks/jdk/build/.configure-support/generated-configure.sh: line 135451: C:/dev/repos/llvm-project/build_llvm/install_local/bin: Is a directory
/cygdrive/c/dev/repos/java/forks/jdk/build/.configure-support/generated-configure.sh: line 135452: C:/dev/repos/llvm-project/build_llvm/install_local/bin: Is a directory
/cygdrive/c/dev/repos/java/forks/jdk/build/.configure-support/generated-configure.sh: line 135453: C:/dev/repos/llvm-project/build_llvm/install_local/bin: Is a directory
...
Well, that could be the problem! I think I need to fix the llvm-config path in Cygwin by appending /llvm-config to LLVM_CONFIG.
bash configure --with-hsdis=llvm LLVM_CONFIG=C:/dev/repos/llvm-project/build_llvm/install_local/bin/llvm-config --with-llvm=C:/dev/repos/llvm-project/build_llvm/install_local/
Sure enough, that was the problem! The bash configure output (below) now looks good and make build-hsdis now works. The fix for this would be to ensure bash configure fails if LLVM_CONFIG is set to the directory instead of the executable!
checking what hsdis backend to use... 'llvm'
checking for LLVM_CONFIG... C:/dev/repos/llvm-project/build_llvm/install_local/bin/llvm-config [user supplied]
checking for number of cores... 8
...
$ make build-hsdis
Building target 'build-hsdis' in configuration 'windows-x86_64-server-release'
Creating support/hsdis/hsdis.dll from 1 file(s)
Finished building target 'build-hsdis' in configuration 'windows-x86_64-server-release'
Notice from the new build command line in build\windows-x86_64-server-release\support\hsdis\BUILD_HSDIS_link.cmdline that there are now many .lib files supplied to the linker! These are the lib files that I was inspecting with dumpbin so my earlier hypothesis was wrong (there were no additional .lib files required, the ones I was looking at were simply not being passed to the linker).
/cygdrive/c/dev/repos/java/forks/jdk/build/windows-x86_64-server-release/fixpath exec
/cygdrive/c/progra~2/micros~3/2019/enterp~1/vc/tools/msvc/1429~1.301/bin/hostx86/x64/link.exe
-nologo
-libpath:/cygdrive/c/dev/repos/llvm-project/build_llvm/install_local//lib
-dll
-debug
"-pdb:/cygdrive/c/dev/repos/java/forks/jdk/build/windows-x86_64-server-release/support/hsdis/hsdis.pdb"
"-map:/cygdrive/c/dev/repos/java/forks/jdk/build/windows-x86_64-server-release/support/hsdis/hsdis.map"
"-implib:/cygdrive/c/dev/repos/java/forks/jdk/build/windows-x86_64-server-release/support/hsdis/hsdis.lib"
-libpath:/cygdrive/c/progra~2/micros~3/2019/enterp~1/vc/tools/msvc/1429~1.301/atlmfc/lib/x64
-libpath:/cygdrive/c/progra~2/micros~3/2019/enterp~1/vc/tools/msvc/1429~1.301/lib/x64
-libpath:/cygdrive/c/progra~2/wi3cf2~1/netfxsdk/4.8/lib/um/x64
-libpath:/cygdrive/c/progra~2/wi3cf2~1/10/lib/100190~1.0/ucrt/x64
-libpath:/cygdrive/c/progra~2/wi3cf2~1/10/lib/100190~1.0/um/x64
-out:/cygdrive/c/dev/repos/java/forks/jdk/build/windows-x86_64-server-release/support/hsdis/hsdis.dll
/cygdrive/c/dev/repos/java/forks/jdk/build/windows-x86_64-server-release/support/hsdis/hsdis-llvm.obj
/cygdrive/c/dev/repos/java/forks/jdk/build/windows-x86_64-server-release/support/hsdis/hsdis.dll.res
llvmx86targetmca.lib llvmmca.lib llvmx86disassembler.lib llvmx86asmparser.lib llvmx86codegen.lib llvmcfguard.lib llvmglobalisel.lib llvmx86desc.lib llvmx86info.lib llvmmcdisassembler.lib llvmselectiondag.lib llvminstrumentation.lib llvmasmprinter.lib llvmdebuginfomsf.lib llvmcodegen.lib llvmtarget.lib llvmscalaropts.lib llvminstcombine.lib llvmaggressiveinstcombine.lib llvmtransformutils.lib llvmbitwriter.lib llvmanalysis.lib llvmprofiledata.lib llvmdebuginfodwarf.lib llvmobject.lib llvmtextapi.lib llvmmcparser.lib llvmmc.lib llvmdebuginfocodeview.lib llvmbitreader.lib llvmcore.lib llvmremarks.lib llvmbitstreamreader.lib llvmbinaryformat.lib llvmsupport.lib llvmdemangle.lib
Now running make install-hsdis copies hsdis-amd64.dll into /build/windows-x86_64-server-release/jdk/bin. The LLVM hsdis backend can now be used to disassemble instructions:
$ ./java -XX:CompileCommand="print java.lang.String::checkIndex" -version
CompileCommand: print java/lang/String.checkIndex bool print = true
============================= C2-compiled nmethod ==============================
----------------------------------- Assembly -----------------------------------
Compiled method (c2) 5912 60 4 java.lang.String::checkIndex (10 bytes)
total in heap [0x00000162f39e3090,0x00000162f39e3308] = 632
relocation [0x00000162f39e31e8,0x00000162f39e3200] = 24
main code [0x00000162f39e3200,0x00000162f39e3280] = 128
stub code [0x00000162f39e3280,0x00000162f39e3298] = 24
oops [0x00000162f39e3298,0x00000162f39e32a0] = 8
metadata [0x00000162f39e32a0,0x00000162f39e32a8] = 8
scopes data [0x00000162f39e32a8,0x00000162f39e32c0] = 24
scopes pcs [0x00000162f39e32c0,0x00000162f39e3300] = 64
dependencies [0x00000162f39e3300,0x00000162f39e3308] = 8
[Disassembly]
--------------------------------------------------------------------------------
[Constant Pool (empty)]
--------------------------------------------------------------------------------
[Verified Entry Point]
# {method} {0x000001628800f2f0} 'checkIndex' '(II)V' in 'java/lang/String'
# parm0: rdx = int
# parm1: r8 = int
# [sp+0x30] (sp of caller)
0x00000162f39e3200: movl %eax, -0x7000(%rsp)
0x00000162f39e3207: pushq %rbp
0x00000162f39e3208: subq $0x20, %rsp
0x00000162f39e320c: testl %r8d, %r8d
0x00000162f39e320f: jl 0x2f
0x00000162f39e3211: cmpl %r8d, %edx
0x00000162f39e3214: jae 0x16
0x00000162f39e3216: vzeroupper
0x00000162f39e3219: addq $0x20, %rsp
0x00000162f39e321d: popq %rbp
0x00000162f39e321e: cmpq 0x338(%r15), %rsp ; {poll_return}
0x00000162f39e3225: ja 0x29
0x00000162f39e322b: retq
0x00000162f39e322c: movl %edx, %ebp
0x00000162f39e322e: movl %r8d, (%rsp)
0x00000162f39e3232: movl $0xffffffe4, %edx
0x00000162f39e3237: nop
0x00000162f39e3238: vzeroupper
0x00000162f39e323b: callq -0x7a80f40 ; ImmutableOopMap {}
;*invokestatic checkIndex {reexecute=0 rethrow=0 return_oop=0}
; - java.lang.String::checkIndex@5 (line 4554)
; {runtime_call UncommonTrapBlob}
0x00000162f39e3240: movl %edx, %ebp
0x00000162f39e3242: movl %r8d, (%rsp)
0x00000162f39e3246: movl $0xffffffcc, %edx
...
References
Here are some of the bugs/questions I looked at when investigating these failures. Stack overflow taught me about dumpbin and C++ decorated names/ the undname tool.
A previous post explored how to use LLVM as the backend disassembler for hsdis. The instructions for how to use GNU binutils (the currently supported option) are straightforward. Listing them here for completeness (assuming you have cloned the OpenJDK repo into your ~/repos/java/jdk folder). Note that they depend on more recent changes. See the docs on the Java command for more info about the -XX:CompileCommand option.
# Download and extract GNU binutils 2.37
cd ~
curl -Lo binutils-2.37.tar.gz https://ftp.gnu.org/gnu/binutils/binutils-2.37.tar.gz
tar xvf binutils-2.37.tar.gz
# Configure the OpenJDK repo for hsdis
cd ~/repos/java/jdk
bash configure --with-hsdis=binutils --with-binutils-src=~/binutils-2.37
# Build hsdis
make build-hsdis
To deploy the built hsdis library on macOS:
cd build/macosx-aarch64-server-release
# Copy the hsdis library into the JDK bin folder
cp support/hsdis/libhsdis.dylib jdk/bin/hsdis-aarch64.dylib
To deploy the built hsdis library on Ubuntu Linux (open question: is this step even necessary?):
cd build/linux-x86_64-server-release
# Copy the hsdis library into the JDK bin folder
cp support/hsdis/libhsdis.so jdk/bin/
Update 2024-03-13: use the make install-hsdis command to copy the hsdis binaries into the new OpenJDK build. This will ensure that the hsdis binary is copied to lib/hsdis-adm64.so (this file name should be used in place of any others that listed by find . -name *hsdis*).
Now we can disassemble some code, e.g. the String.checkIndex method mentioned in PR 5920.
# Disassemble some code
jdk/bin/java -XX:CompileCommand="print java.lang.String::checkIndex" -version
To see how to disassemble the code for a class, we can use the basic substitution cipher class from the post on Building HSDIS in Cygwin as an example. Download, compile and disassemble it using the commands below. Note that these commands save the .java file to a temp folder to make cleanup much easier. Also note the redirection to a file since the output can be voluminous.
cd jdk/bin
mkdir -p temp
cd temp
curl -Lo BasicSubstitutionCipher.java https://raw.githubusercontent.com/swesonga/scratchpad/main/apps/crypto/substitution-cipher/BasicSubstitutionCipher.java
../javac BasicSubstitutionCipher.java
../java -XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly -XX:+LogCompilation BasicSubstitutionCipher > disassembled.txt
open disassembled.txt
The previous post described how LLVM can be configured as the disassembly backend for hsdis. Here, I explain the process it took for me to figure out the details of the change adding support for LLVM. One of the first things to do when learning these details of this change is to build it. Since I’m using my own fork of the OpenJDK repo, I need to add the upstream repo to my remotes. This makes it possible to fetch commits from PRs submitted to the upstream repo.
cd ~/repos/forks/jdk
git remote add upstream https://github.com/openjdk/jdk
git fetch upstream
The LLVM-backend PR has only 1 commit (as of this writing). Create a new branch then cherry-pick that commit (I was on commit 77757ba9 when I wrote this.
git checkout -b hsdis-backend-llvm
git cherry-pick effac9b87ecb3cdc8d3d149b9dcd72ee1ea88fec
Some conflicts need to be resolved:
Performing inexact rename detection: 100% (88356/88356), done.
Auto-merging make/autoconf/spec.gmk.in
Auto-merging make/autoconf/jdk-options.m4
CONFLICT (content): Merge conflict in make/autoconf/jdk-options.m4
Auto-merging make/Hsdis.gmk
error: could not apply effac9b87ec... Create hsdis backend using LLVM
The files view of PR 5920 shows that the change to make/autoconf/jdk-options.m4 is mostly adding another branch to the if-else statements checking the hsdis backend. Lines 841-854 of PR 5920 can therefore be added just before the else on line 890 to resolve the conflict. The diff from my branch can be seen here.
Building the Changes on macOS ARM64
Install LLVM using homebrew (if it is not already installed).
brew install llvm
Set the the LDFLAGS and CPPFLAGS environment variables then run printenv | grep -i flags to verify that the flags have been set correctly. Exporting CC and CXX is crucial since that is how to let bash configure know that we need a custom compiler for the build!
# export LDFLAGS="-L/opt/homebrew/opt/llvm/lib"
# export CFLAGS="-I/opt/homebrew/opt/llvm/include"
export CC=/opt/homebrew/opt/llvm/bin/clang
export CXX=$(CC)++
bash configure --with-hsdis=llvm LLVM_CONFIG=/opt/homebrew/opt/llvm/bin
Run make build-hsdis in the root folder of the jdk repo.
If the proper flags have not been set, make will fail with the error below. Run make --debug=v for additional information on what make is doing.
saint@Saints-MBP-2021 jdk % make build-hsdis
Building target 'build-hsdis' in configuration 'macosx-aarch64-server-release'
/Users/saint/repos/java/forks/jdk/src/utils/hsdis/llvm/hsdis-llvm.cpp:58:10: fatal error: 'llvm-c/Disassembler.h' file not found
#include <llvm-c/Disassembler.h>
^~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[3]: *** [/Users/saint/repos/java/forks/jdk/build/macosx-aarch64-server-release/support/hsdis/hsdis-llvm.o] Error 1
make[2]: *** [build-hsdis] Error 2
ERROR: Build failed for target 'build-hsdis' in configuration 'macosx-aarch64-server-release' (exit code 2)
After all that fidgeting around, the fix is as simple as updating your path to include LLVM <insert facepalm / clown>. This is what installing LLVM using brew ends with:
...
==> llvm
To use the bundled libc++ please add the following LDFLAGS:
LDFLAGS="-L/opt/homebrew/opt/llvm/lib -Wl,-rpath,/opt/homebrew/opt/llvm/lib"
llvm is keg-only, which means it was not symlinked into /opt/homebrew,
because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.
If you need to have llvm first in your PATH, run:
echo 'export PATH="/opt/homebrew/opt/llvm/bin:$PATH"' >> ~/.zshrc
For compilers to find llvm you may need to set:
export LDFLAGS="-L/opt/homebrew/opt/llvm/lib"
export CPPFLAGS="-I/opt/homebrew/opt/llvm/include"
My MacBook didn’t even have a ~/.zshrc file. Setting the PATH using the suggestion above fixed the build errors!
echo 'export PATH="/opt/homebrew/opt/llvm/bin:$PATH"' >> ~/.zshrc
Now open a new terminal and configure the repo (no need for LLVM_CONFIG).
% bash configure --with-hsdis=llvm
% make build-hsdis
Interestingly, running make images does not work on subsequent attempts?! After further investigation, it turns out that the clang compiler installed by brew cannot successfully compile the OpenJDK sources. Why does it issue warnings that Apple’s clang compiler does not?
In file included from /Users/saint/repos/java/forks/jdk/src/hotspot/cpu/aarch64/abstractInterpreter_aarch64.cpp:31:
In file included from /Users/saint/repos/java/forks/jdk/src/hotspot/share/runtime/frame.inline.hpp:42:
In file included from /Users/saint/repos/java/forks/jdk/src/hotspot/cpu/aarch64/frame_aarch64.inline.hpp:31:
In file included from /Users/saint/repos/java/forks/jdk/src/hotspot/cpu/aarch64/pauth_aarch64.hpp:28:
/Users/saint/repos/java/forks/jdk/src/hotspot/os_cpu/bsd_aarch64/pauth_bsd_aarch64.inline.hpp:29:10: fatal error: 'ptrauth.h' file not found
#include <ptrauth.h>
^~~~~~~~~~~
1 error generated.
make[3]: *** [/Users/saint/repos/java/forks/jdk/build/macosx-aarch64-server-release/hotspot/variant-server/libjvm/objs/abstractInterpreter_aarch64.o] Error 1
m
To work around this, first build the JDK using Apple’s clang. Next, add brew’s LLVM installation to the PATH, then configure for hsdis. Finally, build hsdis:
# Warning: ensure /opt/homebrew/opt/llvm/bin is not in the PATH
cd ~/repos/java/forks/jdk
bash configure
make images
# Now add brew's LLVM to the PATH before running bash configure
export OLDPATH=$PATH
export PATH="/opt/homebrew/opt/llvm/bin:$PATH"
bash configure --with-hsdis=llvm
make build-hsdis
make install-hsdis
export PATH=$OLDPATH
# Why doesn't install-hsdis do this?
cd build/macosx-aarch64-server-release
cp support/hsdis/libhsdis.dylib jdk/bin/
The JVM did not appear to be generating the disassembly even with this approach. A quick search for hsdis not printing assembly macOS leads to this post mentioning the error Could not load hsdis-amd64.dylib; library not loadable; PrintAssembly is disabled. This reminds me that theRealAph had pointed out that the library seems to be built with the wrong name, so the runtime doesn’t find it. So I just needed to specify that file name when copying the hsdis dylib in the last step!
cp support/hsdis/libhsdis.dylib jdk/bin/hsdis-aarch64.dylib
Building the Changes on Windows x86-64
Install the 64-bit Windows LLVM. Configure the OpenJDK repo using both the --with-hsdis and LLVM_CONFIG options as shown. I needed to use the 8.3 path name (obtained using the command suggested on StackOverflow) for value of the LLVM_CONFIG parameter.
bash configure --with-hsdis=llvm LLVM_CONFIG=C:/PROGRA~1/LLVM/bin
Unfortunately, this is not sufficient to enable building on Windows as detailed by this error:
$ make build-hsdis
Building target 'build-hsdis' in configuration 'windows-x86_64-server-release'
Creating support/hsdis/hsdis.dll from 1 file(s)
/usr/bin/bash: x86_64-w64-mingw32-g++: command not found
make[3]: *** [Hsdis.gmk:135: /..../build/windows-x86_64-server-release/support/hsdis/hsdis-llvm.obj] Error 127
make[2]: *** [make/Main.gmk:530: build-hsdis] Error 2
ERROR: Build failed for target 'build-hsdis' in configuration 'windows-x86_64-server-release' (exit code 2)
Jorn fixed this so we can add Jorn’s upstream JDK, fetch its commits, then cherry pick the commit with the fix.
git remote add jorn https://github.com/JornVernee/jdk/
git fetch jorn
git cherry-pick 8de8b763c9159f84bcc044c04ee2fac9f2390774
Some conflicts in make/Hsdis.gmk need to be resolved. This is straightforward since Jorn’s change splits the existing binutils Windows code into the first branch of an if-statement then adds support for the LLVM backend in the else case. The resolved conflicts are in my fork in the branch. The repo should now be configured with the additional --with-llvm option added by Jorn.
bash configure --with-hsdis=llvm LLVM_CONFIG=C:/PROGRA~1/LLVM/bin --with-llvm=C:/PROGRA~1/LLVM
Running make build-hsdis results in errors about missing LLVM includes.
$ make build-hsdis
Building target 'build-hsdis' in configuration 'windows-x86_64-server-release'
Creating support/hsdis/hsdis.dll from 1 file(s)
d:\.....\jdk\src\utils\hsdis\llvm\hsdis-llvm.cpp(58): fatal error C1083: Cannot open include file: 'llvm-c/Disassembler.h': No such file or directory
make[3]: *** [Hsdis.gmk:142: /cygdrive/d/dev/repos/java/forks/jdk/build/windows-x86_64-server-release/support/hsdis/hsdis-llvm.obj] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [make/Main.gmk:530: build-hsdis] Error 2
Let’s try setting CC and CXX then rerunning the above configure command.
export CC=C:/PROGRA~1/LLVM/bin/clang.exe
export CXX=C:/PROGRA~1/LLVM/bin/clang++.exe
Turns out a Microsoft compiler is required!
configure: Will use user supplied compiler CC=C:/PROGRA~1/LLVM/bin/clang.exe
checking resolved symbolic links for CC... no symlink
configure: The C compiler (located as C:/PROGRA~1/LLVM/bin/clang.exe) does not seem to be the required microsoft compiler.
configure: The result from running it was: "clang: error: no input files"
configure: error: A microsoft compiler is required. Try setting --with-tools-dir.
configure exiting with result code 1
But let’s see what happens if we change the toolchain type to clang:
# This command does not work
bash configure --with-hsdis=llvm LLVM_CONFIG=C:/PROGRA~1/LLVM/bin --with-llvm=C:/PROGRA~1/LLVM --with-toolchain-type=clang
I guess they were serious about that since clang is not valid on this platform.
configure: Toolchain type clang is not valid on this platform.
configure: Valid toolchains: microsoft.
configure: error: Cannot continue.
configure exiting with result code 1
Indeed, clang is not a valid toolchain for Windows as declared in make/autoconf/toolchain.m4. Open question: how is the VALID_TOOLCHAIN_windows actually checked? So we can now unset the environment variables.
unset CC
unset CXX
This brought me back to the first thing I should have done when I saw the “No such file or directory” error – verifying that the file existed on disk! This is all there is there:
$ ls C:/PROGRA~1/LLVM/include/llvm-c
Remarks.h lto.h
Well, turns out this is the issue that led Jorn to build LLVM manually. I now know what the needed header files being referred to are. So let’s build LLVM using Jorn’s steps.
git clone https://github.com/llvm/llvm-project.git
cd llvm-project
mkdir build_llvm
cd build_llvm
cmake ../llvm -D"LLVM_TARGETS_TO_BUILD:STRING=X86" -D"CMAKE_BUILD_TYPE:STRING=Release" -D"CMAKE_INSTALL_PREFIX=install_local" -A x64 -T host=x64
cmake --build . --config Release --target install
The last command fails with the error below!??? Why can’t anything just simply work?
Building Opts.inc...
'..\..\RelWithDebInfo\bin\llvm-tblgen.exe' is not recognized as an internal or external command,
operable program or batch file.
C:\Program Files\Microsoft Visual Studio\2022\Preview\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(243,5): error MSB8066: Custom build for 'D:\dev\repos\llvm-project\build_llvm\CMakeFiles\dd1f7b42098
1667d7f617e96802947d3\Opts.inc.rule;D:\dev\repos\llvm-project\build_llvm\CMakeFiles\9fbf2dc5caba7f0c75934f43d12abdf5\RcOptsTableGen.rule;D:\dev\repos\llvm-project\llvm\tools\llvm-rc\CMakeLists.txt' exited wit
h code 9009. [D:\dev\repos\llvm-project\build_llvm\tools\llvm-rc\RcOptsTableGen.vcxproj]
Switch to my Surface Book 2 and LLVM builds just fine!
bash configure --with-hsdis=llvm LLVM_CONFIG=C:/dev/repos/llvm-project/build_llvm/install_local/bin --with-llvm=C:/dev/repos/llvm-project/build_llvm/install_local/
Interestingly, this fails with the same errors I saw on macOS:
$ make build-hsdis
Building target 'build-hsdis' in configuration 'windows-x86_64-server-release'
Creating support/hsdis/hsdis.dll from 1 file(s)
hsdis-llvm.obj : error LNK2019: unresolved external symbol LLVMCreateDisasm referenced in function "public: __cdecl hsdis_backend::hsdis_backend(unsigned __int64,unsi...,char const *,int)" (??0hsdis_backend@@QEAA@_K0PEAE0P6APEAXPEAXPEBD2@Z2P6AH23ZZ23H@Z)
...
hsdis-llvm.obj : error LNK2019: unresolved external symbol LLVMInitializeX86Disassembler referenced in function LLVMInitializeNativeDisassembler
c:\dev\repos\java\forks\jdk\build\windows-x86_64-server-release\support\hsdis\hsdis.dll : fatal error LNK1120: 9 unresolved externals
make[3]: *** [Hsdis.gmk:142: /cygdrive/c/dev/repos/java/forks/jdk/build/windows-x86_64-server-release/support/hsdis/hsdis.dll] Error 1
The PATH environment variable probably needs to be adjusted to work around this.
Update 2022-02-08: the problem above is that bash configure is invoked with the wrong LLVM_CONFIG option – the actual llvm-config executable name is missing. See Troubleshooting hsdis LLVM backend MSVC Linker Errors for details.
To specify a backend for hsdis, the OpenJDK repo needs to be configured with the --with-hsdis option. As of commit 77757ba9, LLVM is not yet supported as an hsdis disassembly backend. Therefore, this error from make/autoconf/jdk-options.m4 is displayed. Here’s an example on the Windows platform:
$ bash configure --with-hsdis=llvm
...
checking what hsdis backend to use... invalid
configure: error: Incorrect hsdis backend "llvm"
configure exiting with result code 1
There has been an effort to enable using LLVM as the hsdis disassembler’s backend. To use this change, check out this branch with those changes (and some conflict resolution to incorporate more recent changes).
hsdis LLVM backend on macOS ARM64
To test the LLVM backend for hsdis on macOS, install LLVM using brew (Apple’s LLVM does not have the llvm-c include files):
# install LLVM
brew install llvm
Now build the OpenJDK. This should use Apple’s compiler since we have not made any configuration changes.
cd ~/repos/java/jdk
bash configure
make images
Now add brew’s LLVM bin directory to the PATH and run bash configure again passing the --with-hsdis=llvm option as shown below. The configuration process will detect the clang++ compiler installed by brew and set it up for use when the build-hsdis target is executed.
# Now add brew's LLVM to the PATH before running bash configure
export OLDPATH=$PATH
export PATH="/opt/homebrew/opt/llvm/bin:$PATH"
bash configure --with-hsdis=llvm
make build-hsdis
make install-hsdis
export PATH=$OLDPATH
The install-hsdis target does not appear to be copying the hsdis library to the jdk/bin folder so these commands are required:
cd build/macosx-aarch64-server-release
cp support/hsdis/libhsdis.dylib jdk/bin/hsdis-aarch64.dylib
We can now test hsdis as described in the post about Building hsdis in Cygwin.
hsdis LLVM backend on Windows x86-64
To test the LLVM backend for hsdis, we need to first clone and builld LLVM because the LLVM installer does not come with the include files needed to build the changes in PR 5920. These instructions are from Jorn.
git clone https://github.com/llvm/llvm-project.git
cd llvm-project
mkdir build_llvm
cd build_llvm
cmake ../llvm -D"LLVM_TARGETS_TO_BUILD:STRING=X86" -D"CMAKE_BUILD_TYPE:STRING=Release" -D"CMAKE_INSTALL_PREFIX=install_local" -A x64 -T host=x64
cmake --build . --config Release --target install
Now we can configure the OpenJDK repo for hsdis, and build both the JDK and hsdis.
bash configure --with-hsdis=llvm \
LLVM_CONFIG=C:/dev/repos/llvm-project/build_llvm/install_local/bin \
--with-llvm=C:/dev/repos/llvm-project/build_llvm/install_local/
make build-hsdis
make images
hsdis LLVM backend on Windows ARM64
Open question: is this supported?
Testing the hsdis LLVM backend
The String.checkIndex method of PR 5920 is a good candidate for testing the hsdis LLVM backend. The -XX:CompileCommand option can be used to print the generated assembler code after compilation of the specified method.
java -XX:CompileCommand="print java.lang.String::checkIndex" -version
Tips
- To view the value of variables in GNU make, use the info control function, e.g.
$(info Debug Info HSDIS_CFLAGS = $(HSDIS_CFLAGS)) - To view the command lines being executed by make as well as the value of the variables in use:
make LOG=debug build-hsdis - Autoconf macros are defined using the AC_DEFUN macro. The JDKOPT_SETUP_HSDIS macro (modified by PR 5920) is defined using AC_DEFUN_ONCE, which is for macros that should only be called once.
- To address warnings like
ld: warning: dylib (/opt/homebrew/opt/llvm/lib/libunwind.dylib) was built for newer macOS version (12.0) than being linked (11.0) update MACOSX_VERSION_MIN in make/autoconf/flags.m4.
The previous post on Building HSDIS in Cygwin required running this command to actually build the hsdis DLL.
make OS=Linux MINGW=x86_64-w64-mingw32 BINUTILS=~/binutils-2.37
As it turns out, this make command fails because of a bug in the GNU binutils source code. This is the error I got:
...
x86_64-w64-mingw32-gcc -c -DHAVE_CONFIG_H -O -I. -I/home/User/binutils-2.37/libiberty/../include -W -Wall -Wwrite-strings -Wc++-compat -Wstrict-prototypes -Wshadow=local -pedantic -D_GNU_SOURCE /home/User/binutils-2.37/libiberty/rust-demangle.c -o rust-demangle.o
/home/User/binutils-2.37/libiberty/rust-demangle.c:78:3: error: unknown type name ‘uint’
78 | uint recursion;
| ^~~~
/home/User/binutils-2.37/libiberty/rust-demangle.c: In function ‘demangle_path’:
/home/User/binutils-2.37/libiberty/rust-demangle.c:81:37: error: ‘uint’ undeclared (first use in this function); did you mean ‘int’?
81 | #define RUST_NO_RECURSION_LIMIT ((uint) -1)
| ^~~~
...
make[2]: *** [Makefile:1230: rust-demangle.o] Error 1
...
At this point, I wasn’t sure which version I used to build successfully. Searching for that error (and binutils to narrow things down) led to this bug in the sourceware.org Bugzilla that appears to be the exact bug I ran into: 28207 – error: unknown type name ‘uint’ (78 | uint recursion;) avr-gcc mingw32 Windows Build (sourceware.org). Fortunately, one Alan helpfully points out that this bug fixed on the binutils-2.37 branch with commit 999566402e3.
To figure out where the binutils git repo is, I click on the Browse button in Bugzilla then navigate to the binutils product category, which has a link to the list of bugs for the binutils component. A re-opened bug seems likely to have a link to some commits. I select 26865 – windres: –preprocessor option won’t respect space in file path (sourceware.org) and sure enough, there is a link to a commit on the binutils repo. We can now view the history of rust-demangle.c. To find the commit in question, click on any commitdiff to get the URL format then replace the hash in the URL with 999566402e3 to reveal the aforementioned fix for the unknown type name uint error.
Cloning binutils Repo
I’m used to GitHub where looking at the repo structure implies that you’re at a URL you can copy and trim to clone. In this other web view, the URL to clone is listed above the shortlog:
git clone https://sourceware.org/git/binutils-gdb.git
Tracing the Bug
At this point, it makes sense to verify that the 2.37 sources I downloaded actually contain the bug. Observe that:
- the tags section contains a binutils-2_37 tag described as “Official GNU Binutils 2.37 Release” and committed on Sun, 18 Jul 2021 16:46:54 +0000 (17:46 +0100).
- the fix for the build error shows a fix committed by Alan on Mon, 19 Jul 2021 11:32:21 +0000 (21:02 +0930)
- the bug fix that introduced the error was committed on Thu, 15 Jul 2021 15:51:56 +0000 (16:51 +0100)
Therefore, using binutils older than 2.37 should work just fine. However, it may still be necessary to run “rm -fr build” in the hsdis folder to enable 2.36 to be picked up when you run make (otherwise 2.37 is still baked into some of configure’s output).
Hsdis is an externally loadable disassembler plugin. It lets you see which assembly instructions the JVM generates for your Java code. On 64-bit Windows, it is a binary called hsdis-amd64.dll (and hsdis-i386.dll on 32-bit platforms). This binary needs to be in the same directory as jvm.dll. Some good resources out there on building the hsdis binary for the OpenJDK include:
For Cygwin, the latter resource (from 2012?) is all we need. I like that Gunnar’s blog post covered how to use hsdis after building it so this writeup aims to combine both blogs into a simple Cygwin install-build-disassemble set of instructions.
Building hsdis for 64-bit JVMs
- Install Cygwin with the gcc-core, make, and mingw64-x86_64-gcc-core packages by launching the setup executable using this command (no need to bother selecting packages in the UI since you have already specified them on the command line)
setup-x86_64.exe -P gcc-core -P mingw64-x86_64-gcc-core -P make
- Launch the Cygwin64 terminal
- Clone the OpenJDK repo to get the hsdis sources (if you have not yet set up a Windows OpenJDK Development Environment).
mkdir ~/repos
cd ~/repos
git clone https://github.com/openjdk/jdk
- Run these commands to download GNU binutils and build hsdis (Update 2022-01-07: version downgraded to 2.36 to avoid build failures investigated in Fixing Hsdis Compile Failure in GNU binutils).
cd ~
curl -Lo binutils-2.36.tar.gz https://ftp.gnu.org/gnu/binutils/binutils-2.36.tar.gz
tar xvf binutils-2.36.tar.gz
cd ~/repos/jdk/src/utils/hsdis
make OS=Linux MINGW=x86_64-w64-mingw32 BINUTILS=~/binutils-2.36
- Copy the hsdis binary to the locally built java bin folder
cp src/utils/hsdis/build/Linux-amd64/hsdis-amd64.dll build/windows-x86_64-server-release/jdk/bin/
Testing hsdis
I have created a basic substitution cipher, which we can compile and disassemble using the commands below. Note that these commands save the .java file to a temp folder to make cleanup much easier. Also note the redirection to a file since the output can be voluminous.
cd build/windows-x86_64-server-release/jdk/bin
mkdir -p temp
cd temp
curl -Lo BasicSubstitutionCipher.java https://raw.githubusercontent.com/swesonga/scratchpad/main/apps/crypto/substitution-cipher/BasicSubstitutionCipher.java
../javac BasicSubstitutionCipher.java
../java -XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly -XX:+LogCompilation BasicSubstitutionCipher > BasicSubstitutionCipher.disassembled.txt
Once the disassembly completes, we can view the instructions generated in the BasicSubstitutionCipher.disassembled.txt file.
One open question in this setup is why the installed GNU binutils cannot be used to build hsdis. Seems strange to have to build them from source when the binutils Cygwin package was also installed in step 1 above.