The previous post covered Building libffi for Windows x64 with Visual C++. In this post, I detail the instructions needed to build for the ARM64 platform (building the zero variant of the HotSpot JVM for the Windows ARM64 platform was my overall objective). I used the same Windows x64 machine for this build. As in the previous post, Visual C++ and MSYS are prerequisites. Get the sources from GitHub:
cd /c/repos
git clone https://github.com/libffi/libffi.git
cd libffi
git checkout v3.4.8
MSYS Prerequisites
Launch MSYS2 and install automake and libtool using these commands:
pacman -S automake
pacman -S libtool
The Visual C++ compiler needs to be available in the path as well. Run cl without any parameters to ensure the compiler is available. If it is available, it must be the ARM64 compiler to ensure we cross-compile! It most likely won’t be by default. If it isn’t, add it to the path as follows:
With the MSYS prerequisites installed, run the autogen.sh script:
user@machine /d/repos/libffi
$ ./autogen.sh
This creates a configure script in the root of the repository. Run it using bash. This command is the main difference between ARM64 and x86_64. Notice that I need to specify various include paths for the ARM64 compiler and linker that were not required in the x86_64 case.
Run make in the root of the repo. The generated LIB and DLL files should be in the aarch64-w64-mingw32/.libs/ subdirectory of the repo root. There will also be ffi.h and ffitarget.h include files in the aarch64-w64-mingw32/include/ subdirectory of the repo root. These 4 files are typically what will be required by other projects with a libffi dependency (like OpenJDK).
$ ls -1 aarch64-w64-mingw32/.libs/
libffi.la
libffi.lai
libffi_convenience.la
libffi_convenience.lib
libffi-8.dll*
libffi-8.exp
libffi-8.lib
$ ls -1 aarch64-w64-mingw32/include/
ffi.h
ffitarget.h
Makefile
Background Investigation Details
Investigating Configure Errors
My initial attempt at building libffi for Windows ARM64 started on the wrong path, based on this quote from libffi/libffi at v3.4.8.
To build static library for ARM64 with MSVC using visual studio solution, msvc_build folder have aarch64/Ffi_staticLib.sln required header files in aarch64/aarch64_include/
I thought this meant that it would be much faster for me to build libffi since I wouldn’t need all these bash configure stuff. The solution informed me that I needed to upgrade the toolset:
I then changed the architecture (in the Configuration Manager dropdown on the standard VS toolbox) from x64 to ARM64. There are a bunch of compiler errors!
1>D:\repos\libffi\src\closures.c(1015,30): error C2039: 'ftramp': is not a member of 'ffi_closure'
1> D:\repos\libffi\msvc_build\aarch64\aarch64_include\ffi.h(306,16):
1> see declaration of 'ffi_closure'
...
1>D:\repos\libffi\src\prep_cif.c(248,16): error C2065: 'FFI_BAD_ARGTYPE': undeclared identifier
How could a needed field be missing??!! I tried replacing ffi.h with the one from the x64 build but it was clearly wrong because it had architecture-specific code like this:
/* Specify which architecture libffi is configured for. */
#ifndef X86_WIN64
#define X86_WIN64
#endif
I then checked out the commit that added support for Windows AArch64.
I then tried to configure OpenJDK using this command but the configure script failed!
date; time bash configure --with-jvm-variants=zero --with-libffi=/cygdrive/c/repos/libffi --openjdk-target=aarch64-unknown-cygwin --with-debug-level=slowdebug --with-jtreg=/cygdrive/c/java/binaries/jtreg/jtreg-7.5.1+1 --with-gtest=/cygdrive/c/repos/googletest --with-extra-ldflags=-profile --with-boot-jdk=/cygdrive/c/java/binaries/jdk/x64/jdk-24+36; time /cygdrive/c/repos/scratchpad/scripts/java/cygwin/build-jdk.sh windows aarch64 slowdebug
At this point, I had the build tools installed with the C++ compiler in C:\progra~2\micros~3\2022\buildt~1\vc\tools\msvc\1443~1.348\bin\hostx64\arm64\cl.exe. I opened the VS Installer and installed the ARM64 compiler tools. This was necessary because this script was not present on my machine:
Running vcvarsamd64_arm64.bat initialized the environment for ‘x64_arm64’ (cross-compilation targeting ARM64). I then ran dumpbin to see which symbols were in the .lib file VS generated.
cd /d C:\repos\libffi
dumpbin /all /out:ffi-arm64.txt libffi.lib
cd /d D:\repos\libffi
dumpbin /all /out:ffi-x64.txt libffi.lib
The symbols were very different, which was my sign that I just needed to try building for ARM64 in MSYS2. I also upgraded VS some of the paths use 14.44 and others were 14.43. I started MSYS2 then added the arm64 compiler to the PATH. I tried the long path again but only the 8.3 filename format path worked.
export PATH="/c/Program\ Files/Microsoft\ Visual\ Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/arm64/:$PATH"
export PATH="$PATH:/c/Program\ Files/Microsoft\ Visual\ Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/arm64/"
# Only this one works.
$ export PATH="$PATH:/c/Progra~1/MIB055~1/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/arm64/"
$ where cl.exe
I then switched the repo back to v3.4.8 and ran autogen.sh. This time I specified the –target option to request a aarch64 build. See Specifying Target Triplets (Autoconf) for an overview of the target triplets.
I asked Copilot Which autoconf macro outputs “checking whether the C compiler works” and it said that’s the AC_PROG_CC macro. That string showed up in 3 spots in the codebase but they weren’t what I was looking for. The “checking for C compiler version” was in the generated configure script though.
# Provide some information about the compiler.
printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for C compiler version" >&5
set X $ac_compile
ac_compiler=$2
for ac_option in --version -v -V -qversion -version; do
{ { ac_try="$ac_compiler $ac_option >&5"
case "(($ac_try" in
*\"* | *\`* | *\\*) ac_try_echo=\$ac_try;;
*) ac_try_echo=$ac_try;;
esac
This explained where those odd arguments in the config.log snippet were coming from. The question was now how this was different from the x64 case where it just worked? The diff showed that I was actually still on 3.3-rc0 so I needed to rerun autogen.sh on v3.4.8. I didn’t think I needed the --target option since the correct compiler was selected (as far as I could tell from the --verbose output above).
The configure files were identical in both scenarios. However, there was a key difference in the config logs! Here is a snippet from the working x64 build’s config.log. Notice that the version detection errors were present in this case too (that was a red herring)!
configure:4679: /d/repos/libffi/msvcc.sh -m64 -version >&5
cl : Command line warning D9002 : ignoring unknown option '-version'
cl : Command line error D8003 : missing source filename
configure:4690: $? = 0
configure:4710: checking whether the C compiler works
configure:4732: /d/repos/libffi/msvcc.sh -m64 -DFFI_BUILDING_DLL conftest.c >&5
configure:4736: $? = 0
configure:4787: result: yes
configure:4679: /c/repos/libffi/msvcc.sh -marm64 -version >&5
cl : Command line warning D9002 : ignoring unknown option '-version'
cl : Command line error D8003 : missing source filename
configure:4690: $? = 0
configure:4710: checking whether the C compiler works
configure:4732: /c/repos/libffi/msvcc.sh -marm64 -DFFI_BUILDING_DLL conftest.c >&5
LINK : fatal error LNK1104: cannot open file 'MSVCRT.lib'
configure:4736: $? = 0
configure:4777: result: no
The linker error was really what I needed to address here. I created this conftest.c file to address the command line compilation issue:
int main (void)
{
return 0;
}
$ cl -MD -W3 conftest.c
Microsoft (R) C/C++ Optimizing Compiler Version 19.44.35207.1 for ARM64
Copyright (C) Microsoft Corporation. All rights reserved.
conftest.c
Microsoft (R) Incremental Linker Version 14.44.35207.1
Copyright (C) Microsoft Corporation. All rights reserved.
/out:conftest.exe
conftest.obj
LINK : fatal error LNK1104: cannot open file 'MSVCRT.lib'
How does OpenJDK get around this? Interestingly, this was when I noticed that the OpenJDK log also had all the version checking errors (-v -V –version, etc). This is the snippet from OpenJDK’s config.log (notice the -libpaths):
configure:105502: checking whether the C compiler works
configure:105524: /cygdrive/d/java/forks/TheShermanTanker/jdk/build/windows-aarch64-zero-slowdebug/fixpath exec /cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/tools/msvc/1443~1.348/bin/hostx64/arm64/cl.exe -I/cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/tools/msvc/1443~1.348/include -I/cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/tools/msvc/1443~1.348/atlmfc/include -I/cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/auxili~1/vs/include -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100226~1.0/ucrt -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100226~1.0/um -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100226~1.0/shared -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100226~1.0/winrt -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100226~1.0/cppwinrt -I/cygdrive/c/progra~2/wi3cf2~1/netfxsdk/4.8/include/um -I/cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/tools/msvc/1443~1.348/include -I/cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/tools/msvc/1443~1.348/atlmfc/include -I/cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/auxili~1/vs/include -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100226~1.0/ucrt -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100226~1.0/um -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100226~1.0/shared -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100226~1.0/winrt -I/cygdrive/c/progra~2/wi3cf2~1/10/include/100226~1.0/cppwinrt -I/cygdrive/c/progra~2/wi3cf2~1/netfxsdk/4.8/include/um conftest.c -link -libpath:/cygdrive/c/progra~1/mib055~1/2022/enterp~1/vc/tools/msvc/1443~1.348/lib/arm64 -libpath:/cygdrive/c/progra~2/wi3cf2~1/10/lib/100226~1.0/ucrt/arm64 -libpath:/cygdrive/c/progra~2/wi3cf2~1/10/lib/100226~1.0/um/arm64 -profile >&5
Microsoft (R) C/C++ Optimizing Compiler Version 19.43.34810 for ARM64
Copyright (C) Microsoft Corporation. All rights reserved.
conftest.c
Microsoft (R) Incremental Linker Version 14.43.34810.0
Copyright (C) Microsoft Corporation. All rights reserved.
/out:conftest.exe
-libpath:c:\progra~1\mib055~1\2022\enterp~1\vc\tools\msvc\1443~1.348\lib\arm64
-libpath:c:\progra~2\wi3cf2~1\10\lib\100226~1.0\ucrt\arm64
-libpath:c:\progra~2\wi3cf2~1\10\lib\100226~1.0\um\arm64
-profile
conftest.obj
configure:105528: $? = 0
configure:105579: result: yes
Searching that codebase for libpath led to the location where the -libpath arguments are built in jdk/make/autoconf/toolchain_microsoft.m4. I should do the same thing and set the LDFLAGS.
$ cl -MD -W3 conftest.c -link -libpath:/c/Progra~1/MIB055~1/2022/Enterprise/VC/Tools/MSVC/14.44.35207/lib/arm64 -libpath:/c/progra~2/wi3cf2~1/10/lib/100226~1.0/ucrt/arm64 -libpath:/c/progra~2/wi3cf2~1/10/lib/100226~1.0/um/arm64
Microsoft (R) C/C++ Optimizing Compiler Version 19.44.35207.1 for ARM64
Copyright (C) Microsoft Corporation. All rights reserved.
conftest.c
Microsoft (R) Incremental Linker Version 14.44.35207.1
Copyright (C) Microsoft Corporation. All rights reserved.
/out:conftest.exe
-libpath:C:/Progra~1/MIB055~1/2022/Enterprise/VC/Tools/MSVC/14.44.35207/lib/arm64
-libpath:C:/progra~2/wi3cf2~1/10/lib/100226~1.0/ucrt/arm64
-libpath:C:/progra~2/wi3cf2~1/10/lib/100226~1.0/um/arm64
conftest.obj
That succeeded so I tried to set the LDFLAGS for libffi.
Looks like the other paths are being dropped by the script. Further inspection of the script reveals that it has a -L option for these libraries. I tried the -link option but something wasn’t working so I moved on to -L. These are the libraries I needed:
With the above command, the next issue was around cross compiling:
configure: loading site script /etc/config.site
checking build system type... x86_64-w64-mingw32
checking host system type... x86_64-w64-mingw32
checking target system type... x86_64-w64-mingw32
continue configure in default builddir "./x86_64-w64-mingw32"
....exec /bin/sh ../configure "--srcdir=.." "--enable-builddir=x86_64-w64-mingw32" "mingw32"
configure: loading site script /etc/config.site
checking build system type... x86_64-w64-mingw32
checking host system type... x86_64-w64-mingw32
checking target system type... x86_64-w64-mingw32
checking for gsed... sed
checking for a BSD-compatible install... /usr/bin/install -c
checking whether sleep supports fractional seconds... yes
checking filesystem timestamp resolution... 0.01
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking xargs -n works... yes
checking for gcc... /c/repos/libffi/msvcc.sh -marm64 -L /c/Progra~1/MIB055~1/2022/Enterprise/VC/Tools/MSVC/14.44.35207/lib/arm64 -L /c/progra~2/wi3cf2~1/10/lib/100226~1.0/ucrt/arm64 -L /c/progra~2/wi3cf2~1/10/lib/100226~1.0/um/arm64
checking whether the C compiler works... yes
checking for C compiler default output file name... conftest.exe
checking for suffix of executables... .exe
checking whether we are cross compiling... configure: error: in '/c/repos/libffi/x86_64-w64-mingw32':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use '--host'.
See 'config.log' for more details
At least this error message let me know what I needed to do to.
Next error after that change in the checking how to run the C++ preprocessor step, specifically error: C++ preprocessor "cl -nologo -EP" fails sanity check.
configure:14431: checking how to run the C++ preprocessor
configure:14498: result: cl -nologo -EP
configure:14512: cl -nologo -EP -DFFI_BUILDING_DLL conftest.cpp
conftest.cpp
conftest.cpp(12): fatal error C1034: limits.h: no include path set
configure:14512: $? = 2
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "libffi"
| #define PACKAGE_TARNAME "libffi"
| #define PACKAGE_VERSION "3.4.8"
| #define PACKAGE_STRING "libffi 3.4.8"
| #define PACKAGE_BUGREPORT "http://github.com/libffi/libffi/issues"
| #define PACKAGE_URL ""
| #define PACKAGE "libffi"
| #define VERSION "3.4.8"
| #define LT_OBJDIR ".libs/"
| /* end confdefs.h. */
| #include <limits.h>
| Syntax error
configure:14512: cl -nologo -EP -DFFI_BUILDING_DLL conftest.cpp
conftest.cpp
conftest.cpp(12): fatal error C1034: limits.h: no include path set
configure:14512: $? = 2
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "libffi"
| #define PACKAGE_TARNAME "libffi"
| #define PACKAGE_VERSION "3.4.8"
| #define PACKAGE_STRING "libffi 3.4.8"
| #define PACKAGE_BUGREPORT "http://github.com/libffi/libffi/issues"
| #define PACKAGE_URL ""
| #define PACKAGE "libffi"
| #define VERSION "3.4.8"
| #define LT_OBJDIR ".libs/"
| /* end confdefs.h. */
| #include <limits.h>
| Syntax error
configure:14547: error: in '/c/repos/libffi/aarch64-w64-mingw32':
configure:14549: error: C++ preprocessor "cl -nologo -EP" fails sanity check
See 'config.log' for more details
The configure script now completed! I had a feeling I would need to keep adding paths like this during the build process.
...
checking size of long double... 0
checking whether byte ordering is bigendian... no
checking assembler .cfi pseudo-op support... no
checking whether compiler supports pointer authentication... no
checking for _ prefix in compiled symbols... no
configure: versioning on shared library symbols is no
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating include/Makefile
config.status: creating include/ffi.h
config.status: creating Makefile
config.status: creating testsuite/Makefile
config.status: creating man/Makefile
config.status: creating doc/Makefile
config.status: creating libffi.pc
config.status: creating fficonfig.h
config.status: executing buildir commands
config.status: create top_srcdir/Makefile guessed from local Makefile
config.status: build in aarch64-w64-mingw32 (HOST=)
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing include commands
config.status: executing src commands
real 1m29.429s
user 0m32.473s
sys 0m35.396s
Investigating Build Errors
Just as I suspected, there were build errors when I ran make. Specifically, 8 of these C1083 errors:
libtool: compile: /c/repos/libffi/msvcc.sh -marm64 -I "/c/Progra~1/MIB055~1/2022/Enterprise/VC/Tools/MSVC/14.44.35207/include" -L "/c/Progra~1/MIB055~1/2022/Enterprise/VC/Tools/MSVC/14.44.35207/lib/arm64" -L "/c/progra~2/wi3cf2~1/10/lib/100226~1.0/ucrt/arm64" -L "/c/progra~2/wi3cf2~1/10/lib/100226~1.0/um/arm64" -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -DFFI_BUILDING_DLL -O2 -c ../src/prep_cif.c -DDLL_EXPORT -DPIC -o src/.libs/prep_cif.obj
C:/repos/libffi/include\ffi.h(66): fatal error C1083: Cannot open include file: 'stddef.h': No such file or directory
That file lives in C:\Program Files (x86)\Windows Kits\10\Include\10.0.22621.0\ucrt. The OpenJDK build includes these 5 paths (among many others) but I didn’t think I’d need the RT-related paths. I added the other 3 to the configure command then ran make again.
This search for commits did not yield anything but a web search of ksarm64.h – Search led me to the [Arm64/Windows] Missing ksarm64.h ? · Issue #7409 · dotnet/runtime GitHub issue, which said that ksarm64.h is part of the Windows SDK. ksarm64.h isn’t include in Windows SDK – Developer Community was the pointer about where it lives: c/progra~2/wi3cf2~1/10/include/100226~1.0/shared. I had excluded this path because I wanted a minimal set of include paths. This was the next command I tried. I should have exported these paths to an environment variable like I have at the top but I just kept moving forward.
libffi/msvcc.sh at v3.4.8 · libffi/libffi uses cygpath -ma, which outputs mixed absolute paths (windows form with forward slashes). Here is the corrected configure command (without the /cygdrive path prefixes):
This resolved the error about the spaces but then failed with:
Microsoft (R) Library Manager Version 14.44.35207.1
Copyright (C) Microsoft Corporation. All rights reserved.
LINK : fatal error LNK1181: cannot open input file 'src\.libs\prep_cif.obj'
Here’s the next iteration of the configure script:
...
libtool: compile: /c/repos/libffi/msvcc.sh -marm64 -I "/c/Progra~1/MIB055~1/2022/Enterprise/VC/Tools/MSVC/14.44.35207/include" -I "/c/progra~2/wi3cf2~1/10/include/100226~1.0/ucrt" -I "/c/progra~2/wi3cf2~1/10/include/100226~1.0/um" -I "/c/progra~2/wi3cf2~1/10/include/100226~1.0/shared" "-L/c/Progra~1/MIB055~1/2022/Enterprise/VC/Tools/MSVC/14.44.35207/lib/arm64" "-L/c/progra~2/wi3cf2~1/10/lib/100226~1.0/ucrt/arm64" "-L/c/progra~2/wi3cf2~1/10/lib/100226~1.0/um/arm64" -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -O2 -c ../src/prep_cif.c -DDLL_EXPORT -DPIC -o src/.libs/prep_cif.obj
D:/repos/dups/libffi/src/prep_cif.c(219): warning C4273: 'ffi_prep_cif': inconsistent dll linkage
D:/repos/dups/libffi/src/prep_cif.c(225): warning C4273: 'ffi_prep_cif_var': inconsistent dll linkage
D:/repos/dups/libffi/src/prep_cif.c(257): warning C4273: 'ffi_prep_closure': inconsistent dll linkage
D:/repos/dups/libffi/src/prep_cif.c(268): warning C4273: 'ffi_get_struct_offsets': inconsistent dll linkage
...
libtool: compile: /c/repos/libffi/msvcc.sh -marm64 -I "/c/Progra~1/MIB055~1/2022/Enterprise/VC/Tools/MSVC/14.44.35207/include" -I "/c/progra~2/wi3cf2~1/10/include/100226~1.0/ucrt" -I "/c/progra~2/wi3cf2~1/10/include/100226~1.0/um" -I "/c/progra~2/wi3cf2~1/10/include/100226~1.0/shared" "-L/c/Progra~1/MIB055~1/2022/Enterprise/VC/Tools/MSVC/14.44.35207/lib/arm64" "-L/c/progra~2/wi3cf2~1/10/lib/100226~1.0/ucrt/arm64" "-L/c/progra~2/wi3cf2~1/10/lib/100226~1.0/um/arm64" -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -O2 -c ../src/types.c -DDLL_EXPORT -DPIC -o src/.libs/types.obj
D:/repos/dups/libffi/src/types.c(77): error C2491: 'ffi_type_void': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(81): error C2491: 'ffi_type_uint8': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(82): error C2491: 'ffi_type_sint8': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(83): error C2491: 'ffi_type_uint16': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(84): error C2491: 'ffi_type_sint16': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(85): error C2491: 'ffi_type_uint32': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(86): error C2491: 'ffi_type_sint32': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(87): error C2491: 'ffi_type_uint64': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(88): error C2491: 'ffi_type_sint64': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(90): error C2491: 'ffi_type_pointer': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(92): error C2491: 'ffi_type_float': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(93): error C2491: 'ffi_type_double': definition of dllimport data not allowed
D:/repos/dups/libffi/src/types.c(111): error C2491: 'ffi_type_longdouble': definition of dllimport data not allowed
This seemed pretty odd, considering these errors didn’t show up for x64. I didn’t see any defines related to DLLs. Upon further inspection, I realized that I had removed the CPPFLAGS variable somewhere along the way! Restoring it finally got the job done! No make errors at all, phew!
I needed to build the zero variant of the HotSpot JVM for the Windows platform recently. libffi is one of the prerequisites for the zero variant. It provides “a portable, high level programming interface to various calling conventions.” I decided to build libffi/libffi at v3.4.8 since it looks like the latest version. I used a Windows x64 machine for this entire process. Visual C++ and MSYS need to be installed to do this. Launch MSYS2 and get the sources from GitHub:
Install automake and libtool using these commands:
pacman -S automake
pacman -S libtool
The Visual C++ compiler needs to be available in the path as well. Run cl without any parameters to ensure the compiler is available. It most likely won’t be by default. If it isn’t, add it to the path as follows:
Note that the name of the Visual C++ linker is link.exe, which clashes with the built in “link” command. Prepending the C++ compiler path means that the built-in link command will not be available. Appending the C++ compiler path means that the linker cannot be invoked without specifying its full path.
Generating the configure file
With the MSYS prerequisites installed, run the autogen.sh script:
user@machine /d/repos/libffi
$ ./autogen.sh
This creates a configure script in the root of the repository. Run it using bash:
Running configure takes about a minute and a half on my 24-core (32 logical processor) machine with 128GB RAM.
Building the Source Code
Simply run make in the root of the repo. The generated LIB and DLL files should be in the x86_64-w64-mingw32/.libs/ subdirectory of the repo root. There will also be ffi.h and ffitarget.h include files in the x86_64-w64-mingw32/include/ subdirectory of the repo root. These 4 files are typically what will be required by other projects with a libffi dependency (like OpenJDK).
$ ls -1 x86_64-w64-mingw32/.libs/
libffi.la
libffi.lai
libffi_convenience.la
libffi_convenience.lib
libffi-8.dll*
libffi-8.exp
libffi-8.lib
$ ls -1 x86_64-w64-mingw32/include/
ffi.h
ffitarget.h
Makefile
My Motivation for Building libffi
I was trying to configure an OpenJDK build (at commit c3de94cee12471) using this command line:
...
checking if hsdis should be bundled... no
checking for --enable-libffi-bundling... disabled, default
checking for LIBFFI... checking for ffi.h... no
configure: error: Could not find libffi!
configure exiting with result code 1
...
checking for --enable-libffi-bundling... disabled, default
checking if libffi works... no
configure: error: Found libffi but could not link and compile with it.
configure exiting with result code 1
This was my hint that I probably need to build libffi first. libffi/README.md at v3.4.8 · libffi/libffi explains that the configure script can be generated by running autogen.sh. I first need to fix the line endings. This copilot prompt “convert all existing files in a repo from windows to unix line endings” gets me the solution:
# Tells Git to convert CRLF to LF on commit
# but not the other way around on checkout.
git config core.autocrlf input
# resets the working directory and re-checks
# out the files using the current core.autocrlf setting
git reset --hard
Now autogen.sh can be executed. I didn’t read the instructions all the way through to see what prerequisites are required. Even so, which ones can I get away without?
user@machine /cygdrive/d/repos/libffi
$ ./autogen.sh
autoreconf-2.71: export WARNINGS=
autoreconf-2.71: Entering directory '.'
autoreconf-2.71: configure.ac: not using Gettext
autoreconf-2.71: running: aclocal -I m4
Can't exec "aclocal": No such file or directory at /usr/share/autoconf2.7/Autom4te/FileUtils.pm line 274.
autoreconf-2.71: error: aclocal failed with exit status: 1
configure: loading site script /etc/config.site
checking build system type... x86_64-w64-mingw32
checking host system type... x86_64-w64-mingw32
checking target system type... x86_64-w64-mingw32
continue configure in default builddir "./x86_64-w64-mingw32"
....exec /bin/sh ../configure "--srcdir=.." "--enable-builddir=x86_64-w64-mingw32" "mingw32"
configure: loading site script /etc/config.site
checking build system type... x86_64-w64-mingw32
checking host system type... x86_64-w64-mingw32
checking target system type... x86_64-w64-mingw32
checking for gsed... sed
checking for a BSD-compatible install... /usr/bin/install -c
checking whether sleep supports fractional seconds... yes
checking filesystem timestamp resolution... 0.01
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking xargs -n works... yes
checking for gcc... /d/repos/libffi/msvcc.sh -m64
checking whether the C compiler works... no
configure: error: in '/d/repos/libffi/x86_64-w64-mingw32':
configure: error: C compiler cannot create executables
See 'config.log' for more details
In Cygwin, that command failed with “configure: error: cannot run /bin/sh ./config.sub“. What could be going wrong in the configure script? M365 Copilot prompt: “change build system type in msys2” refers to gcc – Configuration x86_64-pc-msys not supported – Stack Overflow but those flags seem unnecessary given my platform. I tried removing some of the compiler setting flags to no avail:
$ time bash configure CPPFLAGS="-DFFI_BUILDING_DLL" CPP="cl -nologo -EP" CXXCPP="cl -nologo -EP" --disable-docs --prefix=/d/temp/libffi
configure: loading site script /etc/config.site
checking build system type... x86_64-w64-mingw32
checking host system type... x86_64-w64-mingw32
checking target system type... x86_64-w64-mingw32
continue configure in default builddir "./x86_64-w64-mingw32"
....exec /bin/sh ../configure "--srcdir=.." "--enable-builddir=x86_64-w64-mingw32" "mingw32"
configure: loading site script /etc/config.site
checking build system type... x86_64-w64-mingw32
checking host system type... x86_64-w64-mingw32
checking target system type... x86_64-w64-mingw32
checking for gsed... sed
checking for a BSD-compatible install... /usr/bin/install -c
checking whether sleep supports fractional seconds... yes
checking filesystem timestamp resolution... 0.01
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking xargs -n works... yes
checking for gcc... no
checking for cc... no
checking for cl.exe... no
checking for clang... no
configure: error: in '/d/repos/libffi/x86_64-w64-mingw32':
configure: error: no acceptable C compiler found in $PATH
See 'config.log' for more details
The config.log file is in the x86_64-w64-mingw32 folder in the repo root. What I should have verified is that I could run cl.exe in MSYS before trying any of this stuff. That was the primary reason for launch ucrt64.exe from a developer command prompt. Unfortunately, that didn’t work for whatever reason.
user@machine UCRT64 /d/repos/libffi
$ cl
-bash: cl: command not found
user@machine UCRT64 /d/repos/libffi
$ echo $PATH
/ucrt64/bin:/usr/local/bin:/usr/bin:/bin:/c/Windows/System32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
I tried manually fixing the path as follows but this didn’t work (cl.exe could still not be found):
The dir command can show the short name equivalents of a file name, e.g. dir /x C:\Program Files.
dir /x C:\
...
05/24/2025 11:42 AM <DIR> PROGRA~1 Program Files
04/09/2025 01:31 AM <DIR> PROGRA~2 Program Files (x86)
...
dir /x "C:\Program Files"
11/30/2023 04:40 PM <DIR> MIB055~1 Microsoft Visual Studio
Sure enough, I could now find cl.exe and the configure script worked!
$ where cl.exe
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.43.34808\bin\Hostx64\x64\cl.exe
$ bash configure \
CC="/d/repos/libffi/msvcc.sh -m64" \
CXX="/d/repos/libffi/msvcc.sh -m64" \
CPPFLAGS="-DFFI_BUILDING_DLL" \
LD=link CPP="cl -nologo -EP" CXXCPP="cl -nologo -EP" \
--disable-docs \
--prefix=/d/temp/libffi
configure: loading site script /etc/config.site
checking build system type... x86_64-w64-mingw32
checking host system type... x86_64-w64-mingw32
checking target system type... x86_64-w64-mingw32
continue configure in default builddir "./x86_64-w64-mingw32"
....exec /bin/sh ../configure "--srcdir=.." "--enable-builddir=x86_64-w64-mingw32" "mingw32"
configure: loading site script /etc/config.site
checking build system type... x86_64-w64-mingw32
checking host system type... x86_64-w64-mingw32
checking target system type... x86_64-w64-mingw32
checking for gsed... sed
checking for a BSD-compatible install... /usr/bin/install -c
checking whether sleep supports fractional seconds... yes
checking filesystem timestamp resolution... 0.01
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking xargs -n works... yes
checking for gcc... /d/repos/libffi/msvcc.sh -m64
checking whether the C compiler works... yes
checking for C compiler default output file name... conftest.exe
checking for suffix of executables... .exe
checking whether we are cross compiling... no
checking for suffix of object files... obj
checking whether the compiler supports GNU C... no
checking whether /d/repos/libffi/msvcc.sh -m64 accepts -g... yes
checking for /d/repos/libffi/msvcc.sh -m64 option to enable C11 features... unsupported
checking for /d/repos/libffi/msvcc.sh -m64 option to enable C99 features... unsupported
checking for /d/repos/libffi/msvcc.sh -m64 option to enable C89 features... unsupported
checking whether /d/repos/libffi/msvcc.sh -m64 understands -c and -o together... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of /d/repos/libffi/msvcc.sh -m64... none
checking whether the compiler supports GNU C++... no
checking whether /d/repos/libffi/msvcc.sh -m64 accepts -g... yes
checking for /d/repos/libffi/msvcc.sh -m64 option to enable C++11 features... unsupported
checking for /d/repos/libffi/msvcc.sh -m64 option to enable C++98 features... unsupported
checking dependency style of /d/repos/libffi/msvcc.sh -m64... none
checking dependency style of /d/repos/libffi/msvcc.sh -m64... none
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for fgrep... /usr/bin/grep -F
checking for non-GNU ld... link
checking if the linker (link) is GNU ld... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... no, using cp -pR
checking the maximum length of command line arguments... 8192
checking how to convert x86_64-w64-mingw32 file names to x86_64-w64-mingw32 format... func_convert_file_msys_to_w32
checking how to convert x86_64-w64-mingw32 file names to toolchain format... func_convert_file_msys_to_w32
checking for link option to reload object files... -r
checking for file... file
checking for objdump... objdump
checking how to recognize dependent libraries... file_magic ^x86 archive import|^x86 DLL
checking for dlltool... dlltool
checking how to associate runtime and link libraries... func_cygming_dll_for_implib
checking for ranlib... ranlib
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking command to parse /usr/bin/nm -B output from /d/repos/libffi/msvcc.sh -m64 object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... no
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... no
checking for dlfcn.h... no
checking for objdir... .libs
checking for /d/repos/libffi/msvcc.sh -m64 option to produce PIC... -DDLL_EXPORT -DPIC
checking if /d/repos/libffi/msvcc.sh -m64 PIC flag -DDLL_EXPORT -DPIC works... yes
checking if /d/repos/libffi/msvcc.sh -m64 static flag works... yes
checking if /d/repos/libffi/msvcc.sh -m64 supports -c -o file.obj... yes
checking if /d/repos/libffi/msvcc.sh -m64 supports -c -o file.obj... (cached) yes
checking whether the /d/repos/libffi/msvcc.sh -m64 linker (link) supports shared libraries... yes
checking dynamic linker characteristics... Win32 ld.exe
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... cl -nologo -EP
checking whether the /d/repos/libffi/msvcc.sh -m64 linker (link) supports shared libraries... no
checking for /d/repos/libffi/msvcc.sh -m64 option to produce PIC... -DDLL_EXPORT -DPIC
checking if /d/repos/libffi/msvcc.sh -m64 PIC flag -DDLL_EXPORT -DPIC works... yes
checking if /d/repos/libffi/msvcc.sh -m64 static flag works... yes
checking if /d/repos/libffi/msvcc.sh -m64 supports -c -o file.obj... yes
checking if /d/repos/libffi/msvcc.sh -m64 supports -c -o file.obj... (cached) yes
checking whether the /d/repos/libffi/msvcc.sh -m64 linker (link) supports shared libraries... no
checking dynamic linker characteristics... Win32 ld.exe
checking how to hardcode library paths into programs... immediate
checking for readelf... readelf
checking size of size_t... 8
checking for C compiler vendor... microsoft
checking whether C compiler accepts -O2... yes
checking CFLAGS for most reasonable warnings...
checking whether to enable maintainer-specific portions of Makefiles... no
checking for sys/memfd.h... no
checking for memfd_create... no
checking for egrep... (cached) /usr/bin/grep -E
checking for memcpy... no
checking for alloca.h... no
checking size of double... 8
checking size of long double... 8
checking whether byte ordering is bigendian... no
checking assembler .cfi pseudo-op support... no
checking assembler supports pc related relocs... yes
checking whether compiler supports pointer authentication... no
checking for _ prefix in compiled symbols... no
configure: versioning on shared library symbols is no
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating include/Makefile
config.status: creating include/ffi.h
config.status: creating Makefile
config.status: creating testsuite/Makefile
config.status: creating man/Makefile
config.status: creating doc/Makefile
config.status: creating libffi.pc
config.status: creating fficonfig.h
config.status: executing buildir commands
config.status: create top_srcdir/Makefile guessed from local Makefile
config.status: build in x86_64-w64-mingw32 (HOST=)
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing include commands
config.status: executing src commands
I could now run make as instructed by the readme. Here is the tail of the resulting output:
I manually copied these files to set up the libffi repo for building OpenJDK (the expected LIB filename does not have the -8 suffix by default). I’m guessing make install or something like that is the proper way to do this but I had what I needed so this was good enough for me.
I tested this file to ensure I could compile it. The linking step failed (cl.exe without the -c option).
$ cl -c -I include ffi_test.c
Microsoft (R) C/C++ Optimizing Compiler Version 19.43.34810 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
ffi_test.c
$ cl -I include ffi_test.c
Microsoft (R) C/C++ Optimizing Compiler Version 19.43.34810 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
ffi_test.c
Microsoft (R) Incremental Linker Version 14.43.34810.0
Copyright (C) Microsoft Corporation. All rights reserved.
/out:ffi_test.exe
ffi_test.obj
ffi_test.obj : error LNK2019: unresolved external symbol __imp_ffi_call referenced in function main
ffi_test.exe : fatal error LNK1120: 1 unresolved externals
I tried manually running link.exe but this failed because the wrong link.exe is called.
$ where link.exe
C:\software\msys64\usr\bin\link.exe
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.43.34808\bin\Hostx64\x64\link.exe
Prepending the compiler path to $PATH resolved this.
$ cl -I include ffi_test.c -link -libpath:lib libffi.lib
Microsoft (R) C/C++ Optimizing Compiler Version 19.43.34810 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
ffi_test.c
Microsoft (R) Incremental Linker Version 14.43.34810.0
Copyright (C) Microsoft Corporation. All rights reserved.
/out:ffi_test.exe
-libpath:lib
libffi.lib
ffi_test.obj
$ ./ffi_test.exe
At this point, things were in good enough shape to build OpenJDK. However, I could not successfully run bash configure ... in Cygwin (to build OpenJDK) now. Perhaps it’s because I had been mucking around with the Cygwin setup. I tried removing automake and libtool but that didn’t fix the problem.
The 8353009: Improve documentation for Windows AArch64 builds PR has a comment stating that “the BuildTools distribution of Visual Studio do not include aarch64-hosted compilers, so to be able to run native builds without the Prism emulation, you need to install the full Visual Studio, including the IDE.” This post describes how I determined this to be false.
Running bash configure --with-boot-jdk=<PATH> failed with the error that it could not find a C compiler:
...
checking for cacerts file... default
checking for cacerts source... default
checking for --enable-unlimited-crypto... enabled, default
checking for jni library path... default
configure: Using default toolchain microsoft (Microsoft Visual Studio)
configure: Found Visual Studio installation at /cygdrive/c/progra~2/micros~2/2022/BuildTools using well-known name
configure: Found Microsoft Visual Studio 2022
configure: Trying to extract Visual Studio environment variables for aarch64
configure: using /cygdrive/c/progra~2/micros~2/2022/BuildTools/vc/auxiliary/build/vcvarsarm64.bat
configure: Setting extracted environment variables for aarch64
checking that Visual Studio variables have been correctly extracted... ok
checking for cl... [not found]
configure: error: Could not find a C compiler.
configure exiting with result code 1
The TOOLCHAIN_FIND_COMPILER macro calls the UTIL_LOOKUP_TOOLCHAIN_PROGS macro to find the C compiler. I verified that the last argument is “cl” with an AC_MSG_NOTICE. At this point, I compared the TOOLCHAIN_PATH in config.log with that on a different ARM64 machine with a full VS install. Sure enough, it didn’t contain the bin/hostarm64/arm64 path with the buildtools setup, even though the path exists on disk. TOOLCHAIN_PATH is coming from VS_PATH in toolchain_microsoft.m4. Here is the build\windows-aarch64-server-slowdebug\configure-support\vs-env-aarch64\set-vs-env.sh file.
Notice that VS_PATH only has what VS_ENV_CMD added to the PATH! This was a clue that I need to take another step back – I realized that I couldn’t even run cl.exe in the developer command prompt! Then again, the command line for the terminal is:
Changing the host architecture to arm64 did not help. I launched the VS installer and noticed that the “Desktop development with C++” workload was not installed so I must have been missing additional components.
Visual Studio Build Tools 2022 LTSC 17.12 Workloads
I didn’t want to install the whole workload though, just the necessary individual components. I noticed the C++ Build Tools core features component wasn’t installed so I selected it. The Windows Universal C Runtime component is automatically selected as well:
Visual Studio Build Tools 2022 LTSC 17.12 Individual Components
Once the installation completed, I could run cl.exe in the developer command prompt!
**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.12.7
** Copyright (c) 2022 Microsoft Corporation
**********************************************************************
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools>cl
Microsoft (R) C/C++ Optimizing Compiler Version 19.42.34441 for ARM64
Copyright (C) Microsoft Corporation. All rights reserved.
usage: cl [ option... ] filename... [ /link linkoption... ]
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools>
The VS installer log in %TEMP% contained these components:
Copilot informed me that the caret was the way to split a command across multiple lines in the Windows Command Prompt. This was the final command I used to complete the 8353009: Improve documentation for Windows AArch64 builds PR.
I was recently trying to build the Prime95 Mersenne search software in Visual Studio 2022 when I got error messages about missing a gmp.h dependency.
1>C:\repos\gimps\p95v3019b13.source\common.h(23,10): error C1083: Cannot open include file: 'gmp.h': No such file or directory
...
This got me started trying to figure out how to build the GMP sources on Windows. It was easy to do in the MSYS MINGW64 shell. Use these steps:
cd /c/repos/gmp
curl -Lo gmp-6.3.0.tar.xz https://gmplib.org/download/gmp/gmp-6.3.0.tar.xz
unxz --keep gmp-6.3.0.tar.xz
tar xf gmp-6.3.0.tar
cd gmp-6.3.0
./configure
make
Such distractions aside, there is a link to download gmp-6.3.0.tar.xz:
cd /c/repos/gmp
curl -Lo gmp-6.3.0.tar.xz https://gmplib.org/download/gmp/gmp-6.3.0.tar.xz
file gmp-6.3.0.tar.xz
The file command outputs gmp-6.3.0.tar.xz: XZ compressed data, checksum CRC64. Thisrepresents XZ data compression, which is unfamiliar to me (haven’t run into this often). The unxz command can be used to decompress the file with the --keep option to avoid removing the source file.
unxz --keep gmp-6.3.0.tar.xz
tar xf gmp-6.3.0.tar
# Search for gmp.h
cd gmp-6.3.0
find . -name "gmp.h"
On an MS-DOS system DJGPP can be used to build GMP, and on an MS Windows system Cygwin, DJGPP and MINGW can be used. All three are excellent ports of GCC and the various GNU tools.
Trying building it in the MSYS MINGW64 Shell. The end of the ./configure output is shown below. The host type and install prefix are different from the Cygwin environment’s.
config.status: linking mpn/x86_64/k8/gmp-mparam.h to gmp-mparam.h
config.status: executing libtool commands
configure: summary of build options:
Version: GNU MP 6.3.0
Host type: x86_64-w64-mingw32
ABI: 64
Install prefix: /mingw64
Compiler: gcc
Static libraries: yes
Shared libraries: no
The make command succeeds in the MSYS MINGW64 Shell, running for 4 minutes. I can ignore Cygwin for now. Let’s try the Tutorial on GMP (colorado.edu). Copy the example into a file called mpz_simple1.c then use the command from the tutorial to compile it. Interestingly, I don’t need the -I and -L arguments from the tutorial. The gmp library must already be installed.
cd /c/repos/scratchpad/apps/gmp/tutorial
gcc -o mpz_simple1 mpz_simple1.c -lgmp
To see how gmp.h and the libraries are found, run these commands:
Poking around in file explorer shows 2022-01-05 timestamps for gmp.h and libgmp.*. Looks like these were indeed installed with MSYS. How do I automatically output the timestamps for each result of find? bash – How to loop through file names returned by find? – Stack Overflow suggests this command:
find . -name "*gmp*" | while IFS= read -r file; do ls -l $file; done
At this point, all we have seen is how to build GMP in the MSYS MINGW64 shell. We have also verified that we can build a sample GMP program, the Tutorial on GMP (colorado.edu). The Cygwin and Visual Studio environments can be investigated another time.
To change the active code page, go to Control Panel > Region. Click on the “Change system locale…” button in the Administrative tab.
The Region Dialog Box
The Region Settings dialog will pop up. Select a different locale e.g. Japanese (Japan).
Reboot when prompted. You can verify (even before rebooting) that the active and OEM code pages have changed. Locales like Kiswahili (Kenya) and English (India) did not change the code page values (and therefore didn’t prompt to reboot).
After rebooting, I delete the build directory then configure and build OpenJDK again. This time the build fails with these errors:
ERROR: Build failed for target 'images' in configuration 'windows-x86_64-server-slowdebug' (exit code 2)
Stopping javac server
=== Output from failing command(s) repeated here ===
* For target hotspot_variant-server_libjvm_gtest_objs_test_json.obj:
test_json.cpp
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(357): error C2143: syntax error: missing ')' before ']'
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(355): error C2660: 'JSON_GTest::test': function does not take 1 arguments
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(49): note: see declaration of 'JSON_GTest::test'
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(355): note: while trying to match the argument list '(const char [171])'
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(357): error C2143: syntax error: missing ';' before ']'
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(357): error C2059: syntax error: ']'
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(357): error C2017: illegal escape sequence
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(357): error C2059: syntax error: ')'
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(363): error C2143: syntax error: missing ')' before ']'
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(361): error C2660: 'JSON_GTest::test': function does not take 1 arguments
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(49): note: see declaration of 'JSON_GTest::test'
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(361): note: while trying to match the argument list '(const char [174])'
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(363): error C2143: syntax error: missing ';' before ']'
d:\java\forks\jdk\test\hotspot\gtest\utilities\test_json.cpp(363): error C2059: syntax error: ']'
... (rest of output omitted)
* All command lines available in /cygdrive/d/java/forks/jdk/build/windows-x86_64-server-slowdebug/make-support/failure-logs.
=== End of repeated output ===
No indication of failed target found.
HELP: Try searching the build log for '] Error'.
HELP: Run 'make doctor' to diagnose build problems.
The Visual C++ compiler’s behavior when reading source files depends on whether or not source files have a byte-order mark.
By default, Visual Studio detects a byte-order mark to determine if the source file is in an encoded Unicode format, for example, UTF-16 or UTF-8. If no byte-order mark is found, it assumes that the source file is encoded in the current user code page, unless you’ve specified a code page by using /utf-8 or the /source-charset option.
This can be easily tested using hexdump in Cygwin. Launch notepad and open the test.txt file created by these commands. The File > Save as dialog has an Encoding dropdown that write a byte-order marker for any of the UTF options. Running hexdump will display the byte-order markers.
echo abc123 > test.txt
hexdump -C test.txt
Inspect the OpenJDK source file failing to build confirms that there is no BOM in the file. (can this be done on GitHub?)
diff --git a/make/autoconf/flags-cflags.m4 b/make/autoconf/flags-cflags.m4
index c0c78ce95b6..bbb0426c368 100644
--- a/make/autoconf/flags-cflags.m4
+++ b/make/autoconf/flags-cflags.m4
@@ -560,7 +560,9 @@ AC_DEFUN([FLAGS_SETUP_CFLAGS_HELPER],
TOOLCHAIN_CFLAGS_JVM="-qtbtable=full -qtune=balanced -fno-exceptions \
-qalias=noansi -qstrict -qtls=default -qnortti -qnoeh -qignerrno -qstackprotect"
elif test "x$TOOLCHAIN_TYPE" = xmicrosoft; then
- TOOLCHAIN_CFLAGS_JVM="-nologo -MD -Zc:preprocessor -Zc:strictStrings -Zc:inline -MP"
+ # The -utf8 option sets source and execution character sets to UTF-8 to enable correct
+ # compilation of all source files regardless of the active code page on Windows.
+ TOOLCHAIN_CFLAGS_JVM="-nologo -MD -Zc:preprocessor -Zc:strictStrings -Zc:inline -MP -utf-8"
TOOLCHAIN_CFLAGS_JDK="-nologo -MD -Zc:preprocessor -Zc:strictStrings -Zc:inline -Zc:wchar_t-"
fi
The build still fails but this time the error is from the java.desktop tree.
ERROR: Build failed for target 'images' in configuration 'windows-x86_64-server-slowdebug' (exit code 2)
=== Output from failing command(s) repeated here ===
* For target support_native_java.desktop_libfreetype_afblue.obj:
afblue.c
d:\java\forks\jdk\src\java.desktop\share\native\libfreetype\src\autofit\afblue.c(1): error C2220: the following warning is treated as an error
d:\java\forks\jdk\src\java.desktop\share\native\libfreetype\src\autofit\afblue.c(1): warning C4819: The file contains a character that cannot be represented in the current code page (932). Save the file in Unicode format to prevent data loss
d:\java\forks\jdk\src\java.desktop\share\native\libfreetype\src\autofit\afscript.h(1): warning C4819: The file contains a character that cannot be represented in the current code page (932). Save the file in Unicode format to prevent data loss
d:\java\forks\jdk\src\java.desktop\share\native\libfreetype\src\autofit\afblue.c(257): warning C4819: The file contains a character that cannot be represented in the current code page (932). Save the file in Unicode format to prevent data loss
... (rest of output omitted)
* For target support_native_java.desktop_libfreetype_afcjk.obj:
afcjk.c
...
When running this test code, I noticed this warning (first message displayed):
2023-05-31 12:31:33,686 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Checking for Loadable Native Libraries
The Apache Hadoop 3.3.5 – Native Libraries Guide explains that there is a NativeLibraryChecker that can be run using the command bin/hadoop checknative -a to show which native libraries can/cannot be loaded.
saint@ubuntuvm:~/java/binaries/hadoop/hadoop-3.3.5$ find . -name lib*.so
./lib/native/libhadoop.so
./lib/native/libhdfspp.so
./lib/native/libhdfs.so
./lib/native/libnativetask.so
saint@ubuntuvm:~/java/binaries/hadoop/hadoop-3.3.5$ uname -a
Linux ubuntuvm 5.19.0-41-generic #42~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 18 17:40:00 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
saint@ubuntuvm:~/java/binaries/hadoop/hadoop-3.3.5$ bin/hadoop checknative -a
2023-05-31 13:36:04,467 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Native library checking:
hadoop: false
zlib: false
zstd : false
bzip2: false
openssl: false
ISA-L: false
PMDK: false
2023-05-31 13:36:04,711 INFO util.ExitUtil: Exiting with status 1: ExitException
Diagnosing Native Library Load Errors
My assumption when seeing that none of these native libraries could be loaded was that I needed to install all those dependencies. I started with lib64z.
saint@ubuntuvm:~/java/binaries/hadoop/hadoop-3.3.5$ sudo apt install lib64z1
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
gcc-12-base:i386 krb5-locales libc6:i386 libc6-amd64:i386 libcom-err2:i386 libcrypt1:i386
libgcc-s1:i386 libgssapi-krb5-2 libgssapi-krb5-2:i386 libidn2-0:i386 libk5crypto3 libk5crypto3:i386
libkeyutils1:i386 libkrb5-3 libkrb5-3:i386 libkrb5support0 libkrb5support0:i386 libnsl2:i386
libnss-nis:i386 libnss-nisplus:i386 libssl3 libssl3:i386 libtirpc3:i386 libunistring2:i386
Suggested packages:
glibc-doc:i386 locales:i386 krb5-doc krb5-user krb5-doc:i386 krb5-user:i386
The following NEW packages will be installed:
gcc-12-base:i386 krb5-locales lib64z1:i386 libc6:i386 libc6-amd64:i386 libcom-err2:i386
libcrypt1:i386 libgcc-s1:i386 libgssapi-krb5-2:i386 libidn2-0:i386 libk5crypto3:i386
libkeyutils1:i386 libkrb5-3:i386 libkrb5support0:i386 libnsl2:i386 libnss-nis:i386
libnss-nisplus:i386 libssl3:i386 libtirpc3:i386 libunistring2:i386
The following packages will be upgraded:
libgssapi-krb5-2 libk5crypto3 libkrb5-3 libkrb5support0 libssl3
5 upgraded, 20 newly installed, 0 to remove and 85 not upgraded.
Need to get 10.3 MB/12.2 MB of archives.
After this operation, 38.1 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Interestingly, rerunning checknative still showed false for all the native libraries! Next step was to inspect how the checknative argument is handled. It invokes the hadoop/NativeLibraryChecker.java class, which in turn calls the hadoop/NativeCodeLoader.java. One of the most important observations in the latter file is the additional debug logging available when the library doesn’t load!
The debug output is much now more informative! Notice the warning about the possible platform mismatch of the native library!
saint@ubuntuvm:~/java/binaries/hadoop/hadoop-3.3.5$ bin/hadoop --loglevel DEBUG checknative -a
2023-05-31 14:47:32,624 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
2023-05-31 14:47:32,625 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /home/saint/java/binaries/hadoop/hadoop-3.3.5/lib/native/libhadoop.so.1.0.0: /home/saint/java/binaries/hadoop/hadoop-3.3.5/lib/native/libhadoop.so.1.0.0: cannot open shared object file: No such file or directory (Possible cause: can't load AARCH64-bit .so on a AMD 64-bit platform)
2023-05-31 14:47:32,625 DEBUG util.NativeCodeLoader: java.library.path=/home/saint/java/binaries/hadoop/hadoop-3.3.5/lib/native
2023-05-31 14:47:32,625 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2023-05-31 14:47:32,836 DEBUG util.Shell: setsid exited with exit code 0
Native library checking:
hadoop: false
zlib: false
zstd : false
bzip2: false
openssl: false
ISA-L: false
PMDK: false
2023-05-31 14:47:32,847 DEBUG util.ExitUtil: Exiting with status 1: ExitException
1: ExitException
at org.apache.hadoop.util.ExitUtil.terminate(ExitUtil.java:381)
at org.apache.hadoop.util.ExitUtil.terminate(ExitUtil.java:369)
at org.apache.hadoop.util.NativeLibraryChecker.main(NativeLibraryChecker.java:154)
2023-05-31 14:47:32,856 INFO util.ExitUtil: Exiting with status 1: ExitException
To determine the architecture for which the shared library was compiled, I started with the objdump -f command as suggested by a StackOverflow post. However, it outputs architecture: UNKNOWN!, which isn’t very useful. The file command from the same post proves to be exactly what I need.
saint@ubuntuvm:~/java/binaries/hadoop/aarch64/hadoop-3.3.5$ objdump -f lib/native/libhadoop.so
lib/native/libhadoop.so: file format elf64-little
architecture: UNKNOWN!, flags 0x00000150:
HAS_SYMS, DYNAMIC, D_PAGED
start address 0x0000000000005b80
saint@ubuntuvm:~/java/binaries/hadoop/aarch64/hadoop-3.3.5$ file lib/native/libhadoop.so
lib/native/libhadoop.so: symbolic link to libhadoop.so.1.0.0
saint@ubuntuvm:~/java/binaries/hadoop/aarch64/hadoop-3.3.5$ file lib/native/libhadoop.so.1.0.0
lib/native/libhadoop.so.1.0.0: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=19fbe9b0a7449eb05b687721548251af752b869f, with debug_info, not stripped
Turns out I was using an x86-64 Ubuntu VM instead of the aarch64 Ubuntu VM I had created so naturally, hadoop couldn’t load the aarch64 hadoop native library! For the VM I had been using, I needed to get the hadoop build by running:
Checking the loading status of the native libraries now indicates that the hadoop native library can be successfully loaded:
saint@ubuntuvm:~/java/binaries/hadoop/x64/hadoop-3.3.5$ bin/hadoop checknative -a
2023-05-31 14:58:40,869 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native
2023-05-31 14:58:40,877 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
2023-05-31 14:58:40,887 WARN erasurecode.ErasureCodeNative: Loading ISA-L failed: Failed to load libisal.so.2 (libisal.so.2: cannot open shared object file: No such file or directory)
2023-05-31 14:58:40,887 WARN erasurecode.ErasureCodeNative: ISA-L support is not available in your platform... using builtin-java codec where applicable
2023-05-31 14:58:41,035 INFO nativeio.NativeIO: The native code was built without PMDK support.
Native library checking:
hadoop: true /home/saint/java/binaries/hadoop/x64/hadoop-3.3.5/lib/native/libhadoop.so.1.0.0
zlib: true /lib/x86_64-linux-gnu/libz.so.1
zstd : true /lib/x86_64-linux-gnu/libzstd.so.1
bzip2: true /lib/x86_64-linux-gnu/libbz2.so.1
openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open shared object file: No such file or directory)!
ISA-L: false Loading ISA-L failed: Failed to load libisal.so.2 (libisal.so.2: cannot open shared object file: No such file or directory)
PMDK: false The native code was built without PMDK support.
2023-05-31 14:58:41,056 INFO util.ExitUtil: Exiting with status 1: ExitException
Switching to the aarch64 Ubuntu VM also showed the aarch64 hadoop native library being successfully loaded on that platform. In hindsight, the 386 architecture references when I installed lib64z could have been a warning sign if I wasn’t just blasting my way through running these commands.
Building firefox with debug symbols has been a rather tricky endeavor for me for quite some time. The documentation on how to do this seems to indicate that adding these lines to your mozconfig file should be sufficient:
should do the trick. However, I have (for months now) been running into this make error: no rule to make nspr4.pdb needed by export whenever I try to build with debug symbols. Bug 338224 has a pending patch for this issue. In the mean time, the following trick seems to solve the problem for me: force the -Zi compiler option into the compiler flags variable. For the nspr files, the PDB file will be generated as desired. For the other files, the -Zi option will be redundant but this fix gets you up and running ASAP. Here’s the associated patch file:
diff --git a/nsprpub/configure.in b/nsprpub/configure.in
--- a/nsprpub/configure.in
+++ b/nsprpub/configure.in
@@ -2827,18 +2827,18 @@ if test -n "$_SAVE_DEBUG_FLAGS"; then
fi
if test -n "$MOZ_OPTIMIZE"; then
CFLAGS="$CFLAGS $_OPTIMIZE_FLAGS"
CXXFLAGS="$CXXFLAGS $_OPTIMIZE_FLAGS"
fi
if test -n "$MOZ_DEBUG_SYMBOLS"; then
- CFLAGS="$CFLAGS $_DEBUG_FLAGS"
- CXXFLAGS="$CXXFLAGS $_DEBUG_FLAGS"
+ CFLAGS="$CFLAGS $_DEBUG_FLAGS -Zi"
+ CXXFLAGS="$CXXFLAGS $_DEBUG_FLAGS -Zi"
fi
if test -n "$MOZ_OPTIMIZE"; then
OBJDIR_TAG=_OPT
else
OBJDIR_TAG=_DBG
fi
Here is the mozconfig file I used with this patch:
