llvm_package_15.0.2\llvm-project-llvmorg-15.0.2\libcxx\test\std\thread\thread.mutex\thread.mutex.requirements\thread.sharedtimedmutex.requirements\thread.sharedtimedmutex.class\try_lock_shared_for.pass.cpp - The system cannot find the path specified.
llvm_package_15.0.2\llvm-project-llvmorg-15.0.2\libcxx\test\std\thread\thread.mutex\thread.mutex.requirements\thread.sharedtimedmutex.requirements\thread.sharedtimedmutex.class\try_lock_shared_until.pass.cpp - The system cannot find the path specified.
llvm_package_15.0.2\llvm-project-llvmorg-15.0.2\libcxx\test\std\thread\thread.mutex\thread.mutex.requirements\thread.sharedtimedmutex.requirements\thread.sharedtimedmutex.class\try_lock_until_deadlock_bug.pass.cpp - The system cannot find the path specified.
These files still exist on disk though! They are displayed if you dir their containing directory but are not found if you dir their full paths! They cannot be deleted using del either. Interestingly, pressing tab after the directory path will autocomplete the file names.
C:\> dir D:\dev\repos\llvm\dups\llvm-project\llvm\utils\release\llvm_package_15.0.2\llvm-project-llvmorg-15.0.2\libcxx\test\std\thread\thread.mutex\thread.mutex.requirements\thread.sharedtimedmutex.requirements\thread.sharedtimedmutex.class\
Volume in drive D is DATAVOL1
Volume Serial Number is 8800-8693
Directory of D:\dev\repos\llvm\dups\llvm-project\llvm\utils\release\llvm_package_15.0.2\llvm-project-llvmorg-15.0.2\libcxx\test\std\thread\thread.mutex\thread.mutex.requirements\thread.sharedtimedmutex.requirements\thread.sharedtimedmutex.class
10/16/2022 01:22 PM <DIR> .
10/04/2022 03:29 AM <DIR> ..
10/04/2022 03:29 AM 2,461 try_lock_shared_for.pass.cpp
10/04/2022 03:29 AM 2,423 try_lock_shared_until.pass.cpp
10/04/2022 03:29 AM 2,146 try_lock_until_deadlock_bug.pass.cpp
3 File(s) 7,030 bytes
C:\> dir D:\dev\repos\llvm\dups\llvm-project\llvm\utils\release\llvm_package_15.0.2\llvm-project-llvmorg-15.0.2\libcxx\test\std\thread\thread.mutex\thread.mutex.requirements\thread.sharedtimedmutex.requirements\thread.sharedtimedmutex.class\try_lock_shared_for.pass.cpp
Volume in drive D is DATADRIVE1
Volume Serial Number is 548C-FFC9
Directory of D:\dev\repos\llvm\dups\llvm-project\llvm\utils\release\llvm_package_15.0.2\llvm-project-llvmorg-15.0.2\libcxx\test\std\thread\thread.mutex\thread.mutex.requirements\thread.sharedtimedmutex.requirements\thread.sharedtimedmutex.class
File Not Found
C:\> del D:\dev\repos\llvm\dups\llvm-project\llvm\utils\release\llvm_package_15.0.2\llvm-project-llvmorg-15.0.2\libcxx\test\std\thread\thread.mutex\thread.mutex.requirements\thread.sharedtimedmutex.requirements\thread.sharedtimedmutex.class\try_lock_shared_for.pass.cpp
The system cannot find the path specified.
These files can be viewed in file explorer. Something that caught my eye when examining their properties is that their locations started with the \\?\ prefix! That seems unusual for files on my local drive.
File Locations Starting with \\?\
Resource monitor does not show any images with associated handles when searching for “try_lock”. Neither does searching for “\?\D:\dev\repos\llvm\”. I tried using Process Explorer’s “Find Handle or DLL…” command as well. There also don’t appear to be any child processes for the cmd.exe process I was using (a Developer 2019 Command Prompt).
Next idea, open Process Monitor and see what’s happening when dir and rmdir are executed. I used the Path contains thread.sharedtimedmutex.class filter. The deletes are showing up as SetDispositionInformationFile events and seem to be using the RemoveDirectoryW function.
The RemoveDirectory function marks a directory for deletion on close. Therefore, the directory is not removed until the last handle to the directory is closed.
Notice the NOT EMPTY result of the SetDispositionInformationFile event. I believe this comes from RemoveDirectoryW. There’s the question of how the 3 files are printed to the command line. The FindNextFile API is used to search for files.
Ah, in the middle of this investigation, PowerShell.exe dies and so does Windows Terminal. All my tabs, everything, gone! Aaaargh… Windows Event Viewer has an Information level event showing that powershell.exe crashed due to a System.InvalidOperationException. This is then followed by another Information event with the WER source and P1 problem signature Microsoft.WindowsTerminal_1.15.2713.0_x64__8wekyb3d8bbwe. Then comes the Error level event with the Application Hang source and General explanation that “The program WindowsTerminal.exe version 1.15.2209.28003 stopped interacting with Windows and was closed. To see if more information about the problem is available, check the problem history in the Security and Maintenance control panel.” The ExeFileName is cut off below but simply append “\WindowsTerminal.exe” to the package name to reconstruct it. Looks like I need to avoid PowerShell. And why is there no crash dump created for it???
One upside of this crash is that it lets me confirm that it is not the cmd.exe process that is hanging onto those files. I terminate explorer.exe and when I run new task in Task Manager, it asks me to create a Windows Hello pin. What is happening?? Now moving on to opening these files: Notepad++ acts as though nothing happened when you File->Open and select one of them. Notepad opens it though! Running cat in Git Bash also dumps its contents:
cat /d/dev/repos/llvm/dups/llvm-project/llvm/utils/release/llvm_package_15.0.2/llvm-project-llvmorg-15.0.2/libcxx/test/std/thread/thread.mutex/thread.mutex.requirements/thread.sharedtimedmutex.requirements/thread.sharedtimedmutex.class/try_lock_shared_for.pass.cpp
//===----------------------------------------------------------------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
...
I’m suspecting Windows Defender but don’t have any definitive proof. According to Enable attack surface reduction (ASR) rules, this command in an admin powershell should do the trick if the problem was Windows Defender:
Well, looks like rm from Git Bash works just fine as does deleting from file explorer. Unfortunately, this unsolved issue is probably going to continue to cause pain in command prompt batch files like the LLVM build script.
hsdis is a plugin for disassembling code dynamically generated by the Java Virtual Machine. On Linux & MacOS, it uses GNU binutils. Support for the LLVM disassembly backend was recently added to hsdis in https://github.com/openjdk/jdk/pull/7531. This was motivated by the fact that GNU binutils is not distributed with the JDK (due to licensing reasons mentioned at https://github.com/openjdk/jdk/pull/5920#issuecomment-942398786) and the LLVM disassembly may be preferrable in certain circumstances. Unfortunately, the official Windows LLVM distribution does not have the header files necessary to build the hotspot disassembler. This prevents Windows developers from easily using the LLVM disassembler backend because they now have to build LLVM themselves as well – see hsdis LLVM backend for Windows ARM64 and Building LLVM for Windows ARM64, for example. In this post, we investigate why the LLVM Windows build does not have the necessary header files. The llvm-c directory in Windows build contains these 2 files only:
C:\Program Files\LLVM\include\llvm-c>dir
Volume in drive C is OSDisk
Volume Serial Number is c070-2ac0
Directory of C:\Program Files\LLVM\include\llvm-c
01/08/2022 11:54 AM <DIR> .
01/08/2022 11:54 AM <DIR> ..
09/24/2021 10:18 AM 29,760 lto.h
09/24/2021 10:18 AM 9,632 Remarks.h
2 File(s) 39,392 bytes
2 Dir(s) 62,273,200,128 bytes free
I created a local LLVM build (see Building LLVM with CMake) and confirmed that it has all the header files.
C:\dev\repos\llvm-project\build_llvm\install_local\include\llvm-c>dir /w
Volume in drive C is OSDisk
Volume Serial Number is 0087-4c48
Directory of C:\dev\repos\llvm-project\build_llvm\install_local\include\llvm-c
[.] [..] Analysis.h
BitReader.h BitWriter.h blake3.h
Comdat.h Core.h DataTypes.h
DebugInfo.h Deprecated.h Disassembler.h
DisassemblerTypes.h Error.h ErrorHandling.h
ExecutionEngine.h ExternC.h Initialization.h
IRReader.h Linker.h LLJIT.h
lto.h Object.h Orc.h
OrcEE.h Remarks.h Support.h
Target.h TargetMachine.h [Transforms]
Types.h
28 File(s) 382,361 bytes
3 Dir(s) 59,158,138,880 bytes free
Does this problem still exist in the latest Windows LLVM release? I went to Releases · llvm/llvm-project (github.com) to find the latest LLVM installer for Windows but couldn’t find it. Turns out it’s because the 15.0.1 release is only 14 hours old so some of the assets probably haven’t been uploaded. Notice that 15.0.0 has 47 assets. I can successfully download and install LLVM-15.0.0-win64.exe and see that the header files are still missing.
Interestingly, trying to install LLVM-15.0.0-win32.exe before uninstalling LLVM-15.0.0-win64.exe gives this dialog and clicking Yes uninstalls before the actual installation of the 32-bit build starts!
LLVM is already installed.
I assumed that would happen at this stage:
All the same, these dialogs have strings that can lead us to the sources that create the installer! The installer looks very similar to the one from Building the Elmer Install Folder so searching the llvm codebase for “ncis ” gives only a handful of hits leading to the key discovery of build_llvm_release.bat! (later learn that this needs to be executed in a (2019) developer command prompt so that the ninja command can be found). That script requires 7zip though. The script fails on my machine because it can’t find 7zip. Failure seems to be coming from the for-statement (see for | Microsoft Learn for usage). The for command uses the escape character (^) as explained at set | Microsoft Learn.
C:\dev\repos\llvm-project\llvm\utils\release> build_llvm_release.bat 15.0.0
Check 7-zip version and/or administrator permissions.
'7z.exe' is not recognized as an internal or external command,
operable program or batch file.
You need to modify the paths below:
Revision: llvmorg-15.0.0
Package version: 15.0.0
Build dir: C:\dev\repos\llvm-project\llvm\utils\release\llvm_package_15.0.0
Press any key to continue . . .
-- Looking for CrashReporterClient.h
-- Looking for CrashReporterClient.h - not found
-- Looking for pfm_initialize in pfm
-- Looking for pfm_initialize in pfm - not found
-- Could NOT find ZLIB (missing: ZLIB_LIBRARY ZLIB_INCLUDE_DIR)
CMake Error at C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.24/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find LibXml2 (missing: LIBXML2_INCLUDE_DIR)
Call Stack (most recent call first):
C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.24/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.24/Modules/FindLibXml2.cmake:108 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
cmake/config-ix.cmake:156 (find_package)
CMakeLists.txt:774 (include)
-- Configuring incomplete, errors occurred!
See also "C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/CMakeFiles/CMakeOutput.log".
See also "C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/CMakeFiles/CMakeError.log".
Looking through FindPackageHandleStandardArgs.cmake leads me to the simple realization that the wrong define is being used on the command line. Could this be because I’m using a newer CMake? I’ve been using the VS 2022 Preview Developer Command Prompt thus far. My VS 2019 (16.11.19) installation uses CMake 3.20. Both FindLibXml2.cmake in 3.20 and FindLibXml2.cmake in 3.24 require the LIBXML2_INCLUDE_DIR variable. However, they also claim (at the top) to set these variables.
in my build folder (build_llvm), there is a CPackConfig.cmake file that sets variables like CPACK_PACKAGE_FILE_NAME and CPACK_NSIS_DISPLAY_NAME. Since it is NSIS Wiki (sourceforge.io) in use, I wonder about running the package target myself in a manner similar to that used to create my local build. I switch back to a previous build directory (created without the build_llvm_release.bat) and run:
cmake --build . --config Release --target package
The resulting failure below indicates that NSIS is required.
MSBuild version 17.4.0-preview-22466-03+48ab5664b for .NET Framework
PipSqueak.vcxproj -> C:\dev\repos\llvm-project\build_llvm\unittests\Support\DynamicLibrary\Release\PipSqueak.dll
SecondLib.vcxproj -> C:\dev\repos\llvm-project\build_llvm\unittests\Support\DynamicLibrary\Release\SecondLib.dll
obj.llvm-tblgen.vcxproj -> C:\dev\repos\llvm-project\build_llvm\utils\TableGen\obj.llvm-tblgen.dir\Release\obj.llvm-tblgen.lib
LLVMDemangle.vcxproj -> C:\dev\repos\llvm-project\build_llvm\Release\lib\LLVMDemangle.lib
...
verify-uselistorder.vcxproj -> C:\dev\repos\llvm-project\build_llvm\Release\bin\verify-uselistorder.exe
yaml-bench.vcxproj -> C:\dev\repos\llvm-project\build_llvm\Release\bin\yaml-bench.exe
yaml2obj.vcxproj -> C:\dev\repos\llvm-project\build_llvm\Release\bin\yaml2obj.exe
EXEC : CPack error : Cannot find NSIS compiler makensis: likely it is not installed, or not in your PATH [C:\dev\repos\llvm-project\build_llvm\package.vcxproj]
EXEC : CPack error : Could not read NSIS registry value. This is usually caused by NSIS not being installed. Please install NSIS from http://nsis.sourceforge.net [C:\dev\repos\llvm-proje
ct\build_llvm\package.vcxproj]
EXEC : CPack error : Cannot initialize the generator NSIS [C:\dev\repos\llvm-project\build_llvm\package.vcxproj]
After installing NSIS, the previous command successfully creates an LLVM for Windows installer.
...
verify-uselistorder.vcxproj -> C:\dev\repos\llvm-project\build_llvm\Release\bin\verify-uselistorder.exe
yaml-bench.vcxproj -> C:\dev\repos\llvm-project\build_llvm\Release\bin\yaml-bench.exe
yaml2obj.vcxproj -> C:\dev\repos\llvm-project\build_llvm\Release\bin\yaml2obj.exe
CPack: Create package using NSIS
CPack: Install projects
CPack: - Install project: LLVM [Release]
CMake Warning (dev) at C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.24/Modules/GNUInstallDirs.cmake:243 (messa
ge):
Unable to determine default CMAKE_INSTALL_LIBDIR directory because no
target architecture is known. Please enable at least one language before
including GNUInstallDirs.
Call Stack (most recent call first):
C:/dev/repos/llvm-project/llvm/cmake/modules/LLVMInstallSymlink.cmake:5 (include)
C:/dev/repos/llvm-project/build_llvm/tools/llvm-ar/cmake_install.cmake:48 (include)
C:/dev/repos/llvm-project/build_llvm/tools/cmake_install.cmake:39 (include)
C:/dev/repos/llvm-project/build_llvm/cmake_install.cmake:71 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
CPack: Create package
CPack: - package: C:/dev/repos/llvm-project/build_llvm/LLVM-16.0.0git-win64.exe generated.
This installer generates the LLVM includes on disk as expected. The issue must therefore be confined to the installer generated by the script.
Reviewing Ninja NSIS Packaging
At this point, I ran build_llvm_release.bat to create an installer. Once packaging is complete, the install_manifest.txt file can be used to determine which files are in the installer. The batch file also runs lots of tests and this was annoying when trying to generate installers. Once the tests failed on the build I was creating and I had CTRL+C’d a couple of times, I ran ninja package myself (taken from the batch file)
C:\dev\repos\llvm-project\llvm\utils\release\llvm_package_15.0.0\build32_stage0>ninja package
[0/1] Run CPack packaging tool...CPack: Create package using NSIS
CPack: Install projects
CPack: - Install project: LLVM []
CMake Warning (dev) at C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.24/Modules/GNUInstallDirs.cmake:243 (message):
Unable to determine default CMAKE_INSTALL_LIBDIR directory because no
target architecture is known. Please enable at least one language before
including GNUInstallDirs.
Call Stack (most recent call first):
C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/llvm-project/llvm/cmake/modules/LLVMInstallSymlink.cmake:5 (include)
C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/tools/llvm-ar/cmake_install.cmake:40 (include)
C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/tools/cmake_install.cmake:39 (include)
C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/cmake_install.cmake:114 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
CPack: Create package
CPack: - package: C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/LLVM-15.0.0-win32.exe generated.
I then try to find a package target in build.ninja. Search for CMakeFiles\package.util.+ include since we’re interested in include files. There are some interesting differences in the include directories of the build created manually from the local install and the one created by the script, e.g.
Try searching in build.ninja for the 2 header files the installer creates in the (broken) shipping LLVM for Windows build. Nothing there but searching the file system for remarks.h gives interesting results, e.g. the existence of an NSIS project file: project.nsi. Looks like there are some tutorials showing how to create .nsi files at Invoking NSIS run-time commands on compile-time – NSIS (sourceforge.io). The way NSIS is used with CPack when building is documented at Packaging With CPack — Mastering CMake
… but I completely missed the fact that the 2nd didn’t have these lines from the first.
if(CMAKE_INSTALL_COMPONENT STREQUAL "llvm-headers" OR NOT CMAKE_INSTALL_COMPONENT)
file(INSTALL DESTINATION "${CMAKE_INSTALL_PREFIX}/include" TYPE DIRECTORY FILES
"C:/dev/repos/llvm-project/llvm/include/llvm"
"C:/dev/repos/llvm-project/llvm/include/llvm-c"
FILES_MATCHING REGEX "/[^/]*\\.def$" REGEX "/[^/]*\\.h$" REGEX "/[^/]*\\.td$" REGEX "/[^/]*\\.inc$" REGEX "/license\\.txt$")
endif()
if(CMAKE_INSTALL_COMPONENT STREQUAL "llvm-headers" OR NOT CMAKE_INSTALL_COMPONENT)
file(INSTALL DESTINATION "${CMAKE_INSTALL_PREFIX}/include" TYPE DIRECTORY FILES
"C:/dev/repos/llvm-project/build_llvm/include/llvm"
"C:/dev/repos/llvm-project/build_llvm/include/llvm-c"
FILES_MATCHING REGEX "/[^/]*\\.def$" REGEX "/[^/]*\\.h$" REGEX "/[^/]*\\.gen$" REGEX "/[^/]*\\.inc$" REGEX "/cmakefiles$" EXCLUDE REGEX "/config\\.h$" EXCLUDE)
endif()
Search the codebase for “llvm-headers” and find the llvm-header component definition. That whole code block is gated by the LLVM_INSTALL_TOOLCHAIN_ONLY variable! This is explicitly turned off in build_llvm_release.bat! I rerun the batch file and see tests failing after the build succeeds. CTRL+C to kill the processes so that I can get to the root issue: does turning off that flag fix the includes? makensis fails, probably because I killed the build and some things might still have been in use?
C:\dev\repos\llvm-project\llvm\utils\release\llvm_package_15.0.0\build32_stage0>ninja package
[0/1] Run CPack packaging tool...CPack: Create package using NSIS
CPack: Install projects
CPack: - Install project: LLVM []
CMake Warning (dev) at C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.24/Modules/GNUInstallDirs.cmake:243 (message):
Unable to determine default CMAKE_INSTALL_LIBDIR directory because no
target architecture is known. Please enable at least one language before
including GNUInstallDirs.
Call Stack (most recent call first):
C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/llvm-project/llvm/cmake/modules/LLVMInstallSymlink.cmake:5 (include)
C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/tools/llvm-ar/cmake_install.cmake:40 (include)
C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/tools/cmake_install.cmake:39 (include)
C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/cmake_install.cmake:128 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
CPack: Create package
CPack Error: Problem running NSIS command: "C:/Program Files (x86)/NSIS/makensis.exe" "C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/_CPack_Packages/win32/NSIS/project.nsi"
Please check C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/_CPack_Packages/win32/NSIS/NSISOutput.log for errors
CPack Error: Problem compressing the directory
CPack Error: Error when generating package: LLVM
FAILED: CMakeFiles/package.util
cmd.exe /C "cd /D C:\dev\repos\llvm-project\llvm\utils\release\llvm_package_15.0.0\build32_stage0 && "C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cpack.exe" --config ./CPackConfig.cmake"
ninja: build stopped: subcommand failed.
NSISOutput.log failed due to an “Internal compiler error #12345: error mmapping datablock to 17235001.” However, the include files are now present in the source directory being packaged by NSIS.
Turning Off Tests
There are many tests that the build script runs and some of them are failing. Testing is not on my critical path since all I need is to generate installers so I modify the scripts to enable me to package the build without running all the tests. I then start my build without tests and go to bed only to wake up the next morning to find that I need to rerun it because there are no running programs when I log in. Event Viewer doesn’t show any reboot-related events and sure enough, Task Manager shows over 9 days of uptime still. Turns out the Desktop Window Manager crashed (C:\WINDOWS\system32\dwm.exe)! Curse you dwmcore.dll. Well, time to install those updates I’ve been putting off and reboot before jumping back in. Now on the new Windows 10.0.22621.521. The build still fails:
-- LLVM host triple: i686-pc-windows-msvc
-- LLVM default target triple: i686-pc-windows-msvc
-- Using Release VC++ CRT: MD
-- Looking for os_signpost_interval_begin
-- Looking for os_signpost_interval_begin - not found
CMake Error at C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find Python3 (missing: Python3_EXECUTABLE Interpreter) (Required
is at least version "3.6")
Reason given by package:
Interpreter: Cannot use the interpreter "C:/Python310/python.exe"
Call Stack (most recent call first):
C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.20/Modules/FindPython/Support.cmake:3165 (find_package_handle_standard_args)
C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.20/Modules/FindPython3.cmake:485 (include)
CMakeLists.txt:817 (find_package)
-- Configuring incomplete, errors occurred!
See also "C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/CMakeFiles/CMakeOutput.log".
See also "C:/dev/repos/llvm-project/llvm/utils/release/llvm_package_15.0.0/build32_stage0/CMakeFiles/CMakeError.log".
When I interrupted the tests before modifying the batch file to skip them, I noticed that they were being run by %LOCALAPPDATA%\Microsoft\WindowsApps\python3.9.exe. This is still present on my machine. Ah, turns out I’m now using the 2019 developer command prompt (and therefore an older CMake). The only difference between CMake 3.20 FindPython3.cmake and CMake 3.24 FindPython3.cmake is a comment about static libraries, so this failure is a mystery.
Diagnosing Build Failures
Since this issue also bit me when I moved to my Surface Book, it is worth understanding why it happens.
Missing CMake in Visual Studio 17.3.4 Developer Command Prompt
Here is the VS 2022 Preview vs VS 2022 Enterprise path to CMake:
C:\Program Files (x86)\Microsoft Visual Studio\Installer> where cmake
C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe
C:\dev\repos\llvm-project\llvm\utils\release> where cmake
INFO: Could not find files for the given pattern(s).
The View Logs link opens the Documents folder under This PC – not particularly useful. Interestingly though, clicking on the Modify button shows a Total space required 1.63 GB. How is there space required before I’ve selected anything? Something similar happens with 16.11.19 though. Without making any individual component selections, I start the install process. CMake gets (re-?)installed as shown below. This fixes the setup warnings as well and cmake is now usable in the VS2022 command prompt.
Missing Python3 in VS 17.3.4 Developer Command Prompt
This is the error I got when trying to build LLVM on my Surface Book 2 in the VS 2022 developer command prompt:
CMake Error at C:/Program Files/CMake/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:164 (message):
Could NOT find Python3 (missing: Python3_EXECUTABLE Interpreter) (Required
is at least version "3.6")
Reason given by package:
Interpreter: Cannot use the interpreter "C:/Python310/python.exe"
Call Stack (most recent call first):
C:/Program Files/CMake/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:445 (_FPHSA_FAILURE_MESSAGE)
C:/Program Files/CMake/share/cmake-3.17/Modules/FindPython/Support.cmake:2437 (find_package_handle_standard_args)
C:/Program Files/CMake/share/cmake-3.17/Modules/FindPython3.cmake:309 (include)
CMakeLists.txt:817 (find_package)
Uninstalling CMake enables the command line to pick up the CMake distributed with Visual Studio. Python3 is now found successfully in the path below (I’ve shortened it using %LOCALAPPDATA%).
-- Found Python3: %LOCALAPPDATA%/Microsoft/WindowsApps/python3.8.exe (found suitable version "3.8.10", minimum required is "3.6") found components: Interpreter
Missing Python3 in VS 16.11.19 Developer Command Prompt
Interestingly, I still get the same error in VS 2019 despite uninstalling CMake 3.17. My earlier hypothesis is therefore invalid.
CMake Error at C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find Python3 (missing: Python3_EXECUTABLE Interpreter) (Required
is at least version "3.6")
Reason given by package:
Interpreter: Cannot use the interpreter "C:/Python310/python.exe"
Call Stack (most recent call first):
C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.20/Modules/FindPython/Support.cmake:3165 (find_package_handle_standard_args)
C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.20/Modules/FindPython3.cmake:485 (include)
CMakeLists.txt:817 (find_package)
C:/Python310/python.exe -c "import sys; sys.stdout.write('.'.join([str(x) for x in sys.version_info[:3]]))"
I comment out the ERROR_QUIET line to reveal the stdout and stderr output from python since the return code from the python process is causing the CMake error to be raised. Running with --trace-expand --trace-redirect=cmake_trace.txt now reveals the root cause (paths below cleaned up using %LOCALAPPDATA%):
Python path configuration:
PYTHONHOME = '%LOCALAPPDATA%\Programs\Python\Python310-32'
PYTHONPATH = (not set)
program name = 'C:/Python310/python.exe'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = 'C:\\Python310\\python.exe'
sys.base_prefix = '%LOCALAPPDATA%\\Programs\\Python\\Python310-32'
sys.base_exec_prefix = '%LOCALAPPDATA%\\Programs\\Python\\Python310-32'
sys.platlibdir = 'lib'
sys.executable = 'C:\\Python310\\python.exe'
sys.prefix = '%LOCALAPPDATA%\\Programs\\Python\\Python310-32'
sys.exec_prefix = '%LOCALAPPDATA%\\Programs\\Python\\Python310-32'
sys.path = [
'C:\\Python310\\python310.zip',
'%LOCALAPPDATA%\\Programs\\Python\\Python310-32\\DLLs',
'%LOCALAPPDATA%\\Programs\\Python\\Python310-32\\lib',
'C:\\Python310',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00003174 (most recent call first):
<no Python frame>
django – init_fs_encoding: failed to get the Python codec of the filesystem encoding – Stack Overflow is a hint that the PYTHONHOME is wrong. Sure enough, I didn’t change it in build_llvm_release.bat so the paths in the configuration above do not exist! This now raises another question: how on earth does this work in VS 2022? I notice on my desktop that python.exe does not even appear in the CMake tracing output! The difference in behavior stems from the fact that the find_program command in Modules/FindPython/Support.cmake · v3.20.0 finds python 3.10 first in the VS 2019 environment. This path is then assigned to _Python3_EXECUTABLE, preventing the 3.8 path from being used. One important difference between CMake 3.20 and 3.23 that I notice is FindPython: fix typo error (fff8d5b2) · Commits · CMake / CMake · GitLab (kitware.com). Since the fix for the build_llvm_release.bat script is straightforward and it is clear that there are some CMake implementation differences at work, we no longer need to dig into why this behavior could be happening.
Python Hangs
One of my build attempts successfully completes stage0 but hangs when CMake tries to detect the python version. Manually running the same command (copied from Process Explorer) also hangs. Even %LOCALAPPDATA%/Microsoft/WindowsApps/python3.9.exe --version hangs. Inspecting the full dump created by Task Manager reveals that python3.9.exe made a call to get (what looks like) the Package.InstalledLocation Property (Windows.ApplicationModel) – Windows UWP applications | Microsoft Learn
...
-- Looking for os_signpost_interval_begin
-- Looking for os_signpost_interval_begin - not found
Windows becomes pretty unusable as I investigate this behavior (mouse doesn’t work, changes program in focus but can’t click on anything). A reboot fixes these issues (e.g. version now works). Can’t believe we have to deal with this in 2022???
7z x clang+llvm-15.0.1-aarch64-linux-gnu.tar.xz
7z x clang+llvm-15.0.1-arm64-apple-darwin21.0.tar.xz
7z x clang+llvm-15.0.1-x86_64-apple-darwin.tar.xz
tar xf clang+llvm-15.0.1-aarch64-linux-gnu.tar
tar xf clang+llvm-15.0.1-arm64-apple-darwin21.0.tar
tar xf clang+llvm-15.0.1-x86_64-apple-darwin.tar
Here are the directories in the include folder before the installer is created. There are also 28 include files in the include/llvm-c/ directory as desired.
Directory of llvm\utils\release\llvm_package_15.0.0\build32_stage0\_CPack_Packages\win64\NSIS\LLVM-15.0.0-win64\include
clang
clang-c
clang-tidy
lld
lldb
llvm
llvm-c
Outstanding Questions
Why does the NSIS project fail to build? Why are there test failures and build errors?
Why does the Linux build have ompt-multiplex.h and the aarch64-unknown-linux-gnu directory?
How is the Windows ARM64 installer generated?
Why doesn’t the Windows build have c++, flang, mlir, mlir-c, and polly?
How do we get symbols to the Python app in the Microsoft Store?
8253757: Add LLVM-based backend for hsdis by magicus · Pull Request #7531 makes it possible to easily use LLVM as the hsdis backend. An LLVM installation is required for this. The official LLVM builds for the Windows platform do not work for building hsdis because they do not have all the prerequisite LLVM include files. See Building LLVM for Windows ARM64 – Saint’s Log (swesonga.org) for instructions on how to build LLVM for ARM64 Windows (on an x64 Windows host). To configure OpenJDK for LLVM as an hsdis backend on Windows ARM64, use this command:
The JDK and hsdis can then be built as usual with these commands:
make images
make build-hsdis
make install-hsdis
cp /cygdrive/d/dev/software/llvm-aarch64/bin/LLVM-C.dll build/windows-aarch64-server-slowdebug/jdk/bin/
The generated JDK can then be deployed to an ARM64 machine like the Surface Pro X. To test LLVM’s disassembly, use the -XX:CompileCommand flag on the ARM64 machine:
The path given to --with-llvm needs to be a Cygwin path if building in Cygwin. Otherwise, the build-hsdis target will fail with this error: c:\...\jdk\src\utils\hsdis\llvm\hsdis-llvm.cpp(58): fatal error C1083: Cannot open include file: 'llvm-c/Disassembler.h': No such file or directory. I caught this by inspecting build\windows-aarch64-server-release\make-support\failure-logs\support_hsdis_hsdis-llvm.obj.cmdline after the build failed. This was the only include that didn’t have Cygwin paths: -IC:/dev/repos/llvm-project/build_llvm_AArch64/install_local/include
Investigating Missing Disassembly
My first disassembly attempt did not work – only abstract disassembly was displayed:
I verified that hsdis-aarch64.dll was present in the JDK’s bin folder. That was the only issue I had seen before that caused this behavior so I dug around to find the code that loads the hsdis DLL. A search for the “hsdis-” DLL prefix in the sources reveals the hsdis_library_name string used in the Disassembler::dll_load method. Notice that there is a Verbose flag that can display what is happening when loading the hsdis DLL!
void* Disassembler::dll_load(char* buf, int buflen, int offset, char* ebuf, int ebuflen, outputStream* st) {
int sz = buflen - offset;
int written = jio_snprintf(&buf[offset], sz, "%s%s", hsdis_library_name, os::dll_file_extension());
if (written < sz) { // written successfully, not truncated.
if (Verbose) st->print_cr("Trying to load: %s", buf);
return os::dll_load(buf, ebuf, ebuflen);
} else if (Verbose) {
st->print_cr("Try to load hsdis library failed: the length of path is beyond the OS limit");
}
return NULL;
}
Error: VM option 'Verbose' is develop and is available only in debug version of VM.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
hsdis-aarch64.dll is not being loaded because LLVM-C.dll cannot be found! Still learning the need for reading the full instructions to avoid unnecessary pain.
I was trying to test using LLVM as a backend for hsdis on the Windows ARM64 platform as implemented in PR 5920. I downloaded LLVM 13 and tried to use it in the build. Unfortunately, it didn’t have all the prerequisite include files and so building your own LLVM installation was the approach suggested for Windows. This post explicitly outlines the instructions needed to build LLVM for the Windows ARM64 platform on a Windows x64 host machine.
The first requirement is an LLVM build with native llvm-nm.exe and llvm-tblgen.exe binaries. These can be downloaded (I think) or generated by building LLVM for the native x64 platform as specified in the instructions from Jorn.
git clone https://github.com/llvm/llvm-project.git
cd llvm-project
mkdir build_llvm
cd build_llvm
cmake ../llvm -D"LLVM_TARGETS_TO_BUILD:STRING=X86" -D"CMAKE_BUILD_TYPE:STRING=Release" -D"CMAKE_INSTALL_PREFIX=install_local" -A x64 -T host=x64
cmake --build . --config Release --target install
Once that build successfully completes, we can then build LLVM for the Windows ARM64 platform with the commands below. Notice that we specify paths to the native llvm-nm and llvm-tblgen binaries to prevent the build from trying to use their ARM64 equivalents (which won’t run on the host).
cd llvm-project
mkdir build_llvm_AArch64
cd build_llvm_Aarch64
cmake ../llvm -DLLVM_TARGETS_TO_BUILD:STRING=AArch64 \
-DCMAKE_BUILD_TYPE:STRING=Release \
-DCMAKE_INSTALL_PREFIX=install_local \
-DCMAKE_CROSSCOMPILING=True \
-DLLVM_TARGET_ARCH=AArch64 \
-DLLVM_NM=C:/repos/llvm-project/build_llvm/install_local/bin/llvm-nm.exe \
-DLLVM_TABLEGEN=C:/repos/llvm-project/build_llvm/install_local/bin/llvm-tblgen.exe \
-DLLVM_DEFAULT_TARGET_TRIPLE=aarch64-win32-msvc \
-A ARM64 \
-T host=x64
date; time \
cmake --build . --config Release --target install ; \
date
Once the build completes, the LLVM ARM64 files will be in the build_llvm_AArch64/install_local folder in the llvm-project repo. That build should have all the necessary header files and static libraries required for LLVM projects targeting Windows on ARM64. See the general cmake options and the LLVM-specific cmake options for details on the various flags and variables.
cd llvm-project
mkdir build_llvm_AArch64
cd build_llvm_AArch64
cmake ../llvm -D"LLVM_TARGETS_TO_BUILD:STRING=AArch64" \
-D"CMAKE_BUILD_TYPE:STRING=Release" \
-D"CMAKE_INSTALL_PREFIX=install_local_AArch64" \
-D"CMAKE_CROSSCOMPILING=True" \
-D"LLVM_TARGET_ARCH=AArch64" \
-A x64 \
-T host=x64
cmake --build . --config Release --target install
This still results in errors about conflicting machine types:
c:\...\llvm-project\build_llvm_aarch64\install_local_aarch64\\lib\llvmaarch64disassembler.lib : warning LNK4272: library machine type 'x64' conflicts with target machine type 'ARM64'
That’s when I tried adding the LLVM_TABLE_GEN from a Windows x64 LLVM build I had generated earlier. I accidentally omitted the options prefixed with a # below because I didn’t include the trailing slash after adding the llvm-tblgen.exe option.
The build still succeeded and generated AArch64 .lib files in the LLVM installation! Interestingly, they still had the x64 machine type in the header.
$ dumpbin /headers build_llvm_AArch64_2\install_local_AArch64_2\lib\LLVMAArch64AsmParser.lib
Microsoft (R) COFF/PE Dumper Version 14.29.30133.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file build_llvm_AArch64_2\install_local_AArch64_2\lib\LLVMAArch64AsmParser.lib
File Type: LIBRARY
FILE HEADER VALUES
8664 machine (x64)
...
I had no choice but to reexamine my understanding of what the -A flag does. It is used to specify the platform name but it’s only after digging into the CMAKE_GENERATOR_PLATFORM docs that I noticed that this was the target platform! This also made me realize that I hadn’t noticed that the x64 C++ compiler was being used all along!
-- The C compiler identification is MSVC 19.29.30133.0
-- The CXX compiler identification is MSVC 19.29.30133.0
-- The ASM compiler identification is MSVC
-- Found assembler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - works
Setting -A to AArch64 causes MSBuild to fail with an error about an unknown platform. So -A just might be the argument I need to get ARM64 libraries built.
"C:\dev\repos\llvm-project\build_llvm_AArch64_3\CMakeFiles\3.17.3\VCTargetsPath.vcxproj" (default target) (1) ->
(_CheckForInvalidConfigurationAndPlatform target) ->
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(820,5): error : The BaseOutputPath/OutputPath property is not set for project 'VCTargetsPath.vcxproj'. Please check to make sure that you have specified a valid combination of Configuration and Platform for this project. Configuration='Debug' Platform='AArch64'. You may be seeing this message because you are trying to build a project without a solution file, and have specified a non-default Configuration or Platform that doesn't exist for this project. [C:\dev\repos\llvm-project\build_llvm_AArch64_3\CMakeFiles\3.17.3\VCTargetsPath.vcxproj]
So I tried using -A ARM64 instead. I noticed that we now have the ARM64 C++ compiler selected! This is something I should have been paying attention to from the beginning, crucial for cross-compilation.
-- The C compiler identification is MSVC 19.29.30133.0
-- The CXX compiler identification is MSVC 19.29.30133.0
-- The ASM compiler identification is MSVC
-- Found assembler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/arm64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/arm64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/arm64/cl.exe - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/arm64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/arm64/cl.exe - works
Unfortunately, the build still failed with an error from gen-msvc-exports.py. Taking a look at gen-msvc-exports.py, it looks like it is trying to run llvm-nm.exe (for the target platform).
Generating export list for LLVM-C
Traceback (most recent call last):
File "C:/dev/repos/llvm-project/llvm/tools/llvm-shlib/gen-msvc-exports.py", line 116, in <module>
main()
File "C:/dev/repos/llvm-project/llvm/tools/llvm-shlib/gen-msvc-exports.py", line 112, in main
gen_llvm_c_export(ns.output, ns.underscore, libs, ns.nm)
File "C:/dev/repos/llvm-project/llvm/tools/llvm-shlib/gen-msvc-exports.py", line 72, in gen_llvm_c_export
check_call([nm, '-g', lib], stdout=dumpout_f)
File "C:\dev\tools\Anaconda3\lib\subprocess.py", line 359, in check_call
retcode = call(*popenargs, **kwargs)
File "C:\dev\tools\Anaconda3\lib\subprocess.py", line 340, in call
with Popen(*popenargs, **kwargs) as p:
File "C:\dev\tools\Anaconda3\lib\subprocess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\dev\tools\Anaconda3\lib\subprocess.py", line 1307, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
OSError: [WinError 216] This version of %1 is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for 'C:\dev\repos\llvm-project\build_llvm_AArch64_3\CMakeFi
les\02a88fa656bb9cf8b9ffd0e0debe57ae\libllvm-c.exports.rule;C:\dev\repos\llvm-project\build_llvm_AArch64_3\CMakeFiles\8ebc0efbf04134b25d0f37561fba0d55\LLVM-C.def.rule;C:\dev\repos\llvm-project\build_llvm_AArch64_
3\CMakeFiles\509fcb3f8bb132e9c560e15e8d25cb45\LLVM-C_exports.rule;C:\dev\repos\llvm-project\llvm\tools\llvm-shlib\CMakeLists.txt' exited with code 1. [C:\dev\repos\llvm-project\build_llvm_AArch64_3\tools\llvm-shl
ib\LLVM-C_exports.vcxproj]
date; time cmake ../llvm -D"LLVM_TARGETS_TO_BUILD:STRING=AArch64" \
-D"CMAKE_BUILD_TYPE:STRING=Release" \
-D"CMAKE_INSTALL_PREFIX=install_local" \
-D"CMAKE_CROSSCOMPILING=True" \
-D"LLVM_TARGET_ARCH=AArch64" \
-D"LLVM_NM=C:\dev\repos\llvm-project\build_llvm\install_local\bin\llvm-nm.exe" \
-D"LLVM_TABLEGEN=C:\dev\repos\llvm-project\build_llvm\install_local\bin\llvm-tblgen.exe" \
-D"LLVM_DEFAULT_TARGET_TRIPLE=aarch64-win32-msvc" \
-A ARM64 \
-T host=x64
date; time \
cmake --build . --config Release --target install; \
date
These build commands work! Dumpbin shows that the generated .lib files have ARM64 headers!
$ dumpbin /headers C:\dev\repos\llvm-project\build_llvm_AArch64_3\Release\lib\LLVMAArch64Disassembler.lib
Microsoft (R) COFF/PE Dumper Version 14.29.30133.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file C:\dev\repos\llvm-project\build_llvm_AArch64_3\Release\lib\LLVMAArch64Disassembler.lib
File Type: LIBRARY
FILE HEADER VALUES
AA64 machine (ARM64)
Unfortunately, the JDK project that got me started down this path still doesn’t build. Cygwin shows defines like -DLLVM_DEFAULT_TRIPLET='”aarch64-pc-windows-msvc”‘ being passed to the compiler, which then complains:
C:/.../src/utils/hsdis/llvm/hsdis-llvm.cpp(217): error C2015: too many characters in constant
The quotes in the commands therefore needed to be dropped. This caused build failures since the paths used back-slashes!
Building Opts.inc...
'C:devreposllvm-projectbuild_llvminstall_localbinllvm-tblgen.exe' is not recognized as an internal or external command,
operable program or batch file
This is now the part where I find a nice document on the LLVM site with the 3-liner for this task 😀
