Saint's Log

Trying to Build LLVM for Windows using the Release Script

The post on Tracking Down Missing Headers in LLVM for Windows identified which headers were missing from the LLVM for Windows builds. Let us fix this. Clone the LLVM repo and install the prerequisites as explained at Getting Started with the LLVM System using Microsoft Visual Studio — LLVM 18.0.0git documentation.

• 7-Zip
• Visual Studio 2022

The build_llvm_release.bat script in the LLVM repo is used to create a release build for Windows. Running it without any arguments displays a sample command line to use to build LLVM.

cd \repos && mkdir llvm
cd \repos\llvm\llvm-project\llvm\utils\release

C:\> cd \repos\llvm\llvm-project\llvm\utils\release
C:\repos\llvm\llvm-project\llvm\utils\release> build_llvm_release.bat
--version option is required
=============================
Script for building the LLVM installer on Windows,
used for the releases at https://github.com/llvm/llvm-project/releases

Usage: build_llvm_release.bat --version <version> [--x86,--x64, --arm64]

Options:
--version: [required] version to build
--help: display this help
--x86: build and test x86 variant
--x64: build and test x64 variant
--arm64: build and test arm64 variant

Note: At least one variant to build is required.

Example: build_llvm_release.bat --version 15.0.0 --x86 --x64

Let us build version 17.0.1, which is the latest LLVM release. Open an administrator Developer Command Prompt then run:

:: set PATH=%PATH%;C:\Program Files\7-Zip
build_llvm_release.bat --version 17.0.1 --x64

The first error is caused by mv not being a standard command in the Windows command prompt. There are ways to create aliases, e.g. alias – Aliases in Windows command prompt – Stack Overflow but given that this is a batch file, why not just use the built-in function? I fix this in the script.

C:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_17.0.1> mv llvm-project-* llvm-project   || exit /b 1
'mv' is not recognized as an internal or external command,
operable program or batch file.

The build directory needs to be deleted before restarting the build with the fix. Otherwise, the script will fail.

rmdir /s /q llvm_package_17.0.1

Installing Python

The next things that fails on my machine is setting the python environment. I should have started with Getting Started with the LLVM System using Microsoft Visual Studio — LLVM 18.0.0git documentation. I head to Welcome to Python.org and download Python 3.11.5. The installer says I already have it:

This command confirms that it is installed.

C:\repos\llvm\llvm-project\llvm\utils\release> \Python311\python.exe --version
Python 3.11.0

However, from a new command window, it looks like it hasn’t been set up.

C:\repos> python --version
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

Just running python loads the Microsoft Store:

I proceed with the Customize Installation option in the open Python installer, this time selecting “for all users (requires admin privileges)”.

I’m not sure why the “Install Python 3.11 for all users” option is not checked on the “Advanced Options” page.

Setup completes successfully. The What’s New In Python 3.11 — Python 3.11.5 documentation page is interesting.

Opening a new command prompt then running python still gives the same behavior as before. where python shows that the new installation appears 2nd.

C:\repos> where python
C:\Users\saint\AppData\Local\Microsoft\WindowsApps\python.exe
C:\Python311\python.exe

The PATH environment variable has these paths in reverse order so I don’t understand what is happening. Looks like the easiest way forward is to just install python from the Microsoft Store – this at least ensures that python scripts can run. After a failure due to the python command not being found, closer inspection of build_llvm_release.bat reveals that it needs the PYTHON_HOME environment variable to be set. However, that is set on line 358 using the python_dir variable, which is in turn the first argument to :set_environment. The call to set_environment is from :do_build_32 (or do_build_64), which uses a hard-coded python path! The script should first validate the python installation and exit early if it cannot be found! I should have reported/fixed these when I worked on Tracking Down Missing Headers in LLVM for Windows – Saint’s Log (swesonga.org) last year.

Build Errors

With the Python path fixed, the build proceeds but the C++ compilation fails. A few months ago I got the error below build version 15.0.7. Unfortunately, I didn’t explicitly note the compiler version.

[5834/6907] Building CXX object tools\lldb\source\Plugins\InstrumentationRuntime\MainThreadChecker\CMakeFiles\lldbPluginInstrumentationRuntimeMainThreadChecker.dir\InstrumentationRuntimeMainThreadChecker.cpp.obj
FAILED: tools/lldb/source/Plugins/InstrumentationRuntime/MainThreadChecker/CMakeFiles/lldbPluginInstrumentationRuntimeMainThreadChecker.dir/InstrumentationRuntimeMainThreadChecker.cpp.obj
C:\PROGRA~1\MICROS~3\2022\COMMUN~1\VC\Tools\MSVC\1437~1.328\bin\Hostx64\x64\cl.exe 
 /nologo
 /TP
 -DGTEST_HAS_RTTI=0
 -DUNICODE
 -D_CRT_NONSTDC_NO_DEPRECATE
 -D_CRT_NONSTDC_NO_WARNINGS
 -D_CRT_SECURE_NO_DEPRECATE
 -D_CRT_SECURE_NO_WARNINGS
 -D_ENABLE_EXTENDED_ALIGNED_STORAGE
 -D_HAS_EXCEPTIONS=0
 -D_SCL_SECURE_NO_DEPRECATE
 -D_SCL_SECURE_NO_WARNINGS
 -D_UNICODE
 -D__STDC_CONSTANT_MACROS
 -D__STDC_FORMAT_MACROS
 -D__STDC_LIMIT_MACROS
 -IC:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\build64_stage0\tools\lldb\source\Plugins\InstrumentationRuntime\MainThreadChecker
 -IC:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\llvm-project\lldb\source\Plugins\InstrumentationRuntime\MainThreadChecker
 -IC:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\llvm-project\lldb\include
 -IC:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\build64_stage0\tools\lldb\include
 -IC:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\build64_stage0\include
 -IC:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\llvm-project\llvm\include
 -IC:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\llvm-project\llvm\..\clang\include
 -IC:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\build64_stage0\tools\lldb\..\clang\include
 -IC:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\llvm-project\lldb\source
 -IC:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\build64_stage0\tools\lldb\source
 -DLIBXML_STATIC
 /Zc:inline
 /Zc:__cplusplus
 /Oi
 /bigobj
 /permissive-
 /W4
 -wd4141
 -wd4146
 -wd4244
 -wd4267
 -wd4291
 -wd4351
 -wd4456
 -wd4457
 -wd4458
 -wd4459
 -wd4503
 -wd4624
 -wd4722
 -wd4100
 -wd4127
 -wd4512
 -wd4505
 -wd4610
 -wd4510
 -wd4702
 -wd4245
 -wd4706
 -wd4310
 -wd4701
 -wd4703
 -wd4389
 -wd4611
 -wd4805
 -wd4204
 -wd4577
 -wd4091
 -wd4592
 -wd4319
 -wd4709
 -wd4324
 -w14062
 -we4238
 /Gw
 /MD
 /O2
 /Ob2
 /DNDEBUG  
 -wd4018
 -wd4068
 -wd4150
 -wd4201
 -wd4251
 -wd4521
 -wd4530 
 /EHs-c-
 /GR-
 -std:c++14
 /showIncludes
 /Fotools\lldb\source\Plugins\InstrumentationRuntime\MainThreadChecker\CMakeFiles\lldbPluginInstrumentationRuntimeMainThreadChecker.dir\InstrumentationRuntimeMainThreadChecker.cpp.obj
 /Fdtools\lldb\source\Plugins\InstrumentationRuntime\MainThreadChecker\CMakeFiles\lldbPluginInstrumentationRuntimeMainThreadChecker.dir\lldbPluginInstrumentationRuntimeMainThreadChecker.pdb
 /FS
 -c C:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\llvm-project\lldb\source\Plugins\InstrumentationRuntime\MainThreadChecker\InstrumentationRuntimeMainThreadChecker.cpp
C:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_15.0.7\llvm-project\lldb\source\Plugins\InstrumentationRuntime\MainThreadChecker\InstrumentationRuntimeMainThreadChecker.cpp : fatal error C1083: Cannot open compiler generated file: '': Invalid argument

Today, I got a similar error in InstrumentationRuntimeMainThreadChecker.cpp AND and this internal compiler error using Microsoft (R) C/C++ Optimizing Compiler Version 19.38.33133 for x64!

FAILED: tools/clang/lib/Analysis/FlowSensitive/CMakeFiles/obj.clangAnalysisFlowSensitive.dir/HTMLLogger.cpp.obj
C:\PROGRA~1\MICROS~3\2022\COMMUN~1\VC\Tools\MSVC\1438~1.331\bin\Hostx64\x64\cl.exe  /nologo /TP ... -c C:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_17.0.1\llvm-project\clang\lib\Analysis\FlowSensitive\HTMLLogger.cpp
C:\repos\llvm\llvm-project\llvm\utils\release\llvm_package_17.0.1\llvm-project\clang\lib\Analysis\FlowSensitive\HTMLLogger.cpp(332): fatal error C1001: Internal compiler error.
(compiler file 'msc1.cpp', line 1587)
 To work around this problem, try simplifying or changing the program near the locations listed above.
If possible please provide a repro here: https://developercommunity.visualstudio.com
Please choose the Technical Support command on the Visual C++
 Help menu, or open the Technical Support help file for more information

Additional troubleshooting is obviously required to make progress here. I might try to create a narrowed down testcase for the internal compiler error or perhaps just report it as is and let the Visual C++ team figure that out.