As of this post, neither the build instructions on Elmer’s webpage nor the Elmer GitHub repo include instructions for how to build Elmer on Windows. Here is a succinct set of instructions (discovered in the previous post, which detailed the stumbling around that led me a solution).
Clone the Elmer source code.
cd \repos
mkdir fem
cd fem
git clone git://www.github.com/ElmerCSC/elmerfem
Switch to the parent directory of the elmer repo using cd /repos/fem. Note that the elmer repo directory will have siblings such as build and install, so the separate parent directory (fem in this example) simplifies the directory organization.
Create the required directories and switch to the build directory.
# create folders required for building a local install
mkdir -p bundle_msys2/bin
mkdir -p bundle_qt5/bin
mkdir -p platforms
mkdir build
cd build
Deploy the prerequisite binaries for Elmer (Solver, Mesh2D, and GUI):
Build Elmer. When cmake completes, a message will be displayed confirming that generation is done and that build files have been written to the build/ directory. Run make to start compiling the source code or make install to compile the sources then create a local installation in the install/ directory. Note that there are some folders (created using mkdir -p below) that don’t appear to be used in a local build but the build still expects them to exist. You can also specify a debug build by adding the -DCMAKE_BUILD_TYPE=Debug define.
Diffusion is a key step in wafer processing for microprocessor manufacturing. Fick’s laws are therefore an important component in understanding diffusion. Fortunately, there are great resources online for an overview of Fick’s law. I started with the Khan Academy video on Fick’s law of diffusion (embedded below). It was informative but did not go into the level of detail I had hoped for.
Some digging around led to a series of lectures from the Mechanical Engineering’s Fertig Research Group at the University of Wyoming. This video on the mathematics of diffusion (Fick’s 1st law) was the level of detail I was hoping for and was therefore a good supplement to the Khan Academy diffusion overview.
And here’s the video on Fick’s 2nd law (again showing how to derive it and some brief comments about PDEs). I wonder if there is some accessible visualization that could be done of various solutions to the diffusion PDE.
The instructions for building the Elmer source code are really simple! I decided to try them on Windows. The Developer Command Prompt is necessary for cmake (as far as I can tell). Note that C, C++, and Fortran compilers are required for building Elmer.
cd \dev\repos
mkdir fem
git clone git://www.github.com/ElmerCSC/elmerfem
mkdir build
cd build
cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install ../elmerfem
I discovered that a Fortran compiler is required when I got this error on my first build attempt:
-- Building for: Visual Studio 17 2022
-- The Fortran compiler identification is unknown
-- The C compiler identification is MSVC 19.32.31326.0
-- The CXX compiler identification is MSVC 19.32.31326.0
CMake Error at CMakeLists.txt:34 (PROJECT):
No CMAKE_Fortran_COMPILER could be found.
Line 34 of CMakeLists.txt – PROJECT(Elmer Fortran C CXX) – uses the PROJECT cmake command to set the project name to “Elmer” and specify the programming languages required, hence the build failure above.
Unfortunately, that wasn’t sufficient to address the build failure. Interestingly, someone else ran into this exact same issue at windows – The MinGW gfortran compiler is not able to compile a simple test program – Stack Overflow. Sad times though when StackOverflow does not have an answer! Their solution for specifying a custom compiler is much cleaner – simply define the CMake variable when invoking cmake!
The MinGW-w64 downloads looked promising. Since I already had Cygwin installed, I installed the GFortran package. The path to the GFortran compiler can be retrieved using the Cygwin command cygpath -w `which gfortran` and passed to CMake. That still didn’t work.
At least that showed the mingw Fortran compiler package name mingw64-x86_64-gcc-fortran. Interestingly, that package is marked already installed!
Via MSYS2
Since Cygwin didn’t simply work, I decided to try installing MSYS2 (before resorting to uninstalling the Cygwin gcc-fortran package). The Fortran compiler is installed by MSYS2. Once setup completes, CMake also fails when using the MinGW Fortran compiler!
Since none of the compilers work, let’s take a closer look at the error:
$ cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=C:/dev/software/gcc/bin/gfortran.exe ../elmerfem
-- The Fortran compiler identification is unknown
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - failed
-- Check for working Fortran compiler: C:/dev/software/gcc/bin/gfortran.exe
-- Check for working Fortran compiler: C:/dev/software/gcc/bin/gfortran.exe - broken
CMake Error at C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.22/Modules/CMakeTestFortranCompiler.cmake:61 (message):
The Fortran compiler
"C:/dev/software/gcc/bin/gfortran.exe"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: D:/dev/repos/fem/build/CMakeFiles/CMakeTmp
Run Build Command(s):C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/devenv.com CMAKE_TRY_COMPILE.sln /build Debug /project cmTC_4528a &&
Microsoft Visual Studio 2022 Version 17.3.0 Preview 1.0 [...].
Copyright (C) Microsoft Corp. All rights reserved.
The operation could not be completed. The parameter is incorrect.
Use:
devenv [solutionfile | projectfile | folder | anyfile.ext] [switches]
...
To get a sense of what could be going wrong, I opened the folder containing the temporary project CMake is trying to build. Its contents are deleted before CMake terminates. However, the build was slow enough for me to copy all the files into another temp folder to repro this failure. Running the devenv.com command above fails with the same error.
Interestingly, loading the solution in Visual Studio results in an error because one of the projects cannot be loaded! However, that project file has a .vfproj extension (which seems specific to the Intel Fortran compiler, e.g. as described at Cannot open vfproj file in visual studio 2017 – Intel Communities).
Looks like it’s the CMakeTestFortranCompiler.cmake file that is generating Intel Fortran projects. The first check that file is:
if(CMAKE_Fortran_COMPILER_FORCED)
# The compiler configuration was forced by the user.
# Assume the user has configured all compiler information.
set(CMAKE_Fortran_COMPILER_WORKS TRUE)
return()
endif()
The CMAKE_Fortran_COMPILER_FORCED define can be used to bail out of the custom configuration so define it when invoking cmake:
We now get a new error! Finally making some progress!
cmake -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE ../elmerfem
-- The Fortran compiler identification is unknown
CMake Deprecation Warning at cmake/Modules/FindMKL.cmake:2 (CMAKE_MINIMUM_REQUIRED):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
Call Stack (most recent call first):
CMakeLists.txt:308 (FIND_PACKAGE)
-- ------------------------------------------------
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - not found
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
CMake Error at C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find BLAS (missing: BLAS_LIBRARIES)
Call Stack (most recent call first):
C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.22/Modules/FindBLAS.cmake:1337 (find_package_handle_standard_args)
CMakeLists.txt:433 (FIND_PACKAGE)
-- Configuring incomplete, errors occurred!
See also "D:/dev/repos/fem/build/CMakeFiles/CMakeOutput.log".
See also "D:/dev/repos/fem/build/CMakeFiles/CMakeError.log".
This finally gets us past the missing package issues and on to more Fortran compiler errors!
-- Found LAPACK: D:/dev/Software/msys64/mingw64/lib
-- Checking whether D:/dev/Software/msys64/mingw64/bin/gfortran.exe supports PROCEDURE POINTER
-- Checking whether D:/dev/Software/msys64/mingw64/bin/gfortran.exe supports PROCEDURE POINTER -- no
CMake Error at CMakeLists.txt:477 (MESSAGE):
Fortran compiler does not seem to support the PROCEDURE statement.
Support for PROCEDURE Statements
CMakeLists.txt:475 is this line INCLUDE(testProcedurePointer). The included script tests the Fortran compiler but does not explain why the test fails. To see the details, append the string : ${OUTPUT} to the end of the string “Checking whether ${CMAKE_Fortran_COMPILER} supports PROCEDURE POINTER — no” (just before the closing quote). The error message now contains additional information – the same error from earlier! Opening the solution in Visual Studio confirms that yet another unsupported .vfproj has been generated.
Change Dir: D:/dev/repos/fem/build/CMakeFiles/CMakeTmp
Run Build Command(s):C:/Program Files/Microsoft Visual Studio/2022/Preview/Common7/IDE/devenv.com CMAKE_TRY_COMPILE.sln /build Debug /project cmTC_77a33 &&
Microsoft Visual Studio 2022 Version 17.3.0 Preview 1.0 [...].
Copyright (C) Microsoft Corp. All rights reserved.
The operation could not be completed. The parameter is incorrect.
Use:
devenv [solutionfile | projectfile | folder | anyfile.ext] [switches]
<Updated VS, unfortunately changing the CMake version>. This is the CMakeLists.txt generated for the solution:
That does not work though (in my developer command prompt)
CMake Error: CMake was unable to find a build program corresponding to "MinGW Makefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
CMake Error: CMake was unable to find a build program corresponding to "MinGW Makefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
Looks like I need to try this process in MSYS2.
Custom Generator in MSYS
Running which cmake in MSYS did not find cmake so here’s the version I installed.
$ pacman -Ss cmake
...
mingw64/mingw-w64-x86_64-cmake 3.23.2-1
A cross-platform open-source make system (mingw-w64)
...
$ pacman -S mingw64/mingw-w64-x86_64-cmake
This doesn’t result in being able to run cmake.exe (even though it exists on disk in D:\dev\Software\msys64\mingw64\bin). Time to hit the docs again: msys2 cmake – Search (bing.com) -> Using CMake in MSYS2 – MSYS2. No red flags there… How about a search for the exact error message: msys bash: cmake: command not found – Search (bing.com) -> c++ – CMake is not found when running through make – Stack Overflow. Aha! The answer there about launching MSYS2 using mingw32.exe leads me to inquire about how I’m launching MSYS2. Turns out I’m launching using the last shortcut below (which launches “D:\dev\Software\msys64\msys2_shell.cmd -msys“) instead of MinGW x64.lnk (which launches “D:\dev\Software\msys64\msys2_shell.cmd -mingw64“). Sure enough, which cmake now shows /mingw64/bin/cmake.
Retrying the command line now makes progress! Notice the Fortran compiler is successfully detected (and the GNU C++ compiler is also selected).
$ cmake -G "MinGW Makefiles" -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE -DBLAS_LIBRARIES=D:/dev/Software/msys64/mingw64/lib -DLAPACK_LIBRARIES=D:/dev/Software/msys64/mingw64/lib ../elmerfem
-- The Fortran compiler identification is GNU 12.1.0
-- The C compiler identification is GNU 12.1.0
-- The CXX compiler identification is GNU 12.1.0
...
The build fails but things are very promising now. The error is because Qt is missing:
-- Building ElmerGUI
-- ------------------------------------------------
CMake Deprecation Warning at ElmerGUI/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
CMake Warning at ElmerGUI/CMakeLists.txt:19 (find_package):
By not providing "FindQt5.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "Qt5", but
CMake did not find one.
Could not find a package configuration file provided by "Qt5" with any of
the following names:
Qt5Config.cmake
qt5-config.cmake
Add the installation prefix of "Qt5" to CMAKE_PREFIX_PATH or set "Qt5_DIR"
to a directory containing one of the above files. If "Qt5" provides a
separate development package or SDK, be sure it has been installed.
-- ------------------------------------------------
CMake Error at D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindQt4.cmake:1314 (message):
Found unsuitable Qt version "" from NOTFOUND, this code requires Qt 4.x
Call Stack (most recent call first):
ElmerGUI/CMakeLists.txt:42 (FIND_PACKAGE)
Installing Qt5 does not address the build failure. The new error message:
-- Building ElmerGUI
-- ------------------------------------------------
CMake Deprecation Warning at ElmerGUI/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
-- ------------------------------------------------
-- Qt5 Windows packaging
-- [ElmerGUI] Qt5: 1
-- [ElmerGUI] Qt5 Libraries: Qt5::OpenGL Qt5::Xml Qt5::Script Qt5::Gui Qt5::Core
-- ------------------------------------------------
CMake Warning (dev) at D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to `find_package_handle_standard_args` (OpenGL)
does not match the name of the calling package (Qwt). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindOpenGL.cmake:443 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
ElmerGUI/cmake/Modules/FindQwt.cmake:10 (INCLUDE)
ElmerGUI/CMakeLists.txt:61 (FIND_PACKAGE)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Found OpenGL: opengl32
CMake Warning (dev) at D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to `find_package_handle_standard_args` (Qt3) does
not match the name of the calling package (Qwt). This can lead to problems
in calling code that expects `find_package` result variables (e.g.,
`_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindQt3.cmake:213 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
D:/dev/Software/msys64/mingw64/share/cmake/Modules/FindQt.cmake:160 (include)
ElmerGUI/cmake/Modules/FindQwt.cmake:11 (INCLUDE)
ElmerGUI/CMakeLists.txt:61 (FIND_PACKAGE)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Could NOT find Qt3 (missing: QT_QT_LIBRARY QT_INCLUDE_DIR)
CMake was unable to find desired Qt version: 3. Set advanced values QT_QMAKE_EXECUTABLE and QT3_QGLOBAL_H_FILE.
-- [ElmerGUI] Qwt: FALSE
-- [ElmerGUI] QWT_LIBRARY: QWT_LIBRARY-NOTFOUND
-- [ElmerGUI] QWT_INCLUDE_DIR: QWT_INCLUDE_DIR-NOTFOUND
-- ------------------------------------------------
CMake Warning (dev) at D:/dev/Software/msys64/mingw64/lib/cmake/Qt5Core/Qt5CoreMacros.cmake:44 (message):
qt5_use_modules is not part of the official API, and might be removed in Qt
6.
Call Stack (most recent call first):
D:/dev/Software/msys64/mingw64/lib/cmake/Qt5Core/Qt5CoreMacros.cmake:431 (_qt5_warn_deprecated)
ElmerGUI/Application/CMakeLists.txt:216 (QT5_USE_MODULES)
This warning is for project developers. Use -Wno-dev to suppress it.
-- ------------------------------------------------
-- BLAS library: D:/dev/Software/msys64/mingw64/lib
-- LAPACK library: D:/dev/Software/msys64/mingw64/lib
-- ------------------------------------------------
-- Fortran compiler: D:/dev/Software/msys64/mingw64/bin/gfortran.exe
-- Fortran flags: -fallow-argument-mismatch -O2 -g -DNDEBUG
-- ------------------------------------------------
-- C compiler: D:/dev/Software/msys64/mingw64/bin/cc.exe
-- C flags: -O2 -g -DNDEBUG
-- ------------------------------------------------
-- CXX compiler: D:/dev/Software/msys64/mingw64/bin/c++.exe
-- CXX flags: -O2 -g -DNDEBUG
-- ------------------------------------------------
-- ------------------------------------------------
-- Package filename: elmerfem-9.0--20220612_Windows-AMD64
-- Patch version: 9.0-
CMake Error at cpack/ElmerCPack.cmake:99 (INSTALL):
INSTALL FILES given directory "D:/dev/Software/msys64/mingw64/lib" to
install.
Call Stack (most recent call first):
CMakeLists.txt:660 (INCLUDE)
-- Configuring incomplete, errors occurred!
See also "D:/dev/repos/fem/build/CMakeFiles/CMakeOutput.log".
See also "D:/dev/repos/fem/build/CMakeFiles/CMakeError.log".
Does this need Qt3? The ElmerGUI documentation says Qt4 (4.8 or higher). FindQt.cmake:160 (in bold above) appears to indicate that only Qt versions 3 and 4 are supported in MinGW. The mix of warnings and “could not find” makes it hard to know exactly what is wrong. The last error, for example, appears to be about the installation files directory. So is there anything wrong with Qt? I’ll assume not.
The cmake docs on installing files doesn’t point to anything peculiar in this scenario but this is a hint that my LAPACK_LIBRARIES variable is most likely wrong. Let’s drop it altogether:
# Clean up old make files
# rm -fr *
cmake -G "MinGW Makefiles" -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE -DBLAS_LIBRARIES=D:/dev/Software/msys64/mingw64/lib ../elmerfem
The build still fails but right before the error, notice the LAPACK library now has a DLL instead of a directory (below)!
So now it makes sense to drop the BLAS_LIBRARIES definition as well!
# Clean up old make files
# rm -fr *
cmake -G "MinGW Makefiles" -DWITH_ELMERGUI:BOOL=TRUE -DWITH_MPI:BOOL=FALSE -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_Fortran_COMPILER=D:/dev/Software/msys64/mingw64/bin/gfortran.exe -DCMAKE_Fortran_COMPILER_FORCED:BOOL=TRUE ../elmerfem
This build step now succeeds as indicated by the selection of libopenblas.dll.a as the BLAS and LAPACK library.
-- ------------------------------------------------
-- BLAS library: D:/dev/Software/msys64/mingw64/lib/libopenblas.dll.a
-- LAPACK library: D:/dev/Software/msys64/mingw64/lib/libopenblas.dll.a
-- ------------------------------------------------
-- Fortran compiler: D:/dev/Software/msys64/mingw64/bin/gfortran.exe
-- Fortran flags: -fallow-argument-mismatch -O2 -g -DNDEBUG
-- ------------------------------------------------
-- C compiler: D:/dev/Software/msys64/mingw64/bin/cc.exe
-- C flags: -O2 -g -DNDEBUG
-- ------------------------------------------------
-- CXX compiler: D:/dev/Software/msys64/mingw64/bin/c++.exe
-- CXX flags: -O2 -g -DNDEBUG
-- ------------------------------------------------
-- ------------------------------------------------
-- Package filename: elmerfem-9.0--20220612_Windows-AMD64
-- Patch version: 9.0-
-- Configuring done
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
QWT_INCLUDE_DIR (ADVANCED)
used as include directory in directory D:/dev/repos/fem/elmerfem/ElmerGUI/Application
...
used as include directory in directory D:/dev/repos/fem/elmerfem/ElmerGUI/Application
QWT_LIBRARY (ADVANCED)
linked by target "ElmerGUI" in directory D:/dev/repos/fem/elmerfem/ElmerGUI/Application
...
Looks like I now need to define QWT_INCLUDE_DIR and QWT_LIBRARY. Hmm, I don’t think I even installed QWT.
$ pacman -S mingw64/mingw-w64-x86_64-qwt-qt5
resolving dependencies...
looking for conflicting packages...
Packages (1) mingw-w64-x86_64-qwt-qt5-6.2.0-5
Total Download Size: 29.17 MiB
Total Installed Size: 175.53 MiB
:: Proceed with installation? [Y/n] y
:: Retrieving packages...
mingw-w64-x86_64-qwt-qt5-6.2.0-5-any 29.2 MiB 1136 KiB/s 00:26 [###...###] 100%
(1/1) checking keys in keyring [###...###] 100%
(1/1) checking package integrity [###...###] 100%
(1/1) loading package files [###...###] 100%
(1/1) checking for file conflicts [###...###] 100%
(1/1) checking available disk space [###...###] 100%
:: Processing package changes...
(1/1) installing mingw-w64-x86_64-qwt-qt5 [#########################################################################################] 100%
Optional dependencies for mingw-w64-x86_64-qwt-qt5
mingw-w64-x86_64-qt5-tools [installed]
Now that QWT is installed, we can set the include directory as follows:
CMake finally succeeds! The output ends with these lines:
-- Generating done
-- Build files have been written to: D:/dev/repos/fem/build
The generated Makefile has targets such as ElmerGUI, elmersolver, AdvectionDiffusion, FluxSolver, etc. The strange thing is that it has a line that sets SHELL = cmd.exe and so a Windows command prompt is launched when you run make.
#==================================================================
# Target rules for targets named ElmerGUI
# Build rule for target.
ElmerGUI: cmake_check_build_system
$(MAKE) $(MAKESILENT) -f CMakeFiles\Makefile2 ElmerGUI
.PHONY : ElmerGUI
# fast build rule for target.
ElmerGUI/fast:
$(MAKE) $(MAKESILENT) -f ElmerGUI\Application\CMakeFiles\ElmerGUI.dir\build.make ElmerGUI/Application/CMakeFiles/ElmerGUI.dir/build
.PHONY : ElmerGUI/fast
Now we see the expected SHELL = /bin/sh and running make actually causes code to start building! What a journey! I will write another post with simplified instructions for how to build Elmer (on Windows).
$ make
[ 0%] Building C object matc/src/CMakeFiles/matc.dir/c3d.c.obj
[ 0%] Building C object matc/src/CMakeFiles/matc.dir/clip.c.obj
[ 0%] Building C object matc/src/CMakeFiles/matc.dir/dri_ps.c.obj
[ 0%] Building C object matc/src/CMakeFiles/matc.dir/eig.c.obj
...
After learning about Czoralski crystal growth last month, I saw a mention of simulating this growth process and realized that this simulation would be a great candidate for a high performance computing project. A search for such code on GitHub.com revealed arvedes’ simple example for transient Czochralski growth simulation with Elmer. I’d never heard of this program before but it is open source and on GitHub as the elmerfem repo! Since I am a total newbie to finite element analysis, I found a video to introduce me to the field.
As for Elmer, there is a decent (as far as I can tell) set of webinars on YouTube ranging from an Introduction to Elmer to how to use Elmer in various scientific applications. Looks like a promising place to begin exploring this tool to see what it can do and how it has been used.
Crystal defects play an important role in semiconductor fabrication. One type of defect is a Frenkel defect. Understanding such defects involves determining the vacancy concentration as given by Arrhenius function. I reviewed several videos to help me understand this equation:
Background Concepts
I took a detour to remind myself about activation energy, electron volts, and Boltzmann’s constant (all of which feature when studying Arrhenius function).
Line Defects
Another type of crystal defect is a line defect, e.g. edge dislocation. These videos contain additional information about edge dislocations.
Area Defects
A stacking fault is an extra plane of atoms. Some resources about stack faults:
Gettering is a process by which impurities and defects diffuse through the crystal (controlling where defects occur). This can be used to improve yield in semiconductor manufacturing as explained in this video:
The previous post outlined my introduction to materials science with interest stemming from applications in microfabrication. Reading section 2.2 of Fabrication Engineering at the Micro- and Nanoscale left me curious for more information about crystal structures. A YouTube search for “face centered cubic structure” led me to the videos below, which proved sufficient for gaining a basic understanding of crystal structures.
These are also discussed in section 2.2 of the text and are explained in these videos. Interestingly, neither of the videos mentioned the fact that the plane notation also denotes a vector (from the origin) that is perpendicular to that plane!
An important concept when working with materials is how to represent their properties. Phase diagrams are often used for this. I have found watching lectures to be an easier way of getting into a field as new (to me) as microfabrication. This Intro to Phase Diagrams {Texas A&M: Intro to Materials} video, for example, was an easier introduction to the topic than the notes in the microfabrication book I was reading.
This video was also my first introduction to the types of issues studied in the materials science space. The next topic in the microfabrication book I was reading was crystallography. I wanted to get an overview of the area before delving into the microfabrication aspect of crystallography. A YouTube search led me to this Lecture – Intro to Crystallography from my alma mater (interestingly, from the materials science department again)!
After watching these videos, I did a quick search for material science in the Amazon books section, hoping to see the types of topics people study in this field. Materials Science and Engineering: An Introduction looks like a great candidate (cost aside)! Looks like this is an area folks in semiconductor manufacturing need to have a handle on… More to come on crystallography as it pertains to semiconductors.
It has been a while since I wrote graphics/rendering code. The bugs are very different from those I typically write/fix since so I thought I might as well share the types of issues I dealt with. Here are some of the pitfalls I encountered:
Not checking all OpenGL API results. Continuing execution when vertex/fragment shader compilation failed, for example, wastes a ton of time debugging downstream failures.
Assuming successful shader compilation means that you can assign values to every shader uniform you declared! If the uniform is not actually used to generate the shader’s output, the compiler (which I learned lives in the graphics driver) can eliminate the uniform, thereby causing attempts to set it to fail.
Using glVertexAttribPointer instead of glVertexAttribIPointer to pass integer IDs needed by a fragment shader. Wasted so much time on this because I was feeling schedule pressure and didn’t carefully read the documentation. Since I set the normalized parameter to GL_FALSE, the IDs were being converted into floats directly without normalization. TODO: study this behavior now to see exactly which floats ended up in the shader.
Passing a count of 0 to glDrawArrays. The count argument specifies the number of indices to be rendered. Took me a while to figure out why nothing was showing up after some refactoring that I did. Turns out the number of vertices in the class I created was 0. An assertion here would have saved a ton of time.
Mismatched vertex attribute formats. Spiky rendered output instead of a shape like a cube/sphere makes this one rather easy to detect. In my case, I was using 2 structs and one had an extra ID field that ended up being vertex data when the other type of struct was passed to the rendering code.
Passing GL_TEXTURE0 to a sampler2D shader instead of 0! This was a typo that I didn’t catch in course slides.
When storing vertex shader data into a buffer texture, the OpenGL APIs were used to create and bind the buffer: glCreateBuffers, glNamedBufferStorage, glCreateTextures, glTextureBuffer, glBindImageTexture. A snippet of the vertex shader code to write into the buffer is shown below. Unfortunately, the shader compilation failed with this error: 0(69) : error C1115: unable to find compatible overloaded function “imageStore(struct image1D4x32_bindless, int, struct DebugData)”.
I needed up having to define a const int numVec4sPerStruct = 6; and call imageStore for each member of the struct, e.g. imageStore(bufferTexture, gl_VertexID * numVec4sPerStruct + 1, debugData.vPosition);. See related discussions here and here.
Invalid Shader Data in Buffers
There were all 0s in the output written into the texture/shader storage buffers. I tried using imageBuffer instead of image1D to avoid getting all zeros when reading the texture image buffer. The code looked correct but I couldn’t explain why zeros were being read back despite the rendered output looking correct. To figure out why this could be happening, I initialized the texture memory to a known value (integer value -1, which turns out to be a NaN when interpreted as a float). This made it easier to explain the random crap that was being displayed (hint from stack overflow) since it made it obvious that many memory locations were not being written to. Here is a snippet of the fragment shader:
Initializing the storage to all 0xFF bytes made it possible to conclude that the wrong data was being written to the host, more specifically that the wrong locations were being written to! Who knew structs and alignment were a thing (TODO: link to alignment discussion in GLSL spec)! The host needed to interpret the shader storage using this struct:
struct FragmentData
{
unsigned int uniqueFragmentId;
unsigned int fourBytesForAlignment1;
TextureCoord2f texCoords;
VertexCoord4f glFragCoord;
unsigned int faceId;
unsigned int fourBytesForAlignment2;
unsigned int fourBytesForAlignment3;
unsigned int fourBytesForAlignment4;
};
Also see discussion in GLSL spec about std430 vs std140 for shader block storage!
C++ Bugs
Some of the bugs I introduced were also plain old C++ bugs (not OpenGL specific), e.g.
Uninitialized variables (the fovy float had a NaN value).
Copy/pasting code and missing a key fact that both Gouraud and Phong shading code paths were calling the Gouraud shader (even though the scene state output in the console showed the state had been correctly updated to Phong). That’s what you get for copy pasting and not having tests…
Wrong array indexing logic. In the bug below (commit 3cfe07aea6914a91), I was multiplying the indices by elementSize but that is wrong because lines 2-5 from the bottom already have a built in multiplication by the element size. Noticed this from the disassembly.
int VolumeDataset3D::GetIndexFromSliceAndCol(uint32_t slice, uint32_t column)
{
const int sizeOf1Row = width * sizeof(uint8_t);
int index = slice * sizeOf1Row + column;
return index;
}
const int elementSize = sizeof(VertexDataPositionedByte);
for (uint32_t slice = 0; slice < depth - 1; slice++)
{
for (uint32_t col = 0; col < width - 1; col++)
{
int index = elementSize * GetIndexFromSliceAndCol(slice, col);
int rightIndex = elementSize * GetIndexFromSliceAndCol(slice, col + 1);
int diagIndex = elementSize * GetIndexFromSliceAndCol(slice + 1, col + 1);
int bottomIndex = elementSize * GetIndexFromSliceAndCol(slice + 1, col);
VertexDataPositionedByte cellData[4] =
{
((VertexDataPositionedByte*)vertexData2D.data)[bottomIndex],
((VertexDataPositionedByte*)vertexData2D.data)[diagIndex],
((VertexDataPositionedByte*)vertexData2D.data)[rightIndex],
((VertexDataPositionedByte*)vertexData2D.data)[index],
};
I ran into (currently unexplained) crashes in both the Intel and nVidia OpenGL drivers (I used Windows only). There were also crashes (on a specific commit) from a nullref/access violation on my HP desktop but not on my Surface Pro. Found out later that the desktop was actually right to crash but the difference in behavior was certainly troubling.
Longer than expected pauses were observed during GC in JDK 7 as explained on the Buffered Logging hotspot-dev mailing list:
Some folks noticed much longer than expected
pauses that seemed to coincide with GC logging in the midst of a GC
safepoint. In that setup, the GC logs were going to a disk file (these were
often useful for post-mortem analyses) rather than to a RAM-based tmpfs
which had been the original design center assumption. The vicissitudes of
the dirty page flushing policy in Linux when
IO load on the machine (not necessarily the JVM process doing the logging)
could affect the length and duration of these inline logging stalls.
A buffered logging scheme was then implemented by us (and independently by
others) which we have used successfully to date to avoid these pauses in
high i/o
multi-tenant environments.
Note that Unified JVM Logging was introduced in JDK 9 whereas asynchronous logging was introduced in JDK17 in PR 3135. As per the Java docs, “logging messages are output synchronously” by default whereas in “asynchronous logging mode, log sites enqueue all logging messages to an intermediate buffer and a standalone thread is responsible for flushing them to the corresponding outputs.” The AWS Developer Tools Blog has an excellent writeup about how and why they implemented this feature as well as an overview of unified logging (e.g. run java -Xlog:'gc*=info:stdout' to see logging output from log_info_p, which in my case includes output from the G1InitLogger).
Starting the Backport
This is a relatively straightforward backport. Clone the jdk11u-dev repo (or your fork as appropriate). The repo was at commit 86d39a69 when I started the backport.
git clone https://github.com/openjdk/jdk11u-dev
cd jdk11u-dev/
To see the exact same outcomes, switch to that commit (if desired).
git checkout 86d39a69
To backport this feature to JDK11, cherry-pick the commit from PR 3135 onto a new branch. We need to add the upstream as a remote to enable cherry-picking PR commits.
I used Visual Studio for conflict resolution with this strategy:
Take Incoming (Source)
Inspect the diff using Compare with Unmodified… to ensure that the changes being pulled are sensible.
The rest of this section can be skipped. I am including the details of the validation of the conflict resolution strategy (i.e. ensuring nothing undesirable is getting pulled in). The advantage of the strategy outlined above is that changes that are required by the code we want to backport are most likely going to be present after conflict resolution.
None of these changes would be present if only the changes from the PR 3135 commit were used. These lists are generated from the blame view are therefore likely omit any delete-only diffs.
Conflict Resolution: logConfiguration.cpp
This is the list of unrelated changes (i.e. changes not in commit from PR 3135) after taking the incoming changes to logConfiguration.cpp includes (potentially partial) changes from:
Comparing the current and incoming globals.hpp reveals a significant rewriting of this file between the jdk and jdk11u-dev repos. To resolve the conflict, copy only the change from the PR 3135 commit to the target (local) globals.hpp by selecting the checkmark next to the conflict in the Visual Studio merge editor then manually fix up the last line.
Conflict Resolution: init.hpp
jdk and jdk11u-dev also have non-trivial changes to init.hpp so the Merge… command is necessary here.
Pick all the #includes from the source (conflict 1)
Pick all the changes from the target (conflict 2)
Add the new line to the merged file: AsyncLogWriter::initialize();
Conflict Resolution: thread.cpp
The Merge… command is again necessary here due to the significant number of changes between the source and target versions. Take the single line from the source and accept the merge:
cl.do_thread(AsyncLogWriter::instance());
Conflict Resolution: hashtable.hpp
Use the Merge… command once more to resolve the changes between the source and target versions. Take the single line from the source and accept the merge:
template class BasicHashtable;
Addressing Build Errors
Now that all conflicts have been resolved, build the code before committing anything. Here are additional issues that need to be resolved.
Missing ‘runtime/nonJavaThread.hpp’
D:\dev\repos\java\forks\jdk11u-dev\src\hotspot\share\logging/logAsyncWriter.hpp(31): fatal error C1083: Cannot open include file: 'runtime/nonJavaThread.hpp': No such file or directory
nonJavaThread.hpp is a file now in the upstream JDK repo. Blame shows that PR 2390 moved it out of thread.hpp. Fix:
D:\dev\repos\java\forks\jdk11u-dev\src\hotspot\share\logging/logAsyncWriter.hpp(111): error C2143: syntax error: missing ';' before '<'
D:\dev\repos\java\forks\jdk11u-dev\src\hotspot\share\logging/logAsyncWriter.hpp(111): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
D:\dev\repos\java\forks\jdk11u-dev\src\hotspot\share\logging/logAsyncWriter.hpp(144): error C3646: '_stats': unknown override specifier
D:\dev\repos\java\forks\jdk11u-dev\src\hotspot\share\logging/logAsyncWriter.hpp(144): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
D:\dev\repos\java\forks\jdk11u-dev\src\hotspot\share\logging/logAsyncWriter.hpp(155): error C3668: 'AsyncLogWriter::pre_run': method with override specifier 'override' did not override any base class methods
D:\dev\repos\java\forks\jdk11u-dev\src\hotspot\share\logging/logAsyncWriter.hpp(156): error C2039: 'pre_run': is not a member of 'NonJavaThread'
Fix: Remove the pre_run method from logAsyncWritter.hpp.
Stream Errors
./src/hotspot/share/logging/logAsyncWriter.cpp(108): error C2660: 'stringStream::as_string': function does not take 1 arguments
D:\dev\repos\java\jdk11u-dev\src\hotspot\share\utilities/ostream.hpp(220): note: see declaration of 'stringStream::as_string'
./src/hotspot/share/logging/logAsyncWriter.cpp(108): error C2661: 'AsyncLogMessage::AsyncLogMessage': no overloaded function takes 2 arguments
Fix:git cherry-pick b08595d8443bbfb141685dc5eda7c58a34738048 and resolve the conflict (year on copyright line) using Take Incoming (Source).
Unknown class AutoModifyRestore
./test/hotspot/gtest/logging/test_asynclog.cpp(205): error C2065: 'AutoModifyRestore': undeclared identifier
./test/hotspot/gtest/logging/test_asynclog.cpp(205): error C2275: 'size_t': illegal use of this type as an expression
./build/windows-x86_64-normal-server-release/hotspot/variant-server/libjvm/gtest/objs/BUILD_GTEST_LIBJVM_pch.cpp: note: see declaration of 'size_t'
./test/hotspot/gtest/logging/test_asynclog.cpp(205): error C3861: 'saver': identifier not found
cd src/hotspot/share/utilities/
curl -Lo autoRestore.hpp https://raw.githubusercontent.com/openjdk/jdk/195c45a0e11207e15c277e7671b2a82b8077c5fb/src/hotspot/share/utilities/autoRestore.hpp
# Now include autoRestore.hpp in test_asynclog.cpp
Atomic Errors
./src/hotspot/share/logging/logAsyncWriter.cpp(172): error C2039: 'release_store_fence': is not a member of 'Atomic'
D:\dev\repos\java\jdk11u-dev\src\hotspot\share\runtime/atomic.hpp(51): note: see declaration of 'Atomic'
./src/hotspot/share/logging/logAsyncWriter.cpp(172): error C3861: 'release_store_fence': identifier not found
./src/hotspot/share/logging/logConfiguration.cpp(114): error C3861: 'disable_outputs': identifier not found
./src/hotspot/share/logging/logConfiguration.cpp(278): error C2039: 'disable_outputs': is not a member of 'LogConfiguration'
D:\dev\repos\java\forks\jdk11u-dev\src\hotspot\share\logging/logConfiguration.hpp(39): note: see declaration of 'LogConfiguration'
./src/hotspot/share/logging/logConfiguration.cpp(279): error C2065: '_n_outputs': undeclared identifier
./src/hotspot/share/logging/logConfiguration.cpp(293): error C2065: '_outputs': undeclared identifier
./src/hotspot/share/logging/logConfiguration.cpp(296): error C3861: 'delete_output': identifier not found
./src/hotspot/share/logging/logConfiguration.cpp(298): error C2248: 'LogOutput::set_config_string': cannot access protected member declared in class 'LogOutput'
D:\dev\repos\java\forks\jdk11u-dev\src\hotspot\share\logging/logOutput.hpp(63): note: see declaration of 'LogOutput::set_config_string'
D:\dev\repos\java\forks\jdk11u-dev\src\hotspot\share\logging/logConfiguration.hpp(31): note: see declaration of 'LogOutput'
Line 114 is simple the method call disable_outputs(); Since that method body is present in the file, it must be missing in the header file. The correct logConfiguration.hpp shows that 8255756: Disabling logging does unnecessary work is necessary. (This error might have been visible earlier in the process!)
Once the build succeeds on Windows, validate the changes by building on macOS.
Undeclared identifier ‘primitive_hash’
/Users/saint/repos/java/forks/jdk11u-dev/src/hotspot/share/utilities/hashtable.hpp:326:36: error: use of undeclared identifier 'primitive_hash'
unsigned (*HASH) (K const&) = primitive_hash<K>,
^
/Users/saint/repos/java/forks/jdk11u-dev/src/hotspot/share/utilities/hashtable.hpp:327:46: error: use of undeclared identifier 'primitive_equals'
bool (*EQUALS)(K const&, K const&) = primitive_equals<K>
Fix:
diff --git a/src/hotspot/share/utilities/hashtable.hpp b/src/hotspot/share/utilities/hashtable.hpp
index 30483b2f36..5e4c414490 100644
--- a/src/hotspot/share/utilities/hashtable.hpp
+++ b/src/hotspot/share/utilities/hashtable.hpp
@@ -30,6 +30,7 @@
#include "oops/oop.hpp"
#include "oops/symbol.hpp"
#include "runtime/handles.hpp"
+#include "utilities/resourceHash.hpp"
// This is a generic hashtable, designed to be used for the symbol
// and string tables.
Default Member Initializer is a C++11 Extension
/Users/saint/repos/java/forks/jdk11u-dev/src/hotspot/share/logging/logAsyncWriter.hpp:149:33: error: default member initializer for non-static data member is a C++11 extension [-Werror,-Wc++11-extensions]
const size_t _buffer_max_size = {AsyncLogBufferSize / (sizeof(AsyncLogMessage) + vwrite_buffer_size)};
^
Fix:
diff --git a/src/hotspot/share/logging/logAsyncWriter.cpp b/src/hotspot/share/logging/logAsyncWriter.cpp
index 0231be78a9..d9f9ddda5b 100644
--- a/src/hotspot/share/logging/logAsyncWriter.cpp
+++ b/src/hotspot/share/logging/logAsyncWriter.cpp
@@ -82,7 +82,8 @@ void AsyncLogWriter::enqueue(LogFileOutput& output, LogMessageBuffer::Iterator m
AsyncLogWriter::AsyncLogWriter()
: _initialized(false),
- _stats(17 /*table_size*/) {
+ _stats(17 /*table_size*/),
+ _buffer_max_size(AsyncLogBufferSize / (sizeof(AsyncLogMessage) + vwrite_buffer_size)) {
if (os::create_thread(this, os::asynclog_thread)) {
_initialized = true;
} else {
diff --git a/src/hotspot/share/logging/logAsyncWriter.hpp b/src/hotspot/share/logging/logAsyncWriter.hpp
index 313dd6de06..c4e28e5676 100644
--- a/src/hotspot/share/logging/logAsyncWriter.hpp
+++ b/src/hotspot/share/logging/logAsyncWriter.hpp
@@ -146,7 +146,7 @@ class AsyncLogWriter : public NonJavaThread {
// The memory use of each AsyncLogMessage (payload) consists of itself and a variable-length c-str message.
// A regular logging message is smaller than vwrite_buffer_size, which is defined in logtagset.cpp
- const size_t _buffer_max_size = {AsyncLogBufferSize / (sizeof(AsyncLogMessage) + vwrite_buffer_size)};
+ const size_t _buffer_max_size;
AsyncLogWriter();
void enqueue_locked(const AsyncLogMessage& msg);
‘override’ keyword is a C++11 extension
/Users/saint/repos/java/forks/jdk11u-dev/src/hotspot/share/logging/logAsyncWriter.hpp:154:14: error: 'override' keyword is a C++11 extension [-Werror,-Wc++11-extensions]
void run() override;
^
...
Notice that the order of the parameters passed to Atomic::cmpxchg was also changed so we need to ensure that the arguments are swapped (since they were written when the new Atomic::cmpxchg was already in place). Move the first argument into the last spot.
